Statistical Analysis Methods (SAM)
Mission
Improve analysis methods and understanding of subjective experiments.
Improve the statistical analysis of objective media quality predictors/models.
Working Methods
- To join the SAM mailing list (vqeg-sam@googlegroups.com) please send an e-mail to the chair at lucjan.janowski@agh.edu.pl
- Chair: Lucjan Janowski
- Vice-chairs: Zhi Li, Ioannis Katsavounidis, Patrick Le Callet
We have biweekly conference calls on Monday at 17:00 CET. The actions and minutes from those meetings can be found in this document. Please check the document to validate when is the next call and what software is used to call in. Zhi took the time to prepare a github repo where he deposited his code (in Python) - here is the link: https://github.com/Netflix/sureal
Documents:
- SAM meeting minutes and meeting information: this document
- Proposed revisions and merger of P.913, P.911, and P.910 (sent to ITU-T for consideration)
Goal
The SAM group addresses problems related to how to better analyze and improve data quality coming from subjective experiments and how to consider uncertainty in objective media quality predictors/models development.
The long-term goals are:
- Improve methods used to draw conclusions from subjective experiments
- Understand the process of expressing opinion in a subjective experiment
- Improve subjective experiment design to facilitate analysis and applications
- Improve the analysis of objective model performances
The mid-term goals are:
- Popularize the analysis related to the subject model by publishing a white paper and ITU recommendation modification
- Revisit standardised methods for the assessment of the performance of objective model performances
The short-term goals are:
- Fix a stable method of parameter estimation for the subject model based on Maximum Likelihood Estimation (MLE) method proposed by Zhi Li (Netflix). Please check Zhi's presentation for more details.
Uniform JSON-based Subjective Data Format
Here is the repository with tools for the suJSON format. The format's name expands to "A Uniform JSON-based Subjective Data Format". Its main goal is to make it easier to store, exchange and analyze subjective data gathered in any type of subjective testing.
The repository contains a source code of a set of tools. All are aiming at automating some basic data conversion tasks (from and into suJSON). It is also a place for discussions so please do not hesitate to visit and leave a comment.
For a brief introduction to the concept, you are encouraged to go through this presentation.