Statistical Analysis Methods (SAM)

Mission

Improve analysis methods and understanding of subjective experiments.

Improve the statistical analysis of objective media quality predictors/models.

Working Methods

To join the SAM mailing list (vqeg-sam@googlegroups.com) please send an e-mail to the chair at lucjan.janowski@agh.edu.pl
Chair: Lucjan Janowski
Vice-chairs: Zhi Li, Ioannis Katsavounidis, Patrick Le Callet

We have biweekly conference calls on Monday at 17:00 CET. The actions and minutes from those meetings can be found in this document. Please check the document to validate when is the next call and what software is used to call in. Zhi took the time to prepare a github repo where he deposited his code (in Python) - here is the link: https://github.com/Netflix/sureal

Documents:

SAM meeting minutes and meeting information: this document
Proposed revisions and merger of P.913, P.911, and P.910 (sent to ITU-T for consideration)

Goal

The SAM group addresses problems related to how to better analyze and improve data quality coming from subjective experiments and how to consider uncertainty in objective media quality predictors/models development.

The long-term goals are:

Improve methods used to draw conclusions from subjective experiments
Understand the process of expressing opinion in a subjective experiment
Improve subjective experiment design to facilitate analysis and applications
Improve the analysis of objective model performances

The mid-term goals are:

Popularize the analysis related to the subject model by publishing a white paper and ITU recommendation modification
Revisit standardised methods for the assessment of the performance of objective model performances

The short-term goals are:

Fix a stable method of parameter estimation for the subject model based on Maximum Likelihood Estimation (MLE) method proposed by Zhi Li (Netflix). Please check Zhi's presentation for more details.

Uniform JSON-based Subjective Data Format

Here is the repository with tools for the suJSON format. The format's name expands to "A Uniform JSON-based Subjective Data Format". Its main goal is to make it easier to store, exchange and analyze subjective data gathered in any type of subjective testing.

The repository contains a source code of a set of tools. All are aiming at automating some basic data conversion tasks (from and into suJSON). It is also a place for discussions so please do not hesitate to visit and leave a comment.

For a brief introduction to the concept, you are encouraged to go through this presentation.