Video Quality Experts Group (VQEG)

No Reference Metrics (NORM)

The No Reference (NR) Metrics group (NORM) is an open collaborative for developing NR metrics for monitoring visual service quality. NORM encourages knowledge sharing on all aspects of NR metric research and development. Current projects include:

NORM coordinates work using the VQEG reflector

#1) NR Metric Integration

The NORM project seeks to merge multiple NR metrics into a single model that is accurate enough to be included in an ITU Recommendation.

VQEG discussions from 2017-2019 indicated industry need for NR metrics that use the decoded video at the point of measurement to estimate:

We seek a broad scope: camera capture, encoding, decoding, transcoding, scaling, transmission, aesthetics and artistic intent, image enhancement, monitor, and display device.

Users must be able to modify the scope to ignore specific impairments. For example, broadcasters want MOS to ignore artistic intent and aesthetics. The omitted factors should not influence MOS.

The performance goals are robust accuracy with a minimum of operational restriction. Use cases include Video on Demand (VoD), live broadcast services, social media, first responder video, medical, and AI vision systems (autonomous vehicles). Access to the bitstream is beneficial, but not required.

NORM is an open collaborative group that intends to make all of its work public, royalty free.

Working Methods:

#2) Improved Complexity Metric

This effort began by clarifying SI and TI. The ITU-T Rec. P.910 definitions of SI and TI had contained ambiguities. Agreement was reached on how to eliminate these ambiguities. Our proposal was submitted to ITU-T Study Group 12, to revise P.910.

Work continues on developing an improved metric that assesses the coding complexity of videos. Design specifications include:

Working Documents:

Finished Project: Video Quality Metadata Standard

Full reference video quality metrics are readily available in most modern transcoding pipelines. Including full-reference video quality metrics as metadata in compressed bitstreams would take very little space and provides a more accurate and “green” way of estimating source video quality.

To realize this vision, we must establish a standard format to save such metadata at both elementary video bitstream level and system layer. Both hardware (device) makers and service providers have a lot to gain by offering such metadata in their compressed bitstreams.

NR metrics would still be needed in situations like the following:

Working Documents:

Finished Project: Clarifying SI and TI metrics

The VQEG NORM group worked to clarify the traditional SI and TI metrics in ITU-T Rec. P.910. The ITU updated P.910 to include these clarifications. 


VQEG is Co-Chaired by: Margaret Pinson, NTIA/ITS and Kjell Brunnström, RISE Research Institute of Sweden AB
The VQEG website is hosted by ITS.Click here to contact the VQEG webmaster.