References¶

VMAF is an on-going project. It has gone through substantial updates since its inception, and even more so after its open sourcing on Github in June 2016. This page attempts to maintain a (non-exhaustive) list of references on VMAF, including tech blogs, academic papers, presentations, etc. VMAF also has a Wikipedia page.

Tech Blogs¶

• Toward a practical perceptual video quality metric, June 6, 2016 -- tech blog with VMAF's open sourcing on Github.
• Dynamic Optimizer — a perceptual video encoding optimization framework, March 6, 2018 -- tech blog describing how VMAF is used in an codec-agnostic encoding optimization framework.
• Optimized shot-based encodes: now streaming!, March 9, 2018 -- tech blog describing systems design for the Dynamic Optimizer.
• VMAF: the journey continues, October 25, 2018 -- second tech blog on VMAF focus on new features and best practices.
• Toward a better quality metric for the video community, December 7, 2020 -- third tech blog on VMAF focus on speed optimization, new API design and the introduction of a codec evaluation-friendly NEG mode.
• CAMBI, a banding artifact detector, October 12, 2021 -- tech blog introducing the CAMBI algorithm to detect banding artifacts.

Academic Papers¶

Note that not all ideas in the academic papers below are implemented in the current version of VMAF open-source package (or not yet).

• A. Aaron, Z. Li, M. Manohara, J.Y. Lin, E.C.-H. Wu, and C.-C. J. Kuo, Challenges in cloud based ingest and encoding for high quality streaming media, in Proc. IEEE International Conference on Image Processing, pp. 1732–1736, 2015.
• J. Y. Lin, T. J. Liu, E. C.-H. Wu and C. C. J. Kuo, A fusion-based video quality assessment (FVQA) index, Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific, Siem Reap, 2014.
• J. Y. Lin, R. Song, C.-H. Wu, T. Liu, H. Wang, C.-C. Jay Kuo, MCL-V: A streaming video quality assessment database, Journal of Visual Communication and Image Representation, Volume 30, 2015, Pages 1-9, ISSN 1047-3203,
• J. Y. Lin, C.-H. Wu, I. Katsavounidis, Z. Li, A. Aaron and C.-C. J. Kuo, EVQA: An ensemble-learning-based video quality assessment index, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Turin, 2015.
• H. Sheikh and A. Bovik, Image information and visual quality. IEEE Transactions on Image Processing. 15 (2): 430–444, 2006.
• S. Li, F. Zhang, L. Ma, K. N. Ngan, Image quality assessment by separately evaluating detail losses and additive impairments. IEEE Transactions on Multimedia. 13 (5): 935–949, 2011.
• Z. Li and C. Bampis, Recover subjective quality scores from noisy measurements, in Proc. Data Compression Conference, April 2017.
• C. G. Bampis, Z. Li, I. Katsavounidis and A. C. Bovik, Recurrent and dynamic models for predicting streaming video quality of experience, in IEEE Transactions on Image Processing, vol. 27, no. 7, pp. 3316-3331, July 2018.
• C. G. Bampis, A. C. Bovik, Learning to predict streaming video QoE: distortions, rebuffering and memory, in arXiv e-print, 2017.
• C. G. Bampis, Z. Li, and A. C. Bovik, SpatioTemporal feature integration and model fusion for full reference video quality assessment, in arXiv e-print, 2018.
• J. Li, L. Krasula, P. Le Callet, Z. Li, Y. Baveye, Quantifying the influence of devices on quality of experience for video streaming, in Proc. Picture Coding Symposium (PCS), San Francisco, 2018.

The papers below independently evaluate the performance of VMAF.

• R. Rassool, VMAF reproducibility: validating a perceptual practical video quality metric, 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Cagliari, 2017, pp. 1-2.
• C. Lee, S. Woo, S. Baek, J. Han, J. Chae and J. Rim, Comparison of objective quality models for adaptive bit-streaming services, 2017 8th International Conference on Information, Intelligence, Systems & Applications (IISA), Larnaca, 2017.
• N. Barman, S. Schmidt, S. Zadtootaghaj, M. Martini, S. Möller, An evaluation of video quality assessment metrics for passive gaming video streaming, 23rd Packet Video Workshop 2018 (PV 2018).

White Papers¶

• Z. Li, On VMAF’s property in the presence of image enhancement operations, July 13, 2020 (Updated Dec. 11, 2020), available [online]: https://tinyurl.com/y34mgafa.

Presentations¶

• Measuring perceptual video quality at scale by A. Aaron, at Demuxed 2016.
• More efficient encoding for mobile video by A. Aaron, at Video@Scale 2017.
• Measure perceptual video quality with VMAF by Z. Li, at Netflix Industry Workshop: Video Encoding at Scale, 2017 IEEE International Conference on Image Processing (ICIP), Beijing, 2017.
• A VMAF model for 4K by Z. Li, T. Vigier and P. Le Callet, at Video Quality Experts Group (VQEG) Meeting in Madrid, March 2018.
• Quantify VMAF model variability using bootstrapping by Z. Li and I. Katsavounidis, at Video Quality Experts Group (VQEG) Meeting in Madrid, March 2018.
• VMAF: the journey continues by Z. Li, at Streaming Media West, Huntington Beach, CA, November 2018.
• Analysis tools in the VMAF open-source package by Z. Li and C. Bampis, at Video Quality Experts Group (VQEG) Meeting in Mountain View, CA, November 2018.
• Toward a better quality metric for the video community By Z. Li, at Video@Scale, November 2020.