Image and Video Quality Resources

On these pages I have been compiling a list of online resources primarily aimed at researchers in the image/video quality area, which may also be useful to the larger image processing and computer vision communities. I have included publicly available image and video databases, both with (annotated) and without subjective ratings, eye tracking databases, and links to standards. I don't update it as much as I used to, but do let me know if you are aware of other interesting online resources that I could include here.

Please also refer to the following papers for in-depth analysis of these databases:

Tables summarizing the test material and subjective experiments for most of the image and video quality databases are available for download in comma-separated value (.csv) file format.

Annotated Image Quality Databases

Aesthetics and Attributes Database (AADB) (2016)
10,000 photographs aesthetic quality ratings and attribute assignments provided by 5 observers.

Aesthetic Visual Analysis (AVA) Database (2012)
Over 250,000 photographs with aesthetics scores and other labels.

IRCCyN/IVC Watermarking Databases (grayscale) (2007-2011)
Six separate databases deadling with watermarking and encryption distortions. They comprise Broken Arrow (BA), Fourier Subband (FSB), Enrico, Meerwald (MW), Selective Encryption (SE), and H.264 watermarking databases.

Categorical Image Quality (CSIQ) Database (2010)
Distortions include JPEG and JPEG2000 compression, global contrast decrements, additive pink Gaussian noise, and Gaussian blur.

Colourlab Image Database: Image Quality (CID:IQ) (2014)
Distortions include JPEG and JPEG2000 compression, Poisson noise, blurring, and two gamut mapping algorithms.

CUHK Content-based Photo Quality (CUHKPQ) Database (2011)
17,673 photographs with quality classifications based on 10 observers.

ESPL-LIVE HDR Subjective Image Quality Database (2016)
1800 images created by tone-mapping and exposure fusion algorithms, with and without post-processing. Subjective ratings were obtained via crowdsourcing from over 5000 observers.

JPEG XR Database (2009)
Comparing the quality of JPEG XR compression to JPEG and JPEG2000.

Konstanz Image Quality Assessment Database (KonIQ-10k) (2018)
10,073 images with 1.2 million ratings from 1,467 crowd workers.

LIVE Image Quality Assessment Database (2006)
Release 2 of this popular database. Distortions include JPEG, JPEG2000, white noise, Gaussian blur, and simulated Rayleigh fading channel (JPEG2000 bitstream).

LIVE Multiply Distorted Image Quality Database (2012)
Two multiple distortion scenarios: 1) images are first blurred and then compressed by a JPEG encoder. 2) images are first blurred and then corrupted by white Gaussian noise.

LIVE In the Wild Image Quality Challenge Database (2015)
1162 images captured using a variety of mobile devices with authentic distortions. Subjective ratings were obtained via crowdsourcing from over 8000 observers.

MCL-JCI Database (2016)
50 source images compressed using JPEG at quality factors (QF) ranging from 1 to 100. Stair quality functions (SQF) and just-noticeable difference (JND) data obtained from more than 150 volunteers are available.

Tampere Image Database (TID) (2008, 2013)
TID2013 is an extension of TID2008. Currently the largest image quality database available in the public domain, both in terms of test images and number of subjects. It contains a wide variety of distortions, including various types of noise, blur, JPEG and JPEG2000 compression, transmission errors, local image distortions, luminance and contrast changes.

VCL@FER Image Quality Database (2012)
Distortions include white Gaussian noise (AWGN), Gaussian blur, JPEG2000 and JPEG compression.

UniMiB IVL Databases (2014)
20 reference images with single and multiple distortions.

Waterloo Exploration Database & Toolbox (2016)
Database contains 4744 source natural images, together with 94880 distorted images. Also includes a toolbox for a new methodology to compare IQA models.

Annotated Video Quality Databases

AVT-VQDB-UHD-1 (2019)
16 UHD video sequences encoded with H.264, HEVC and VP9 and frame rate variations, rated by 29 observers.

BVI-HD Perceptual Video Quality Database (2018)
22 unique HD video sequences at frame rates up to 120 Hz, rated by 51 observers.

BVI-HFR High Frame Rate Video Database (2015)
32 reference and 384 distorted sequences, created using both original High Efficiency Video Coding (HEVC) and HEVC with synthesis mode, rated by 86 observers.

EPFL-PoliMI Video Quality Assessment Database (2009)
Test conditions focus on H.264 compressed videos corrupted by simulated packet loss due to transmission over an error-prone network.

ETFOS CIF & VGA Video Quality Databases (ECVQ, EVVQ) (2012)
90 test videos with distortions from H.264/AVC and MPEG-4 compression; DMOS based on 40 subjects.

IVP Subjective Video Quality Database (2011)
Progressive HD video compressed with MPEG-2, Dirac wavelet, and H.264 codecs as well as H.264 streams affected by simulated packet loss. DMOS are provided separately for expert and non-expert observers.

Konstanz Natural Video Database (KoNViD-1k) (2017)
1200 videos with subjective data and attribute evaluation.

LIVE Video Quality Database (2010)
Test conditions include MPEG-2 compression, H.264 compression, simulated transmission of H.264 compressed bitstreams through error-prone IP wired and wireless networks.

LIVE Mobile Video Quality Database (2012)
Test conditions focus on H.264 compressed videos with distortions such as packet loss, frame freeze, and rate adaptation.

LIVE Mobile Stall Video Database (2016)
176 videos generated from 24 reference videos with 26 unique stalling events, rated by 54 subjects.

LIVE Public-Domain Subjective Mobile Stall Video Database-II (2017)
174 videos generated from 24 reference videos with 26 unique stalling events. Continuous-time and overall subjective scores from 54 subjects.

LIVE-Qualcomm Subjective Mobile In-Capture Video Quality Database (2017)
208 videos captured using 8 different mobile devices with six common in-capture distortion categories, assessed by 39 subjects.

LIVE Netflix Video Quality of Experience Database (2017)
112 videos containing typical distortions encountered in adaptive streaming, evaluated by over 55 human subjects on a mobile device.

LIVE Video Quality Challenge (VQC) Database (2018)
585 videos captured using 101 different devices with wide ranges of levels of complex, authentic distortions. Crowdsourcing was used to collect an average of 240 quality ratings per video.

MCL-V Database (2015)
12 uncompressed HD source video clips with two distortion types (compression and image scaling) and four distortion levels.

MCL-JCV Database (2016)
24 source clips compressed using H.264/AVC at quality factors (QF) ranging from 1 to 51. Just-noticeable difference (JND) data obtained from about 50 volunteers are available.

MMSP Scalable Video Database (SVD) (2010)
The test conditions include two scalable video codecs using multiple spatial and temporal resolutions. The database only includes the sources together with the software and process for creating the test conditions, rather than including the test videos as such. Subjects performed paired comparisons in side-by-side viewing sessions.

Poly@NYU Video Quality Databases (2008-2012)
4 separate but related tests using videos with different frame rates and quantization parameters.

Poly@NYU Packet Loss Database (2009)
Small test on the impact of packet loss in H.264 videos. Test clips are only 2~seconds long.

Roma Tre ReTRiEVED Video Quality Database (2014)
Test video sequences comprise practical transmission scenarios with different packet loss rates, jitter, delay, and throughput, using a network simulator.

TUM 1080p25 Dataset (2010)
1080p25 video sequences of the SVT test set, encoded with H.264/AVC and Dirac.

TUM 1080p50 Dataset (2012)
1080p50 video sequences of the SVT test set encoded with H.264/AVC. Subjective tests were conducted in 4 different scenarios (different displays, with and without sound).

VQEG FR-TV Phase I Database (2000)
The oldest public quality database (interestingly it came out several years before the first image quality database). Consequently, test conditions focus on MPEG-2 compression and transmission and even include some analog distortions.

VQEG HDTV Database (2010)
Test conditions include MPEG-2 and H.264 compression as well as different types of network impairments. 5 of the 6 sets in the HDTV test are being released via the Consumer Digital Video Library (CDVL); the sixth set is not public.

Annotated 3D Image/Video Quality Databases

IRCCyN/IVC 3D Image Quality Database (2008)
The first public-domain database on 3D image quality. Test conditions include JPEG and JPEG2000 compression as well as blur.

IRCCyN/IVC DIBR Image Database (2011)
Comparison of 7 depth image-based rendering algorithms on frames extracted from 3 video sequences (ACR and pair-comparison data is available).

IRCCyN/IVC NAMA3DS1-COSPAD1 3D Video Quality Database (2012)
Test conditions include H.264 and JPEG2000 compression as well as common image processing operations like downsampling or sharpening.

IVP Anaglyph Image Database (2013)
25 stereo image pairs and 100 anaglyph images.

IVY Lab Stereoscopic 3D Image Database (2013)
120 stereo image pairs evaluated by 18 subjects.

LIVE 3D Image Quality Database (2012)
Phase I contains 20 pristine stereopairs with symmetrical distortions, while phase II contains 8 pristine stereopairs with both symmetrical and asymmetrical distortions. Distortions are similar to the LIVE database above.

MCL 3D Database (2014)
9 image-plus-depth sources undergoing depth image-based rendering (DIBR) with different distortions.

MMSP 3D Image Quality Assessment Database (2010)
The test conditions represent different inter-camera distances. All images are JPEG-compressed.

MMSP 3D Video Quality Assessment Database (2010)
The first public-domain database on 3D video quality. The test conditions represent different camera distances. All videos are slightly cropped and compressed.

VCL@FER 3D Video Quality Database (2016)
8 scenes from the IRCCyN/IVC NAMA3DS1-COSPAD1 3D Video Quality Database with 22 degradation types.

Waterloo-IVC 3D Image Quality Database (2015)
10 stereo scenes with three types of symmetrical and asymmetrical distortions.

Eye Tracking Databases

ADSC PASCAL animal classes Eye Tracking (PET) Database (2015)
Eye movement data from 40 participants for six animal-centric classes from the PASCAL Visual Object Classes (VOC) Challenge 2012, for free-viewing and visual search tasks.

Actions in the Eye (2012)
Eye movement data from 16 participants for 1857 videos, with one group performing action recognition, the other free viewing.

Coutrot datasets (2013-2016)
Multiple video datasets, including a large study on viewing of faces with 405 participants.

DIEM Project (2011)
Eye movement data from up to 42 participants for 85 videos.

GazeCom (2010)
Eye movement data from 54 subjects for 18 outdoor scenes (HD), 2 Hollywood trailers (SD), and static images taken from the outdoor scenes.

GIPSA Dynamic Natural Scenes (2014)
Eye movement data from 72 subjects for 60 videos in 4 auditory conditions.

GIPSA Meeting Scenes (2015)
Eye movement data from 40 subjects for 15 videos of 4 persons having a meeting (with and without sound).

IRCCyN/IVC Datasets:
Eyetracker 2006 05 (Image 1) Dataset (2006) contains eye tracking data of up to 40 observers for 27 images. The same database is also available here.
Berkeley (Image 2) Dataset (2006) contains eye tracking data from 25 observers for 84 images from the Berkeley Segmentation Dataset.
LIVE (Image 3) Dataset (2013) contains eye tracking data from 21 observers for 29 images from the LIVE Image Quality Database listed above.
3DGaze Dataset (2013) contains 18 stereoscopic images and the associated fixation density map, disparity map, depth map, and the raw eye tracking data.
SD 2008 11 (Video 1) Dataset (2009) contains eye tracking data from 37 observers for 51 H.264-compressed SD videos.
SD 2009 12 (Video 2) Dataset (2010) is the one listed above; it contains subjective ratings as well as eye tracking data from 30 observers for 100 SD videos in a quality rating task.

LIVE DOVES (A Database Of Visual Eye movements) (2009)
Fixation points of 29 observers for 101 images from van Hateren's Natural Image Database, obtained using a dual-Purkinje eye tracker.

McGill ImgSal (2013)
Eye movement data of 21 observers for 235 images. Also includes labels of salient regions from 19 observers.

MIT CSAIL Saliency Dataset (2009)
Eye tracking data and fixation maps of 15 viewers for 1003 images.

MIT Low-Resolution Saliency Dataset (2011)
Eye tracking data of 8 viewers for 1544 images at different resolutions.

NUS EyeCrowd Dataset (2014)
500 images of crowded scenes with eye tracking data from 16 viewers and annotation data on faces.

NUS FiWI (Fixations in Webpage Images) Dataset (2014)
149 webpages and eye tracking data from 11 subjects.

Object and Semantic Images and Eye-tracking (OSIE) Dataset (2014)
700 images with eye-tracking data of 15 viewers and annotation data of 5551 segmented objects with fine contours and 12 semantic attributes.

Pascal Objects Eye Tracking (POET) Dataset (2014)
Eye fixations of 5 viewers for 6270 images from the PASCAL Visual Object Classes (VOC) Challenge 2012.

SALICON (Saliency in Context) Dataset (2015)
10,000 training images and 5,000 validation images with saliency ground-truth.

Semiautomatic Visual-Attention Modeling (SAVAM) Database (2014)
Eye movement data of 50 viewers for 41 HD and UHDTV sequences.

SBU Gaze-Detection-Description Dataset (2013)
Eye fixations of 3 participants for 1000 images from the Pascal Visual Object Classes (VOC) dataset and of 8 participants for 104 images from the SUN09 dataset. Also includes descriptions, pre-trained object detectors, and associated bounding boxes.

SFU Video Dataset (2012)
Eye tracking data of 15 observers for 12 video sequences.

Stuttgart Video Eyetracking Benchmark Dataset (2014)
Eye tracking data of 25 observers for 11 video sequences.

Toronto Dataset (2006)
Eye movement data of up to 11 observers for 120 color images.

TU Delft Datasets
Image Quality Dataset 1 (2009) contains saliency maps for 29 reference images obtained using 20 observers.
Image Quality Dataset 2 (2011) contains eye tracking data from 75 observers for 160 JPEG-compressed images.
Image Quality Interactions (2011) contains eye tracking data from 14 observers for 54 images with varying quality for two different tasks (free viewing and quality scoring).
Video Quality Task Effect (2012) contains eye tracking data from 14 observers for 54 videos with varying quality for two different tasks (free viewing and quality scoring).

UCSB VIU Dataset (2014)
Eye movements and perceptual judgments from 161 observers viewing 800 natural images while completing 4 different tasks (free viewing, saliency judgment, and cued object search).

USC CRCNS Video Dataset (2004-2006)
Eye tracking data of 8 observers for 25 minutes of video (in original sequence as well as cut into short clips and scrambled).

USC Visual Attention Guided Bit Allocation (VAGBA) Dataset (2011)
Fixation points of 14 observers for 50 HD video clips in raw uncompressed format.

Also of interest: MIT Saliency Benchmark (2012).

Other Databases

Qualinet Databases
List of 200+ available multimedia databases collected as part of the QUALINET project.

Consumer Digital Video Library (CDVL)
A growing collection of high-quality uncompressed videos, some of which are annotated with subjective ratings.

YouTube UGC Dataset
A large scale dataset (around 1500 video clips) containing YouTube User Generated Content (UGC) intended for video compression and quality assessment research.

ITU-T Test Signals
A collection of voice, audio, image, and video data referenced in various ITU-T recommendations.

ARRI Raw Color Sequence Dataset
High-resolution color image sequences captured with a professional digital cinema camera in raw format.

Consolidated List of Test Video Clip Resources
Collection of links and references to test videos (both compressed and uncompressed).

Computer Vision Test Images
Collection of links to various databases for computer vision applications.

TU Delft Perceived Ringing Datasets
Maps of perceived ringing for 16 JPEG-compressed images from 12 observers, plus ringing annoyance ratings for 55 JPEG-compressed images from 20 observers.

UFRJ Blurred Image Database
585 images representing realistic blur (out-of-focus, motion, etc.) with ratings from users.

Colourlab Image Database: Perceptual Projection Sharpness (CID:PPS)
7 original images distorted by 6 levels of blur.

Spectral Image Database for Quality (SIDQ)
9 original 160-band hyperspectral images of scenes representing pseudo-flat surfaces of different materials with 5 spectral distortions.


Video Quality Experts Group (VQEG)
A group of experts from various backgrounds and affiliations working in the field of video quality assessment. VQEG combines the expertise and resources found in several ITU Study Groups to work towards a common goal.

Video Services Forum (VSF), QoE Activity Group
This group is defining metrics for characterizing content quality of video streams and describing the relationship between transport losses and video payload as a function of content. This is joint work with the the ATIS IPTV interoperability Forum (IIF).