Image and Video Quality Resources
On these pages I am compiling a list of online resources primarily aimed at researchers in the image/video quality area, which may also be useful to the larger image processing and computer vision communities. I have included publicly available image and video databases, both with (annotated) and without subjective ratings, eyetracking databases, and links to standards.
Do let me know if you are aware of other interesting online resources that I could include here.
Annotated Image Quality Databases
- IRCCyN/IVC Image Quality Database (2005)
- Test conditions include JPEG, JPEG2000 and LAR compression as well as blur.
- LIVE Image Quality Assessment Database (2006)
- Release 2 of this popular database. Distortions include JPEG, JPEG2000, white noise, Gaussian blur, and simulated Rayleigh fading channel (JPEG2000 bitstream).
- Tampere Image Database (2008)
- This database contains a large amount of test images with many distortions, including different types of noise, blur, JPEG and JPEG2000 compression and transmission with errors, local distortions, luminance and contrast changes. A paired comparison method is used in the subjective experiments to derive MOS.
- MICT Image Quality Evaluation Database (2008)
- This database focuses on image compression using JPEG and JPEG2000.
IRCCyN collected ratings for the same images using an LCD screen in a separate experiment. - IRCCyN/IVC Art Image Quality Database (2009)
- Half of this dataset are "art" images. Test conditions include JPEG, JPEG2000 and LAR compression.
- Categorical Image Quality (CSIQ) Database (2009)
- This database comprises JPEG compression, JPEG-2000 compression, global contrast decrements, additive pink Gaussian noise, and Gaussian blurring.
- Wireless Imaging Quality (WIQ) Database (2009)
- This database focuses on gray-scale JPEG compressed images and distortions introduced by a simulated wireless link.
- MMSP 3D Image Quality Assessment Database (2010)
- The first 3D image quality database. The test conditions represent different camera distances.
Annotated Video Quality Databases
- VQEG FR-TV Phase I Database (2000)
- The oldest public database on video quality. Test conditions focus on MPEG-2 compression.
- IRCCyN/IVC 1080i Database (2008)
- Test with HD video compressed using H.264. SAMVIQ MOS is available for part of the database.
- IRCCyN/IVC SD RoI Database (2009)
- Test with SD video compressed using H.264, with and without transmission errors.
- IRCCyN/IVC Eyetracker SD Database (2009)
- Test with SD H.264 video with transmission errors. The database also includes eyetracking data from the subjects.
- Poly@NYU Video Quality Database (2008-2010)
- 3 separate but related tests using videos with different frame rates and quantization parameters.
- Poly@NYU Packet Loss Database (2009)
- Small test on the impact of packet loss in H.264 videos.
- EPFL-PoliMI Video Quality Assessment Database (2009)
- Test conditions focus on H.264 compressed videos corrupted by simulated packet loss due to transmission over an error-prone network.
- LIVE Wireless Video Quality Assessment Database (2009)
- Test conditions focus on H.264 AVC/MPEG-4 Part 10 compressed videos transmitted over a wireless channel.
- LIVE Video Quality Database (2010)
- Test conditions include MPEG-2 compression, H.264 compression, simulated transmission of H.264 compressed bitstreams through error-prone IP wired and wireless networks.
- MMSP 3D Video Quality Assessment Database (2010)
- The first 3D video quality database. The test conditions represent different camera distances.
- MMSP Scalable Video Database (2010)
- Test conditions include 2 scalable video codecs using multiple spatial and temporal resolutions. Subjects performed paired comparison in side-by-side viewing sessions.
- VQEG HDTV Database (2010)
- Being released progressively via the Consumer Digital Video Library (CDVL).
Test conditions include MPEG-2 and H.264 compression as well as different types of network impairments.
Summary of Annotated Databases
| Database | Year | Type | Total | SRC | HRC | Resolution | Method | Ratings | Correlation |
| IRCCyN/IVC | 2005 | Image | 235 | 10 | 25 | DSIS | 15 | 65% | |
| LIVE | 2006 | Image | 800 | 29 | 27 | 768x512 | ACR | 32 | 88% |
| TID | 2008 | Image | 1700 | 25 | 68 | 512x384 | Custom | 144 | 55% |
| MICT | 2008 | Image | 168 | 14 | 12 | 768x512 | ACR | 16 | 61% |
| MICT/IRCCyN | 2008 | Image | 168 | 14 | 13 | 768x512 | ACR | 27 | 62% |
| IRCCyN Art | 2009 | Image | 120 | 8 | 15 | 512x512 | DSIS | 20 | |
| CSIQ | 2009 | Image | 866 | 30 | 29 | Custom | 35 | ||
| WIQ | 2009 | Image | 80 | 7 | 512x512 | DSCQS | 30 | 64% | |
| MMSP 3D | 2010 | Image | 60 | 10 | 6 | 1920x1080 | ACR | 17 | |
| VQEG FR-TV I | 2000 | Video | 320 | 20 | 16 | 480i,576i | DSCQS | 81 | 79% |
| IRCCyN/IVC 1080i | 2008 | Video | 192 | 24 | 7 | 1080i | ACR,SAMVIQ | 27 | |
| IRCCyN/IVC SD RoI | 2009 | Video | 84 | 6 | 14 | SD | ACR | 25 | |
| Eyetracker SD | 2009 | Video | 100 | 20 | 4 | SD | ACR | 30 | |
| Poly@NYU | 2008-2010 | Video | 320 | 11 | 50 | CIF/QCIF | ACR | 20 | |
| Poly@NYU PL | 2009 | Video | 17 | 17 | 1 | QVGA | ACR | 32 | |
| EPFL-PoliMI | 2009 | Video | 156 | 16 | 9 | CIF/4CIF | ACR | 40 | |
| LIVE Wireless | 2009 | Video | 160 | 10 | 16 | 768x480 | SSCQE | 29 | 86% |
| LIVE | 2010 | Video | 150 | 10 | 15 | 768x432p | ACR | 29 | 37% |
| MMSP 3D | 2010 | Video | 30 | 6 | 5 | 1920x1080 | ACR | 17 | |
| MMSP SVD | 2010 | Video | 72 | 3 | 24 | 720p | Custom | 16 | |
| VQEG HDTV | 2010 | Video | 675 | 45 | 15 | 1080i/p | ACR | 24 | 78% |
| Total: | Total number of test images or videos. |
| SRC: | Number of source (reference) images or videos. |
| HRC: | Number of test conditions (a.k.a. hypothetical reference ciruits). |
| Resolution: | Image or video format or size in pixels. i/p indicates interlaced/progressive. |
| Method: | Subjective testing method used. See ITU-R Rec. BT.500 and ITU-T Rec. P.910 for details. |
| Ratings: | Average number of valid subjective ratings per test image/video. |
| Correlation: | Approximate correlation between PSNR and MOS (where available). |
Eyetracking Databases
- USC CRCNS Dataset (2004-2006)
- Eyetracking data of 8 observers for 25 minutes of video (in original sequence as well as cut into short clips and scrambled).
- IRCCyN/IVC Image Datasets (2006)
- Dataset 1 contains eyetracking data for 27 images.
Dataset 2 contains eyetracking data from 25 observers for 84 images from the Berkeley Segmentation Dataset. The images in both datasets are uncompressed. - RUG Dataset (2008)
- Eyetracking data of 31 observers for 100 images.
- IRCCyN/IVC Video Datasets (2009)
- Dataset 1 contains eyetracking data from 37 observers for 51 H.264-compressed SD videos.
Dataset 2 is the one mentioned above; it contains subjective ratings as well as eyetracking data from 30 observers for 100 SD videos. - BTH VAIQ Dataset (2009)
- Gaze patterns of 15 observers and saliency maps for 42 reference images from the IRCCyN/IVC, MICT, and LIVE databases listed above.
- LIVE DOVES Dataset (2009)
- Fixation points of 29 observers for 101 images from van Hateren's Natural Image Database, obtained using a dual-Purkinje eye tracker.
- TU Delft Datasets (2009-2011)
- Dataset 1 contains saliency maps for 29 reference images obtained using 20 observers.
Dataset 2 contains eyetracking data from 75 observers for 160 JPEG-compressed images. Interactions contains eyetracking data from 14 observers for 54 images with varying quality for two different tasks (free viewing and quality scoring).
- USC iLab Video Dataset (2009)
- Fixation points of 14 observers for 50 HD video clips in raw uncompressed format.
- MIT CSAIL Dataset (2009)
- Eye tracking data and fixation maps of 15 viewers for 1003 images.
- NUSEF Dataset (2010)
- Eye fixations of 75 viewers for 758 images.
- INB Video Dataset (2010)
- Eye movement data from 54 subjects for 18 outdoor scenes (HD), 2 Hollywood trailers (SD), and static images taken from the outdoor scenes.
- DIEM Dataset (2010)
- Eye movement data from over 250 participants for 85 videos.
| Database | Year | Type | Scenes | Viewers |
| USC CRCNS | 2004-6 | Video | 25' | 8 |
| IRCCyN/IVC 1 | 2006 | Image | 27 | |
| IRCCyN/IVC 2 | 2006 | Image | 84 | 25 |
| TU Delft 1 | 2009 | Image | 29 | 20 |
| TU Delft 2 | 2009 | Image | 160 | 75 |
| TU Delft I | 2011 | Image | 54 | 14 |
| MIT CSAIL | 2009 | Image | 1003 | 15 |
| LIVE DOVES | 2009 | Image | 101 | 29 |
| NUSEF | 2010 | Image | 758 | 75 |
| IRCCyN/IVC 1 | 2009 | Video | 51 | 37 |
| IRCCyN/IVC 2 | 2009 | Video | 100 | 30 |
| USC iLab | 2009 | Video | 50 | 14 |
| DIEM | 2010 | Video | 85 | 250 |
| INB | 2010 | Both | 20 | 54 |
Other Databases
- TU Delft Perceived Ringing Datasets
- Maps of perceived ringing for 16 JPEG-compressed images from 12 observers, plus ringing annoyance ratings for 55 JPEG-compressed images from 20 observers.
- Consumer Digital Video Library (CDVL)
- A growing collection of high-quality uncompressed videos, some of which are annotated with subjective ratings.
- Uncompressed Colour Image Database (UCID)
- A dataset of images that were captured and are available in uncompressed form. The current version of the database (v2) has over 1300 images together with ground truth for image retrieval benchmarking.
- Camera Imaging Quality Database
- A set of image quality measurements of commercial digital cameras and lenses.
- The Official HD Video Clip List
- Collection of links to various HD clips (both compressed and uncompressed).
- Consolidated List of Test Video Clip Resources
- Collection of links and references to test videos (both compressed and uncompressed).
- Computer Vision Test Images
- Collection of links to various databases for computer vision applications.
Standards
- Video Quality Experts Group (VQEG)
- A group of experts from various backgrounds and affiliations working in the field of video quality assessment. VQEG combines the expertise and resources found in several ITU Study Groups to work towards a common goal.
- Video Services Forum (VSF), QoE Activity Group
- This group is defining metrics for characterizing content quality of video streams and describing the relationship between transport losses and video payload as a function of content. This is joint work with the the ATIS IPTV interoperability Forum (IIF).