Image Manipulation Quality Assessment Challenge

VCIP 2025

Introduction

With the rapid growth of image sharing on social media platforms, filter-altered images have become a dominant form of visual communication. Unlike traditional distortions such as noise or compression, filter-based manipulations are often used for aesthetic enhancement, and their impact on perceived image quality is highly subjective, non-monotonic, and content-dependent. This presents new challenges for computational image quality assessment (IQA), which conventional models are not well equipped to handle.

This proposed competition invites participants to benchmark their models on our publicly available IMQA dataset. The competition aims to foster innovation in both subjective and objective quality assessment techniques tailored to manipulation-based image changes. It also encourages the exploration of new perceptual metrics and learning-based approaches that go beyond distortion-focused IQA.

Dataset

The dataset includes 360 images manipulated by various filters and over 7,000 image quality scores provided by 20 human subjects. Our dataset provides a valuable resource for understanding how filters influence perceived image quality and enables the development of IQA models that better align with human preferences in aesthetic-driven scenarios.

The dataset, training labels, test data and baseline model can be downloaded here. Test data can be found at ./Labels/mos_fold_test.csv.

More details about the dataset can be found at our paper.

Submission Instructions

  output.csv format:
  image_name	mos
  Act2_clahe_1	0.372379619
  Act4_toning_2	0.705094234
  Act5_toning_1	0.375190078
  Act6_clahe_2	0.511680965
  Ani1_clahe_1	0.479970751
  Ani1_toning_2	0.303433496
  Ani2_toning_2	0.671735144
  Ani4_pr_1	0.458737355
  Fo2_clahe_1	0.677228593

Please submit your results as a CSV file by emailing it to liuj137@cardiff.ac.uk

To ensure your submission is processed correctly, please use the following subject line:

"Challenge Submission - [Your Name/Team Name]"