Unveiling Perceptual Artifacts: A Fine-Grained Benchmark for Interpretable AI-Generated Image Detection

TL;DR

X-AIGD is a fine-grained benchmark for interpretable AI-generated image (AIGI) detection. Where prior AIGI benchmarks treat the task as binary real-vs-fake classification and offer little to no localized evidence for the decision, X-AIGD ships pixel-level, categorized artifact annotations across three tiers — low-level distortions, high-level semantic errors, and cognitive-level counterfactuals — so detectors can be scored on what artifact they identified, where, not just on overall accuracy. The framing is that interpretability gaps in AIGI detection trace back to dataset design: existing corpora lack artifact diversity and localized labels, and the paper attempts to fix both at once.

Key claims

• Current AIGI detection benchmarks predominantly score binary classification and lack interpretable, convincing evidence for their decisions [§1].
• The interpretability gap stems from dataset design rather than detector design: existing benchmarks have limited artifact diversity and lack detailed, localized annotations [§1].
• X-AIGD provides pixel-level, categorized annotations of perceptual artifacts spanning three tiers — low-level distortions, high-level semantics, and cognitive-level counterfactuals [§Abstract].
• The three-tier annotation scheme is designed to enable fine-grained interpretability evaluation, not just per-image accuracy, and to support analysis of model decision-making processes [§Abstract].

Method

The paper introduces X-AIGD, a benchmark dataset and accompanying evaluation protocol for explainable AI-generated image detection. The core artifact is a corpus of synthetic images with pixel-level segmentation-style annotations of perceptual artifacts, categorized into three levels:

1. Low-level distortions — texture, edge, lighting, geometric irregularities at the pixel/patch scale.
2. High-level semantics — anatomical errors, object-part inconsistencies, scene-composition violations.
3. Cognitive-level counterfactuals — content that is locally coherent but globally implausible (impossible physics, contradictory shadows, semantic anachronisms).

Detectors are evaluated not only on whether they correctly flag an image as AI-generated, but on whether their attribution / explanation aligns with the human-annotated artifact regions and their categories. The retrieved abstract (from alphaxiv mirror) does not expose the specific generator mix used to source images, the annotation team’s protocol, or the headline detector baselines and numbers; the dataset details and quantitative comparisons are in the full paper.

Results

The retrieved abstract does not surface quantitative results. The contribution is positioned as the benchmark itself plus the resulting analysis of model decision-making, rather than a new detector that tops a leaderboard.

Why it’s interesting

The paper sits at an unusual intersection for this wiki: it is a VLM-side benchmark whose subject is the failure modes of the diffusion-side generators most of the wiki covers. The three-tier taxonomy (low-level distortion / high-level semantics / cognitive counterfactuals) is the same axis along which ERNIE-Image-Aes: Robust Image Aesthetics Scoring with Balanced Category Generalization documents the bias modes of existing aesthetics scorers — both papers are arguing that the next-generation visual evaluator needs categorical annotations, not a scalar. It also complements VLM-as-Evaluator‘s open question on which judge backbones reliably ground their decisions in visible evidence: a detector that scores well on X-AIGD’s pixel-aligned interpretability metric is, by construction, one whose attribution survives external inspection. Of note for Luma’s internal generation work: the cognitive-level counterfactual tier is the one current AIGI detectors are weakest on, and the one closest to what frontier image generators (Nano Banana 2, GPT-Image-2, ChatGPT Images 2.0) still fail at — making this dataset usable both as a detector benchmark and as a diagnostic for generator failure modes.

See also

• VLM-as-Evaluator — VLM-as-judge concept; X-AIGD’s pixel-aligned interpretability metric is one way to validate that a VLM evaluator’s stated evidence actually matches a human-annotated artifact region.
• ERNIE-Image-Aes: Robust Image Aesthetics Scoring with Balanced Category Generalization — ERNIE-Image-Aes documents categorical bias modes of legacy scalar aesthetic scorers; X-AIGD does the same on the detection side.
• When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generators — analyses why modern T2I models fail as reliable training-data generators; X-AIGD’s cognitive-counterfactual tier is the labelled-evidence companion to that argument.
• IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation — IGenBench decomposes prompts into atomic yes/no checks for infographic reliability; X-AIGD decomposes artifacts into a three-tier taxonomy for detection — adjacent decomposition strategies for adjacent reliability problems.