EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing

TL;DR

EditReward is a VLM-based reward model for instruction-guided image editing, trained on EditReward-Data — a new 200K-pair human-preference corpus annotated by trained experts under a rigorous protocol. The model achieves state-of-the-art correlation with human raters on GenAI-Bench, AURORA-Bench, ImagenHub, and a new companion EditReward-Bench (multi-way preference prediction), outperforming a wide range of VLM-as-judge baselines. As a downstream demonstration, the authors use EditReward to score the ~46K ShareGPT-4o-Image edit samples, retain the top 20K, and fine-tune Step1X-Edit on the curated subset — closing some of the gap between open-source and closed-source editors (GPT-Image-1, Seedream, Nano Banana).

Key claims

The bottleneck for open-source image editors catching up to closed-source models (GPT-Image-1, Seedream, Google Nano Banana) is not architecture but the lack of a reliable reward model to scale up high-quality synthetic training data [§1].
EditReward-Data is a 200K-pair preference dataset for instruction-guided image editing, manually annotated by trained experts following a rigorous protocol — distinct from prior VLM-as-judge reward sources in being human-labeled at this scale [§1, Abstract].
EditReward is a VLM-based scalar reward model (not a frozen VLM-as-judge), and it achieves SOTA correlation with human preferences on GenAI-Bench, AURORA-Bench, ImagenHub, and the new EditReward-Bench, beating a wide range of VLM-as-judge models [§4.2, Abstract].
EditReward-Bench is introduced as a more difficult evaluation than prior benchmarks because it adds multi-way preference ranking (more than binary preferences) [§4.2, project page].
As a data-curation gate, EditReward scores ~46K ShareGPT-4o-Image samples and selects the top-20K subset; fine-tuning Step1X-Edit on this curated subset yields a measurable improvement over the same model trained on the unfiltered corpus [§5, project page].

Method

EditReward has three artifacts: (1) EditReward-Data, a 200K preference-pair dataset for instruction-guided image editing, with high-quality manual annotations and diverse source models for the (instruction, source-image, edit-A, edit-B) tuples; (2) EditReward, a VLM trained on EditReward-Data as a scalar reward head over (source image, instruction, edited image), trained to align with the binary/ranked human preferences; (3) EditReward-Bench, a held-out preference benchmark that adds multi-way ranking rather than just pairwise.

The downstream pipeline used to validate the reward model is the canonical “verification-gate then fine-tune” recipe: EditReward scores every sample of ShareGPT-4o-Image (~46K instruction-edit pairs); the top 20K are kept; Step1X-Edit is fine-tuned on this curated subset. The comparison is to fine-tuning the same Step1X-Edit base on the unfiltered ~46K to isolate the effect of the reward-model gate.

Results

EditReward achieves state-of-the-art human correlation on GenAI-Bench, AURORA-Bench, ImagenHub, and EditReward-Bench, outperforming a wide range of VLM-as-judge baselines [§4.2, Abstract].
Data-curation experiment: filtering ShareGPT-4o-Image from 46K down to the top-20K samples ranked by EditReward, then fine-tuning Step1X-Edit on the curated subset, improves the editing model [§5, project page].

Why it’s interesting

EditReward fills a specific gap in the wiki’s VLM-as-Evaluator taxonomy: every prior filed instance is either a frozen VLM-as-judge at benchmark time (PICABench: How Far Are We from Physically Realistic Image Editing? PICAEval with ROI grounding, UniREditBench: A Unified Reasoning-based Image Editing Benchmark GPT-4.1 with dual text+GT-image references, IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation Gemini-2.5-Pro with prompt-decomposed atomic yes/no questions), or a context-adaptive VLM reward trained from distilled reasoning traces (Unified Personalized Reward Model for Vision Generation UnifiedReward-Flex, SFT+DPO on 90K synthetic traces). EditReward is the first filed instance of a VLM reward model trained at scale (200K pairs) from human expert annotations rather than distillation — pushing back on the “VLM-judges-VLMs” feedback loop that pervades the rest of the taxonomy.

It also sharpens the Synthetic Training Data thesis in an actionable way: the rate-limiting step for catching up to closed-source editors is not the generator architecture but the verification gate. The 46K → 20K filtering experiment on ShareGPT-4o-Image is structurally identical to the gate-and-curate pipelines in PICABench: How Far Are We from Physically Realistic Image Editing? (PICA-100K’s GPT-5 auto-annotation) and UniREditBench: A Unified Reasoning-based Image Editing Benchmark (UniREdit-Data-100K’s Gemini quality filter), but with the gate’s reliability measured against ground-truth human preferences rather than assumed. Worth tracking whether Step1X-Edit + EditReward-curated data crosses the closed-source threshold on the editing benchmarks the team cares about.