Human detectors are surprisingly powerful reward models

TL;DR

HuDA is a zero-shot reward model for video generation that combines an off-the-shelf human detector’s confidence (a per-frame, worst-window aggregation over ViTDet’s “person” class) with a BLIP-based temporal prompt-alignment score over LLM-generated action phase captions. With no training data and no human-anomaly annotations, it beats VBench-2.0’s trained human-anomaly detector (77.4 vs 72.7 accuracy) at predicting human preference between generated video pairs, and as the reward signal in GRPO post-training of Wan 2.1 14B it wins 73% of pairwise comparisons against the base model on hard human-action prompts while preserving prompt faithfulness. The transfer surprise: GRPO with HuDA also improves animal and human-object-interaction generation, suggesting the human-detector confidence acts as a generic plausibility signal for non-rigid bodies.

Key claims

HuDA is the sum of a Human-detection score (worst-window minimum of per-frame ViTDet “person” confidence) and a temporal-Prompt-alignment score (mean BLIP similarity between uniformly sampled frames and Llama-3.2-8B-generated phase captions), weighted with α favoring H-score [§3.2].
The H-score component alone is hackable: a static video that nails the easy phase of an action gets a near-perfect human-detector score, which is what the temporal P-score is designed to penalize [§1, §3.2].
HuDA outperforms VLM-as-a-judge (55.0), DanceGRPO’s reward (51.5), and VBench-2.0’s trained human-anomaly detector (72.7) at 77.4% accuracy on a held-out human preference test, despite using no training data while VBench-2.0 uses HumanRefiner + extra real/generated supervision [Table 1].
GRPO post-training of Wan 2.1 14B with HuDA achieves 73% win-rate over the base model on hard human-action prompts, with win-rates also positive on medium and easy splits [§4.3, Fig. 4].
Prompt faithfulness is preserved (win-rate within ±1% vs base Wan 2.1), so the gains are not bought at the cost of ignoring the prompt [§4.3].
The reward generalizes off-domain: GRPO with HuDA also improves generation of animal videos and human-object interactions, without any task-specific signal [Abstract, §4.4].
Ablations: H-score alone underperforms full HuDA (HuDA wins 75% of comparisons); prompt-video similarity as P-score is weaker (57%); optical-flow P-score is worst (90% wins for full HuDA); KL-divergence in GRPO does not help (0.961 vs 0.975 HuDA score) [Table 2, Table 3].
The authors choose GRPO over DPO citing on-policy training, continuous reward signal, and multi-objective reward support; they follow DanceGRPO in dropping the KL term [§2, §3.3].

Method

HuDA decomposes “is this a plausible human doing the prompted action?” into two zero-shot signals computed over a generated video:

H-score (appearance). Run a closed-vocabulary object detector (ViTDet) on every frame; for each fixed-size temporal window of W frames, sum the highest “person” detection probability per frame, then take the minimum across windows. The minimum-over-average aggregation captures the worst stretch of the video — the failure modes (missing limbs, distorted poses) typically manifest in a small temporal region, and averaging would hide them.
P-score (motion). Prompt Llama-3.2-8B to decompose the action prompt into N=4 phase captions (e.g. a lunge → standing → bending → holding → returning), each spanning a fraction of the assumed action duration. Compute BLIP similarity between each phase caption and the corresponding uniformly-sampled frame; average. This rewards videos that progress through the action rather than statically rendering the easy phase.

The overall reward is H + α·P with α tuned to favor H. For RL post-training, Wan 2.1 1.3B and 14B variants are fine-tuned with GRPO (no KL term, following DanceGRPO) using HuDA as the single continuous reward signal; rollouts generate multiple videos per prompt, advantages are normalized within each prompt-group. Training uses A100s; medium+hard prompts for 1.3B, hard-only for 14B (high-variance prompts are more sample-efficient for GRPO).

Results

Reward-model accuracy (predicting human preference between Wan 2.1 outputs): HuDA 77.4 > VBench-2.0 72.7 > VLM-as-a-judge 55.0 > DanceGRPO reward 51.5 [Table 1].
GRPO post-training win-rates over baselines (Wan 2.1 14B + HuDA vs Wan 2.1 14B): 73% on hard prompts, with gains on medium and easy as well; faithfulness win-rate within ±1% [§4.3, Fig. 4].
Ablations on reward design [Table 2]: HuDA vs Only H-score → 75% win; vs Prompt-video similarity → 57%; vs optical-flow P-score → 90%. H-score is the dominant signal but P-score is needed to prevent reward hacking.
GRPO design [Table 3]: HuDA-score 0.975 for the default, 0.961 with KL-divergence (Wan’s original GRPO), 0.936 without prompt expansion.
Out-of-domain transfer: GRPO with HuDA also improves animal video generation and human-object interactions despite never being optimized for them [§4.4, Abstract].

Why it’s interesting

Two things stand out for a video-generation team. First, the result is a direct counterexample to the “scale a learned reward model on annotated preference data” default — an off-the-shelf detector + a frozen VLM + an LLM-decomposed prompt, all zero-shot, beats VBench-2.0’s specifically-trained anomaly detector at predicting human preference. Second, the GRPO recipe on Wan 2.1 is recognizably the same one as Reasoning RL: continuous shaped reward, no KL, group-relative advantages — but on a video diffusion policy rather than an LLM, with a hand-designed perception-stack reward in place of a verifier. The 73% win-rate over Wan 2.1 14B is a strong signal that RL-from-perception-priors is a viable post-training axis for video generators. The animal/HOI transfer is the part most worth investigating: it implies the human detector’s confidence is functioning as a general “non-rigid body plausibility” prior, not a human-specific signal.