HumanScore: Benchmarking Human Motions in Generated Videos

TL;DR

HumanScore is a benchmark for the biomechanical plausibility of human motion in AI-generated videos. It targets the gap between visual realism (VBench, T2V-CompBench) and physical plausibility by fitting an OpenSim-compatible articulated skeleton to each generated clip and scoring it against literature-derived anatomical, kinematic, and kinetic limits. Across 51 curated motion types × 2 intensities × 13 frontier video models (Sora 2, Veo 3.1 fast, Kling 2.5 Turbo Pro, Seedance 1.0 Pro fast, HunyuanVideo 1.5, Wan 2.2/2.6, etc.), Seedance 1.0 Pro fast and HunyuanVideo 1.5 co-lead at 91.1 overall; CogVideoX-5B trails at 74.8; real-video reference scores 94.3. Spearman correlation with human pairwise preference is “close to 1.0 across all evaluation dimensions” — the metric tracks human judgement of motion realism while existing VBench axes only correlate strongly on anatomy and kinematics, not kinetics.

Key claims

HumanScore is the first systematic framework that evaluates human-motion realism in generated videos via biomechanics rather than appearance metrics [§1, Contributions].
The metric set is structured as a three-tier biomechanics hierarchy — anatomical / kinematic / kinetic — with two interpretable metrics per tier (extra limbs, bone-length stability; joint range of motion, self-collision; kinematic extremes, motion smoothness) [§3.3, Fig. 6].
The motion list is curated semi-automatically from Kinetics-700: CLIP+SBERT cosine-similarity de-duplication at threshold 0.8 + farthest-point sampling, LLM-driven balancing across movement families, empirical feasibility check, yielding 51 motion types × 3 difficulty levels × 2 intensities (102 prompts) [§3.1].
Prompts are standardized along five components — Scene / Motion / Intensity / Description / Camera — with hard constraints “in an indoor studio with a neutral background”, “full-body”, “Locked, static camera; the subject stays centered and unobstructed”, “A single person” — to decouple subject motion from camera motion and avoid prompt-following ambiguity [§3.2, Fig. 5].
Bone-length, range-of-motion, self-collision, kinematic-extremes, and motion-smoothness metrics share a biomechanics-aware monocular fitting pipeline (Peiffer 2025): 87-keypoint MeTRAbs 3D lifting → optimization to a per-frame skeleton → OpenSim-compatible export; the global scale/bone-length rigidity is removed so per-frame bone lengths can vary, which is exactly what the bone-length metric measures [§3.3].
Extra-limb detection uses HADM (anomalous-limb detector) per frame with added temporal-consistency checks tuned for motion-blur duplicates [§3.3].
Self-collision uses PromptHMR-fitted SMPL-X meshes plus a BVH triangle–triangle intersection test, with adjacent faces excluded and dual-threshold filtering to suppress numerical noise and cloth contact [§3.3].
Leaderboard headline numbers (overall, on a 0-100 scale where real videos score 94.3): Seedance 1.0 Pro fast 91.1, HunyuanVideo 1.5 91.1, KlingAI 2.5 Turbo Pro 90.8, Ray 3.0 87.7, Kandinsky 5.0 pro 86.5, Sora-2 86.1, Wan 2.2 86.1, Veo 3.1 fast 85.9, Hailuo 02 85.8, PixVerse 5.5 85.5, Wan 2.6 84.8, Pika v2.2 81.6, CogVideoX-5B 74.8 [Table 3.3, §4.1].
Per-dimension strengths split: HunyuanVideo 1.5 leads anatomy, KlingAI 2.5 leads kinematic, KlingAI 2.5 also leads kinetic; the open-source HunyuanVideo 1.5 ties the proprietary Seedance for overall first place [§4.1].
Spearman rank correlation between HumanScore model-level win ratios and a human pairwise-preference study (AI and biomechanics researchers) is “close to 1.0 across all evaluation dimensions” [§4.2, Fig. 7].
Real-world videos collected from the internet for the same motion list score 94.3 overall — strictly above all 13 generators, validating the metric’s real-vs-AI discrimination, though residual jitter from monocular fitting prevents a perfect 100 even on real footage [§4.3].
Cross-benchmark correlation analysis: HumanScore anatomy and kinematic dimensions correlate strongly with VBench imaging-quality / aesthetic / subject-consistency axes (Spearman 0.669–0.962), but kinetic correlates weakly (0.186–0.349) — appearance metrics miss motion dynamics specifically [§5.1, Table 2].

Method

HumanScore evaluates a video in three stages: (1) generate the video using a standardized 5-part prompt for one of 51 curated motions × 2 intensities, (2) extract human structure with two backbones — HADM for extra-limb detection on raw frames, and a Peiffer 2025–style monocular fitting pipeline (87-point MeTRAbs 3D lifting → articulated-skeleton optimization → OpenSim export) for the remaining five metrics, plus PromptHMR-fitted SMPL-X meshes for self-collision — and (3) score against literature-derived limits. The fitting step is modified relative to Peiffer 2025 by removing the global scale / bone-length rigidity constraint, so per-frame bone lengths can vary freely; the bone-length metric then measures the relative deviation from the median bone length over time. Joint ROM uses biomechanics-standard angle limits (Rajagopal 2016) relaxed by a tolerance factor (Catelli 2019). Self-collision uses a BVH triangle–triangle test on SMPL-X meshes with adjacent-face exclusion and dual mild/severe thresholding to filter cloth-contact noise. Kinetic correctness substitutes Newton’s second law (F = ma) for the unobservable force signal: joint angular velocities computed via central differences and segment-COM linear velocities via forward kinematics are compared against per-DoF capability limits (Sun 2017); angular acceleration and jerk are compared against per-DoF limits (Grimmer 2020) for the smoothness metric. All six metric scores are normalized to 0–100 (higher is better) and aggregated by per-dimension mean.

Results

Overall ranking on 102 generated clips per model (Seedance 1.0 Pro fast 91.1 ≈ HunyuanVideo 1.5 91.1 > KlingAI 2.5 Turbo Pro 90.8 > Ray 3.0 87.7 > … > CogVideoX-5B 74.8 < real videos 94.3) — the open-source HunyuanVideo 1.5 ties the strongest proprietary model [Table 3.3].
HumanScore-derived model win ratios correlate with human researcher pairwise preferences at Spearman ρ “close to 1.0 across all evaluation dimensions” [§4.2, Fig. 7].
VBench cross-correlation (Spearman): Anatomy 0.669/0.679/0.808 against VBench imaging/aesthetic/subject-consistency; Kinematic 0.927/0.898/0.962; Kinetic 0.349/0.328/0.186. Appearance-centric scores capture pose validity but miss temporal dynamics [Table 2].
Real-vs-AI separation: real videos score 94.3 average, above all 13 generators; residual jitter from monocular fitting (depth ambiguity, occlusion, motion blur) is the source of the < 100 ceiling on real footage [§4.3].
Egocentric → not measured: HumanScore is single-subject exocentric only by prompt design (locked static camera, single person, full-body, neutral indoor studio).

Why it’s interesting

HumanScore fills the specific gap that Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning flagged on the general physics side: video models that ace VBench can still produce inhuman motion, and existing appearance benchmarks miss this almost entirely on the kinetic axis (Spearman 0.19–0.35). Where Physion-Eval relies on 10,990 human annotations of physics glitches to expose the gap, HumanScore replaces the human with a procedural OpenSim pipeline grounded in biomechanics literature — interesting on its own and a near-direct test of the VLM-as-Evaluator rule-based-vs-VLM-judge debate framed by A Very Big Video Reasoning Suite: a rule-based scorer over a deterministic substrate (human anatomy) correlates with researchers at Spearman ≈ 1.0, the same shape VBVR found for its puzzle-style tasks. For Luma the immediate value is the will-flagged one: pose / SMPL fitting is unloved on the generation side, but as a reward or eval signal it side-steps the depth-ambiguity / occlusion failures that block direct skeleton conditioning, because the fitting noise is shared across compared models and the metric reads out a relative score. The Spearman-≈1.0 human correlation also says this could plug into a UnifiedReward-style (Unified Personalized Reward Model for Vision Generation) RL post-training stack as a frozen, deterministic, free-per-call reward, complementing a VLM judge. Open questions left by the paper: whether the metric survives multi-person scenes, hand-held camera, or non-locked subjects (none of which are in the curated prompt set); and whether the OpenSim-fit noise on real videos (~5.7 points off 100) leaves enough headroom to detect future model improvements past ~94.