Skip to content
Bud's Wiki

Mixture of Frames Policy: Multi-Frame Action Denoising for Bimanual Mobile Manipulation

TL;DR

Section titled “TL;DR”

Mixture of Frames Policy (MoF) is a diffusion policy for long-horizon bimanual mobile manipulation that denoises the action chunk in multiple coordinate frames simultaneously (base, left hand, right hand, relative trajectory) rather than committing to any single hand-designed reference frame. An MoE variant learns to route per subtask between frame-specialist experts, and empirically outperforms the best oracle single-frame choice by ~3 points and vanilla Diffusion Policy by ~16 points averaged across tasks. The submission frames “which frame is best?” as a task-and-subtask-dependent question that should be answered by the policy at inference time, not by the engineer at design time.

Key claims

Section titled “Key claims”
  • No single hand-designed action frame wins across tasks: Base averages 61.0, Left 55.6, Right 55.0, Rel-Traj 55.8, with the oracle picking the best-per-task hitting 63.8 while the worst-per-task collapses to 48.8 — a 15.0 percentage-point gap depending on frame choice [F1].
  • MoF-MoE (learned routing over frame experts) beats every single-frame policy and every naive combination baseline: 66.8 average vs Oracle-Frame 63.8, MoF-Ensemble 65.9, Single-Frame Ensemble 55.9, MoE-DP 55.1, vanilla Diffusion Policy 50.3 [F2].
  • Ablations show three load-bearing components: replacing the relative-trajectory frame with a canonical variant costs 2.5 points, removing best-frame supervision costs 6.2 points, removing the auxiliary loss costs 8.0 points, and adding an orthogonal-projection variant costs 7.0 points [F3].
  • On per-subtask visualization (Pouring, Serving, Kitchen Cleanup), MoF-MoE dynamically switches which frame-expert is dominant across subtasks within a single episode rather than picking one frame globally [project page, task visualizations].

Method

Section titled “Method”

The core observation is that a bimanual mobile robot has several natural coordinate frames — mobile base, left end-effector, right end-effector, and relative-trajectory frames — and the “best” frame for representing an action chunk depends on the local subtask (e.g. a bimanual pour is naturally expressed in a hand-relative frame, while approaching a target is naturally base-relative). Rather than picking one, MoF trains a diffusion policy that denoises the action chunk in multiple frames in parallel and combines them.

Two combination schemes are reported: MoF-Ensemble simply averages/aggregates the per-frame denoising outputs, and MoF-MoE treats each frame as an expert with a learned router that produces per-subtask weights, plus best-frame supervision and an auxiliary loss during training. The MoE variant is the headline recipe; ablations indicate all three training-side pieces (best-frame supervision, auxiliary loss, canonical rel-traj frame) are load-bearing.

Results

Section titled “Results”
  • MoF-MoE 66.8 vs Diffusion Policy 50.3 averaged over three long-horizon bimanual tasks (Pouring, Serving, Kitchen Cleanup) — a 16.5-point absolute gain from the frame-selection axis alone [F2].
  • Beats the oracle-per-task frame choice (66.8 vs 63.8), i.e. per-subtask adaptive frame selection strictly dominates any fixed per-task frame — the gain the paper is claiming can only be recovered by an inference-time mixture, not by better engineering upfront [F2].
  • The MoE routing beats the ensemble (66.8 vs 65.9) by a small but consistent margin, suggesting the router is doing more than aggregating variance across frames [F2].
  • Naive alternatives underperform badly: Single-Frame Ensemble (ensemble of DPs trained per frame) sits at 55.9 and MoE-DP (MoE without the frame decomposition) at 55.1, both close to vanilla DP baseline — the ensembling gain is specifically from frame diversity rather than model diversity [F2].

Why it’s interesting

Section titled “Why it’s interesting”

MoF stakes out a structurally new lever in the VLA Models recipe space distinct from every recipe currently on that page: not more action pretraining (π*0.6: a VLA That Learns From Experience (RECAP)), not cleaner teleop data (Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models), not a stronger grounding VLM (Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models), not a bigger world model (Causal World Modeling for Robot Control (LingBot-VA)), but a representation choice — which coordinate frame the action head denoises in — treated as a per-subtask MoE routing decision rather than a design-time pick. The oracle-frame → best-per-subtask gain (63.8 → 66.8) is small but meaningful because it’s evidence that even after choosing the right frame per task, further gains exist from switching frames within an episode; that’s a claim about the geometry of action distributions that hasn’t been isolated on the wiki. Complements Human-to-Robot Retargeting papers like ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control and Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots which grapple with cross-embodiment frame mismatches from a data / kinematics angle — MoF suggests the frame problem may partly be an architectural problem the policy should learn to solve.

See also

Section titled “See also”