Skip to content
Bud's Wiki

FLASH: Fast Learning via GPU-Accelerated Simulation for High-Fidelity Deformable Manipulation in Minutes

TL;DR

Section titled “TL;DR”

FLASH is the full technical write-up of the Fan Shi group’s custom GPU-native deformable-object simulator previously used as the rollout backend in SILR (Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement). The system redesigns a non-smooth Newton contact solver for fine-grained GPU parallelism via an inertia-dominated approximation that keeps the Schur complement sparse under multi-environment execution, scaling to 3M+ DoF at 30 FPS on a single RTX 5090 and training deployable zero-shot sim-to-real cloth-folding policies in 50–600 minutes on one GPU. Headline closed-loop result: 85.8% success rate across 106 back-to-back real-world towel folds on the AdamU humanoid after sim-only training, with no real demonstrations collected.

Key claims

Section titled “Key claims”
  • A lightweight inertia-dominated approximation to the non-smooth Newton contact system eliminates the dense Schur-complement coupling that limits multi-environment scaling in the full implicit formulation, preserving sparsity across block-diagonal multi-env assembly [§IV-B1, §IV-B3, Eq. 10].
  • Multi-environment throughput beats Newton (VBD) and Isaac Sim (FEM) at matched quasi-static settings; at 256 parallel envs, FLASH reports 185.1 (vs 480.6 for Newton, 47.8 for Isaac Sim; Genesis diverges at ≥32 envs) in the per-step time table [Table II]. Note: Newton is actually faster than FLASH at the 128/256 scale in raw numbers — FLASH’s advantage is combined fidelity+throughput, not pure speed.
  • In open-loop sleeve-folding on a shared trajectory, FLASH visually matches real-world garment behavior most closely of the four GPU simulators (Isaac Sim shear-distortion, Genesis elastic snapping, Newton over-stiff bending) [Fig. 4, §V-A].
  • Sim-only training to deployable real-world policies takes 50 min (desktop single-arm towel), 50 min (humanoid single-arm towel), 150 min (dual-arm towel), 600 min (T-shirt), 600 min (shorts) on a single RTX 5090 [§V-D].
  • Zero-shot sim-to-real transfer with no real demonstrations: 85.8% success (91/106) across a one-hour continuous AdamU dual-arm evaluation on towel folding, with emergent reactive recovery against missed grasps and human interference [§VI-A1, §VI-A2].
  • The simulator parameters have predictable physical semantics: increasing gripper speed induces stronger inertial coupling; increasing bending stiffness widens the shoulder; extreme low-iteration regimes degrade behavior gracefully rather than producing numerical blow-ups [§V-B].

Method

Section titled “Method”

FLASH builds on the GPU-friendly projective-dynamics local–global pipeline, replacing the fully implicit contact metric with an inertia-dominated inverse approximation (Eq. 10) that keeps the Schur-complement sparse. Each non-smooth Newton iteration solves a saddle-point system via (i) Schur-complement assembly, (ii) constraint-increment solve, and (iii) position correction with constraint projections. Multi-environment training is handled by block-diagonal assembly of system matrices and stacking of primal variables into a single large sparse system, so hardware parallelism is handled by standard sparse linear algebra primitives rather than hand-written env-level kernels. Rendering is tightly coupled with simulation to produce depth images with correct self-occlusion that are spatially aligned to real hand-eye-calibrated hardware. Policy learning uses a teacher-student distillation framework: privileged-state heuristic teachers (end-effector keypoint grasping + reactive recovery for missed grasps, FSM for multi-stage garments) generate rollouts, which are distilled into a vision-based student policy conditioned on stacked proprioceptive + depth history, with extensive physics and visual domain randomization during distillation.

Results

Section titled “Results”
  • Throughput at 256 parallel envs (quasi-static settings): FLASH 185.1 ms/step, Isaac Sim 47.8, Newton 480.6, Genesis diverges [Table II].
  • Scaling: 3M+ DoF at 30 FPS on a single RTX 5090; sim-only policy wall-clock times 50–600 min per task on same GPU [§I, §V-D].
  • Robustness: tolerates ±cm placement translations and arbitrary azimuthal rotations spanning reachable workspace, with policy-level reactive recovery under human interference [§VI-A1].
  • Long-run deployment: 85.8% (91/106) over one hour continuous AdamU dual-arm towel folding at ≤5 cm goal-pose alignment within 40 s per trial [§VI-A2].

Why it’s interesting

Section titled “Why it’s interesting”

FLASH is the simulator-side companion paper to SILR (Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement), which had already filed FLASH as its MPPI rollout backend and reported that FLASH beat Newton and Isaac Sim at matched K under the SILR harness — this paper is the full technical justification for why FLASH wins (inertia-dominated Schur sparsity + block-diagonal multi-env assembly). The two papers together make the Fan Shi group’s case that a hand-built, task-targeted physics simulator is still the load-bearing primitive for contact-rich deformable manipulation, both as an inference-time refinement engine (SILR) and as a pure-sim training-data engine (this paper’s 50–600 min distilled policies with 85.8% real-world success). Contrasts with the WFM-as-rollout-backend branch of Simulator-in-the-Loop Refinement (DriftWorld, SIMPACT) and with the “simulation-for-evaluation-only” stance of The Role of Simulation in Scalable Robotics, Genesis World 1.0, and the Path Forward Genesis — FLASH is a sharp datapoint that specialized analytic physics still wins on the specific slice (deformable contact-rich manipulation) where general-purpose simulators have known fidelity gaps and video WFMs can’t resolve friction stability.

See also

Section titled “See also”