SLAM on Robocap via cuVSLAM, packaged with rattler-build (Claude Code build report)

TL;DR

Pablo Velagomez (Rerun) reports getting visual SLAM running on the “Robocap” wearable capture device using NVIDIA’s cuVSLAM for GPU-accelerated multi-camera tracking, with the integration written almost entirely by Claude Code. The interesting parts are the packaging story (cuVSLAM CUDA binaries shipped via rattler-build so they install across repos and on DGX Spark ARM64 out of the box) and the zero-setup UX (git clone && pixi run track-robocap auto-downloads a 100MB dataset from HuggingFace). Still visual-only (no IMU fusion), loop closure needs work. Repo: rerun-io/robocap.

Key claims

cuVSLAM (NVIDIA’s GPU-accelerated multi-camera visual tracker) runs on the Robocap device and produces real-time trajectories, landmarks, and pose graphs viewable in Rerun [tweet body].
Packaging compiled CUDA binaries via rattler-build (pixi/conda ecosystem) gave cross-repo reuse and ARM64 (DGX Spark) compatibility without per-host setup [tweet body].
The author wrote “basically zero code” themselves — Claude Code produced the integration by pattern-matching against existing examples in surrounding repos [tweet body].
Current setup is visual-only (not visual-inertial); loop closure is acknowledged as needing debugging [tweet body].
Planned next steps: Gradio interface, integration into daggr, and an exoego-forge dataset extension; longer-term a layered pipeline (raw sensors → SLAM → human pose → depth) leveraging Rerun’s incremental-layer RRD support [tweet body].

Method

The Robocap (a wearable / robot-mounted multi-camera capture rig in the rerun-io ecosystem) feeds streams into cuVSLAM, which runs feature tracking, pose estimation, and landmark/pose-graph maintenance on GPU. Output trajectories, landmarks, and pose graphs are logged into Rerun for live 3D visualization alongside video playback. The cuVSLAM CUDA binaries are packaged as a rattler-build artifact so any consumer repo can depend on them via pixi without rebuilding from source; the same artifact is consumed on DGX Spark (ARM64). A pixi run track-robocap task auto-downloads a 100MB sample dataset from HuggingFace so the demo runs from a clean clone.

Results

No quantitative results — this is a build report, not a paper. The qualitative claim is that integration time was dominated by giving Claude Code enough surrounding examples to copy from, and that rattler-build + pixi materially simplified the CUDA-binary distribution problem that usually blocks “git clone && run” on GPU code.

Why it’s interesting

Two threads worth following. First, this is a concrete, in-the-wild example of the “tell the agent which examples to copy” pattern that papers like Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster and the cluster around The First Fully General Computer Action Model (FDM-1) and Cursor: agent swarm discovers novel solution to Problem Six of the First Proof challenge keep gesturing at — when the surrounding repo gives an LLM enough idioms, the human contribution collapses to specifying the target. Second, it’s a useful pointer for Luma’s own robotics / capture work: cuVSLAM as a drop-in GPU multi-camera tracker, plus Rerun’s incremental-layer RRD format as a way to compose perception pipelines (SLAM → pose → depth) without re-encoding data at each stage.