SuperMap: A Spatio-Temporal SLAM System for Visual-Language Navigation

TL;DR

SuperMap is a training-free 4D spatio-temporal SLAM framework (CMU + collaborators, RSS 2026) that fuses SuperOdometry-based geometric SLAM with open-vocabulary perception (GroundingDINO + SAM2) to build a persistent semantic world model for visual-language navigation. Rather than replacing foundation vision models, it sits above them as a spatial memory layer: per-frame open-vocabulary detections are associated to 3D map objects via a hybrid 2D–3D tracker, and probabilistic geometric-consistency + Bayesian semantic fusion maintain stable instance identities across long time horizons under occlusion and scene change. The output is a queryable 4D scene graph G = (V, Eₛ, Eₜ) with spatial edges (on/beside/under) and temporal edges (per-object trajectory history), serialized as structured text for VLM queries. Validated with a 2-hour continuous CMU campus indoor/outdoor run and evaluated on ScanNet + a new change-detection benchmark; open-sourced as a deployable baseline.

Key claims

Frame-level open-vocabulary perception from foundation models (GroundingDINO + SAM2) cannot by itself maintain consistent object identities or reason about scene evolution — that gap is what SuperMap fills, positioned as complementary rather than a replacement for vision foundation models [§Perception, §Insight].
A three-layer pipeline (geometric SLAM → instance-level spatio-temporal association → topological 4D scene graph) provides a training-free, model-agnostic route to a persistent open-vocabulary semantic world model [§System Architecture].
SuperOdometry supplies pose estimation and a colorized dense 3D reconstruction from RGB + depth/LiDAR + IMU that acts as the geometric prior anchoring all 2D–3D association and global consistency checks [§Geometric Layer].
A hybrid 2D–3D tracker plus probabilistic geometric consistency update and Bayesian semantic fusion maintains stable object identities across long time horizons under occlusions, partial observations, label variability, and scene change [§Instance Layer].
The topological layer abstracts the object map into a scene graph with typed spatial predicates (on/beside/under) and temporal edges (per-object trajectory history), and serializes it as structured text so VLMs can query object semantics, spatial relations, and history compositionally [§Topological Layer].
On ScanNet class-level segmentation SuperMap reaches 27.42 mIoU / 43.50 f-mIoU / 55.48 Acc, outperforming HOV-SG (26.79 / 36.05 / 35.17) and ConceptGraphs (21.62 / 24.32 / 31.05) despite running fully online [§Results, Table: Class-level Segmentation].
On ScanNet instance-level segmentation (mAP50) the gap to prior scene-graph methods is dramatic: SuperMap 63.76 (Chair) / 42.20 (Window) / 62.50 (Refrigerator) vs HOV-SG 4.58 / 0.00 / 0.00 and ConceptGraphs 0.00 across categories — sharpest quantitative evidence that persistent identity tracking is what prior object-level mapping methods were missing [§Results, Table: Instance-level Segmentation].
On a new spatio-temporal change-detection benchmark SuperMap achieves 1.000 recall on appeared-Bucket and disappeared-Chair events, 0.755 on disappeared-Plant, and 0.583 on appeared-Sign, versus DualMap collapsing to 0.000 on 5/6 categories and Khronos not reporting — first filed evidence that appearance/disappearance/relocation of instances can be tracked jointly with mapping at scene scale [§Results, Table: Change Detection Recall].
The system runs fully online with real-time high-frequency geometric SLAM fused with asynchronous open-vocabulary perception, released as an open-source deployable baseline [§Insight, §Contributions].
Generalization is demonstrated on a continuous 2-hour CMU-campus indoor-to-outdoor run without retraining, framed as evidence that the persistent-identity + temporally-consistent-semantic-map recipe scales to campus-length deployments [§Interactive 3D Instance Objects Map].

Method

SuperMap ingests synchronized RGB + depth/LiDAR + IMU streams and operates as a three-layer pipeline. The geometric layer runs SuperOdometry to jointly produce ego-motion and a colorized dense 3D reconstruction; these geometric priors anchor every subsequent 2D–3D association and are used for global map-consistency checks. The instance layer takes per-frame open-vocabulary detections from GroundingDINO + SAM2 and associates them to existing 3D map objects via a hybrid 2D–3D tracker. Two update rules run in parallel: a probabilistic geometric-consistency update over object existence, and a Bayesian semantic-label fusion, jointly maintaining stable instance identities across long horizons under occlusion and label noise. The topological layer abstracts the resulting object map into a 4D scene graph G = (V, Eₛ, Eₜ) where spatial edges encode geometric predicates (on, beside, under) and temporal edges encode per-object trajectory history (positions and appearance events over time); the graph is serialized as structured text so a downstream VLM can issue compositional queries about semantics, spatial relations, and history.

The design is training-free and model-agnostic: no fine-tuning of the underlying detection/segmentation models, and any comparable open-vocabulary detector/segmenter should slot in. The system runs high-frequency geometric SLAM synchronously and open-vocabulary perception asynchronously, letting perception latency be absorbed without stalling the pose/geometry front-end.

Results

ScanNet class-level segmentation (mIoU / f-mIoU / Acc, %): SuperMap 27.42 / 43.50 / 55.48; HOV-SG 26.79 / 36.05 / 35.17; ConceptGraphs 21.62 / 24.32 / 31.05 [§Results, Table 1].
ScanNet instance-level segmentation (mAP50 by class): SuperMap 63.76 / 42.20 / 62.50 / 33.35 / 10.00 (Chair/Window/Refrigerator/Sofa/Door); HOV-SG 4.58 / 0.00 / 0.00 / 30.00 / 9.70; ConceptGraphs 0.00 across all five categories [§Results, Table 2].
Spatio-Temporal Change Detection Recall (new benchmark): SuperMap 1.000 / 0.262 / 0.583 (appeared: Bucket / Cart / Sign) and 0.755 / 0.434 / 1.000 (disappeared: Plant / Trash / Chair); DualMap 0.000 across appeared categories and 0.310 / 0.000 / 0.000 across disappeared categories; Khronos not reporting on these events [§Results, Table 3].
Deployment: continuous 2-hour CMU-campus indoor-to-outdoor run in real time with a single system configuration, no retraining [§Interactive 3D Instance Objects Map].
Code: full release, presented as a deployable baseline including change-detection benchmark, ablations, and runtime profiling [§Contributions, §Open-Source Framework].

Why it’s interesting

SuperMap is the second concrete instance filed of the “explicit 4D scene graph as external symbolic memory for VLM/VLA agents” recipe — a direct sibling to Describe Anything Anywhere At Any Moment (DAAAM, MIT SPARK Lab, CVPR 2026). Both build hierarchical spatio-temporal graphs from RGB(-D) + pose streams, both target long-horizon embodied QA / navigation, both expose the graph to a downstream VLM via structured queries, and both explicitly frame the design as the missing spatial memory layer between vision foundation models and embodied reasoning rather than a competitor to them. The two diverge on the memory-management substrate: DAAAM leans on batched Describe-Anything-Model captions selected by an optimization front-end and uses factor-graph optimization + reconciliation to merge duplicates, while SuperMap leans on probabilistic geometric-consistency + Bayesian semantic fusion within a SuperOdometry SLAM backbone and emphasizes appearance/disappearance/relocation events as a first-class capability with its own benchmark. Neither one directly compares against the other in the retrievable material, but they clearly co-cover the same “queryable 4D SG for embodied AI” design point that Cambrian-S: Towards Spatial Supersensing in Video frames as internal-MLLM machinery — sharpening the external-vs-internal-memory question the wiki has been tracking.

The system is also a load-bearing candidate for the Parametric memory cluster’s external-symbolic pole: under the Memory in the Age of AI Agents Forms × Functions × Dynamics taxonomy, SuperMap is external-structural + experiential + evolution-heavy, the same slot DAAAM occupies but with a heavier weighting on geometric priors and change-tracking. For the VLA / VLN thread (VLA Models), it slots in as a perception + memory backbone that VLAs could sit on top of — comparable to how 3D-Object Perception Transformer (3PT) supplies zero-shot 6-DoF pose to industrial VLAs, SuperMap supplies persistent open-vocabulary object identities and trajectories to a language-conditioned navigation agent. The change-detection benchmark (with strong SuperMap recall vs DualMap collapse) is the sharpest filed argument that the object-persistence axis is a first-class missing capability, not a nice-to-have.