Inside NVIDIA Halos for Robotics: A Full-Stack Functional Safety System for Physical AI

TL;DR

NVIDIA Halos for Robotics is a full-stack functional safety platform that ports NVIDIA’s autonomous-vehicle safety stack — over 18,000 engineering years of work and standards-aligned tooling (ISO 26262, IEC 61508, ISO 13849) — onto IGX Thor for industrial robots, humanoids, and AMRs. The stack has three layers: platform hardware safety (IGX Thor SoC with an IEC 61508 SIL 3-capable Functional Safety Island plus Holoscan Sensor Bridge), Halos OS (Linux or Linux+QNX over an NV Hypervisor, with the Safety Extension Package wiring hardware errors to the FSI and Safety MCU), and Halos Applications including the open-source Outside-In Safety Blueprint. Halos is paired with an ANAB-accredited Inspection Lab that pre-assesses the platform so partners certify only their application logic. Agility (Digit humanoid) is the launch customer.

Key claims

The hardware substrate is IGX Thor, advertised at up to 2,070 FP4 TFLOPS with 14× Neoverse ARM cores, 128 GB memory at 273 GB/s, and a physically isolated IEC 61508 SIL 3-capable Functional Safety Island carrying up to 12K DMIPs with its own I/O, power, and clocks [§What is NVIDIA Halos for Robotics?].
Halos Core ships in two configurations — Linux-only, and Linux + QNX via an NV Hypervisor — where the QNX partition is positioned for higher-integrity safety functions and the Linux partition for AI and application workloads [§The safety operating system: Halos Core].
The Safety Extension Package (SEP) is the software glue that routes SoC hardware errors to the FSI and Safety MCU and includes the Edge Safety Link IEC 61508 SIL 2 protocol that extends the safety chain over Ethernet to sensors via the Holoscan Sensor Bridge [§NVIDIA IGX Thor provides platform safety, §NVIDIA Holoscan Sensor Bridge extends functional safety capabilities].
The Outside-In Safety Blueprint is decomposed into four customizable agents — Sensor Input Processing Pipeline (built on the Metropolis VSS Blueprint), Safety AI Monitor (out-of-distribution / camera-degradation detection), Safety Event Integrator (multi-camera event fusion with staleness checks), and Safety Decision Maker (a finite state machine running on the FSI) [§Outside-in safety].
Automated Trailer Loading is the worked reference use case: the SDM mutes the forklift’s onboard safety only while it is inside the trailer and no worker is in the loading ROI, with SAIM-triggered fall-back when camera inputs go OOD; this is presented as the resolution of the throughput-vs-safety tension inside trailers where onboard sensors can misread cargo as obstacles [§Automated trailer loading reference use case].
Certification is offloaded to the NVIDIA Halos AI Systems Inspection Lab, billed as the first ANAB-accredited ISO/IEC 17020 Inspection Body for AI + functional safety in both AVs and robotics; the Lab issues an Inspection Certificate that partners take to a third-party agency (TÜV Rheinland/SÜD, SGS, exida, CERTX, UL) so they certify only application logic [§Ecosystem safety: The NVIDIA Halos AI Systems Inspection Lab].
Agility Robotics is the launch humanoid partner: Digit’s safe human-detection system is being built on IGX Thor + Halos OS and assessed against IEC 61508, ISO 13849, and ISO/IEC TR 5469 through the Inspection Lab [§Ecosystem safety: The NVIDIA Halos AI Systems Inspection Lab].
Halos OS is in early access; Halos Core for IGX requires registration, and the Outside-In Safety Blueprint is Apache-2.0 at NVIDIA/halos-outside-in-safety with three profiles (base, sil for Isaac Sim closed-loop, and a WIP hil) and Claude-Code-style agent skills (hoisa-deploy-profile) that automate end-to-end deployment [§Get started with NVIDIA Halos for Robotics].

Method

Halos for Robotics is structured as three vertically integrated layers that mirror Halos for AVs. The bottom layer is IGX Thor + Holoscan Sensor Bridge: a SoC with a separate Functional Safety Island, >22,000 safety mechanisms for diagnostic coverage, in-system logic/memory BIST for latent-fault coverage, and partitioning primitives (SMMU in CCPLEX and GPU, NoC firewalls, GFX execution watchdog, clock/voltage/thermal monitors) that support Freedom from Interference and ASIL/SIL decomposition by pairing GPU/CPU, GPU/PVA, or CCPLEX-CPU/FSI-CPU. The Holoscan Sensor Bridge extends the safety chain to sensors over Ethernet using ConnectX RDMA + RTX GPU Direct for low latency, MACsec for device authentication, and an end-to-end IEC 61508 SIL 2 safety protocol.

The middle layer is Halos OS, a successor to DriveOS. Halos Core provides the Safety Extension Package (collecting SoC hardware errors and dispatching them to FSI and Safety MCU firmware), an Error Propagation Layer, and the Edge Safety Link protocol. The Linux+QNX configuration adds an NV Hypervisor that splits IGX Thor into an AI/application Linux VM and a real-time QNX VM for the safety-critical path.

The top layer is Halos Applications. The reference blueprint, Outside-In Safety, is a four-agent pipeline (SIPP → SAIM → SEI → SDM) where the SDM is the only component that must run on the FSI; perception runs on the VSS Blueprint and feeds detection events to the Safety Core via a published event-stream contract. The repo ships three deployment profiles — base (Safety Core on an existing perception feed, MUTE/UNMUTE rendered as the VST halo_safety overlay), sil (Isaac Sim drives a simulated forklift, the safety decision is fed back over ROS), and an in-progress hil — packaged via Docker Compose with NGC-hosted images.

Results

The post is an architecture/program announcement, not a benchmark paper, so the quantitative content is hardware-substrate specifications (the FSI 12K DMIPs, 22,000 safety mechanisms, IEC 61508 SC 3 systematics, IGX Thor’s 2,070 FP4 TFLOPS / 14× Neoverse cores / 128 GB at 273 GB/s) and certification-program scope (Inspection Lab membership of 43+ companies; new joiners include Agility, Lyte AI, Neurealm, Ouster, Peer Robotics joining Boston Dynamics, KION Group, Infineon, TI, NXP, Lattice, Synapticon, Reynolds & Moore, Secedge, FORT Robotics). NVIDIA reports its corpus of safety work as 18,000+ engineering years, 21B+ safety transistors assessed, 7M+ lines of safety-assessed code, 22,000+ platform safety monitors, 330+ AV-safety research papers, and 30+ certificates / assessment reports issued. Third-party assessments by TÜV SÜD and TÜV Rheinland confirm compliance across AV and robotics domains; the Automated Trailer Loading example is described as inspected by TÜV Rheinland.

Why it’s interesting

Halos for Robotics is the first filed wiki artifact to address how commercial physical-AI systems will actually ship to factories and hospitals, sitting orthogonally to the capability-centric debates on VLA Models (π*0.6, Embodied-R1.5, ABC, Spirit) and World Foundation Models — those papers argue about which policy architecture or pretraining data unlocks generalist behavior, while Halos addresses the certification, partitioning, and OOD-fall-back substrate any of those policies would have to integrate with for industrial deployment. The Outside-In Safety Blueprint’s Safety AI Monitor (OOD detection on the perception stack → mandatory fall-back to onboard safety) is the most concrete production-grade answer the wiki has yet seen to the “VLM perception failures” failure modes catalogued by BOP-Ask: Object-Interaction Reasoning for Vision-Language Models and Solving Spatial Supersensing Without Spatial Supersensing. It also complements the launch posture of Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models and ENPIRE: Agentic Robot Policy Self-Improvement in the Real World — NVIDIA’s published roadmap now spans policy (GR00T / world-action models), self-improvement (ENPIRE), and the certified hardware/OS substrate (Halos), making the full vertical legible from outside.