Perceptron Egocentric 1.0 — self-service API with SOTA hand tracking and EgoHome benchmark

TL;DR

Perceptron releases Perceptron Egocentric 1.0 as a self-service API: the frontier for egocentric annotations, now with batch endpoints (up to 1,000 videos/request) and a usage dashboard. Three headline claims on their newly announced EgoHome benchmark (fine-grained egocentric annotation OS benchmark): at 4 fps the system scores 0.787 vs GPT-6 Astra 0.797 at ~1/8 the cost; 8 fps gains further ground at ~1/5 the cost. 3D hand-shape reconstruction is reported 12–25% more accurate than ACE-Ego-Hand on HOT3D and ARCTIC (with and without camera calibration). Segment F1 on WGO-Bench climbs to 0.583 (vs the 0.370 the API shipped with in July). Each hand/gripper gets its own time-aligned action label stream.

Key claims

Perceptron Egocentric 1.0 ships as a self-service API with batch jobs up to 1,000 videos, pollable per-job status, downloadable labels, and a usage dashboard [tweet §1, §7].
On EgoHome (Perceptron’s forthcoming OS benchmark for fine-grained egocentric annotation), the 4 fps default profile scores 0.787 — within 0.010 of GPT-6 Astra — at approximately 1/8 the cost; 8 fps scores further gains at approximately 1/5 Astra’s cost [tweet §2, Fig. Pareto chart].
The new 3D hand reconstruction model improves hand-shape accuracy by 12–25% over ACE-Ego-Hand on HOT3D and ARCTIC, both with and without camera calibration [tweet §3].
Every hand or robot gripper receives its own independent action-label timeline — a per-end-effector breakdown of what each is doing at every step [tweet §4].
Current public benchmarks under-sample <1-second subtasks; Perceptron Egocentric beats GPT-6 Astra on 0.5–1 s segments under permissible subsegment Jaccard, which EgoHome is designed to measure [tweet §5].
Achieves segment F1 0.583 on WGO-Bench, reported as the highest known score [tweet §6].

Method

Tweet-level product announcement with Pareto plot and two benchmark figures; no model-level detail beyond what was already disclosed in the July API launch. The functional primitives — Mk1-backed segmentation + labeling, a hand pose-estimation sidecar for 2D/3D keypoints and left/right identity, optional task-instruction conditioning — carry over from Introducing Perceptron Egocentric API. What is new in the 1.0 release is (a) the EgoHome benchmark itself (short-subtask-heavy, OS-grade), (b) a new 3D hand reconstruction model that supersedes ACE-Ego-Hand, (c) per-hand-per-gripper action-label timelines as first-class output, and (d) self-service batch delivery with up-to-1,000-video jobs.

Results

EgoHome (new benchmark, fine-grained egocentric OS annotation): Perceptron-Egocentric @ 4 fps = 0.787 vs GPT-6 Astra = 0.797 at ~1/8 the cost; Perceptron-Egocentric @ 8 fps = further gains at ~1/5 the cost. Short-subtask slice (0.5–1 s): Perceptron-Egocentric > GPT-6 Astra under permissible subsegment Jaccard.
3D hand shape (HOT3D, ARCTIC): 12–25% more accurate than ACE-Ego-Hand, with/without camera calibration.
WGO-Bench segment F1: 0.583 (highest reported). The July launch reported segment F1 0.370 for the same product, so the 1.0 release is a +0.213 absolute (+58% relative) step on the same benchmark.
Delivery: batch requests up to 1,000 videos, pollable jobs, per-job usage dashboard inspection.

Why it’s interesting

This 1.0 release is a second datapoint for the thesis Perceptron shipped in July: an embodied-VLM-backed annotation pipeline can cover the egocentric video labeling pipeline at substantially lower cost than frontier general VLMs, and the gap widens as the VLM behind it is improved. The WGO-Bench segment F1 climb from 0.370 → 0.583 inside four months (same benchmark, same vendor) is the sharpest filed instance of within-product evaluator quality lifting with Mk1 pretraining updates, complementing the broader Flora / Astra / Opus / Sol cross-vendor sweeps in Flora: An Open Model Suite for Scaling Robotics Data Engines and Argus: An Open-Source Annotator for Robotics Data.

Two things it also changes in the wiki’s active questions. First, it extends VLM-as-Evaluator‘s judge-backbone-sensitivity discussion with a concrete productized instance: at 4 fps Perceptron sits within 0.010 of GPT-6 Astra on their own EgoHome at 1/8 Astra’s cost — the same cost/quality Pareto shape Kite ML’s Robot-eval-bench surfaced for success detection (Robot-eval-bench: Benchmarking Task Success Evaluation for VLMs) and that Flora’s open-weights variant shows at 125× cost reduction versus Astra on different substrates. Second, the per-hand, per-gripper action-timeline output is a stronger labeling-signal shape than the segment-label-only pipelines downstream VLA training stacks consume today (What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?, HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining) — it supplies left/right-decomposed contact structure that cotraining recipes can condition on without re-deriving it.