Introducing HRM-Text: 1B Hierarchical Reasoning Language Model

TL;DR

HRM-Text is a 1.15B-parameter, fully open-weights “hierarchical reasoning” language model from Sapient Intelligence. It is pretrained from scratch on ~40B structured task-completion tokens (≈1/1000 of typical open-pretraining budgets) for roughly $1K of compute in about a day, and on independent April-2026 benchmarks reaches 56.2% MATH, 81.9% ARC-Challenge, 82.2% DROP, 60.7% MMLU — competitive with much larger, much more data-hungry baselines. The architectural bet is that depth-via-recurrence in latent space (two interleaved Transformer stacks, H and L, run for 8 total iterations per forward pass) substitutes for parameter-count scaling on reasoning-heavy benchmarks.

Key claims

1.15B parameters, ~40B pretraining tokens, ~$1K total pretraining cost, ~1 day wall-clock, 0.6 GiB at int4 [§Introducing HRM-Text].
Benchmark numbers reported under “independent verification, April 2026”: MATH 56.2%, DROP 82.2%, ARC-Challenge 81.9%, MMLU 60.7% — base model only, no post-training, no fine-tuning, no RL [§Benchmark Performance].
The architecture is a two-stack hierarchical recurrence: a High-level (H) and a Low-level (L) Transformer module, with the L-stack iterating faster than the H-stack; one forward pass executes 2 H updates and 6 L updates (8 stack iterations total), with cross-level state injection at every step [§Hierarchical Latent Recurrent Architecture].
“Reasoning” is performed in continuous latent space inside the forward pass, not as visible chain-of-thought tokens — the claim is that more compute happens before the first output token, decoupling reasoning depth from output-token budget [§Latent Space Reasoning].
Training data is structured instruction-response pairs (“task completion”) rather than raw next-token prediction over web text — the marketed contrast is “learning how to complete a task” vs predicting filler tokens with equal weight [§Task Completion].
The HF model card describes the run as H_cycles × (L_cycles + 1) recurrent steps with additive state injection (z_L + z_H), giving “effectively unbounded compute depth at bounded parameter count” (sapientinc/HRM-Text-1B model card).

Method

HRM-Text is built on the Hierarchical Reasoning Model (HRM) architecture: instead of a single Transformer stack doing one forward pass, two Transformer modules — H (high-level, “slow”) and L (low-level, “fast”) — share the same input embedding and iterate over it. The L-module updates more frequently than the H-module, and state is added across levels at every recurrent step. For the 1B release the schedule is fixed at 2 H-updates × 6 L-updates = 8 stack iterations per forward pass, executed entirely in continuous latent space before any token is emitted. The two stacks together contain 1.15B parameters, reused across iterations.

Training is “task completion”: curated structured instruction → response pairs rather than raw next-token-prediction over web text, with the stated goal of weighting reasoning steps and solutions over filler. The model is released as a pre-training-only checkpoint (no SFT/DPO/RL), with Apache-2.0 weights on Hugging Face (sapientinc/HRM-Text-1B), 0.6 GiB at int4 quantization for local deployment. The model card notes that some chain-of-thought-style behavior emerges as a side effect of the training mix but is “uneven and below an instruction-tuned model of comparable size.”

Results

Benchmark numbers from the announcement (independent verification, April 2026), base model only:

MATH 56.2% (multi-step mathematical reasoning)
DROP 82.2% (discrete reasoning over paragraphs)
ARC-Challenge 81.9% (science / commonsense)
MMLU 60.7% (57-domain general knowledge)

The blog frames these against models trained on 4–36T tokens (i.e. 100×–900× more data) without naming specific competitors in the body. KuCoin’s reporting of the announcement quotes “130–600× lower training cost” vs comparable standard models, and Threads coverage cites ”~$1K” total budget — both consistent with the 40B-token / 1-day figure.

There is one large caveat the blog states explicitly: HRM-Text has had no post-training. The comparison models on the (unreproduced) headline table have all been fine-tuned and aligned, which substantially shifts benchmark numbers. So the apples-to-apples claim is about architecture efficiency per pretraining token, not about deployment-ready capability.

Why it’s interesting

This is a third concrete datapoint on the Looped Transformers page that isn’t a generation-quality paper. Hyperloop Transformers and ELT: Elastic Looped Transformers for Visual Generation both argue depth-via-recurrence can match a ~2×-parameter non-looped baseline at fixed compute, and Layer-looping scaling laws — Hayden Prairie tweet gives scaling-law evidence for the loop axis itself; HRM-Text is the first filed full open-weight 1B pretraining in that family — with the strongest claim yet about data efficiency (1000× fewer tokens), but the weakest about apples-to-apples baselines (no post-training, headline comparators not fixed). It is also a natural sibling to ELT: Elastic Looped Transformers for Visual Generation‘s “any-time inference” finding: HRM’s 8 fixed iterations per forward pass is the dual of ELT’s elastic-L design, raising the open question of whether HRM’s L_cycles is exploitable as an inference-time compute knob.

Relative to Open foundation-model releases, HRM-Text extends the recent “weights + paper + recipe” pattern (Qwen3-VL, LTX-2, K2.5) with an unusual axis: a brain-inspired alternative to the Transformer-style scaling bet shipped as Apache-2.0. Worth tracking whether the architecture survives a real post-training pass, and whether the data-efficiency claim replicates under controlled re-training.