Ripple-65B Preview — Open-Weight Dense 65B Co-Designed with Ricursive's Chip Platform

TL;DR

Ricursive Intelligence releases Ripple-65B Preview, a 65B-parameter dense Transformer trained from scratch on 12T tokens with 1M context support, positioned as the current U.S. open-weight SOTA on TerminalBench 2.1 (80.52). The lab reports a total R&D cost of $32M — roughly one-tenth of the prior open-weight SOTA — attributing the efficiency to a disciplined pretraining ladder (six iterations from Olmo 3’s baseline, 16× compute efficiency on code at 4B), a dense-not-MoE architectural bet optimized for training simplicity on NVIDIA hardware, and a steep RL climb from a strong SFT base using only 1.4M rollouts (~70× fewer than Reflection Beam). The model is a byproduct of their AI-chip-design platform and ships on Hugging Face plus five inference providers.

Key claims

Ripple-65B Preview reaches 80.52 on TerminalBench 2.1, surpassing larger U.S. open-weight models — Reflection Beam 501B (80.1), Muse Spark Open 816B (76.4), Inkling 975B (63.8), and Nemotron 3 Ultra 550B (56.4) [§Achieving U.S. Open-Weight Frontier].
Total R&D cost including ladder development, data mixing, evals, debugging, synthetic data, posttraining recipe design, and the hero pretraining+posttraining run was $32M — ≈10× cheaper than the previous open-weight SOTA [§Achieving U.S. Open-Weight Frontier].
The architecture is a dense modernized Transformer, not MoE — a deliberate choice to reduce pretraining/posttraining infrastructure complexity, sustain 40% MFU with simple sharding and no custom kernels or quantization, and fit on NVIDIA DGX Spark edge hardware [§A Co-Designed Model Architecture].
Architectural ingredients: thin-and-deep aspect ratio (d_model 5120, 80 layers), overprovisioned attention heads, increased SWA-to-global layer ratio with reduced SWA window size, FFN activation clipping, gated attention, global NoPE, and value residuals [§A Co-Designed Model Architecture].
The RL recipe is mixed and compute-frugal — truncated importance sampling + score centering + linear length penalty over high-quality long-horizon synthetic environments with difficulty continuously ratcheted to the model’s current capability — reaching 80.5 TerminalBench 2.1 in 1.4M rollouts vs ~70× more for Reflection Beam [§Steep Ascent Reinforcement Learning].
The pretraining ladder improved compute efficiency on code (dolma_stack CE) by up to 16× at 4B scale across six iterations: StepFun batch-size/LR laws (v1.0) → architectural+depth scaling (v1.1) → pre+post norm (v1.2) → sigmoid gated attention (v1.3) → final-LR cooldown to zero + global NoPE (v1.5) → value residuals (v1.6) → custom code-upsampled dataset [§Careful Model Laddering].
Ladder v1.4 (2-stream mHC) looked good at small scale but did not improve at 4B and was rejected — a concrete datapoint that small-scale gains don’t always transfer [§Careful Model Laddering].
An internal Ripple variant powers Ricursive’s AI chip-design platform; the public release is a general-purpose derivative [§Introduction, §Conclusion].

Method

Ripple-65B Preview is a 65B-parameter dense decoder-only Transformer with d_model 5120 and 80 layers (thin-and-deep), overprovisioned attention head count, and an SWA-heavy attention schedule with a reduced sliding-window size so the per-layer attention cost stays bounded. The FFN uses activation clipping (as in gpt-oss); attention is sigmoid-gated (following Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free); positional information uses global NoPE; and the residual stream carries value residuals. Pretraining spans 12T tokens with 1M-context support and holds ~40% MFU with vanilla sharding, no custom kernels, and no low-precision tricks.

The pretraining recipe was built by a six-step ladder study starting from Olmo 3’s ladder: at each step the authors ran small-scale ablations with a code-capability focus (dolma_stack CE), kept interventions that scaled to 4B, and rejected those that didn’t — including a 2-stream mHC variant that regressed at 4B despite small-scale wins. Final gains stack to 16× compute efficiency at 4B scale on code vs the Olmo 3 baseline.

Posttraining is a mixed RL recipe atop a strong SFT checkpoint. The three cited ingredients are truncated importance sampling, score centering, and a linear length penalty (per MAI-Thinking-1). Difficulty is curriculum-ratcheted as the model improves, over long-horizon synthetic environments the lab built themselves (no purchased posttraining data). The final 80.5 TB2.1 is reached in 1.4M rollouts.

Results

TerminalBench 2.1: 80.52 — U.S. open-weight SOTA; beats Reflection Beam 501B (80.1), Muse Spark Open 816B (76.4), Inkling 975B (63.8), Nemotron 3 Ultra 550B (56.4) [§Achieving U.S. Open-Weight Frontier].
Rollout efficiency: 1.4M rollouts to reach 80.5, ~70× fewer than Reflection Beam [§Steep Ascent Reinforcement Learning].
Total R&D cost: $32M end-to-end — the blog cites ~$ 150M/month compute commitments by comparable U.S. open-weight labs and Reflection’s reported 4 weeks pretrain + 4 weeks posttrain as context, estimating ~10× cost reduction vs prior open-weight SOTA [§Achieving U.S. Open-Weight Frontier, footnote 1].
Ladder efficiency: 16× compute reduction at 4B model scale on dolma_stack CE vs Olmo 3’s baseline ladder, with the gap widening at larger scales because the Olmo 3 slope is shallower than ladder v1.0+ [§Careful Model Laddering].
Pretraining MFU: ~40% sustained through pretraining and posttraining with simple sharding, no custom kernels, no quantization [§A Co-Designed Model Architecture].
Distribution: Hugging Face, Together AI, GMI Cloud, OpenRouter, Fireworks [§Introduction].

Why it’s interesting

Three angles make Ripple interesting for Luma. First, it’s a dense 65B that beats sparse open-weight models 5–15× its total-parameter count on long-horizon agentic coding — a fresh data point in the dense-vs-MoE tradeoff that the filed agentic-SWE cluster has largely leaned MoE-ward (Kimi K2, MiniMax-M2, DeepSeek-V4). Second, the “we ran the ladder carefully and got 16× efficiency” story is a concrete worked example of the approach advocated in Scaling Laws, Carefully and complements the open-ladder work in Frontier Pre-training at Microsoft AI — including the honest negative result on 2-stream mHC that failed to transfer from small scale to 4B. Third, the vertical-integration angle (model → chip-design platform → chip → model) puts Ripple adjacent to Redwood (AI-designed accelerator in 2 weeks) in a small-but-growing cluster of labs explicitly trying to close the AI-for-AI-research loop at the hardware level.