Contrastive Language Model (CLM) — System One Model for Fast and Generalizable Decision-Making

TL;DR

CLM (Contrastive Language Model) is a Stanford Hazy Research + NVIDIA entry into the same “System One” design space as TypeSafe AI’s Jev and Convai’s Laya, but built from an entirely different primitive: a dual-encoder InfoNCE-trained state encoder + action encoder that scores each candidate action by dot-product against the state embedding. CLM-8B is pre-trained on 60M Nemotron Q&A pairs, mid-trained on 30M synthetic hard negatives, and post-trained on 1M agentic trajectories; the authors claim up to 9× inference-latency reduction vs Jev at comparable zero-shot accuracy on computer-use, gaming, and tool-calling, and — after lightweight fine-tuning — SOTA on DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%), a regime where Jev is reported to fall below random-selection Pass@1. The disaggregated architecture lets state and action embeddings be cached independently, which is the main systems win for repeated-scoring loops. Scaling laws are fit as power laws in training compute, model size, dataset size, projection-head size, and encoder size.

Key claims

Architecture: two encoders (state, action) trained with an InfoNCE contrastive objective that pulls each state toward the ground-truth action embedding and pushes it away from all other candidates; at deployment CLM scores each candidate action by dot-product alignment with the state embedding and returns the highest-scoring one — no autoregressive decode of the answer [§Architecture / thread 1].
Data recipe for CLM-8B: 60M Nemotron Q&A pairs (pre-training) + 30M synthetic hard negatives (mid-training) + 1M agentic trajectories (post-training) [thread 2].
Zero-shot latency: up to 9× faster than Jev on computer-use, gaming, and tool-calling at comparable accuracy; speedups are largest when the candidate action set is large (WikiRacing) or when actions are reused across states (T-Rex Game) [§Zero-Shot Evaluation / thread 5].
Agentic-coding fine-tuning: with lightweight fine-tuning, CLM-8B is claimed as SOTA on DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%); in contrast, Jev is reported to score below the random-selection Pass@1 baseline on the same long-horizon tasks [§Agentic Benchmarks / thread 6].
Serving-side disaggregation: because the state and action encoders are decoupled, state and action embeddings can be cached independently — the state embedding amortizes across many candidate actions in one decision step, and reusable actions can be pre-embedded once [§Latency vs. Jev and Constrained Decoding / thread 4].
Scaling laws: test InfoNCE loss decreases as a power law in training compute, model size, projection-head size, and encoder size, and the authors report that these dimensions must be scaled jointly rather than any one axis alone [§Scaling Laws for Verification / thread 3].
Roadmap: a multimodal CLM-35B is claimed to be in training with more data, compute, and parameters, targeting an early-October release [thread 12].
All weights, data, code, and an interactive playground are released (Contrastive-LM org on HF, github.com/Contrastive-LM/CLM) [thread 11 / blog aside].

Method

CLM is a two-tower retrieval architecture repurposed as an action classifier for agent decision loops. During training, a state encoder $f_s$ and an action encoder $f_a$ are trained jointly with an InfoNCE objective: for each (state, ground-truth-action) pair in a batch, the ground-truth action is the positive and all other actions in the batch are negatives. The two encoders are then used as a zero-shot classifier: given a state and a candidate set of actions, embed each once and take argmax over dot products. This inverts the standard LLM contract — instead of generating an answer token by token, CLM turns the decision into a maximum-inner-product-search over pre-embedded candidates.

The mid-training stage on 30M synthetic hard negatives is what the authors argue unlocks agentic-coding performance where Jev fails — long-horizon tasks require the verifier to distinguish between many plausible actions, and a batch’s other examples are typically not adversarial enough to teach that distinction. The post-training on 1M agentic trajectories is the standard “condition on real deployment traces” step.

The systems story is the disaggregation: because $f_s$ and $f_a$ never interact (only their outputs), you can prefill and cache the state embedding once per turn, then serve many candidate action scores as pure dot products. This is the direct systems analogue of what makes CLIP retrieval cheap at scale — and, at 8B, cheap enough to run in an inner agentic loop.

Results

Headline numbers as reported in the launch post and thread (all self-reported by the authors, no external corroboration at filing time):

Latency: up to 9× faster than Jev on zero-shot computer-use, gaming, and tool-calling, at comparable accuracy [thread 5].
DeepSWE: 81.6% after lightweight fine-tuning [thread 6].
Terminal-Bench 2.1: 87.6% after lightweight fine-tuning [thread 6].
Jev on the same long-horizon coding benchmarks: reported below the random-selection Pass@1 baseline [thread 6].
Scaling-law $R^2$ / exponents on test InfoNCE loss vs compute, model size, dataset size, projection-head size, encoder size are shown as log-log fits in the blog’s scaling-laws figure but not tabulated numerically in the launch post [§Scaling Laws for Verification].

Why it’s interesting

CLM makes the sharpest technical version yet of the “the System One class is separable from any one proprietary training pipeline” argument that Qwen-2.5-1B-RLCD open-sourced as tongue-in-cheek response to System One / Jev launch made informally (Qwen-2.5-1B-RLCD as a decoding recipe) and Laya — Non-Autoregressive System One Decision Model (open-source RLCD) made on the encoder-decoder-head axis (ModernBERT + RLCD with a strictly proper scoring rule). Laya’s win over Jev comes from a bidirectional encoder + RL-with-calibrated-scoring — an architectural rebuttal. CLM’s win comes from an objective rebuttal: swap the autoregressive next-token loss for InfoNCE dot-product retrieval, and the whole action space collapses into an embedding lookup. The three now form a triangle around Jev — Qwen-2.5-1B-RLCD open-sourced as tongue-in-cheek response to System One / Jev launch (inference-time recipe), Laya — Non-Autoregressive System One Decision Model (open-source RLCD) (specialised encoder architecture + RLCD), and CLM (contrastive dual-encoder). None of the three has been benchmarked head-to-head; each attacks a different lever, and the load-bearing question is which one actually holds up on the DeepSWE / Terminal-Bench numbers under third-party evaluation.

The distinct new contribution is the state/action disaggregation and its scaling-law framing. CLM turns the verifier problem into a retrieval problem, and once you’re doing retrieval the deployed inference profile looks like Faiss + a cached prefill — a very different systems shape from either Jev (parallel decoding + short output) or Laya (single forward pass with option-marker heads). The agentic-coding SOTA claim, if it holds, is a strong signal that dual-encoder verifiers can reach or exceed generative verifiers on long-horizon SWE tasks, which would put CLM in direct dialogue with the “verifier reward” cluster in Agentic Software Engineering and with the on-policy-distillation / RL-verifier debate. The scaling-law claim is unusually explicit for a launch post — power laws in projection-head size and encoder size as separate axes, with the prescription to scale jointly, sits cleanly alongside Hyperparameter scaling laws‘s existing “multi-axis co-fit” line (Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design for MoE FFN-vs-attention allocation, Scaling Native Multimodal Pre-Training From Scratch for multimodal composition ratio) but adds retrieval-model geometry as a new dimension.

Two caveats. First, all comparisons are author-run: there is no independent Jev vs CLM head-to-head, and “up to 9×” and “SOTA on DeepSWE” numbers should be treated as launch-day claims until third parties reproduce. Second, the framing that “Jev fails as a verifier for long-horizon tasks, performing below Pass@1” is exactly the kind of pointed empirical statement that other authors will try to reproduce quickly — worth watching the following two weeks.