Laya — Non-Autoregressive System One Decision Model (open-source RLCD)

TL;DR

Laya (Convai Innovations, Apache 2.0) is an open-weights, non-autoregressive System One decision model positioned as a direct answer to TypeSafe AI’s closed Jev. Built on ModernBERT-large (395M) plus a 26M decision head (421M total), it takes an unstructured state plus typed questions (choice / score / noul) and returns typed answers with calibrated probabilities in a single 33 ms forward pass. Training uses Reinforcement Learning for Calibrated Decisions (RLCD) — a REINFORCE-with-group-mean-baseline recipe (GRPO-style) whose reward is a strictly proper scoring rule (log + spherical + RPS), so honest probabilities are the only reward-maximising strategy. Three checkpoints ship together (English, multilingual, typed-decisions-fine-tuned) with a lightweight router. Self-reported vs published Jev numbers claim 7.8× lower single-question latency, +3.9 pp on TypeSafe’s own typed-decisions benchmark, and 3× better ECE — but Jev numbers are third-party, not measured head-to-head, and the base checkpoints sit below the majority-class baseline on the fine-tuning task.

Key claims

Non-autoregressive one-shot decision architecture: ModernBERT-large bidirectional backbone + 2-layer transformer decision head + option-marker scorer + act/escalate head, 421M total parameters, 512-token budget per question, one forward pass per call [§Architecture].
Every option is scored at its own [MASK] token and softmaxed over that question’s options, so the answer space is defined at request time and new schemas require no retraining [§Architecture].
RLCD training uses a strictly proper scoring rule as reward — log + spherical + ranked-probability-score for ordinal questions — under REINFORCE with a group-mean baseline (GRPO-style); Gaussian noise on logits provides exploration; multi-turn conversations use TD(λ=1.0) over prefix slices [§Training].
Training cost: 7,313 updates, 1 epoch, ~1.96 h wall-clock; per-option-count temperature scaling fitted post-hoc [§Training].
Speed on Tesla T4: 39.5 ms for 1 question, 15.9 ms/q at batch 10, 6.8 ms/q at batch 50 for the multilingual checkpoint (103–332 questions/sec batched) [§Speed].
Head-to-head vs Jev 1.13.0 (Jev numbers from third-party benchmarks by AbdelStark and nibzard, not measured in-repo): typed-decisions accuracy 0.766 vs 0.727, AG News 0.950 vs 0.910, DAIR Emotion 0.595 vs 0.480, ECE 0.081 vs 0.246 (3× better), p50 latency 32.8 ms vs 236–276 ms (7.8× faster) [§Laya (with routing) vs Jev].
On DAIR Emotion, Jev is reported to assign zero probability to the true label on 16% of examples [§Laya (with routing) vs Jev].
Router auto-selects the checkpoint per input at microsecond cost, but a cold checkpoint build costs ~7.4 s median on CPU / 10.3 s on T4 — the default max_loaded=1 alternates rebuilds on mixed-language traffic and preloading is a claimed 4.8× speedup at 50% non-English mix [§Quickstart].
Load-bearing honesty note: the 0.766 typed-decisions accuracy belongs only to the laya-typed-decisions checkpoint fine-tuned on that benchmark’s training split; the base laya (0.362) and laya-multilingual (0.342) checkpoints sit below the 0.461 majority-class baseline zero-shot [§typed-decisions, all three checkpoints].
Multilingual coverage across 51 MASSIVE-intent languages macro-averages 0.227 accuracy at 0.733 macro-ECE — clears 3× random on only 23/51 languages, with Khmer at 0.000 accuracy / 0.952 confidence (no confidence signal for unreadable inputs; language routing must happen before the forward pass) [§Languages].
Ships over-confident by default; refitting one temperature per (question-type, option-count) drops mean ECE from 0.466 to 0.081 — recommended per-user-data calibration step [§Limits].
Limits called out explicitly: choice questions degrade beyond ~20 options (fixed 192-token head budget shared across labels), ordinal score is the weakest primitive (SST-5 0.372), and English-only base — multilingual checkpoint is a separate loadable [§Limits].

Method

The core architecture inverts the LLM’s autoregressive contract: instead of a token-by-token distribution over a shared vocabulary, Laya emits one softmax per question-instance option set, scored at question-specific [MASK] positions inserted into the prefix. ModernBERT-large’s bidirectional attention lets every option-marker attend to every other option-marker plus the state and the question, so a single forward pass computes all questions in a call jointly. The act/escalate head sits above the same trunk and gates whether the model should return an answer or defer.

Training is straightforward REINFORCE with a group-mean baseline (the GRPO simplification: no critic, no clipping, average-of-samples as the baseline). The novelty is entirely in the reward: a strictly proper scoring rule (Brier / log / spherical / RPS depending on question type) is the only reward the model receives. Under a strictly proper rule the unique argmax of expected reward is the true posterior — so if the reward is honestly optimised, the emitted probabilities are honest too. Gaussian noise added to logits during rollout provides exploration; TD(λ=1.0) folds multi-turn conversation trajectories into single-episode credit assignment.

The three shipped checkpoints (English 421M, multilingual, typed-decisions-fine-tuned) are all the same architecture; the Router lazy-loads by input language detection with a preload=True flag for server deployments.

Results

Self-measured on a Tesla T4. Speed: 39.5 ms single-question, 158.6 ms for 10 batched (15.9 ms/q), 771 ms for 50 (multilingual: 337 ms at 6.8 ms/q). Reported vs published-Jev numbers on typed-decisions (0.766 vs 0.727, +0.039), AG News (0.950 vs 0.910, +0.040), DAIR Emotion (0.595 vs 0.480, +0.115), ECE (0.081 vs 0.246, 3× better), p50 latency (32.8 ms vs 236–276 ms, 7.8× faster). English task numbers (base laya): AG News 0.947, BoolQ 0.830, DAIR Emotion 0.573 (held out), prompt-injections 0.698, SST-5 0.372.

On the fine-tuning benchmark’s own held-out splits, laya-typed-decisions clears the teacher ceiling of 0.735 on aggregate (0.766) and wins each of four workflows: invoice processing 0.804, security incidents 0.766, customer service 0.764, agent-trace observability 0.730. By primitive: noul 0.857, choice 0.733, score 0.723. The paired ablation is unambiguous — the base and multilingual checkpoints score 0.362 and 0.342 on the same benchmark, both under the 0.461 majority-class baseline. The 0.766 result is a fine-tuning story, not a zero-shot story.

Why it’s interesting

This is the sharpest public rebuttal yet to the “new model class” framing of Introducing System One Models & Jev. Where Qwen-2.5-1B-RLCD open-sourced as tongue-in-cheek response to System One / Jev launch argued in the abstract that any decoder-only LLM can be prompted to score typed JSON slots in parallel, Laya makes the encoder-side argument concretely: take a bidirectional ModernBERT, bolt on an option-marker decision head, train under RLCD with a strictly proper scoring rule, and ship three Apache-2.0 checkpoints under 500M parameters that beat Jev’s self-reported accuracy and calibration numbers by a wide margin at 7.8× lower latency and 4 GB VRAM. This makes concrete the concern raised in the Jev filing that the load-bearing gains — parallel decode, short output, specialised architecture — are readily separable from a proprietary training pipeline. The “output-space specialist model” pole of Tool-Use Agents now has an open-weights datapoint.

Two caveats sharpen the reading. First, the Jev comparison numbers are entirely third-party, drawn from AbdelStark’s and nibzard’s independent benchmarks; there’s no Laya-run head-to-head against a live Jev API. Second, and more load-bearing: the impressive typed-decisions number (0.766, clearing the 0.735 teacher ceiling) is only achieved by the checkpoint that was fine-tuned on that benchmark’s own training split — the base and multilingual checkpoints regress below the majority-class baseline zero-shot. This is honest reporting on Convai’s part (the tables are laid out explicitly), but it re-poses the SimpleTool question from a slightly different angle: how much of Jev’s advertised 20-200× workflow speedup is architecture (parallel decode) vs task-specific fine-tuning that any small model could match given the labelled data? The Portuguese-language launch tweet’s framing (“came above Jev”) is thin — the actual technical position is more modest: this is what a well-executed open baseline for the System One class looks like, and it’s small enough to fine-tune on a workstation.