LFM2 Technical Report

TL;DR

Liquid AI’s LFM2 family is the second-generation Liquid Foundation Model series, designed edge-first: a hardware-in-the-loop architecture search is run with downstream task scores plus on-device TTFT / decode-latency / peak-RSS as direct objectives, and the search repeatedly selects a minimal hybrid of inexpensive gated short-conv blocks interleaved with a small minority of grouped-query attention layers — no SSM, no linear attention, no long convolutions. Released checkpoints span four dense sizes (350M / 700M / 1.2B / 2.6B, 32K context) plus an 8.3B-total / 1.5B-active MoE (LFM2-8B-A1B), and add VL (SigLIP2 + lightweight connector), audio (continuous-feature in, text-or-discrete-audio-token out), and a late-interaction ColBERT retrieval head on the same backbone. Pre-training is 10–12T tokens + 1T long-context mid-training, with a decoupled, tempered Top-K KD objective from an LFM1-7B teacher; post-training is a 3-stage SFT → preference → model-merging pipeline. On Snapdragon-8-Elite and Ryzen-AI-9-HX-370 CPUs the dense models hit up to ~2× prefill and decode throughput vs same-size open baselines under matched llama.cpp Q4_0 quantization.

Key claims

The hardware-in-the-loop search uses three explicit Pareto axes — quality on an internal eval suite, latency (TTFT and p50/p95 decode), and peak RSS at 4K/32K context — with deployability checks on ExecuTorch / llama.cpp / vLLM at fixed quantization and batch settings; STAR-style proxy objectives (perplexity for quality, cache size for efficiency) are explicitly rejected as not transferring to downstream task scores or device metrics [§2.1].
Under realistic edge latency/memory budgets the search consistently picks a minimal hybrid: a majority of gated short-convolution blocks plus a small minority of GQA layers, with no additional SSM / linear-attention / long-convolution operators; augmenting with those operators at the same on-device budget does not improve aggregate quality and typically worsens device metrics [§2.1].
The gated short-conv block is out = W_out · ((B·x) ⊙ Conv1d(C·x)) with depthwise short kernels and input-dependent multiplicative gating, providing fast CPU-friendly local mixing; the global attention layers use GQA + QK-Norm + RoPE under pre-norm RMSNorm [§2.2].
Five dense/MoE configs are released: LFM2-350M (16 layers, 6 attn / 10 conv), 700M (same layout, wider FF), 1.2B (same depth-16 layout, larger heads), 2.6B (30 layers, 8 attn), and LFM2-8B-A1B (24 layers, 32 experts per MoE layer, top-4 routing, first 2 layers dense for stability) [Table 1].
LFM2-8B-A1B uses a normalized sigmoid router with adaptive routing biases for load balancing — the same DeepSeek-V3-style auxiliary-loss-free balancing recipe — and targets the on-device regime where compute-per-token, not weight storage, dominates perceived latency [§2.3].
Pre-training is 10T tokens at 4K context + 1T long-context mid-training at 32K (12T + 1T for the MoE), on a ~75/20/5 English/multilingual/code mixture (60/25/15 for the MoE), with FIM applied to 50% of code samples; tokenizer is a 65,536-vocab byte-level BPE [§3.1, §3.2].
The pre-training distillation objective decomposes the KL into (i) a binary KL on total Top-K mass and (ii) a temperature-scaled conditional KL inside the Top-K set, applying temperature only to the conditional term to avoid the support mismatch that arises when tempering a Top-K-truncated distribution over the full vocabulary [§3.3, Eq. 1–2].
On Snapdragon-8-Elite (Galaxy S25), llama.cpp Q4_0, 1K context: LFM2-350M reaches 1,067 prefill tok/s and 194 decode tok/s vs Granite-4.0-350M at 528/133; LFM2-700M reaches 522/104 vs Qwen3-0.6B at 318/77; LFM2-1.2B 335/70 vs Llama-3.2-1B 229/55 [Table 2].
The 2–4B comparison: LFM2-2.6B and LFM2-8B-A1B both beat Qwen3-4B / Gemma-3-4B / Llama-3.2-3B / SmolLM3-3B / Granite-4.0 H/Tiny on prefill and decode throughput at 1K and 4K context; LFM2-8B-A1B reports prefill comparable to Granite-4.0-H-Tiny but 60-90% higher decode throughput and the authors flag CPU MoE kernels as a known area for further gains [§2.4, Tables 2–3].
On the Ryzen-AI-9-HX-370 laptop CPU, LFM2-350M reaches 7,534 prefill tok/s and 207 decode tok/s at 1K context — the same relative ranking against same-size baselines as on the smartphone CPU, with absolute throughputs substantially higher [Table 3].
The MoE LFM2-8B-A1B is positioned as attaining “3–4B-class quality at roughly 1.5B-class decode cost” via top-4-of-32-expert sparse activation [§2.3].
The SFT corpus is ~5.39M samples for dense / 9.24M for the MoE across 67-79 curated sources; the MoE mixture is shifted heavily toward math (26.2%) and code (10.2%) while dense models lead with general-purpose (26.6%) and instruction-following (17.1%) [Table 4].

Method

The backbone is a decoder-only stack of pre-norm RMSNorm blocks where most positions are gated short-convolution blocks (input-dependent multiplicative gating around a depthwise short Conv1d, with separate linear projections producing the gate, the convolution input, and the output) and a minority are GQA layers with QK-Norm and RoPE. Layer counts and the attention/conv interleave pattern were selected by a hardware-in-the-loop search over a design space that included linear-attention variants (Katharopoulos-style, Gated DeltaNet), state-space models (S4, S5, Liquid-S4, Mamba, Mamba2), Liquid-Time-Constant blocks (CfC), sliding-window attention, and full hybrid operators that retain both a depthwise short convolution and a longer-range linear-attention/SSM branch. Every candidate was compiled to ExecuTorch (8da4w) and llama.cpp (Q4_0) and profiled on a Galaxy S24 Ultra and a Ryzen HX 370, with TTFT, prefill throughput, p50/p95 decode latency, and peak RSS measured at 4K and 32K. The search consistently dropped the linear-attention / SSM branches and retained only the depthwise short-conv submodule plus a small number of GQA layers — the released configurations have 6 attention blocks across 16 conv-heavy layers (350M / 700M / 1.2B), 8 attention blocks across 30 layers (2.6B), and 6 attention blocks across 24 layers (8B-A1B). For the MoE, all layers past the first two have a 32-expert top-4 SwiGLU MoE with normalized sigmoid routing and adaptive routing biases for load balancing.

Pre-training is two-stage: a 10T-token (12T for MoE) phase at 4K context with standard next-token prediction plus a decoupled, tempered Top-K KD loss against LFM1-7B, then a 1T-token mid-training phase at 32K context with naturally-long sources and an accelerated LR-decay schedule. The KD objective writes the KL between teacher and student as a chain rule over (i) whether the next token lies in the teacher’s Top-K set and (ii) the relative probabilities inside the Top-K, then tempers only the inner conditional KL — which keeps temperature-scaled distillation working without the support mismatch that occurs when softmax-with-temperature is applied to a Top-K-truncated distribution over the full vocabulary. With K = vocabulary size this reduces to a lower bound on the full forward KL (the teacher-tail term is unavailable when only Top-K logits are stored).

Post-training is a three-stage pipeline: (1) SFT on a curated mixture of ~67-79 sources mixing general-purpose chat, instruction following, RAG, tool use, math, and long-context data; (2) preference alignment with offline data including on-policy samples generated from the SFT checkpoint; (3) model merging over candidate post-trained checkpoints to improve robustness. Native multimodality is added on top of the same backbone: LFM2-VL uses a SigLIP2 vision encoder and a lightweight connector tuned for accuracy/latency trade-offs on device, LFM2-Audio ingests continuous audio features and autoregressively emits either text or discrete audio tokens for low-latency speech-in/speech-or-text-out, and LFM2-ColBERT-350M adds a dense projection on top of the backbone for max-similarity late-interaction multilingual retrieval.

Results

The headline result is the on-device throughput table. On the Snapdragon-8-Elite (Samsung Galaxy S25) under llama.cpp Q4_0, LFM2-350M delivers 1,067 / 657 prefill tok/s at 1K / 4K context and 194 / 144 decode tok/s — versus Granite-4.0-350M at 528 / 210 prefill and 133 / 71 decode (so roughly 2× prefill, 1.5× decode at 1K; 3× prefill, 2× decode at 4K). LFM2-700M at 522 / 341 prefill and 104 / 80 decode beats Qwen3-0.6B (318 / 136 prefill, 77 / 42 decode) on all four cells. LFM2-1.2B at 335 / 222 prefill, 70 / 56 decode beats Granite-4.0-1B and Qwen3-1.7B on all four cells and matches or exceeds Llama-3.2-1B by 20–50%; Gemma-3-1B is faster on prefill (377 / 295) but LFM2-1.2B is comparable on decode at the same parameter scale [Table 2]. At 2–4B, LFM2-2.6B (143 / 116 prefill, 34 / 30 decode) beats Qwen3-4B / Gemma-3-4B / Llama-3.2-3B / SmolLM3-3B / Granite-4.0-Micro and -H-Micro / -H-Tiny on all four cells; the 8.3B-total LFM2-8B-A1B (85 / 76 prefill, 49 / 42 decode) hits prefill comparable to Granite-4.0-H-Tiny (86 / 86) with 60–90% higher decode throughput and ~2× the decode throughput of dense 4B-class baselines, despite having 8.3B total weights [Table 2].

On the Ryzen-AI-9-HX-370 laptop CPU, the same relative ranking holds with absolute throughputs roughly 5–8× higher: LFM2-350M reaches 7,534 / 5,540 prefill tok/s and 208 / 171 decode tok/s, LFM2-1.2B reaches 2,784 / 2,302 prefill and 100 / 89 decode, LFM2-2.6B 1,335 / 1,171 prefill and 50 / 47 decode, LFM2-8B-A1B 1,320 / 1,185 prefill and 75 / 69 decode [Table 3]. Memory and quality benchmark numbers (downstream task scores against the baselines in Tables 2–3) are detailed in §4.5 (not reproduced here — the snapshot truncated before that section).

Why it’s interesting

This is the strongest published evidence yet that, under realistic on-device CPU budgets, the best-known efficient-attention zoo (Mamba/Mamba2, S4/S5, Liquid-S4, CfC, Gated DeltaNet, linear attention) is dominated by just the depthwise short-conv submodule of those blocks plus a small number of GQA layers — directly contradicting the recent design pattern in Mamba-3: Improved Sequence Modeling using State Space Principles and the broader SSM-hybrid literature, where the SSM/linear-attention branch is treated as load-bearing. The paper also operationalizes Liquid AI’s “edge-first” framing in a way the wiki hasn’t seen elsewhere: the search objective is downstream task score + on-device TTFT/decode/RSS on release runtimes, with proxies like perplexity and KV-cache size explicitly rejected. As a release, LFM2 extends the Open foundation-model releases pattern down to the truly small-and-on-device regime — covering a Pareto sweep (350M–8B) with multiple modalities (text / VL / audio / retrieval) and matched llama.cpp / ExecuTorch / vLLM deployment paths — a useful counterpart to the much larger open MoE drops (Kimi K2.5, HunyuanImage 3.0, GLM-4.5) already filed. The decoupled-tempered-Top-K distillation objective is also a cleaner construction of small-model KD than the de-facto truncated-softmax recipes, and worth comparing against the predictions of Distillation Scaling Laws — whose compute-optimal regime concluded teachers larger than ~2× the student bring diminishing returns, exactly the LFM1-7B → LFM2-350M-to-2.6B setup here.