Trajectory Labs finds PiSSA outperforms vanilla LoRA on three agentic RL benchmarks

TL;DR

Trajectory Labs runs agentic RL with rank-32 LoRA adapters on Qwen3-0.6B (GSM8K), Qwen3-4B (Tau Retail), and Qwen3.6-27B (SuperGPQA), initializing the adapters via PiSSA (principal singular values + vectors of the base weight matrix) instead of the standard Gaussian-A / zero-B scheme. PiSSA finishes ahead of vanilla LoRA on all three benchmarks. On the 0.6B GSM8K probe at matched learning rate, PiSSA moves the policy farther from the base (higher KL) with longer reasoning traces; on SuperGPQA the accuracy gain lands without longer outputs. Framed as an extension of PiSSA (Meng et al., NeurIPS 2024) to agentic RL.

Key claims

Rank-32 PiSSA initialization beats vanilla LoRA on three agentic-RL setups spanning three model sizes and three task types (math, tool-use, multi-domain knowledge) [thread post 3].
At matched LR on Qwen3-0.6B / GSM8K, PiSSA produces higher KL divergence from the base and longer reasoning traces than vanilla LoRA [thread post 4].
On Qwen3.6-27B / SuperGPQA, PiSSA’s accuracy gain arrives without a corresponding increase in output length [thread post 4].
The proposed mechanism: PiSSA’s SVD-based initialization places the adapter on the principal singular directions of the base weight, giving RL a non-random subspace to move in from step 0 [thread post 2].

Method

Standard LoRA freezes the pretrained weight W and trains BA starting from B=0, A~N(0,σ²) — early updates traverse a random subspace. PiSSA (Meng et al., NeurIPS 2024) instead computes the truncated SVD W ≈ U_r S_r V_r^T and sets A = √S_r · V_r^T, B = U_r · √S_r; the adapter BA at step 0 equals the top-r component of W, and the residual W_res = W - BA is frozen. Same architecture as LoRA, different trainable geometry. Trajectory Labs applies this to agentic-RL setups at rank 32 on three Qwen models, comparing against vanilla LoRA at matched rank and (in the GSM8K probe) matched LR. Full writeup with interactive visualizations was linked from the thread; benchmarks span GSM8K (Qwen3-0.6B), Tau Retail (Qwen3-4B), SuperGPQA (Qwen3.6-27B).

Results

PiSSA > vanilla LoRA on all three benchmarks (specific final numbers not reported in the tweet body).
Qwen3-0.6B / GSM8K, matched LR: PiSSA has higher KL divergence from base + longer reasoning traces than vanilla LoRA.
Qwen3.6-27B / SuperGPQA: PiSSA accuracy gain without longer outputs — different failure mode from the GSM8K probe.

Why it’s interesting

This directly contradicts The Path Not Taken: RLVR Provably Learns Off the Principals‘s central negative result. That paper opens the RLVR black box at the parameter level and shows RLVR concentrates updates on off-principal, low-magnitude weights while preserving each layer’s singular-value spectrum — and demonstrates PiSSA (which does the opposite: initializes on principal singular directions) collapses under RLVR, dropping to near-zero accuracy on a math-reasoning setup. The Trajectory Labs experiment reports the opposite qualitative result on the closest comparable setup (Qwen3-0.6B / GSM8K, rank-32 LoRA).

Reconciliation candidates (none confirmed by either side): (i) agentic RL (tool-use / long trajectories on Tau Retail and SuperGPQA) may differ from single-turn RLVR in ways that shift the off-principal bias — but the collision is sharpest on Qwen3-0.6B / GSM8K, which is exactly a single-turn math-reasoning setup; (ii) the RLVR paper’s collapse-mode result held PiSSA rank/LR fixed to the SFT-tuned defaults and may not have swept independently — Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning already argues that reported advantages of advanced LoRA variants over vanilla LoRA collapse under per-method LR sweeps, which cuts against both sides here; (iii) matched-LR comparisons may be the wrong control if PiSSA and LoRA have systematically different optimal LRs. Until the interactive Trajectory writeup and the RLVR paper’s PiSSA collapse-mode setup are compared head-to-head at matched sweep budget, the conflict stands. This is worth flagging on the concept page rather than resolving.

Also intersects LoRA Without Regret and Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning — the µA prescription is Init[B] + α=1, and Thinking Machines Lab finds rank-1 LoRA matches full-parameter RL under the standard peft parametrization; neither result speaks directly to PiSSA-vs-vanilla under RL, and the Trajectory Labs claim would push that comparison up in priority.