Muon is Not That Special: Random or Inverted Spectra Work Just as Well

TL;DR

The paper attacks the now-standard narrative that Muon’s success comes from its geometric structure (matrix-sign update, spectral / Schatten-∞ LMO, second-order interpretation). Three constructions undercut that story: Freon, a Schatten-(quasi)norm family interpolating SGD↔Muon with QDWH iteration, whose best-performing exponents for GPT-2 lie outside the unitarily-invariant LMO regime; Kaon, an “absurd” optimizer that replaces gradient singular values with random noise and yet matches Muon; and an analysis identifying two local quantities — alignment and descent potential — as the actual drivers, with step-size optimality as the mechanism behind Muon’s wins. Net claim: precise geometric structure is practically irrelevant; Muon succeeds by being well-step-sized along directions with adequate alignment, not by tracking any specific ideal geometry.

Key claims

The geometric / LMO narrative around Muon is not load-bearing for its empirical performance — precise geometric structure of the update is not the key factor affecting optimization performance [Abstract, §1].
Freon, a Schatten-(quasi)norm-based optimizer family computed via a novel provably-optimal QDWH iterative approximation, naturally interpolates between SGD and Muon and extrapolates smoothly into the quasi-norm regime (Schatten- $p$ with $p < 1$ ) [Abstract, §Freon].
The best-performing Schatten parameters for GPT-2 lie strictly within the quasi-norm regime, and so cannot be represented by any unitarily-invariant linear minimization oracle (LMO) — i.e. the LMO theory that motivates Muon’s matrix-sign update can’t even express the empirically-optimal point in this family [Abstract].
Kaon, an “absurd” optimizer that replaces the gradient’s singular values with random noise (i.e. discards any coherent spectral / geometric structure), nevertheless matches Muon’s performance and retains classical convergence guarantees [Abstract, §Kaon].
Optimizer performance is controlled by two local quantities — alignment (between update direction and gradient) and descent potential — not by any specific norm geometry; each optimizer must tune its step size around these two quantities [Abstract, §Analysis].
Under a stochastic random feature model, Muon’s empirical success is explained as guaranteeing step-size optimality, not tracking an ideal global geometry [Abstract, §Analysis].

Method

Three constructions, each scoped to falsify a different part of the geometric narrative:

Freon — a parameterized optimizer family $\Delta = -\eta \cdot \mathrm{Freon}_p(G)$ that applies a Schatten- $p$ (quasi-)norm steepest-descent / LMO step to the gradient. For $p = 2$ this reduces to SGD; for $p = \infty$ it reduces to Muon’s matrix-sign update. The transformation is computed via a new QDWH-based iteration with provable optimality among Halley-class iterations; this is what makes the quasi-norm regime ( $p < 1$ ) numerically reachable. GPT-2 sweeps over $p$ are then run; the best $p$ falls below 1, outside the LMO-representable region.
Kaon — keeps the Muon update form (matrix-sign style, applied via Newton-Schulz–like iteration) but replaces the gradient’s singular values with random noise before reconstruction. By construction this destroys any claim that the update tracks a coherent geometric object (LMO solution, Newton step, Schatten-norm steepest descent), yet pretraining performance is on par with Muon. Convergence guarantees are recovered under standard assumptions, removing the “but it still works in theory” objection.
Local-quantity analysis — replaces the global geometric framing with a local one: at each step, what matters is (a) the inner product of the update direction with the gradient (alignment) and (b) a “descent potential” capturing how much loss can be made to drop along this direction with an optimally-chosen step. Under a stochastic random feature model, Muon’s empirical wins are reproduced by showing Muon’s step-size choice is near-optimal along its alignment direction — not that its direction is geometrically privileged.

Results

Headline empirical results, all from GPT-2 pretraining experiments [Abstract; arxiv abs only — full §5 not fetched]:

The optimal Schatten- $p$ for Freon on GPT-2 lies in the quasi-norm regime $p < 1$ , outside the LMO-representable family. This is the load-bearing falsification of the LMO-theory framing.
Kaon (random-singular-values “Muon”) matches Muon’s GPT-2 pretraining loss curve despite having no coherent spectral structure.
The stochastic random feature analysis predicts that Muon’s empirical advantage is recovered by step-size optimality alone — explicitly not by directional geometry.

The paper does not claim to beat Muon; the contribution is interpretive — Muon’s headline performance does not require any of the geometric stories told about it, and the active ingredient is step-size–optimality along an adequately-aligned direction.

Why it’s interesting

This paper is a direct attack on the explanatory frame underneath much of the wiki’s recent muon-cluster reading. The Newton-Muon Optimizer derives Muon as an implicit-Newton method under an isotropic activation assumption and proposes a $(ZZ^\top)^{-1}$ right-preconditioner as the missing piece. Controlled LLM Training on Spectral Sphere (SSO) argues Muon is “half-aligned” with µP — it controls update spectra but not weight spectra — and proposes spectral-sphere retraction for both. Both are explanatory stories grounded in geometry. “Muon is Not That Special” says the geometric framing is not what’s doing the work: Kaon discards spectral structure entirely and still ties Muon, and the empirically optimal Freon exponent isn’t even representable in the LMO framework that Muon belongs to. If correct, the wiki’s “Muon as Newton-type method” / “Muon as half-µP” narratives are post-hoc rationalizations of a step-size-optimality phenomenon, not causal explanations — which is the framing Kyle’s accompanying note (“interpretations of muon might not be explanatory of muon behavior”) points at.

The paper is also a useful sibling to Gram Newton-Schulz: A Fast, Hardware-Aware Newton-Schulz Algorithm for Muon (Gram Newton-Schulz hardware-aware Muon) and When does RandOpt work? (RandOpt working surprisingly well): both point in the same direction — the matrix-sign step is doing less geometric work than its derivations suggest. If Kaon replicates broadly, the wiki’s open question under Training stability at scale (“Is ‘Muon is half-aligned with µP’ the right framing?”) gets a strong “no, the framing is wrong” answer.