Winfree Oscillatory Neural Network

TL;DR

WONN is a phase-oscillator architecture where each neuron is a point on the torus $(S^1)^d$ and computation is the joint synchronization of all neurons under a generalized Winfree dynamics. Each layer maintains a dual phase/frequency state and performs $T$ parameter-shared recurrent updates of the phase before a per-layer phase+frequency transition — structurally the same weight-sharing-across-iterations recipe as looped Transformers, but with the inner block replaced by phase-space coupling instead of attention. The headline claim: WONN is the first synchronization-based architecture to scale competitively to ImageNet-1K (76.78% with 12.28M params), and on Maze-hard reasoning it reaches 80.1% with 0.396M params — 1.47% of HRM’s parameter count — using interaction-energy voting at test time as the selection rule.

Key claims

The architecture replaces Kuramoto-style pairwise-difference coupling $K_{ij}\sin(\theta_j-\theta_i)$ with a separable Winfree form $S(\theta_i)\sum_j c_{ij} I(\theta_j)$ , where sensitivity $S$ and influence $I$ can be fixed trigonometric functions or learnable MLPs; the separation breaks the global phase-shift symmetry of Kuramoto and enables richer asymmetric synchronization patterns [§2.1, Eq. 3].
Each layer maintains a dual state (phase $\Theta$ , frequency $\Omega$ ) and runs $T$ parameter-shared recurrent updates on the phase before a single joint layer-transition update — fast inner-loop synchronization, slower outer-loop frequency evolution, deliberately recalling biological timescale separation [§2.2 Eq. 5, Eq. 7].
A grouped formulation partitions oscillators into $g\times g$ spatial patches and aggregates each patch into a shared influence signal, instantiating a two-level local/global hierarchy controlled by group size $g$ ; coupling $c_{ij}$ is implemented as convolution (local) or attention (global), and all reported WONN models use global attentive coupling [§2.2 Eq. 6].
On CIFAR-10/100 with three model sizes (~3M / 7.5M / 12M params), WONN matches or beats ResNet-50 (23.5M) and ViT-B (85M) at both interaction parameterizations (MLP and trigonometric) [Table 1].
On ImageNet-1K, WONN at 12.28M params reaches 76.78% top-1, exceeding ResNet-50 (76.89% at 25.56M, effectively a tie at half the params) and ViT-B/16 (75.85% at 86.57M); this is claimed as the first synchrony-based architecture to scale competitively to ImageNet-1K [Table 2].
Phase variables consistently bifurcate into a bimodal distribution with peaks separated by $\sim\pi$ on trained image classifiers; one mode aligns with coarse global object regions, the other with fine boundaries — a self-organized phase decomposition that is not imposed by supervision or architecture [§3.2, Fig. 4].
On Maze-hard pathfinding, WONN with point-wise interactions ( $g=1$ ) achieves 76.2% at 0.396M params vs HRM 74.5% at 27M and TRM-Att 85.3% at 7M; LLM baselines (DeepSeek-R1, Claude 3.7, O3-mini-high) all score 0.0% [Table 3].
Test-time energy voting — sample 32 trajectories, select the one with lowest final interaction energy $E = -\frac{1}{2}\sum_{ij} c_{ij}\cos(\theta_i-\theta_j)$ — pushes Maze-hard accuracy to 80.1% at the same 0.396M params [Table 3, §3.3].
WONN’s solution dynamics on Maze-hard are visibly different from HRM’s: WONN generates many diffuse candidate path fragments early, then synchronizes/merges them into a single coherent path through collective alignment; HRM is largely inactive until ~T=4, then has an abrupt insight-like transition at ~T=6 [§3.3, Fig. 5].
On Sudoku (Wang et al. 2019 dataset, 31-42 given digits), WONN at 1.58M params reaches 100.0% test accuracy, matching R-Transformer (0.211M) and exceeding AKOrN (99.8% at 2.98M) and HRM (99.7% at 27.28M) [Table 4].

Method

WONN treats each neuron as a phase oscillator $\theta_i \in S^1$ with a natural frequency $\omega_i$ . Given input $x$ , the frequency state $\Omega_{\text{init}}=\text{Embed}(x)$ carries the input information, while the phase state $\Theta_{\text{init}}$ is randomly initialized — so the input enters the dynamics through the frequencies rather than the phases. Each layer $l$ then performs $T$ parameter-shared recurrent updates of the phase (Eq. 5):

$\theta_i^{(l,t+1)} = \theta_i^{(l,t)} + \Delta t\left[\omega_i^{(l)} + S(\theta_i^{(l,t)})\sum_{j} c_{ij}\, I(\theta_j^{(l,t)})\right]$

with $S, I$ either fixed trigonometric mappings ( $S(\theta)=-\sin\theta$ , $I(\theta)=\cos\theta$ , the separable first-Fourier form of Winfree dynamics) or learnable MLPs. Coupling $c_{ij}$ is global attention in all reported models. After $T$ inner steps, a layer-transition jointly updates phase and frequency to $(\Theta^{(l+1)}, \Omega^{(l+1)})$ (Eq. 7). Phases are represented in tangent / Euclidean embedding form $(\cos\theta, \sin\theta)$ for compatibility with standard convolution and attention operators, then projected back to $S^1$ via $\arctan2$ . A grouped variant aggregates oscillators inside $g\times g$ patches into a shared per-group influence signal (Eq. 6), interpolating between point-wise ( $g=1$ , used for reasoning) and grouped ( $g=2$ , used for vision) coupling.

When natural frequency is excluded and interactions are trigonometric, the dynamics admit a Lyapunov function $E = -\frac{1}{2}\sum_{ij}c_{ij}\cos(\theta_i-\theta_j)$ that decreases along trajectories — this interaction energy doubles as a test-time selection signal for reasoning tasks (energy voting).

Results

CIFAR-10: WONN-12M reaches 95.24% (MLP) / 95.26% (trig) vs ResNet-50 94.22% at 23.5M and ViT-B 92.04% at 85M [Table 1].
CIFAR-100: WONN-12M reaches 76.20% (MLP) / 76.17% (trig) vs ResNet-50 73.54% and ViT-B 71.05% [Table 1].
ImageNet-100: WONN-12M reaches 82.88% vs ResNet-50 81.18% and ViT-B/16 76.36% [Table 2].
ImageNet-1K: WONN-12M reaches 76.78% top-1 at 12.28M params vs ResNet-50 76.89% at 25.56M and ViT-B/16 75.85% at 86.57M; AKOrN at 4.85M only reaches 67.45% [Table 2]. First synchrony-based architecture reported to be competitive at this scale.
Maze-hard: WONN 76.2% at 0.396M (1.47% of HRM’s 27M params); WONN + energy voting (32 trajectories) 80.1% [Table 3]. AKOrN at 1M reaches only 36.2%; all LLM baselines (DeepSeek-R1 671B, Claude 3.7, O3-mini-high) score 0.0%.
Sudoku: WONN 100.0% at 1.58M; matches R-Transformer (100.0% at 0.211M) and exceeds AKOrN (99.8% at 2.98M) and HRM (99.7% at 27.28M) [Table 4].

Why it’s interesting

WONN is the closest the Looped Transformers cluster has come to a paper that isn’t a transformer: weight-sharing across $T$ inner recurrent steps is the same scaffold ELT and Hyperloop use, but the inner block is phase-space coupling, not attention — so it’s a natural test of whether the recent looped-transformer wins (any-time inference, iso-quality-at-half-params, reasoning competence at sub-1M params) come from the loop structure itself or from the attention primitive sitting inside it. The Maze-hard result is the relevant data point: WONN matches HRM-Text: Efficient Pretraining Beyond Scaling‘s hierarchical-recurrence variant at 1.47% of the params, with a qualitatively different solution dynamics (parallel candidate fragments resolved by synchronization vs HRM’s abrupt-transition profile), strengthening The Serial Scaling Hypothesis‘s claim that the looping/recurrence axis matters more than the choice of inner mixer. Energy voting also offers something the deterministic single-trajectory looped models don’t have natively: a principled, model-internal test-time selection rule that doesn’t require an external verifier, closer in spirit to GRAM’s (Generative Recursive Reasoning) stochastic-trajectory branch but driven by physics rather than a variational objective.