ARC Is a Vision Problem!

TL;DR

VARC reframes the Abstraction and Reasoning Corpus (ARC) as image-to-image translation rather than language modeling or recurrent reasoning. The recipe is deliberately textbook: place each ARC grid on a large canvas, apply translation/scale augmentation, run a vanilla ViT (or U-Net) trained from scratch on ARC-only data with per-pixel cross-entropy, and use test-time training on each unseen task’s few demo pairs. An 18M-parameter ViT reaches 54.5% pass@2 on ARC-1; ensembling gets to 60.4%, matching average human performance (60.2%) and beating every from-scratch baseline including HRM and TRM. The paper is a vision-side counter to the LLM and recurrent-reasoning consensus that ARC needs language-modeling priors or hand-built recurrence.

Key claims

A vanilla ViT trained from scratch on ARC-only data with image-style visual priors (canvas, 2D positional embedding, translation + scale augmentation, patchification beyond raw pixels) reaches 54.5% pass@2 on ARC-1 at 18M params; ensembling reaches 60.4%, matching avg-human 60.2% on the same benchmark [Table 3, §5.3].
VARC substantially outperforms the prior best from-scratch ARC methods — HRM (27M, 40.3%) and TRM (7M, 44.6%) — under matched “train on ARC only, no internet-scale pretraining” rules [Table 3, §5.3].
Visual priors are the dominant contributor: stacking 2D positional embedding + canvas-based patchification + translation augmentation + scale augmentation gives +27.7 points over a 1D-position vanilla ViT baseline (32% → 54.5% on ARC-1) [Fig. 7, §5.1].
Scale augmentation alone contributes +6.2 points and is the single largest visual-prior gain — interpreted as evidence that ViT has very little inductive bias for scale invariance, in contrast to translation invariance (which patchification partly supplies) [Fig. 7(f), §5.1].
The canvas / patchification trick expands the effective token vocabulary from ~10 (one per ARC color) to up to 10^(p²) per p×p patch, which the paper argues is necessary to give the model meaningful spatial inductive bias rather than letting it memorize per-pixel colors [§3.3, Fig. 5].
Performing test-time training independently per test task outperforms TTT done jointly across all test tasks by ~10 points, even though the joint setting has access to strictly more demo data — interpreted as offline-training knowledge getting overwritten by cross-task TTT [Fig. 9(b), §5.2].
TTT without offline training still reaches 26.4% on ARC-1, suggesting a meaningful fraction of ARC tasks are solvable tabula-rasa via the canvas + augmentation + TTT recipe alone [Fig. 9(b), §5.2].
ViTs scale predictably with width/depth in the explored regime (6M / 18M / 66M); the 66M model overfits despite higher train accuracy, identifying generalization (not capacity) as the next bottleneck [Table 1, §5.2].
U-Nets also work — the 17M U-Net reaches 47.5% and the 55M U-Net 48.3% — so the result is not ViT-specific but a general property of vision-backbone + image-to-image-translation framing [Table 1, §5.2].
The 2D positional embedding matters even on top of a strong baseline: replacing 2D RoPE with 1D RoPE in the best configuration degrades by 3.5 points (54.5 → 51.0) [§5.1].
Multi-view inference (5–10 random augmented views, majority voting) lifts pass@1 from 35.9 to 49.8 — a much larger gap than typical vision tasks see, attributed to ARC’s pixel-perfect correctness requirement (any single-pixel error fails the whole task) [Table 2, §5.2].
TTT per task takes ~70s on a single GPU [§3.4], setting the practical compute budget for the recipe.

Method

The model is a standard ViT (default 512-wide × 10-deep, 18M params, 2×2 patches on a 64×64 canvas). Each ARC grid is rescaled by an integer factor and translated to a random position on the canvas; the background fills the rest with an 11th “background” color. Per-pixel embeddings are continuous-valued lookups indexed by the 11 colors. Each task gets a learnable task token concatenated to the sequence. The output head is a per-patch linear layer producing per-pixel categorical distributions, trained with per-pixel cross-entropy ([Eq. 1]). 2D positional embeddings are separable (half the channels for x, half for y), applied to both absolute additive embeddings and RoPE.

Training has two stages. Offline training uses the 400-task ARC-1 train set plus 1,000 RE-ARC-synthesized demos per task (~400K total pairs); all tasks share parameters except per-task tokens, and translation + scale augmentation is on. The training set’s inference pairs are held out for validation. Test-time training (TTT) takes one unseen test task, initializes a fresh task token, and fine-tunes the model on the task’s 2–4 demo pairs — augmented by flip / rotation (90°/180°/270°) / color-permutation, each treated as a separate auxiliary task with its own task token, plus the same translation + scale augmentation (without a new task token, since translation/scale are assumed task-invariant). TTT runs per-task (not jointly), at ~70 seconds on a single GPU [§3.4, §4]. Inference then applies the TTT’d model to multiple augmented views of the test input and consolidates by majority voting, retaining the top-2 most-voted output grids for the pass@2 metric [§3.5].

Results

ARC-1 pass@2: vanilla ViT baseline ~32%; +2D positional embedding → 36%; +canvas patchification → 45.4%; +full translation augmentation → 48.3%; +scale augmentation → 54.5% [Fig. 7]. Ensembling (multiple model seeds × multi-view voting, 73M aggregate params) → 60.4%. Compared to the leaderboard: Deepseek R1 (671B) 15.8%, Claude 3.7 21.2%, o3-mini-high 34.5%, GPT-5 44.0%, Grok-4-thinking (1.7T) 66.7%, Bespoke (Grok-4) 79.6%; HRM (27M) 40.3%, TRM (7M) 44.6%. VARC’s 60.4% sits between GPT-5 and Grok-4-thinking while being four orders of magnitude smaller than the LLMs above it [Table 3, §5.3]. On ARC-2 (no ARC-2 training data, only ARC-1 offline + ARC-2 TTT), VARC reaches 8.3% (18M) / 11.1% (73M ensemble), beating HRM (5.0), TRM (7.8), Deepseek R1 (1.3), Claude 3.7 (0.9), and GPT-5 (1.9), losing only to Grok-4-thinking (16.0) and Bespoke (29.4) [Table 3].

Why it’s interesting

VARC is the same-author follow-up to Back to Basics: Let Denoising Generative Models Denoise (JiT / “Back to Basics: Let Denoising Generative Models Denoise”) and continues the lab’s “throw out the language-modeling priors, use plain vision recipes on the raw signal” thesis — JiT did this for diffusion (no VAE, x₀-prediction, raw pixel patches), VARC does it for abstract reasoning (no LLM, no recurrent reasoning module, plain ViT image-to-image). The cross-paper consistency makes the position legible as a research program rather than a one-off result. On the wiki, this is most directly in tension with the Looped Transformers cluster (HRM, TRM, ELT, Hyperloop, HRM-Text, GRAM) and with The Serial Scaling Hypothesis: the looped/recurrent crowd argues ARC-like abstract reasoning requires serial depth that fixed-depth Transformers (TC⁰) cannot supply, while VARC delivers a strictly stronger ARC-1 number than HRM and TRM using a fixed-depth ViT with TTT — if you accept TTT-as-gradient-steps as the missing serial-compute mechanism. That reading aligns VARC with End-to-End Test-Time Training for Long Context and the broader Long context as weights framing: extra task-specific compute lands in weight updates (the TTT inner loop) rather than in deeper unique-parameter stacks or in CoT tokens.

For Luma, the load-bearing claim is the visual-prior ablation: scale augmentation contributes +6.2 points and is irreplaceable by any other component the paper tests. That’s the most generalizable transfer to image/video model design — if a ViT trained on pixel-perfect grid reasoning still needs explicit scale augmentation to generalize, then the standing assumption that natural-image pretrained backbones come pre-equipped with scale invariance is probably overstated. Two open questions the paper leaves: (i) does the bottleneck/manifold story from Back to Basics: Let Denoising Generative Models Denoise (low-rank input projection helps even on raw pixels) transfer to VARC’s canvas representation, where each patch already has up to 10^(p²) configurations — or does the discrete-color setting break the manifold framing? (ii) Does jointly trained-from-scratch TTT (with the outer loop meta-learned through inner-loop steps, End-to-End Test-Time Training for Long Context style) beat VARC’s two-stage offline-then-TTT pipeline?