VecGlypher: Unified Vector Glyph Generation with Language Models

TL;DR

VecGlypher is a single multimodal LLM that emits an SVG <path> string for a glyph directly, conditioned either on free-form style tags (e.g. “Active, Cute, Vintage” + target character) or on 1–8 reference glyph images. The training recipe is two-stage SFT: a large-scale continuation stage on 39K noisy Envato fonts to teach SVG syntax and long-horizon coordinate prediction, then post-training on 2.5K expert-tagged Google Fonts mixing text- and image-referenced samples. The interesting empirical claim is the domain-data gap: even Claude Sonnet 4.5 / GPT-5 / Gemini 2.5 Pro produce broken glyphs from style prompts, while a 27B Gemma3 fine-tuned with the right typography-aware preprocessing closes contours, follows style, and matches the target character — the authors argue glyph programs are simply underrepresented in standard LLM corpora.

Key claims

General-purpose flagship LLMs (GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro) cannot produce typographically valid glyphs from style prompts; budget tiers rarely emit valid SVG at all and Claude Sonnet 4.5 — the strongest baseline — reaches only R-ACC 46.65 / FID 19.59 [Table 8].
A 27B Gemma3 trained with the two-stage recipe reaches R-ACC 100.5, CD 1.72, DINO 94.22, FID 3.46 on cross-family OOD Google Fonts test families — i.e. ≈2× the recognition and ≈83% lower FID than the strongest proprietary LLM [Table 8, §4.4].
Stage 1 large-scale continuation on noisy Envato tags is essential; Envato-only training gets recognition (R-ACC 94) but weak geometry (CD 3.63 / FID 20.43 at 27B), and a single-stage Envato+Google mix is consistently worse than the two-stage recipe [Table 6, §4.3].
Absolute-coordinate SVG serialization beats relative coordinates at scale (27B), even though 4B sometimes gets marginally higher R-ACC with relative coordinates — at low capacity recognition is decoupled from geometry [§4.3, Table 6].
For image-referenced generation, VecGlypher-27B at R-ACC 99.12 / CD 1.18 / FID 2.32 vs DeepVecFont-v2 (R-ACC 37.86 / FID 115.5) and DualVector (R-ACC 49.20 / FID 105.5) — 92% lower CD and ~98% lower FID than the best specialized vector-font baseline [Table 9, §4.4].
Specialized SVG tokenizers (Xing et al. 2025, Yang et al. 2025b) add complexity without measurable gains; subword tokenization of the raw d="" string suffices [§3.4].
Model scale is a primary driver of vector fidelity — at 27B parameters geometry stabilizes; below ~30B current evidence is that quality remains brittle [§4.3, §5].
The training recipe relies on aggressive typography-aware preprocessing: dedup near-identical fonts, drop the top 10% longest pangram renderings, normalize to UPM=1000, keep only the d="" attribute, quantize coordinates to one decimal [§3.3].

Method

A multimodal decoder ingests (i) a style description as a bag of tags or 1–8 reference glyph images rendered with centering and uniform padding, plus (ii) the target character identity, then autoregressively predicts SVG path tokens. The same architecture and decoding procedure handle both modalities — there are no separate vector encoders or post-vectorization optimizers. Each glyph is one SVG <path> element, serialized as a string of drawing commands with one-decimal coordinate precision. The training objective is plain next-token cross-entropy on the path text. Backbone variants reported: Gemma3 4B / 27B and Llama3.3 70B; the 27B is the default operating point.

The data pipeline is the dominant load-bearing component: filter to alphanumeric coverage, drop fonts whose pangram render exceeds the 0.9 length quantile, dedup near-identical fonts, rescale to UPM=1000, drop all path attributes except d="", quantize. This is what reduces sequence length variance enough for stable long-sequence decoding. Stage 1 SFT on the 39K Envato corpus (text-referenced only) teaches SVG syntax and long-horizon geometry on noisy tags; Stage 2 SFT on the 2.5K Google Fonts corpus mixes text- and image-referenced samples to align language and image inputs with glyph appearance. Training uses AdamW with cosine decay, base LR 1e-5, weight decay 0.1, LR scaled by √(GPU scale factor); Envato is one epoch and Google Fonts three.

Results

On cross-family OOD evaluation against general-purpose flagship LLMs (text-referenced, Google Fonts test families):

Model	R-ACC ↑	CD ↓	DINO ↑	FID ↓
Claude Sonnet 4.5	46.65	5.28	88.31	19.59
GPT-5	43.98	6.12	86.92	29.00
VecGlypher 27B (T,I,A)	100.5	1.72	94.22	3.46
VecGlypher 70B (T,A)	100.4	1.68	94.28	3.34

[Table 8] — Budget-tier Gemini 2.5 Flash / GPT-5 Mini are essentially unusable (R-ACC ≈ 5). VecGlypher-70B improves CD/FID over 27B only marginally; on text-referenced the 27B/70B gap is much smaller than the 4B/27B gap.

Against dedicated vector-font baselines (image-referenced):

Model	R-ACC ↑	CD ↓	DINO ↑	FID ↓
DeepVecFont-v2	37.86	14.58	79.41	115.5
DualVector	49.20	16.45	79.57	105.5
VecGlypher 27B (T,I,A)	99.12	1.18	95.82	2.32

[Table 9] — both specialized baselines collapse on unseen families and thin strokes.

Scaling within the recipe: 4B → 27B on Google-only training takes R-ACC from 66.66 → 94.91 and FID from 14.69 → 3.96 with absolute coordinates [Table 6]. Adding Stage-1 Envato continuation then takes 27B from 94.91/3.96 to 101.0/3.47 — Stage 1 is not optional even when targeted Google data is available.

Why it’s interesting

VecGlypher is a clean instance of the “the LLM is the model, the data engineering is the paper” pattern: the headline result is that a fine-tuned 27B Gemma3 beats GPT-5 / Claude Sonnet 4.5 by 2× on recognition because the proprietary models have effectively never seen glyph programs at training time. This is a sharper version of the The flavor of the bitter lesson for computer vision observation about modality gaps — generic web SVG (icons, illustrations) does not transfer to typography because the geometric constraints (closure, winding, consistent stroke logic across characters) are qualitatively stricter than what icon-grade SVG training implicitly enforces. The unified text-or-image conditioning into a single autoregressive model — rather than the dual encoder-decoder + geometry post-optimizer stack of DeepVecFont-v2 / DualVector — is also a structural simplification worth noting for anyone designing structured-output generators (CAD, layout, programmatic 3D); for vector graphics specifically it complements C3Net: Compound Conditioned ControlNet for Multimodal Content Generation‘s ControlNet-style multimodal conditioning by collapsing the entire stack into next-token prediction over the program text.