SpecBundle & SpecForge v0.2: Production-Ready Speculative Decoding Models and Framework

TL;DR

SpecBundle Phase 1 is the first open, community-scale release of production-grade EAGLE-3 draft-model checkpoints — covering 12 mainstream open instruct-tuned target models from Llama-3.1-8B-Instruct to Kimi-K2-Instruct (1T) and Qwen3-Coder-480B-A35B-Instruct, none of which previously had a community EAGLE-3 draft beyond what the original EAGLE authors released. Alongside it, SpecForge v0.2 ships a refactored training framework with a unified Eagle3TargetModel interface that supports both SGLang and Hugging Face Transformers as target backends, ~10× faster data regeneration via parallelism, and unified online/offline training. Drafts are trained on the 1.4M-sample Perfect-Blend dataset (vs. the ~320K ShareGPT+UltraChat used in the original EAGLE papers) with SGLang-regenerated responses to align the draft’s training distribution with the actual target outputs, reportedly delivering up to 4× end-to-end inference speedup.

Key claims

Phase 1 covers 12 target models from 8B to 1T parameters with EAGLE-3 drafts, including 7 (Qwen3-30B-A3B-Instruct-2507, Qwen3-Next-80B-A3B-Instruct-FP8, Qwen3-Coder-30B-A3B-Instruct, Qwen3-Coder-480B-A35B-Instruct, inclusionAI/Ling-flash-2.0, Kimi-K2-Instruct, two Nex-N1 variants) for which no prior community EAGLE-3 draft existed [§Background, table].
SpecBundle draft-model training data is regenerated by running SGLang on each target model rather than relying on raw ShareGPT/UltraChat responses; the team identifies this distribution alignment as the lever that lifts token-acceptance rate in deployment [§Performance].
The training corpus is Perfect-Blend (~1.4M samples, broader coverage of coding and math), in contrast to the ~320K ShareGPT+UltraChat used in the original EAGLE papers; the blog attributes the wider domain coverage to better generalization at scale [§Performance].
SpecForge v0.2 introduces an Eagle3TargetModel interface; a new backend is integrated by implementing only generate_eagle3_data, and both SGLang and Hugging Face Transformers ship as built-in backends [§SpecForge v0.2 / Multi-backend Support].
v0.2 reports ~10× faster data regeneration than v0.1 through data parallelism and asynchronous processing, plus unified online/offline training scripts [§User-friendliness Enhancement].
Reported end-to-end inference speedup over standard decoding baselines is up to 4× across diverse benchmarks; per-model results live on the SpecBundle benchmark website rather than in the blog body [§Performance].
Drafts are positioned as initialization points for downstream RL pipelines via integrations like ReSpec (arXiv 2510.26475) and slime, framing draft-model availability as a bottleneck for RL training efficiency, not only inference [§Motivation].
2026 Q1 roadmap explicitly targets long-context training, VLM support, MTP finetuning, and SpecBundle Phase 2 (reasoning models) + Phase 3 (VLMs) [§Roadmap].

Method

EAGLE-3 (Li et al., arXiv:2503.01840) attaches a small draft module to a target LLM that predicts the next few tokens from intermediate target-model hidden states; the target then verifies the drafted tokens in a single forward, accepting a prefix that matches its own distribution. SpecBundle’s contribution is operational rather than algorithmic: pair each open target model (8B–1T) with a SpecForge-trained EAGLE-3 draft using a two-step recipe — (1) regenerate the draft training dataset by sampling responses from the target through SGLang so that draft inputs match what the target will actually produce at decode time, then (2) train the draft on Perfect-Blend’s 1.4M-sample mixture, which extends well beyond ShareGPT/UltraChat into coding and math.

SpecForge v0.2’s main system change is the Eagle3TargetModel abstraction. A user instantiates the target as e.g. get_eagle3_target_model(pretrained_model_name_or_path=..., backend="sglang", ...); the framework calls a single generate_eagle3_data method per backend, so adding new target-model families no longer requires reimplementing the target inside SpecForge. The data-regeneration stage was rebuilt around data parallelism and asynchronous processing for the ~10× throughput claim, and online/offline training paths were merged into one script to keep the two regimes from drifting apart.

Results

The blog reports headline numbers in two figures (not reproduced inline here): a per-model comparison of SpecBundle drafts vs. existing community EAGLE-3 weights, and absolute speedups for the >100B SpecBundle drafts. Top-line claim is up to 4× end-to-end inference speedup over standard (no-SD) decoding across “diverse benchmarks.” Detailed benchmark tables are deferred to the SpecBundle website rather than included in the post, so the specific batch sizes, lookahead lengths, and acceptance-rate numbers are not in the blog body. Coverage matters as much as the speedup magnitude: SpecBundle is the first public source of EAGLE-3 drafts for 7 of the 12 covered targets, which is the gap the blog explicitly frames as the bottleneck blocking community-wide SD adoption.

Why it’s interesting

This release closes a specific gap that the wiki has been tracking on multiple fronts. Speculative Decoding: Performance or Illusion? showed that under fully-optimized vLLM serving, SD variant choice is workload-dependent and that EAGLE-3 wins on reasoning workloads — but its empirical study was bottlenecked by the lack of EAGLE-3 drafts for many production targets (the same scarcity SpecBundle’s coverage table names). TorchSpec: Speculative Decoding Training at Scale solved the system-level training bottleneck for EAGLE-3 at trillion-parameter scale (Mooncake RDMA disaggregation, ~700 TB of storage avoided, Kimi K2.5 draft in 1500 H200-hours); SpecBundle is the complementary artifact release that actually ships drafts for Kimi-K2, Qwen3-Coder-480B, Qwen3-Next-80B, and others — i.e. the missing draft-weight inventory that any future systematic SD study or production deployment needs. The 1.4M-sample Perfect-Blend recipe also offers a different scaling-axis answer than the original EAGLE work, which is a useful contrast point for the “draft generalization” sub-thread of the inference-efficiency concept.