FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformers

TL;DR

FullDiT2 is Kling/Kuaishou’s efficiency-focused follow-up to the in-context-conditioning paradigm popularized by FullDiT and UNIC: Unified In-Context Video Editing — where varying control signals (camera trajectories, reference images, masks, text) are concatenated with the noisy video latent into a long token sequence and processed by full-attention. Unifying everything in one sequence is conceptually clean but pays a quadratic compute price as task complexity grows. FullDiT2 attacks two specific redundancy sources: token-level redundancy within the context conditions (handled by a dynamic token-selection mechanism) and step-level redundancy in context↔latent interaction throughout the diffusion process (handled by a selective context-caching mechanism). The result is 2–3× per-step speedup on six conditional video editing/generation tasks with minimal — sometimes positive — quality change.

Key claims

In-context conditioning — the FullDiT/UNIC paradigm of concatenating arbitrary control tokens with the noisy video latent and running unified full-attention — faces quadratic computation overhead as task complexity (i.e., number of context-condition tokens) grows, which is the deployment bottleneck this paper targets [§1, Abstract].
Two distinct sources of inefficiency are identified by systematic analysis: (a) inherent redundancy within context-condition tokens — many tokens in the condition block carry overlapping information, and (b) computational redundancy in context↔latent interactions throughout the diffusion process — the cross-attention contribution from condition tokens to the noisy latent does not need to be recomputed at every diffusion step [§1].
Dynamic token selection adaptively identifies the important context-condition tokens at each step, shortening the sequence length seen by the unified full-attention. Distinct from static token pruning because the selection is data- and step-dependent [§3, Abstract].
Selective context caching caches the contribution of condition tokens to the latent across diffusion steps, recomputing only when the cache becomes stale, minimizing redundant condition↔latent interactions over the diffusion trajectory [§3, Abstract].
Across six diverse conditional video editing and generation tasks: 2–3× speedup in averaged per-diffusion-step wall time, with minimal degradation or even higher video generation quality [Abstract].

Method

FullDiT2 starts from the FullDiT / UNIC: Unified In-Context Video Editing architecture — a video DiT receiving [conditions · noisy latent] as a single token sequence — and adds two efficiency mechanisms while keeping the unified-attention backbone unchanged:

Dynamic token selection. A lightweight selection module scores context-condition tokens for importance at each diffusion step (using attention statistics or learned scores; the paper treats this as task-agnostic) and keeps only the top fraction for that step’s full-attention computation. The number kept is the knob trading speed against quality.

Selective context caching. Because the condition tokens don’t change across diffusion steps (only the noisy latent does), the contribution of condition→latent attention is approximately stationary over consecutive steps. The mechanism caches the attended output and only re-evaluates the cache on a schedule (e.g., every k steps or when a staleness criterion fires), removing the bulk of the condition↔latent compute over the multi-step denoising trajectory.

Together, these two interventions attack the two factors of the quadratic cost: sequence length (token selection) and number of steps (caching).

Results

Reported across six diverse conditional video editing and generation tasks — the FullDiT2 project page and abstract assert 2–3× speedup in averaged time cost per diffusion step with minimal quality degradation, and higher quality in some settings [Abstract, project page]. The specific per-task quality metrics and the trade-off curve at different token-selection ratios were not retrievable in detail at filing (arxiv 429); see the project page and the v1 HTML.

Why it’s interesting

This is the efficiency missing-piece for the in-context-conditioning paradigm the Kling team is pushing across UNIC: Unified In-Context Video Editing (editing), OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfer (transfer), and Kling-MotionControl Technical Report (motion control). Unified-attention designs trade simplicity for quadratic blow-up as control signals multiply — FullDiT2 reads as the empirical fix for that. Conceptually similar to KV-cache compression (e.g., Inference-Time Hyper-Scaling with KV Cache Compression) but operating on the control-condition tokens of a diffusion model rather than on autoregressive KV state, and similar in spirit to inference-time activation reuse like TeaCache / First-Block Caching (see the approximate-step-caching candidate concept) — except the cache is over the condition contribution, not the whole DiT block. This makes FullDiT2 a clean datapoint for the “step-cached cross-stream attention” sub-pattern that’s appearing across both LLM serving and diffusion serving in 2026.