Skillful joint probabilistic weather forecasting from marginals

TL;DR

FGN (Functional Generative Network) is the model behind Google DeepMind’s WeatherNext 2 operational forecasts. It produces a 56-member, 15-day global ensemble at 0.25° resolution by sampling parameter perturbations of a GenCast-shaped graph-transformer through a single forward pass per timestep — no iterative denoising. Trained only on per-location marginal CRPS, it nonetheless captures joint spatial and cross-variable structure, beats GenCast on 99.9% of variable/lead-time combinations (avg 6.5% CRPS, 5.8% RMSE improvement), and runs ~8× faster.

Key claims

FGN models aleatoric uncertainty by sampling neural-network parameters from a learned Gaussian via reparameterization, with the noise injected through conditional layer-norm parameters that are shared across spatial dimensions — yielding globally coherent functional variability from a single 32-dim noise vector [§2.2.2, §2.3].
Epistemic uncertainty is captured by deep-ensembling four independently-trained 180M-parameter models; the final 56-member ensemble draws 14 trajectories per model seed [§2.2.1, §3.3].
Training uses the unbiased “fair” CRPS estimator averaged across locations/variables/levels with M=4 samples per step; despite being a purely marginal loss, it learns skillful joints under the architectural constraint of low-dim globally-applied noise [§2.2.3, §6].
FGN achieves better CRPS than GenCast on 99.9% of variable-level-leadtime combinations (avg 6.5%, up to 18%) and better ensemble-mean RMSE on 100% (avg 5.8%, up to 18%) [§5.1, Fig. 2a].
Spread–skill ratios sit close to 1 across all lead times; calibration is as good or better than GenCast over the first 5–7 days, where GenCast is slightly under-dispersed [§5.1, Fig. 2b-f].
On spatially-pooled CRPS (a proxy for joint spatial structure) FGN beats GenCast on 90%+ of (variable, level, pool-size, lead-time) cells with avg 8.7% / 7.5% improvement on avg-pooled / max-pooled CRPS [§5.2, Fig. 3a-b].
Cross-variable derived quantities — 10m wind speed from (u,v) and z300−z500 — also improve, demonstrating learned inter-variable dependencies despite no joint loss [§5.2, Fig. 3c-d].
Power spectra match HRES-fc0 ground truth at high frequencies out to 15 days — FGN avoids the blurring failure mode of low-RMSE deterministic forecasters [§5.2, Fig. 3e-i].
A single 15-day forecast trajectory runs in <1 minute on one TPU v5p, 8× faster than GenCast, because FGN is single-pass rather than iteratively denoised — even though FGN is larger (180M × 4 seeds vs 57M) and emits twice as many frames (6h vs 12h timestep) [§3.3].

Method

The architecture is a graph-transformer encoder/decoder modeled on the GenCast denoiser: a GNN maps the 0.25° lat/lon grid to a 6×-refined icosahedral mesh, a 24-layer graph-transformer with latent dim 768 processes mesh nodes, and a decoder GNN maps back to the grid. Each constituent model has 180M parameters. The state at each step bundles 6 atmospheric variables × 13 pressure levels plus 6 surface variables, predicted autoregressively at 6-hour steps from the two prior states.

The core mechanism is functional variability via conditional-normalization perturbation: a 32-dim Gaussian noise vector ε is passed (per ensemble member, per timestep) into every conditional layer-norm in the network, where it modulates the scale/shift parameters. Because the noise is low-dimensional and globally applied across all spatial positions, and because conditional-norm parameters are shared across mesh/grid nodes, the architecture is forced to express variability as globally coherent functional changes rather than spatially-independent jitter. This is what makes training on a marginals-only CRPS objective yield skillful joints. Compare with AIFS-CRPS (Lang et al., 2024), which also uses conditional normalization for stochasticity but with location-specific conditioning — adding high-frequency spatial noise rather than coherent low-frequency functional perturbations.

Training is multi-stage: ERA5 pre-training on a single-step CRPS loss, fine-tuning on HRES-fc0, then autoregressive fine-tuning over rollouts up to 8 steps. Total compute is roughly 490 TPUv5p/v6e days × 4 seeds. The “fair” Zamo-Naveau CRPS estimator is used to remove the bias of empirical CRPS from finite M.

Results

On the operational benchmark (initialized and evaluated against HRES-fc0 in 2023, 56-member ensembles):

Marginal skill: CRPS better than GenCast on 99.9% of cells (avg 6.5%, max 18%); RMSE better on 100% (avg 5.8%) [Fig. 2a, A.11]. REV on 99.99th-percentile 2m temperature matches GenCast; on 99.99th-percentile 10m wind, FGN beats it [Fig. 2g-h].
Joint structure: avg-pooled CRPS avg 8.7% better, max-pooled 7.5% better, across pool sizes 2² to 32² [Fig. 3a-b]. Cross-variable 10m wind speed and z300−z500 CRPS also improve [Fig. 3c-d].
Power spectra: FGN matches ground-truth spectra at all wavenumbers out to 15 days; GenCast also does well but FGN is slightly closer to truth at high frequencies on 2m temperature and 700hPa specific humidity [Fig. 3e-i].
Tropical cyclones: ensemble-mean track position error and REV of track-probability heatmaps both improve significantly over GenCast (further details in body, Fig. 4–5 region of the paper).
Efficiency: ~1 min per 15-day trajectory on a single TPU v5p, 8× faster than GenCast; all 56 members parallelizable [§3.3].

Why it’s interesting

The mechanism — generating ensembles by sampling a low-dim global noise vector that modulates conditional-norm parameters across an entire spatially-shared backbone — is a clean and general recipe for turning a deterministic backbone into a generative one without iterative denoising. It’s structurally close to how AdaLN-Zero / style-conditioning works in MMDiT-family video and image models, which is where this paper becomes relevant outside meteorology: training a marginals-only objective (per-pixel CRPS or equivalent) on an architecture with parameter-shared conditional modulation appears to be sufficient to learn coherent joints, sidestepping diffusion’s iterative sampling cost. The 8× inference speedup over a diffusion baseline of comparable quality is the headline a video/image team should track. There’s also a methodological lesson worth flagging for downstream work: the architectural inductive bias (low-dim noise, global application, spatial parameter sharing) is doing most of the work — point-wise noise injection in input/output space underperforms.