Do Audio-Visual Large Language Models Really See and Hear?

TL;DR

A mechanistic-interpretability study of Audio-Visual LLMs (Qwen2.5-Omni, Qwen3-Omni, Video-LLaMA, MiniCPM-o) showing that on counterfactual videos — where the audio track is swapped for an unrelated one — audio captioning fidelity drops by up to 56% relative to factual videos, while the model’s internal audio representations remain informative. Probing intermediate states with logit lens recovers correct sound-source tokens (61.4% latent audio understanding vs 23% manifested) and attention-knockout experiments show that deep layers actively suppress audio in favor of vision: blocking visual pathways in those layers recovers audio reasoning. A token-distribution comparison against the base Qwen2.5-VL traces the bias to alignment training rather than architecture — 85% of audio-related tokens generated by the AVLLM are already in the top-3 predictions of the vision-only base model.

Key claims

AVLLMs lose up to 56% relative audio caption fidelity on counterfactual (audio-swapped) videos compared to factual ones, while video caption fidelity is unaffected — direct evidence that aligned-modality benchmark numbers conflate audio reasoning with visual inference [§1, Fig. 2].
Audio tokens receive 40–50% attention from generated tokens in early layers (0–5) but drop to ≈0% in subsequent layers; video tokens show the opposite trajectory, climbing through middle layers (15–30) to 20–40% [§5, Fig. 4].
Logit-lens probing of intermediate audio-token positions decodes interpretable onomatopoeic / sound-source tokens (e.g. “drill”, “engine”, “horse”, “keyboard”), often in multiple languages reflecting Qwen’s Chinese-heavy pretraining; meaningful audio info emerges in the last 5 layers [§6, Fig. 5].
Qwen2.5-Omni achieves 61.4% latent audio understanding (logit-lens decode evaluated by LLM-as-judge) on counterfactual samples vs 23% in the actually-generated caption — a 38-point gap between encoded and expressed audio knowledge [§6].
Attention-knockout causal mediation: blocking generated tokens’ attention to audio (GA) in final layers degrades audio understanding by 35%, confirming deep-layer audio integration; blocking video (GV) in those same layers improves counterfactual audio understanding by 28%, recovering near factual-setting performance — direct evidence that vision actively interferes with audio in deep fusion layers [§7.2, Fig. 6D].
Token-distribution analysis against the base Qwen2.5-VL: KL ≈ 0.4 between AVLLM and base-LVLM next-token distributions; median base rank = 1; 66.06% of audio-describing tokens are unshifted (top-1) and 85.36% are in the top-3 of the vision-only base model — audio contributes ≈15% of the audio-related token decisions [§8].
The visual-dominance pattern persists across Qwen3-Omni, Video-LLaMA, and MiniCPM-o, indicating an architecture-agnostic alignment-training artifact rather than a Qwen2.5-Omni-specific quirk [§7.2, Supplementary D].
Evaluation uses an LLM-as-judge over AudioCaps-derived 500-sample factual/counterfactual splits, with Qwen3-32B as judge; calibrated against humans at Spearman ρ ≈ 0.7 for audio fidelity, ρ ≈ 0.6 for video fidelity [§4].

Method

The paper studies AVLLMs as adapter-based stacks: frozen audio + vision encoders → learned projectors → shared LLM backbone. The evaluation task is audio-visual captioning (“describe what you see and hear”), with modality-specific prompts (“…what you see” / “…what you hear”) used to isolate per-modality reasoning. The 500-sample evaluation set is derived from AudioCaps, half factual (original audio kept) and half counterfactual (audio track swapped with a semantically unrelated AudioCaps clip), forcing the model to commit to one modality when the two disagree.

The interpretability stack runs in four stages. (1) Attention-pattern analysis: tracking average attention from generated tokens to audio / video / query tokens across all layers. (2) Logit-lens probing: projecting hidden states at audio-token positions through the unembedding matrix at every layer to read out the model’s “thought” about the audio. (3) Attention knockout: blocking attention from text-generation positions to audio (GA) or video (GV) tokens in a sliding 9-layer window, then measuring caption-fidelity change — a form of causal mediation that pinpoints where each modality’s information actually enters the output. (4) Token-distribution analysis: feeding the AVLLM’s own generated context back through the AVLLM (with audio) and the base Qwen2.5-VL (without audio), and comparing next-token distributions via KL divergence and base-rank statistics. Evaluation throughout uses an LLM-as-judge prompted with rubrics for objects, actions, temporal ordering, and audio events, scoring audio and video fidelity separately on [0,1].

Results

Headline numbers across the four analyses: 56% relative drop in audio caption fidelity on counterfactual samples; audio attention near-zero past layer ≈5 in deeper layers while audio integration causally happens in those same deep layers (per knockout); 61.4% latent understanding vs 23% expressed; vision-blocking recovers ~28% relative audio caption fidelity in counterfactual samples by removing the suppressive visual signal; KL = 0.4 between AVLLM and base-LVLM output distributions; 85% of audio-describing tokens are top-3 predictions of the vision-only base. The qualitative failure mode is consistent — given a video of a helicopter with a baby-crying audio track, Qwen2.5-Omni generates “hear the sound of a helicopter flying” and describes it as “clear and distinct,” with attention localized on the visible helicopter even though the audio query was the prompt.

Why it’s interesting

This is the mechanistic complement to Audio Hallucination Attacks: Probing the Reliability of Large Audio Language Models (AHA) — both papers document that recent audio-capable models (Qwen2.5/3-Omni, Audio Flamingo 3, MiniCPM-o, Gemini 3/3.1) hallucinate audio events at high rates, but AHA characterizes the failure surface (95+% ASR under audio-injection + implicit + adversarial queries) while this paper traces where in the model the failure happens (early-layer audio attention exists but is suppressed by deep visual fusion). The 61.4% latent vs 23% expressed gap and the “block vision in deep layers → recover audio” intervention are the actionable findings: they suggest the audio-omni team’s mitigation lever may be at the deep-layer cross-modal attention rather than at the audio encoder or projector. The token-distribution-vs-base-LVLM analysis also slots cleanly into the Unified Multimodal Models open question about whether adding audio to a vision-language UMM systematically helps — here the answer for the understanding side is “not really; audio describes itself through the vision prior 85% of the time.” Contrasts with Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing, which builds an audio-omni stack on top of Qwen2.5-Omni-3B and reports MMSU/MMAU numbers but does not probe counterfactual robustness — this paper is what those benchmarks miss.