ASCILINE — Real-Time ASCII Video Rendering Engine

TL;DR

ASCILINE is a Python/FastAPI + JS Canvas engine that streams video as binary-encoded ASCII (or colored-block “pixel mode”) frames over WebSockets at 24–30 FPS, with FFmpeg-driven audio as the master A/V clock. It’s a real-time terminal/HTML5-canvas video renderer with an opt-in adaptive frame codec (RAW / zlib / delta) that cuts wire bytes by ~10–375× on low-motion content. The Slack hook is the obvious one: if a video is just a stream of structured text frames, a language model can in principle emit it — making this a curio for anyone thinking about text-tokenized visual generation.

Key claims

Encodes each video frame as either ASCII characters or colored blocks (“pixel mode”) and streams them as raw Uint8Array over WebSockets, with audio acting as the master clock for A/V sync [README §🛠️ Architecture].
Adaptive codec tags each frame as RAW / zlib / delta; on static screens it reports ~0.3% of legacy bandwidth (≈375× saving), ~11.6% on pixel mode, and never worse than legacy on full-frame motion [README §🗜️ Adaptive Frame Codec].
Supports 5 fidelity modes from B&W DOM to 16M-color “ultra” pixel mode; recommended --cols 200–240 for ASCII, --cols 600–900 for pixel mode at near-HD [README §🎨 Customization].
Bit-exact codec correctness is verified two ways: Python-encoded vectors decoded by the shared codec.js in Node, and a live adaptive-vs-legacy WebSocket diff [README §🗜️ Adaptive Frame Codec].

Method

A FastAPI backend uses OpenCV to decode source video and NumPy to map pixel intensities to a character/color grid at a chosen --cols × --rows resolution; the grid is serialized as a binary framebuffer with a 1-byte codec tag (RAW, zlib, or delta-from-previous-keyframe) and pushed over a WebSocket whose INIT handshake negotiates grid dimensions and FPS. A vanilla-JS frontend buffers frames and blits them to an HTML5 <canvas> via requestAnimationFrame. Audio is extracted by FFmpeg and played as the absolute clock; the visual stream catches up or decimates to stay in sync. An optional lossy temporal-delta mode only re-sends a color cell when it drifts past a tolerance from the viewer’s last-seen value (character plane stays exact).

Results

The headline numbers are bandwidth, not quality: on a 200×80 grid in mode 5, the adaptive codec shrinks static-screen traffic to 0.3% of the legacy protocol (≈375× compression), pixel mode to 11.6% (≈8.6×), and stays within 63% of legacy on worst-case full-frame motion [README §🗜️ Adaptive Frame Codec]. The lossy --quality knob (“lossless / high / balanced / low”) trims another ~15–30% on hard cases at imperceptible quality. No model-quality numbers — the engine is a transport, not a generator.

Why it’s interesting

The Slack framing (“a LLM could generate videos with this”) is the actual hook for our team: the engine collapses a video to a stream of textual tokens with strict spatial structure, which is the same target distribution that pixel-as-token work like Karpathy: pixels may be better LLM inputs than text tokens (DeepSeek-OCR springboard) and DeepSeek-OCR: Contexts Optical Compression argue language models should be reading from, not just writing to. It’s also a useful sanity check against the maximalist “video tokens are unavoidably continuous” position implicit in most of the autoregressive video stack we file (e.g. FARMER: Flow AutoRegressive Transformer over Pixels, VideoMAR: Autoregressive Video Generation with Continuous Tokens) — ASCILINE is the proof-by-existence that you can get watchable 30 FPS video out of a pure discrete-text channel, even if the bitrate is brutal.