CLI-Anything: Making ALL Software Agent-Native

TL;DR

CLI-Anything (HKUDS) is a Claude Code plugin and methodology for auto-generating Python CLIs that wrap arbitrary desktop software so AI agents can drive it from the terminal instead of via GUI screenshots. A /cli-anything slash command runs a 7-phase pipeline (codebase analysis → architecture → implementation → test planning → test writing → docs → pip-installable package) and produces one of ~40+ shipped harnesses today — GIMP (107 tests), Blender (208), Inkscape (202), Audacity (161), LibreOffice (158), OBS Studio, Kdenlive, Shotcut, ComfyUI, etc. — under a unified cli-anything-<software> namespace with a shared REPL skin, a --json flag on every command for machine consumption, and “zero-fallback” backends that call the real application. Its pitch against GUI agents is concrete: replace pixel-clicking with structured CLI calls and use that same CLI to synthesize tasks, evaluators, and benchmarks automatically.

Key claims

Five core design principles drive every generated harness: Authentic Software Integration (real apps render real outputs, no toy backends), Flexible Interaction (every CLI runs in dual REPL + subcommand modes), Consistent UX (shared repl_skin.py across all harnesses), Agent-Native Design (--json on every command), and Zero-Compromise Dependencies (tests fail when backends are missing rather than skip) [README “Core Design Principles”].
A fully automated 7-phase pipeline takes a codebase as input and emits a complete pip-installable harness with unit + E2E tests and docs, with no human-in-the-loop required for an initial harness [README “Fully Automated 7-Phase Pipeline”].
The shipped registry as of 2026-05-20 covers ~40+ software harnesses spanning creative tools (GIMP, Blender, Inkscape, Audacity, MuseScore, OBS Studio, Kdenlive, Shotcut, Sketch, Godot, s&box, ComfyUI, AnyGen), office/productivity (LibreOffice, Calibre, Zotero, Mubu, NotebookLM, Mailchimp, Zoom), dev/graphics tooling (RenderDoc, Nsight Graphics, LLDB, Unreal Insights, Mermaid, Draw.io), and infra (AdGuard Home, Ollama, Dify Workflow, Exa, CloudCompare, 3MF) [README “Project Structure”].
The Browser harness is built on top of “DOMShell MCP + Accessibility Tree” rather than screenshots, making it part of the same agent-native non-vision stance as the rest of the registry [README “Project Structure”].
Each harness is multi-layered: synthetic-data unit tests, real-file/real-software E2E tests, and CLI-subprocess verification of installed entry points; test counts per harness are published (GIMP 107, Blender 208, Inkscape 202, Audacity 161, LibreOffice 158, OBS Studio 153, Kdenlive 155, Shotcut 154, Mailchimp 303 commands / 36 unit tests, etc.) [README “Production-Grade Testing”, per-harness table].
A secondary thesis is that an agent-driven CLI is itself the substrate for synthesizing agent tasks, evaluators, and benchmarks automatically — once GUI software is reachable from code, evaluation can be authored end-to-end in terminal pipelines [README “Replace or Supercharge GUI Agents”].

Method

The plugin’s central artifact is /cli-anything, a Claude Code slash command that takes a codebase (or docs / SDK manuscript) and produces a Python package under cli_anything.*. The harness is generated through seven phases: codebase analysis, architecture design, implementation, test planning, test writing, documentation, and PyPI publishing. Every generated CLI conforms to a shared interface contract — a stateful REPL (entered by running the bare command) that shares branded banners, prompt styling, command history, and progress indicators via repl_skin.py, plus a subcommand mode for scripting and a --json flag on every command. The harness delegates to the real application for execution rather than reimplementing functionality: LibreOffice renders the PDFs, Blender’s bpy does the 3D work, Audacity’s pipeline is invoked through Python wave + sox, GIMP’s image ops are routed through Pillow + GEGL/Script-Fu. The Browser harness is the odd one out, building on DOMShell MCP and the accessibility tree instead of a desktop binary.

Persistent session state with undo/redo is part of the contract (“ReplSkin”), and security passes are visible in the changelog — e.g. routing untrusted XML/SVG/ODF/MLT/MusicXML/CSL through defusedxml, and guarded SQLCipher writes for the Rekordbox harness. The repo itself is structured as one agent-harness/ directory per supported piece of software, each containing an installable Python package under cli_anything.* with its own test suite.

Results

The repo doesn’t publish benchmark numbers; the “result” is the registry itself plus per-harness test counts as a proxy for surface coverage. Headline harness sizes: Blender 208 tests, Inkscape 202, Audacity 161, LibreOffice 158, Kdenlive 155, Shotcut 154, OBS Studio 153, NSLogger 139, Draw.io 138, GIMP 107, Openscreen 101, Ollama 98, Mubu 96, CloudCompare 88, ComfyUI 70, RenderDoc 59, Calibre 58 (38 unit + 20 E2E), MuseScore 56, Nsight Graphics 51, Unreal Insights 50, AnyGen 50, AdGuard Home 36, Mailchimp 36 (over 303 commands), LLDB 27, VideoCaptioner 26, Godot 24, Zoom 22, NotebookLM 21, Sketch 19, ETH2-QuickStart 18, ComfyUI 70, Mermaid 10. The s&box harness is the largest single bundle: 244 tests (157 unit + 17 orchestrator + 50 E2E + 20 exit-code). No agent-side success-rate numbers are reported in the README.

Why it’s interesting

This is the GUI-software-coverage half of the same agent-environment-synthesis story that SETA: Scaling Environments for Terminal Agents (env repo) is the terminal-task-synthesis half of: SETA assumes you already have a terminal-shaped action space and synthesizes the tasks; CLI-Anything generates the action space itself by turning GUI tools into terminal-shaped action spaces. Together they sketch a plausible pipeline for “auto-synthesize a verifier-cheap RL environment over any piece of desktop software” — exactly the shape Luma cares about for graphics tooling (Houdini, Blender, Nuke, FFmpeg pipelines) where successful agent actions are unit-testable but vision-based GUI interaction is brittle. It is also the explicit “replace GUI agents” counter-thesis to vision-based desktop control benchmarks like VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (where frontier VLMs top out at 46.6% Easy / 26.0% Hard and visual grounding is the dominant bottleneck) — CLI-Anything’s bet is that the right move for “GUI software” is not better visual agents but a code generator that translates GUI software into the substrate text agents already excel on. The harness-per-tool design with a unified --json and REPL contract is the practical realization of the same “tool-use loop with structured action space” abstraction articulated on Tool-Use Agents, pushed down to a layer below the agent itself.