Skip to content
Bud's Wiki

ben (@contraben) open-sources 234-step Premiere Pro editing trajectories for computer-use agent SFT

TL;DR

Section titled “TL;DR”

ben (@contraben, Contra) announces an open-source Hugging Face dataset of 234 annotated steps across 4 short-form-social-reel editing trajectories in Adobe Premiere Pro. Each step pairs a screenshot with the professional editor’s spoken reasoning, a structured action, and executable grounding — a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click. Positioned as a preview drop and framed for computer-use agent SFT, reasoning mid-training, tool-use / function-calling training, and benchmarking against human experts. Format follows the AgentNet trajectory schema, extended with a Premiere action taxonomy and multi-path execution.

Key claims

Section titled “Key claims”
  • Dataset contains 234 annotated steps across 4 computer-use trajectories of professional editors building short-form social reels in Adobe Premiere Pro [tweet body].
  • Every step pairs a screenshot with the expert’s own spoken thought, a structured action, and executable grounding — a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click [tweet body].
  • Format extends the AgentNet trajectory schema with a Premiere-specific action taxonomy and multi-path execution [tweet body].
  • Intended uses stated: computer-use agent SFT, reasoning mid-training, tool-use / function-calling training, and benchmarking against a human-expert baseline [tweet body].
  • Data is released under a preview label on Hugging Face, with more to follow [tweet body].

Method

Section titled “Method”

Human data collection with think-aloud protocol: professional video editors are observed step by step while producing short-form social reels in Adobe Premiere Pro, narrating their reasoning as they work. Each observed step is captured as a (screenshot, thought, structured action, executable grounding) tuple. The executable-grounding channel is deliberately multi-path — the same editing intent can be realized as a Premiere MCP tool call, a keyboard shortcut, a menu path, or a coordinate click — following the AgentNet trajectory schema with a Premiere-specific action taxonomy layered on top. The published preview contains 234 such steps across 4 trajectories.

Results

Section titled “Results”

No model training or benchmark numbers are reported in the announcement — this is a dataset release rather than a model paper. The tweet reports 33.5K views at time of filing.

Why it’s interesting

Section titled “Why it’s interesting”

Complements Computer Use Large: 48k screen recordings (~12,300 hours) of professional software and Markov AI gaming dataset — 500+ hours of Valorant / Minecraft / GTA gameplay screen recordings + keystrokes/mouse, both of which scale internet-sourced screen recordings into computer-use training corpora, by going in the opposite direction: a small, deeply-annotated, think-aloud-supervised preview dataset in a single high-value creative-professional app. The multi-path executable grounding (MCP tool call vs shortcut vs menu vs pixel click) is the interesting design choice — it lets a downstream agent train the same semantic action against multiple execution surfaces, which is exactly the failure mode OSGym: Scalable OS Infra for Computer Use Agents flags for OS-agent generalization. Also a data-side counterpart to VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos: where VideoAgentTrek infers (clip, action, thought) tuples from unlabeled tutorials, this drop supplies human-verified (screenshot, spoken thought, structured action, grounding) tuples that could serve as an eval or SFT gold set for such inferred-trajectory pipelines. Notably orthogonal to the video-generation stack (EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning) — Contra is betting on agent-driven human-tool-use rather than end-to-end video-editing models.

See also

Section titled “See also”