Skip to content
Bud's Wiki

Computer Use Large: 48k screen recordings (~12,300 hours) of professional software

TL;DR

Section titled “TL;DR”

Markov AI’s computer-use-large is an open-source CC-BY-4.0 dataset of 48,478 internet-sourced screen-recording videos (~12,300 hours total) of professional software in use, spanning AutoCAD, Blender, Excel, Photoshop, Salesforce, and VS Code. All clips are trimmed to remove intros/outros/talking heads and have audio stripped, leaving only screen-recording content; per-video metadata gives category, trimmed duration, and number of contiguous segments. It targets the same data-scarcity bottleneck that The First Fully General Computer Action Model (FDM-1) called out (the largest prior public computer-use dataset was <20 hours) — but as a labels-free pretraining corpus, not an annotated trajectory set. By March 2026 it had crossed 100K downloads on HuggingFace.

Key claims

Section titled “Key claims”
  • The dataset contains 48,478 trimmed screen-recording videos totaling ~12,300 hours, split into 6 software categories: AutoCAD (10,059 / 2,149 h), Blender (11,493 / 3,624 h), Excel (8,111 / 2,002 h), Photoshop (10,704 / 2,060 h), Salesforce (7,807 / 2,336 h), VS Code (304 / 127 h) [§Dataset Summary].
  • Each video is paired with a metadata.jsonl entry containing only file_name, category, trimmed_duration (seconds), and num_segments (count of contiguous screen-recording spans after trim) — there are no action labels, no click/keystroke annotations, no instruction prompts [§Data Fields].
  • Videos are pre-trimmed to remove “non-screen-recording content (intros, outros, talking heads, transitions)” and audio is stripped — the dataset only ships screen content [§dataset header].
  • Storage is sharded as data/{category}/ with up to 9,999 files per directory (HuggingFace’s per-directory cap), so most categories spill into a {category}_2/ sibling folder [§Data Organization].
  • Intended use is stated as “training and evaluating computer use agents — models that interact with desktop software through GUI actions (clicking, typing, scrolling)” [§Intended Use].
  • License is CC-BY-4.0 [§License].

Method

Section titled “Method”

The dataset is a sourced-and-curated corpus, not a generated one. Videos are scraped from the internet (the dataset card does not name the source platforms but the long-tail subject distribution and segment-splitting metadata field are consistent with YouTube tutorial / livestream content). The processing pipeline ingests raw video, identifies contiguous screen-recording spans (the num_segments field), discards everything else (intros, outros, talking-head cuts, transitions), strips audio, and writes per-category MP4 archives. No per-frame or per-event annotation step is applied — labeling is left entirely downstream.

A separate sibling dataset markov-ai/computer-use (referenced in search results, smaller) ships trajectory-level data with instruction, num_steps, per-step screenshots, and per-step actions strings — i.e. the annotated counterpart. computer-use-large is explicitly the unannotated pretraining-scale companion.

Results

Section titled “Results”

This is a data release, not a model paper — no eval numbers are reported on the dataset card. Indirect evidence of impact:

  • ~12,643 downloads in the last month on HuggingFace (per the dataset header at fetch time).
  • Cumulative >100K downloads as of April 2026 per third-party coverage.
  • The release significantly enlarges the open computer-use data pool: prior to it, the largest publicly available dataset was <20 hours per the framing in The First Fully General Computer Action Model (FDM-1).

Why it’s interesting

Section titled “Why it’s interesting”

This is the open analog of FDM-1’s proprietary 11M-hour screen-recording corpus — much smaller (12K vs 11M hours, a ~1000× gap) but the first publicly-downloadable dataset at the scale where the VPT-for-computers recipe in The First Fully General Computer Action Model (FDM-1) starts to be reproducible. The deliberate choice to ship unlabeled video is the right one if you believe the IDM-pseudo-labeling story: labels are cheap to manufacture given enough data + a small contractor-labeled bootstrap, but the raw screen footage is the scarce resource. It also slots into the Tool-Use Agents design space as a visual / desktop counterpart to the BFCL-style API-call corpora the text-tool-use papers train on, and to the per-task heuristic-solver demonstrations VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents uses for its 17 visual environments. The category mix (Blender + AutoCAD + Photoshop) makes it particularly relevant for Luma — three of the six categories are creative-tool software whose action distributions are not covered by web-browsing or office-app benchmarks.

See also

Section titled “See also”