FineWeb-C: A Community-Driven Dataset for Educational Quality Annotations in 122 Languages

TL;DR

HuggingFace community-coordinated annotation effort that produced 58,185 educational-quality labels over 122 languages on top of FineWeb2, designed to train per-language educational-quality classifiers in the same lineage as the FineWeb-Edu filter for English. 465 contributors participated, with a long-tail distribution (14 “diamond” contributors with 1000+ labels carrying the bulk; one contributor logged 4,614 labels alone). The dataset is shipped as data-is-better-together/fineweb-c on the Hub. The wiki interest is in the upstream of pretraining-data filters that drove the SmolLM3 result, and in how the labeling pipeline is constructed when no single org can speak 122 languages.

Key claims

58,185 educational-quality annotations were collected across 122 languages from 465 distinct contributors, against the FineWeb2 multilingual web corpus [§By the Numbers].
The contributor distribution is heavily long-tailed: 14 Diamond (≥1000 annotations), 18 Gold (500–999), 65 Silver (100–499), 368 Bronze (1–99) [§By the Numbers].
The framing is explicitly to “develop educational quality classifiers to help create better open LLMs in more languages” — i.e. the dataset is the supervised signal for a downstream filter, not a direct training corpus [§intro].
Per-language coverage is uneven and driven by individual annotators: Tatar 3,015, Vietnamese 2,869, Danish 2,573, Tigrinya 1,837 are the top reported counts [§Building a Multilingual Dataset Together].
The dataset is loadable as a single multilingual table or sliced per language via load_dataset("data-is-better-together/fineweb-c", "<language>") [§intro].
The post argues this annotate-then-train-a-classifier-then-filter-the-web loop is a transferable blueprint for multilingual data work that no single company can do alone [§A Blueprint for the Future].

Method

The annotation effort sits in the FineWeb lineage. FineWeb-Edu is HuggingFace’s English-only filter that scored web documents for educational quality (via a Llama-3-trained classifier over Llama-3 labels) and was used in Smol Training Playbook (GPU MODE talk on SmolLM3) as a load-bearing pretraining ingredient. FineWeb-C generalizes that recipe to the 122-language FineWeb2 corpus, but with a crucial change of labeling source: instead of a single frontier model producing the labels, the labels come from a distributed community of native speakers via Argilla / Hub-hosted annotation spaces.

The post does not describe the annotation schema in detail beyond “educational quality,” nor does it report classifier-training results — the artifact released here is the labeled examples (~58k of them), not a trained filter. Downstream classifier-training work and filtered-corpus releases are flagged as community follow-up rather than included in this post.

Results

58,185 annotations across 122 languages, with extreme per-language and per-annotator variance: top-3 annotators (@stefan-it 4,614, @tagay1n 2,094, @hannayukhymenko 1,937) account for ~15% of all labels [§Building a Multilingual Dataset Together, §Acknowledgments].
No per-language classifier accuracy, no filtered-corpus quality measurement, and no downstream-LM evaluation is reported in the post — it is a dataset-release announcement, not a methods paper.

Why it’s interesting

This is the upstream of the data-curation lever that Smol Training Playbook (GPU MODE talk on SmolLM3) credits as the dominant return-on-iteration in SmolLM3 — FineWeb-Edu (English) → SmolLM3 was the recipe; FineWeb-C → per-language educational classifiers → multilingual variants of that recipe is the natural extension, and this post is the dataset that unlocks it. It also lands in active tension with A Bitter Lesson for Data Filtering, which argues that in the high-compute, data-scarce regime the best filter is no filter at all — FineWeb-C is exactly the kind of human-designed quality signal that paper claims is the small-scale-calibrated heuristic the bitter lesson will eventually wash out. Whether per-language community-annotated quality filtering survives at frontier-multilingual scale is therefore a live empirical question. Separately, the labeling architecture itself is the interesting datapoint: it’s a working counter-example to the “synthetic labels from a frontier model” recipe (PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models, Action100M: A Large-scale Video Action Dataset) in a setting — 122 languages, many low-resource — where no frontier model has the coverage to be the labeler.