Skip to content
Bud's Wiki

Twin-2K-500: A Dataset for Building Digital Twins of over 2,000 People Based on Their Answers to over 500 Questions

TL;DR

Section titled “TL;DR”

Twin-2K-500 is a public dataset of 2,058 US participants who each answered over 500 questions across four survey waves (≈2.42 hours per person), designed as a ground-truth testbed for evaluating LLM-based “digital twin” simulations of individual human behavior. It bundles 14 demographic items, 19 personality tests (26 constructs / 279 questions), 11 cognitive-ability tests, 10 economic-preference tests, replications of 11 between-subject + 5 within-subject behavioral-economics experiments, and a pricing study, with the final wave repeating earlier tasks to give a test-retest accuracy baseline against which any LLM-twin prediction can be measured. Published as a Database Report in Marketing Science 44(6):1446–1455 under CC-BY, with the data and code publicly released.

Key claims

Section titled “Key claims”
  • Releases a representative-sample US dataset of N=2,058 participants answering >500 questions across four waves at ≈2.42 hours per person, with both raw responses and a persona_summary quick-overview field per participant [Abstract].
  • Survey content combines 14 demographic questions, 19 personality tests measuring 26 constructs over 279 questions, 11 cognitive-ability tests (85 questions, 11 measures), 10 economic-preferences tests (34 questions, 10 measures), 11 between-subject + 5 within-subject behavioral-economics replications, and the Gui & Toubia (2023) pricing study [§ “We assembled a wide range of measures…”].
  • Wave 4 re-administers tasks from earlier waves, yielding a per-participant test-retest baseline that bounds how well any LLM-twin can score on individual prediction [Abstract].
  • Defines an accuracy metric that generalises from binary to numerical questions as 1 − |pred − truth| / range, ranging in [0, 1] and equal to 1 on exact match [§ Method, accuracy definition].
  • Initial analyses show LLM-built digital twins can predict participant behaviour at both individual and aggregate levels, but the paper frames the dataset itself — not any specific twin recipe — as the contribution [Abstract; §1 Introduction].

Method

Section titled “Method”

The authors recruit a representative US sample (N=2,058) and run a four-wave online survey averaging 2.42 hours per participant. Waves 1–3 cover demographics, personality (Big Five and 18 other inventories), cognitive ability, economic preferences, and replications of canonical between- and within-subject behavioral-economics experiments, plus the pricing study from Gui & Toubia (2023). Wave 4 repeats a subset of earlier items to establish a per-person test-retest ceiling. The released dataset is organised into three folders per the openBIGdata mirror: complete persona records (full survey responses in text form, chunked for processing), per-participant persona_summary quick-look fields, and the raw measure tables. For overlap items present in both waves 1–3 and wave 4, the wave-4 response is used as the canonical value. To benchmark LLM digital twins, the authors define accuracy as 1 − |pred − truth| / range_of_answers, which collapses to standard accuracy in the binary case and gives a graded score for numerical/Likert items.

Results

Section titled “Results”

The paper’s headline “result” is the dataset itself — scale, breadth, and a test-retest ceiling for individual-level prediction. The authors report descriptive statistics and cross-measure correlations across the dozens of constructs collected, and run initial LLM-twin experiments showing predictions are of high enough quality to be useful at both individual and aggregate levels (numerical breakdowns are in the paper proper at Marketing Science 44(6):1446–1455). The release pairs the data with code under CC-BY.

Why it’s interesting

Section titled “Why it’s interesting”

This is the first paper Bud has filed that is explicitly a benchmark for LLM-as-persona-emulator rather than for synthetic content generation — it sits orthogonally to Synthetic Training Data, which collects recipes where models generate the training corpus; Twin-2K-500 instead pins down a real-human ground truth against which any synthetic persona can be scored. The Wave-4 test-retest baseline is the key methodological move: it gives an upper bound on individual-prediction accuracy that is not model-limited but human-noise-limited, so an LLM-twin’s score can be reported relative to humans’ own self-consistency rather than against an aspirational 100%. Compared with EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World — which similarly bets on real-human ground truth as a scarce resource, but for embodied behaviour from egocentric video — Twin-2K-500 covers the cognitive/preference slice of the same problem with survey-style instruments.

See also

Section titled “See also”