Research & Advisory

When can synthetic respondents
be trusted?

Every provider claims superior quality — each against a benchmark it graded itself. The Behavioral Validation Engine (BVE) is our answer: an independent research program that validates synthetic populations against what people actually do, not just what they say in surveys.

11,000+synthetic interviews
3countries
6commercial AI models
2 yrsof real purchase behavior as ground truth

The program

The BVE tests synthetic respondents in three stages: whether they express the personality they are given, whether that personality translates into coherent decision parameters, and — the stage almost no one runs — whether their predictions match real household purchase behavior. The ground truth is behavioral: two years of real transaction data, not another survey.

What the research found

Fidelity is construct-dependent. The same synthetic respondent that is remarkably good at one class of question fails, measurably and repeatably, at another. The boundary has a structure — and it can be mapped for any questionnaire before a single dollar is spent.
More data can make predictions worse. Adding rich persona layers on top of clean behavioral data can reduce accuracy — a counterintuitive interference effect with direct consequences for how synthetic panels should be built.
The human benchmark is not neutral. Survey data carries its own systematic response biases. High agreement with a contaminated benchmark is not accuracy. Validation must ask what the benchmark itself is worth.

The Buyer's Guide

We published our evaluation framework openly: A Buyer's Guide to Synthetic Population Solutions — 20 questions across data provenance, generation architecture, behavioral validation, transparency, bias, temporal dynamics, and governance. Use it on any provider, including us.

READ THE BUYER'S GUIDE →

Applied: the Synthetic Readiness Audit

For research teams that need the applied version — your questionnaire, your benchmarks, your vendor — we run the Synthetic Readiness Audit: an independent, vendor-neutral assessment of where synthetic respondents hold and where they must not be used.

EXPLORE THE AUDIT →

Lead researcher

Adriana Rocha — founder of Wisdom Beyond Technology, inventor of INO-OS, author of Refactoring the Firm, and lead author of the BVE validation studies. Full research at adrianarocha.me.