Writing · August 6, 2026 · Updated October 2026 · W. Stetson Line

Synthetic Data, Then and Now.

Revisiting my 2019 Interchange paper: what held up, and what I underestimated.

In October 2019 I gave a talk at the CDISC US Interchange in San Diego called “Synthetic Data and Predictive Data Modeling: Benefits in Driving Standards and Quality Outcomes.” The paper concluded that although test data is a necessary evil to many, holistically produced synthetic study data has an enormous and largely unrealized potential. Synthetic study data was a quiet corner of the field back then—it isn't anymore. So, it is time to take stock of my 2019 opinions considering today's evolving trends and technologies.

What I argued in 2019

The talk cited three common types of test data:

The paper explained why the mocked-up and borrowed kinds fall short.

Against those, I set the axes of integrity—the congruity real data has because real subjects lived the protocol: cohorts enrolled in a shared window, visits in sequence with dates that agree, assessments inside their visit windows, histories that predate enrollment, and durations that end on or after they start. Common test data rarely meets this integrity bar. Ironically, this makes it unfit for testing what a downstream user might want it for, such as a monitoring report tied to real dates, a patient profile, or a cross-form edit check. I borrowed a statistical concept for test data and asked whether such data had the power to support reliable conclusions about the quality of the study build. In 2019 the answer was often no.

The paper laid out a four-phase method for constructing test data with integrity:

  1. Acquire the study's design metadata.
  2. Set up the synthetic study deliberately: a subject roster, per-subject visit inventories, dataset templates, and terminology.
  3. Conduct the study, generating data chronologically, visit by visit, so that later data can react to earlier events.
  4. Process the output into whatever downstream shape is needed.

Buried in the text, I introduced a concept that would take on greater meaning: in synthetic data generation, edit checks and rules are prescriptive, not proscriptive. A rule written to catch bad data can be reverse engineered to define what should be created. Quality logic, run in the opposite direction, becomes a generator.

What changed in seven years

  1. Conformance rules became executable. Before the 2019 paper I had founded and co-led the CDISC SDS Conformance Rules team, which wrote down rule by rule what “conformant” means. Those efforts laid the foundation for the present CORE rules catalog and the development of an open-source engine that can automate the execution of those rules against data to evaluate conformance to standards. The rules stopped being documentation and became software.
  2. The protocol became machine-readable. USDM and the Digital Data Flow work turned study designs (arms, epochs, visit schedules, activities) into structured data. In 2019 the schedule of assessments was a table in the protocol. A design authored in USDM carries it as data a program can read.
  3. Large language models arrived. They can read rules written in prose, and they made coherent, realistic structure learnable at scale.

Where synthetic study data is now

The method in the 2019 paper was real: its appendix was SAS code that generated a DM domain from a metadata sheet. But it was a craftsman's method. I wrote the supplemental metadata and the rules by hand. What I have built since, Verdatic, is a language-agnostic software platform that uses standards and rules throughout the synthetic data creation process.

Using a study design and CDISC standards, it simulates a multi-domain SDTM study: demographics, vital signs, labs, adverse events, and more. Visit schedules come from the machine-readable design, and treatment arms can carry modeled treatment effects. The 2019 “axes of integrity” are no longer a checklist to aspire to; they are enforced by construction, because subjects are generated as coherent digital twins who experience the protocol chronologically—exactly the “synthetic study conduct” idea, now enhanced by AI with human governance.

This prescriptive-rules concept is now the core of Verdatic. Conformance tooling is built to grade data that already exists. Verdatic runs the official CORE engine as part of making the data. It generates a study, runs the engine, feeds the findings back to the generator, and repeats until the engine has little left to report. A clean verdict says the data has the right shape. It does not say the values are believable, and that is what the October talk takes up.

The generation loop A study design and the CDISC standards feed generation. The generated study's programs are built and run, and the official CDISC CORE engine checks the resulting datasets. Its findings go back into generation, and the loop repeats. Decisions the loop cannot make with confidence go to a person. The datasets and programs go downstream with the engine's verdict. Study design + CDISC standards Generate synthetic study Build and run the programs Official CDISC CORE engine findings fed back A person decides what the loop can't Datasets + programs with the CORE verdict
Each generated study's programs are run, and the official CORE engine checks the datasets they produce. Its findings go back into generation. Decisions the loop cannot make with confidence go to a person.

Verdatic shines in the synthetic data “waterfall,” the downstream half of my 2019 argument. Synthetic data that exists before first patient in lets a team rehearse the pipeline: mapping specifications, value-level metadata, and the generated SAS, R, and Python programs that carry the data through SDTM to ADaM. UAT changes too. Where the paper's coverage diagram showed test subjects scattered thinly across the schedule of assessments, Verdatic now generates a clean twin and a deliberately defective twin from the study's edit checks, one for the PASS test cases and one for the FAIL cases.

What held up, and what I underestimated

The benefits case held up; I'd argue it was conservative. Test data with real power does accelerate study builds, qualification, and downstream program development, and the 2019 use case I flagged for the standards community—using synthetic data to evaluate a new standard's impact before adoption—has only grown more relevant as standards versions are expected to become more dynamic and adaptive to industry requirements.

What I underestimated: In 2019, prescriptive rules meant a study's edit checks, translated by hand into generation logic by someone like me. I did not foresee the official conformance rule set, run by an open engine, steering a generator in a loop, with people supervising the loop instead of feeding it by hand. When the generator can't satisfy a rule, sometimes the data is wrong, and sometimes the rule or the way it is evaluated is. Both show up in synthetic data, before any real data exists.

None of this replaces the need for appropriate human oversight; the judgment calls stay with people. An important caveat: although some progress has been made using synthetic data for modeling placebo arms, the synthetic data described here is for development, testing, training, and demonstration, not regulatory submission.

Where this goes next

On October 6 I was back at the CDISC US Interchange, in Denver, with “Agentic AI Meets Data Simulation: A Framework for Self-Optimizing Clinical Trial Digital Twins.” In 2019 I brought a vision. This time I presented how that vision was realized. In 2019 I discussed ‘axes of integrity’; in 2026 I proposed seven axes for assessing synthetic data quality, and I'd like help gaining consensus for how these are defined and measured.

Message me on LinkedIn with how you would define or measure the seven axes.

Here is the six-minute demo from Verdatic's entry in the CDISC AI Innovation Challenge:

Start a conversation with Clinventive about what synthetic data could do for your study build.

The original paper “Synthetic Data and Predictive Data Modeling: Benefits in Driving Standards and Quality Outcomes,” CDISC US Interchange, San Diego, October 2019 — read the PDF.