In October 2019 I gave a talk at the CDISC US Interchange in San Diego called “Synthetic Data and Predictive Data Modeling: Benefits in Driving Standards and Quality Outcomes.” The paper concluded that although test data is a necessary evil to many, holistically produced synthetic study data has an enormous and largely unrealized potential. Synthetic study data was a quiet corner of the field back then—it isn't anymore. So, it is time to take stock of my 2019 opinions considering today's evolving trends and technologies.
What I argued in 2019
The talk cited three common types of test data:
- Zombie data is cobbled together from past test cycles and dragged forward on the assumption that the old data model still fits the new one.
- Misfit data is borrowed from a similar study, with the gaps patched by hand, and brings potential privacy and unblinding risks.
- Piecemeal data consists of sparse records mocked up per form to satisfy individual test cases, sometimes keyed in by hand.
The paper explained why the mocked-up and borrowed kinds fall short.
Against those, I set the axes of integrity—the congruity real data has because real subjects lived the protocol: cohorts enrolled in a shared window, visits in sequence with dates that agree, assessments inside their visit windows, histories that predate enrollment, and durations that end on or after they start. Common test data rarely meets this integrity bar. Ironically, this makes it unfit for testing what a downstream user might want it for, such as a monitoring report tied to real dates, a patient profile, or a cross-form edit check. I borrowed a statistical concept for test data and asked whether such data had the power to support reliable conclusions about the quality of the study build. In 2019 the answer was often no.
The paper laid out a four-phase method for constructing test data with integrity:
- Acquire the study's design metadata.
- Set up the synthetic study deliberately: a subject roster, per-subject visit inventories, dataset templates, and terminology.
- Conduct the study, generating data chronologically, visit by visit, so that later data can react to earlier events.
- Process the output into whatever downstream shape is needed.
Buried in the text, I introduced a concept that would take on greater meaning: in synthetic data generation, edit checks and rules are prescriptive, not proscriptive. A rule written to catch bad data can be reverse engineered to define what should be created. Quality logic, run in the opposite direction, becomes a generator.
What changed in seven years
- Conformance rules became executable. Before the 2019 paper I had founded and co-led the CDISC SDS Conformance Rules team, which wrote down rule by rule what “conformant” means. Those efforts laid the foundation for the present CORE rules catalog and the development of an open-source engine that can automate the execution of those rules against data to evaluate conformance to standards. The rules stopped being documentation and became software.
- The protocol became machine-readable. USDM and the Digital Data Flow work turned study designs (arms, epochs, visit schedules, activities) into structured data. In 2019 the schedule of assessments was a table in the protocol. A design authored in USDM carries it as data a program can read.
- Large language models arrived. They can read rules written in prose, and they made coherent, realistic structure learnable at scale.
Where synthetic study data is now
The method in the 2019 paper was real: its appendix was SAS code that generated a DM domain from a metadata sheet. But it was a craftsman's method. I wrote the supplemental metadata and the rules by hand. What I have built since, Verdatic, is a language-agnostic software platform that uses standards and rules throughout the synthetic data creation process.
Using a study design and CDISC standards, it simulates a multi-domain SDTM study: demographics, vital signs, labs, adverse events, and more. Visit schedules come from the machine-readable design, and treatment arms can carry modeled treatment effects. The 2019 “axes of integrity” are no longer a checklist to aspire to; they are enforced by construction, because subjects are generated as coherent digital twins who experience the protocol chronologically—exactly the “synthetic study conduct” idea, now enhanced by AI with human governance.
This prescriptive-rules concept is now the core of Verdatic. Conformance tooling is built to grade data that already exists. Verdatic runs the official CORE engine as part of making the data. It generates a study, runs the engine, feeds the findings back to the generator, and repeats until the engine has little left to report. A clean verdict says the data has the right shape. It does not say the values are believable, and that is what the October talk takes up.
Verdatic shines in the synthetic data “waterfall,” the downstream half of my 2019 argument. Synthetic data that exists before first patient in lets a team rehearse the pipeline: mapping specifications, value-level metadata, and the generated SAS, R, and Python programs that carry the data through SDTM to ADaM. UAT changes too. Where the paper's coverage diagram showed test subjects scattered thinly across the schedule of assessments, Verdatic now generates a clean twin and a deliberately defective twin from the study's edit checks, one for the PASS test cases and one for the FAIL cases.
What held up, and what I underestimated
The benefits case held up; I'd argue it was conservative. Test data with real power does accelerate study builds, qualification, and downstream program development, and the 2019 use case I flagged for the standards community—using synthetic data to evaluate a new standard's impact before adoption—has only grown more relevant as standards versions are expected to become more dynamic and adaptive to industry requirements.
What I underestimated: In 2019, prescriptive rules meant a study's edit checks, translated by hand into generation logic by someone like me. I did not foresee the official conformance rule set, run by an open engine, steering a generator in a loop, with people supervising the loop instead of feeding it by hand. When the generator can't satisfy a rule, sometimes the data is wrong, and sometimes the rule or the way it is evaluated is. Both show up in synthetic data, before any real data exists.
None of this replaces the need for appropriate human oversight; the judgment calls stay with people. An important caveat: although some progress has been made using synthetic data for modeling placebo arms, the synthetic data described here is for development, testing, training, and demonstration, not regulatory submission.
Where this goes next
On October 6 I was back at the CDISC US Interchange, in Denver, with “Agentic AI Meets Data Simulation: A Framework for Self-Optimizing Clinical Trial Digital Twins.” In 2019 I brought a vision. This time I presented how that vision was realized. In 2019 I discussed ‘axes of integrity’; in 2026 I proposed seven axes for assessing synthetic data quality, and I'd like help gaining consensus for how these are defined and measured.
Message me on LinkedIn with how you would define or measure the seven axes.
Here is the six-minute demo from Verdatic's entry in the CDISC AI Innovation Challenge:
Start a conversation with Clinventive about what synthetic data could do for your study build.