The seven axes
Each axis asks one question and is reported on its own. There is no total score, because a good result on one axis cannot make up for a poor result on another. Several axes refer to the define: the Define-XML file (define.xml) that describes a study's datasets.
| Axis | The question it asks | What it lets you test |
|---|---|---|
| 1. Structural conformance | Does the data satisfy the published rules for its standard? | Mapping, the define, and any tool that expects conformant input |
| 2. Referential integrity | Are subject, visit and parent-record references consistent across datasets? | Cross-form checks, merges and joins |
| 3. Temporal coherence | Is the study's timeline consistent? | Date and visit-window edit checks, visit derivations |
| 4. Clinical plausibility | Would a clinician believe the values? | Medical review listings, lab units, coding checks |
| 5. Longitudinal realism | Do each subject's values change over time as one person's would? | Change-from-baseline derivations, shift tables, subject profiles |
| 6. Cross-domain clinical consistency | Are the parts of each subject's record clinically consistent? | Safety listings across datasets, medications against indications |
| 7. Deliverable completeness | Is everything the define declares delivered, and does the define match the data? | The define and the package |
Axis 1 is scored from a report by CDISC CORE, CDISC's open-source conformance engine. The scorer reads the report and does not run the rules itself. The other six axes are scored by the scorer, following the draft method.
What you need
- The study's SDTM datasets, saved as XPT files (.xpt, SAS transport version 5), Dataset-JSON files (.json, CDISC's JSON format for datasets) or CSV files (.csv). SAS data sets (.sas7bdat) are not read. A folder or a ZIP file is fine, and the scorer looks inside every subfolder.
- Optional: the define (define.xml). Axis 7 uses it to check that every dataset it declares was delivered, and axis 4 uses it to check coded values against its codelists.
- Optional: a CDISC CORE report in JSON. Axis 1 is scored from it. See Getting a CORE report.
- A recent version of Chrome, Edge, Firefox or Safari on a desktop or laptop computer. There is nothing to install and no account to create.
If the study comes with a facts file (submission.json) in the format the scorer's command-line version uses, the scorer reads it too, and the facts it states take the place of what the scorer worked out from the files. The scorer says when it used one.
How to score a study
- Open the scorer. Go to clinventive.com/fidelity/score.
- Give it the study. Drag the study folder or its ZIP file onto the box, or click Choose files or a ZIP. In Chrome and Edge there is also Choose a folder. The browser then asks whether to let the site view the files in that folder; that permission lets the page read them on your computer. In other browsers, drag the folder onto the box. Dragging asks for no permission. Whichever you use, the page reads the files on your computer and does not send them anywhere (see What stays on your computer).
- Check what it found. The scorer lists the datasets it will score and whether it found a define, a CORE report and a visit schedule. If the same datasets are in two formats, it scores one copy and says which. If something you expected is missing, add the file and score again. If it finds no datasets at all, check that they are .xpt, .json or .csv files. If it stops with an error, send the message through the contact form.
- Read the results. Each axis has its own card. See How to read the results.
- Save the results. Save the scorecard as text saves a file you can read or share. Save the results for other software saves the same results as JSON. Both files are made on your computer.
How to read the results
Each axis card shows one of six labels.
| No problems found | Every check on this axis ran and none found a problem. |
| No problems in the checks that ran | None of the checks that ran found a problem, but some checks were not run. The card lists them as Not run and says why. Those parts of the axis are unknown, not passed. |
| Problems found | At least one check found a problem. The card lists each check, how many times it ran, and how many problems it found, with examples. |
| Not scored | Nothing on this axis could be checked with the files supplied. The card says what was missing, such as a CORE report for axis 1. |
| Measured, no target | Measurements that are not pass or fail, such as how consistent each subject's values are from visit to visit (the intraclass correlation, ICC). There is no right answer. Compare them with other studies. |
| Checklist | Axis 7 lists what was delivered: yes, no, partly, or could not tell. |
A check that was not run says why, on its own row. Each reason is one of six kinds.
| Nothing to check | The study has no records the check applies to. For example, the check on discontinuations attributed to an adverse event has nothing to check in a study where no subject discontinued for that reason. |
| Not supplied | A dataset or file the check reads was not in the files, such as the Subject Visits (SV) dataset or a define. In a CORE V1.0 report, it can also be a coding dictionary a rule needs. Some datasets are optional, and a study may simply have none. |
| No value | A record has no value the check needs, or a value that cannot be read. An event that is still ongoing, for example, has no end date. |
| Dates too imprecise | A date is not precise enough for the check, for example a start recorded only as March 2022 and an end on 15 March 2022. |
| Not run by design | The method does not run the check in this case, for example a range check for a test with no published limit to cite, or a subject outside the reference ranges that ship with the scorer. |
| CORE did not run it | CDISC CORE gave no pass or fail for the rule, because it skipped the rule or failed to run it. A CORE V1.0 report gives the reason for a skip, and the scorer shows it. A CORE 0.17.1 report does not. |
Only Not supplied can be fixed from your side: add the file and score again. A dictionary goes to CORE and not to the scorer, so it means making a new report. The other kinds describe the data, the method or CORE. Variable names written with two dashes, such as --DTC, stand for the same variable in every dataset: LBDTC in LB, VSDTC in VS.
Three rules apply to every card.
- "Not run" is not a pass. A check that was not run is counted separately, and never added to the checks that passed.
- There is no total. Read each axis on its own.
- Versions are printed. Every result names the method, scorer, reference-range and CORE versions it used, so results from different versions are not compared by mistake. Axis 1 also prints a rule set fingerprint (a short code computed from the ID and version of each rule in the CORE report); see Comparing axis 1 results.
Getting a CORE report (axis 1)
Axis 1 is scored from a CORE report. CDISC CORE does not run in a browser. It is a program you start by typing a command on your computer, so you make the report there and then add it to the scorer. If you do not have a CORE report, the other six axes are still scored and axis 1 shows Not scored.
Which CORE to use
CDISC's CORE page lists two versions of CORE, and they run different sets of rules. It lists CORE V1.0 for SDTMIG 3.2, 3.3 and 3.4 and the FDA Business Rules, and CORE 0.17.1 for TIG, USDM and SENDIG, which are other CDISC standards. CORE V1.0 is in Verisian's GitHub account, the location CDISC's page links to, and its page calls it Verisian's version of the CDISC rules engine. CORE 0.17.1 is in cdisc-org, CDISC's GitHub account. It still checks SDTM data with the rules it ships with.
| CORE V1.0 | CORE 0.17.1 | |
|---|---|---|
| Use it if | Your datasets are XPT files and you can install Python | You cannot install Python, or your datasets are Dataset-JSON |
| Rule packages CDISC released on November 3, 2026 | Included | Not included |
| Dataset files it reads | XPT (.xpt) | XPT (.xpt) or Dataset-JSON (.json) |
| Setup | Install Python 3.12, then run two commands | Unzip a download |
For SDTM data, use one of these two versions, not the v1.0.0 release in cdisc-org. Two GitHub releases carry the number 1.0.0, and they are different code. The one in cdisc-org is marked Latest on that account's releases page, and CDISC's CORE page does not list it.
In our test on one SDTMIG 3.4 study in November 2026, the cdisc-org release gave different results from CORE V1.0 on the same files.
Reports from both say version 1.0.0. If your report says 1.0.0 and you made it with a downloaded program instead of Python, make a new one with either version here.
Comparing axis 1 results. CORE V1.0 and CORE 0.17.1 run different sets of rules, so the same data can get different axis 1 counts from each. Results can also differ between two downloads with the same version number, and the CORE report does not name the build. Compare two axis 1 results only when they name the same CORE version and the same rule set fingerprint. A different fingerprint means the rule sets differ. The same fingerprint does not show that two runs used the same engine code, because the fingerprint does not change when only the engine's code changes.
Set up CORE V1.0
- Install Python 3.12 from the Python 3.12.10 page. Under Files, choose Windows installer (64-bit) or macOS 64-bit universal2 installer. On Windows, tick Add python.exe to PATH in the installer. Later 3.12 releases do not have installers. CORE V1.0 does not support Python 3.11 or older, or 3.13 or newer. You can keep other versions of Python installed. On Linux, see the CORE V1.0 instructions.
- Open the CORE V1.0 release page,
which is on GitHub. Scroll down to Assets and download Source code (zip). Unzip it to a short path,
such as
C:\core-v1. In our test on Windows, the install in step 4 failed from a long folder path. The download includes the rules, and no GitHub account or CDISC Library key is needed. - Find the folder that contains
core.py, which is one level inside the unzipped folder, and open a command window there. On Windows, open that folder, typepowershellin the address bar and press Enter. On macOS or Linux, open Terminal, typecdand a space, drag the folder onto the Terminal window, and press Enter. - Run these two commands, one at a time. The first makes a folder called
venv, a private copy of Python for CORE. The second downloads the software CORE needs into it. You do not repeat them for the next study.On Windows:
py -3.12 -m venv venv .\venv\Scripts\python.exe -m pip install -r requirements.txt
On macOS or Linux:
python3.12 -m venv venv ./venv/bin/python -m pip install -r requirements.txt
- Go to Make the report.
Set up CORE 0.17.1
- Open the CORE 0.17.1 release
page, which is on GitHub. Scroll down to Assets and download the file for your computer:
core-windows.zipfor Windows,core-mac-apple-silicon.ziporcore-mac-intel.zipfor a Mac, orcore-ubuntu-latest.zipfor Linux. Unzip it. The download includes the rules, and no GitHub account or CDISC Library key is needed. - Open a command window in the unzipped folder. On Windows, open the folder that contains
core.exe, typepowershellin the address bar and press Enter. On macOS or Linux, open Terminal, typecdand a space, drag the unzipped folder onto the Terminal window, and press Enter. - Go to Make the report.
Make the report
Before you run the command.
- Use full paths, which start from the drive or the top folder, as the boxes ask. In our use of CORE 0.17.1, it did not read datasets through a relative path (one that starts from the current folder).
- For CORE V1.0, the folder that holds the datasets must hold XPT files, with one file for each dataset, named
after it (
ae.xpt). CORE V1.0 skips other formats, such as Dataset-JSON, with a warning. - On macOS, if a security warning stops CORE 0.17.1 from opening,
its quick-start guide
gives this fix, run in its folder:
xattr -rd com.apple.quarantine . chmod +x ./core
- Choose the CORE version you set up and your computer, then fill in the boxes below.
- Copy the command, paste it into a command window opened in the CORE folder, and press Enter. CORE V1.0 took up to about four and a half minutes on a 100-subject study in our tests.
- CORE saves
core-report.jsonin the folder you chose. Go back to the scorer, add that file with the study files, and score again.
If the command does not work, you can still score the study without the report. To ask for help, send the error message through the contact form.
What stays on your computer
The scorer page reads your files in your browser and shows the results there. It does not send the files, or anything about them, to Clinventive or to anyone else.
The page is served with a security policy that tells the browser to block the ways a page sends data in the
background: uploads, data requests and connections (connect-src 'none'), and form submissions
(form-action 'none'). The page loads nothing from other sites, no fonts, images or scripts, and its own
code makes no requests after the page has loaded. To check this yourself:
- Open the scorer, then open your browser's developer tools (F12 on Windows, or Option-Command-I on a Mac) and choose the Network tab.
- Score a study. No new requests appear in the list while the files are read and scored.
The code the page runs is the same scoring code as the command-line version, served as readable files rather than compressed into one, at /fidelity/score/lib/.
Questions
Can I score real study data?
The scorer reads SDTM datasets in the formats listed above, whatever their source, and the files stay on your computer. The axes were written to judge synthetic data.
Why is there no overall score?
A single number would have to trade a structural defect against a measure of realism, and any weighting of the two would be arbitrary. Each axis is reported on its own instead.
What does "draft 0.3" mean?
The method is a draft and may change. Each result names the versions it used, so you can tell which method produced it. The method explains each axis, and the scoring specification says exactly what is computed. Both are open for comment.
Where do the reference ranges for axis 4 come from?
From a reference bundle that ships with the scorer. Its version is printed with every result.
Is there a version for scripts and automated pipelines?
A command-line version runs the same scoring code. It is not published yet.
Who made this?
Clinventive, LLC. The seven axes were proposed by Ward Stetson Line at the 2026 CDISC US Interchange. He co-founded and led the CDISC SDS Conformance Rules team.
How do I send a question or a comment?
Use the contact form on clinventive.com.