Syndat Dashboard
A Dashboard for evaluation & visualization of synthetic patient level data.
Syndat is a python package for evaluation and visualization of synthetic data.
Syndat is a Python library to evaluate and visualize the quality of synthetic patient and clinical trial data. It computes synthetic data quality metrics for tabular datasets and aggregates them into three normalized scores (distribution, correlation and discrimination). Beyond metrics, Syndat visualizes how close synthetic data is to the real data: feature distributions, correlation structures, summary statistics and SHAP-based discrimination plots. It also allows users to generate stratified and interpretable visualisations, including raincloud plots, GOF plots, and trajectory comparisons, offering deeper insights into the quality of synthetic clinical data across different subgroups.
For a browser-based workflow, the SYNDAT Dashboard wraps Syndat in a web application that adds privacy risk estimates (via anonymeter) alongside the quality scores. Use it if you need a privacy assessment; the syndat package itself covers quality only.
Install via pip:
pip install syndat
The Jenson-Shannon distance is a measure of similarity between two probability distributions. In our case, we compute probability distributions for each feature in the datasets and can thus compare the statistic feature similarity of two dataframes.
It is bounded between 0 and 1, with 0 indicating identical distributions.
In addition to statistical similarity between the same features, we also want to make sure to preserve the correlations across different features. The normalized correlation difference measures the similarity of the correlation matrix of two dataframes.
A low correlation difference near zero indicates that the correlation structure of the synthetic data is similar to the real data.
A classifier is trained to discriminate between real and synthetic data. Based on the Receiver Operating Characteristic (ROC) curve, we compute the area under the curve (AUC) as a measure of how well the classifier can distinguish between the two datasets.
An AUC of 0.5 indicates that the classifier is unable to distinguish between the two datasets, while an AUC of 1.0 indicates perfect discrimination.
Exemplary usage:
import pandas as pd
from syndat.metrics import (
jensen_shannon_distance,
normalized_correlation_difference,
discriminator_auc
)
real = pd.DataFrame({
'feature1': [1, 2, 3, 4, 5],
'feature2': ['A', 'B', 'A', 'B', 'C']
})
synthetic = pd.DataFrame({
'feature1': [1, 2, 2, 3, 3],
'feature2': ['A', 'B', 'A', 'C', 'C']
})
print(jensen_shannon_distance(real, synthetic))
>> {'feature1': 0.4990215421876156, 'feature2': 0.22141025172133794}
print(normalized_correlation_difference(real, synthetic))
>> 0.24571345029108108
print(discriminator_auc(real, synthetic))
>> 0.6
For convenience and easier interpretation, a normalized score can be computed for each of the metrics instead:
# JSD score is being aggregated over all features
distribution_similarity_score = syndat.scores.distribution(real, synthetic)
discrimination_score = syndat.scores.discrimination(real, synthetic)
correlation_score = syndat.scores.correlation(real, synthetic)
Scores are defined in a range of 0-100, with a higher score corresponding to better data fidelity.
Visualize real vs. synthetic data distributions, summary statistics and discriminating features:
import pandas as pd
import syndat
real = pd.read_csv("real.csv")
synthetic = pd.read_csv("synthetic.csv")
# plot *all* feature distribution and store image files
syndat.visualization.plot_distributions(real, synthetic, store_destination="results/plots")
syndat.visualization.plot_correlations(real, synthetic, store_destination="results/plots")
# plot and display specific feature distribution plot
syndat.visualization.plot_numerical_feature("feature_xy", real, synthetic)
syndat.visualization.plot_numerical_feature("feature_xy", real, synthetic)
# plot a shap plot of differentiating feature for real and synthetic data
syndat.visualization.plot_shap_discrimination(real, synthetic)
Postprocess synthetic data to improve data fidelity:
import pandas as pd
import syndat
real = pd.read_csv("real.csv")
synthetic = pd.read_csv("synthetic.csv")
# postprocess synthetic data
synthetic_post = syndat.postprocessing.assert_minmax(real, synthetic)
synthetic_post = syndat.postprocessing.normalize_float_precision(real, synthetic)
An example demonstrating how to compute distribution, discrimination, and correlation scores, as well as how to generate stratified visualizations (gof, raincloud and other plots), is available in examples/rct_example.py.
Full API documentation and a quickstart guide are available at syndat.readthedocs.io, including an overview of the synthetic data quality metrics and how to interpret them.
Syndat is developed at the Fraunhofer Institute for Algorithms and Scientific Computing (SCAI) and is used within the NFDI4Health Consortium for the assessment of synthetic health data.
It is currently also being extended as part of the SYNTHIA collaboration, which develops synthetic data methodology for clinical trials and health research.
Syndat is aimed at researchers and data scientists working with sensitive health data who need to check whether a synthetic dataset preserves statistical properties before using it for analysis or model development.
If you use Syndat in your research, please cite as:
@article{Adams_On_the_fidelity_2025,
author = {Adams, Tim and Birkenbihl, Colin and Otte, Karen and
Ng, Hwei Geok and Rieling, Jonas Adrian and
Näher, Anatol-Fiete and Sax, Ulrich and
Prasser, Fabian and Fröhlich, Holger},
title = {On the fidelity versus privacy and utility trade-off of synthetic patient data},
journal = {iScience},
volume = {28},
year = {2025},
doi = {10.1016/j.isci.2025.112382}
}
A Dashboard for evaluation & visualization of synthetic patient level data.