Real World Data (RWD) associated with healthcare can include various types of data such as clinical data, biological samples and analyses, laboratory test results, prescriptions, clinical notes, electronic health records (EHRs), as well as social and demographic data.
The extraction of new knowledge from this data is referred to as “secondary use” because it involves repurposing the data for objectives different from the primary purpose, the provision of healthcare services. The secondary use of health data offers significant opportunities to enhance research and innovation, ultimately improving patient care.
Among these opportunities is the development of prediction algorithms, which leverage large RWD datasets to analyse historical and real-time data, to forecast patient outcomes, identify trends and promote personalised medicine. These prediction algorithms, which utilise Machine Learning (ML) and statistical models, can be applied in areas such as risk stratification and classification, where they can predict disease progression, potential complications, or the probability of treatment success, thereby offering valuable insights to support clinical decision-making.
However, to harness the potential of health data for these applications, several challenges must be addressed. Health data is personal and highly sensitive, governed by legal and ethical issues that prevent its potential sharing for joint processing and analysis. Furthermore, RWD exhibits heterogeneity across different entities, such as variations in type, format, and data quality, which creates technical interoperability issues, while information is often only available in free-text format (e.g., clinical notes), posing an additional challenge due to the difficulty of converting free text into a more processable format.
To address the challenges related to data privacy, federated learning (FL) offers a promising solution. Specifically, the FL approach distributes the predictive model across local databases instead of transferring data to the location of the model. Each local model is trained on its respective database, updating and refining its parameters locally. These updates are then aggregated at a central node to create a unified predictive model. Rather than centralising sensitive patient data, FL ensures data privacy while still enabling the development of accurate, data-driven models. Federated algorithms can enhance prediction accuracy by combining data from various sources while maintaining strict privacy standards.
In the context of the PREPARE project, aiming to improve rehabilitation care by developing reliable patient stratification and prediction tools, utilizing RWD from nine different clinical partners to address diverse pathologies, FL is employed to leverage decentralized clinical sites for training machine learning models without requiring the consolidation of patient data into a central repository. FL necessitates coordinated operations across all decentralised nodes, such as local clinical databases, and the central node responsible for synthesising the final predictive model. Open-source tools provided by the OHDSI community are leveraged for supporting this framework. In detail:
- The Arachne Node software enables essential operations on local nodes, including interfacing with local databases, communicating with the central node, generating descriptive statistics of local databases, and training predictive models on local data.
- The Achilles library, developed in R, facilitates the generation of descriptive statistics for databases, forming the basis for the corresponding functionality within Arachne Node.
- The Arachne Central application supports the central node in inspecting descriptive statistics from local databases, distributing analyses to local nodes for model training, and aggregating updated parameters from the trained models.
- The Ares tool offers a graphical interface for reviewing and comparing descriptive statistics across multiple nodes.
- Lastly, Atlas is utilised for designing scientific analyses on clinical data, enabling researchers to categorise patient groups based on diagnoses, medications, and other characteristics. It also supports the initialisation of predictive models and provides tools for comparing patient cohorts and exploring medical terminology through the Athena vocabulary.
Addressing the challenge of data standardisation, the PREPARE project is transforming the clinical data to the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) format. It is noted that while the transformation is ongoing, initial testing is performed using available synthetic datasets.
To support model training and validation, we deployed a pipeline that trains a Patient-Level Prediction (PLP) model, which serves as the foundation for what we refer to as the Global Model. During the initial training phase, the synthetic Eunomia dataset is divided into two subsets: 75% is used for training, while 25% is reserved for testing the model’s accuracy. Once the Global Model is trained, we use the training subset and split it into smaller partitions. Each partition, a segment of the Eunomia dataset, represents a local database, simulating the different clinical sites. Alongside multiple Arachne node instances, we simulated distributed clinical sites to train PLP models locally, generating Local Models. These Local Models were then uploaded to the Arachne Central, where they were aggregated to create the Federated Global Model.
At the end of this process, the pipeline produces: a single Global Model, multiple Local Models, (equal to the number of the splits performed) and a Federated Global Model created by aggregating the different Local Models.
Finally, we evaluate all models (Global, Local, and Federated) against the same reserved test subset. This uniform evaluation ensures consistent benchmarking and provides insights into the performance of Local Models in comparison to the Global and Federated Global Models. The pipeline also enables cross-validation by allowing Local Models to be tested on subsets from other Local Models, simulating data sharing across clinical sites.
By leveraging this robust testing pipeline, we aim to refine our approach to model training, ensuring scalability and accuracy in real-world scenarios.
In the PREPARE project, the secondary use of clinical data through prediction algorithms development and federated learning, will enable healthcare professionals to make data-driven decisions and improve patient outcomes, all while ensuring privacy and security of the sensitive health data.
Authors: Giorgos Giannios, Vasilis Alepopoulos, Margarita Grammatikopoulou, Spiros Nikolopoulos, Centre for Research and Technology Hellas