Data preparation is unglamorous work. In most project plans it appears as a single line between data collection and analysis, as though it were a brief administrative step rather than the most consequential and time-consuming phase of any data-intensive project. In my experience managing multiomics datasets from Phase 2 and Phase 3 clinical trials at Galderma, data preparation was not a step. It was the majority of the work, and the quality of everything downstream depended entirely on how well it was done.
This post is about what that actually looks like in practice, specifically in the context of multiomics, where the complexity of data preparation is higher than in almost any other research setting. I will draw on both that experience and on the data preparation principles I have since formalised through structured AI project management methodology, because the parallels are more direct than you might expect.
What multiomics data actually is
Multiomics is the simultaneous study of multiple biological layers within the same sample or subject. Where a single-omics study might examine only gene expression, a multiomics study combines data from genomics (DNA sequence and variation), transcriptomics (gene expression at the RNA level), proteomics (protein abundance and modification), metabolomics (small molecule metabolites), and sometimes epigenomics (methylation and chromatin structure) or microbiomics (microbial community composition from tissue or body fluid).
The appeal of multiomics is that no single biological layer tells the complete story. A gene mutation may or may not be expressed. A gene that is expressed may not produce a functional protein. A protein that is present may be inactive due to post-translational modification. Metabolites reflect the net output of all these upstream processes. Each layer adds context that the others cannot supply on their own.
At Galderma, we were applying multiomics to understand disease biology in skin conditions, particularly in Phase 2 and Phase 3 clinical trial settings where the data had to meet regulatory expectations as well as scientific ones. The datasets were large, heterogeneous, and generated by multiple technology platforms across multiple clinical sites. Managing them properly was not a technical problem alone. It was a data governance problem, a logistics problem, an interoperability problem, and a quality problem, all at the same time.
The scale of the challenge: variety, volume, and veracity
The foundational framework for understanding big data challenges describes four dimensions: volume, velocity, variety, and veracity. All four are present in multiomics, but the variety and veracity problems are where the real difficulty lives.
Volume is the most visible challenge. Whole genome sequencing data alone runs to roughly 100 gigabytes per sample in raw form. RNA sequencing adds tens of gigabytes per sample. Proteomics from mass spectrometry generates files that are large and structurally complex. Across hundreds of clinical trial subjects with samples collected at multiple time points, you are managing data measured in terabytes. The infrastructure to store, transfer, version, and process that volume requires deliberate engineering, not improvisation.
Velocity matters less in clinical trial settings than in real-time applications, but it still surfaces in specific ways. Samples arrive from clinical sites on rolling schedules. Assay batches are processed at different times. You are continuously receiving new data while simultaneously preparing earlier batches for analysis. Managing this flow without introducing errors or version conflicts requires systematic tracking from the moment a sample is registered.
Variety is where multiomics becomes genuinely different from other data problems. Each omics layer is generated by a different platform, produces a different file format, and requires a different processing pipeline. Genomics data comes off sequencers in FASTQ format and passes through alignment and variant calling pipelines before it resembles anything analytically useful. RNA-seq data follows a similar path through alignment, quantification, and normalisation. Proteomics data from mass spectrometry requires peptide identification and protein inference steps that are platform and software dependent. Metabolomics data from either mass spectrometry or NMR spectroscopy has its own preprocessing requirements entirely.
When you then need to integrate these layers to answer a biological question, you are combining data that was generated by different technologies, processed by different algorithms, normalised by different methods, and stored in different formats. The integration step does not begin when all the data is ready. It is planned from the start, because the choices made in each individual pipeline affect what can be integrated later.
Veracity is the dimension that is hardest to communicate to people who have not worked with biological data directly. Biological measurements are inherently noisy. Gene expression levels vary between technical replicates. Mass spectrometry signal intensity fluctuates with instrument condition. Metabolite measurements are sensitive to sample handling, storage temperature, and time to processing. These sources of variability do not necessarily indicate error. Some of them are biological signal. But distinguishing biological signal from technical noise requires careful experimental design and rigorous quality control.
The roles that made it work: steward, custodian, owner
At Galderma, I held three distinct roles in relation to the multiomics data, and understanding the difference between them was essential for knowing what decisions I could make and which ones required escalation.
As a data custodian, I was responsible for the safe storage, transfer, and accessibility of the data. This meant maintaining the infrastructure, managing access credentials, ensuring backup and replication, and tracking data movement between analysis environments. The custodian role is primarily technical and administrative. It is about keeping the data physically secure and available.
As a data steward, I was responsible for the integrity and quality of the data, and for enforcing the governance policies that governed its use. This meant maintaining data dictionaries, tracking provenance and lineage, flagging quality issues, enforcing naming conventions and metadata standards, and acting as the point of contact when researchers had questions about what a particular field meant or how a particular value had been derived. The steward role requires both technical understanding and deep knowledge of the data itself.
As a data owner, I had accountability for the data as an asset and authority over decisions about its use, sharing, and access. This included approving access requests, signing off on data sharing agreements, and ensuring that the data was being used in ways consistent with the ethical approvals, informed consent, and regulatory requirements that governed the clinical trials from which it came.
In many organisations these roles overlap or are informally combined. In a regulated clinical trial setting, keeping them distinct matters, because each carries specific accountabilities that need to be traceable. When a question arose about why a particular quality control decision had been made, there needed to be a clear answer about who had made it and on what basis.
Data preparation in practice: the five components
The structured project management methodology I have since studied describes five core components of data preparation: ingestion and merging, cleaning, splitting and selection, augmentation, and transformation. All five are present in multiomics work, though they take specialised forms.
Ingestion and merging
Raw data in a multiomics clinical trial arrives from multiple sources: sequencing facilities, mass spectrometry cores, imaging platforms, electronic data capture systems for clinical metadata, and laboratory information management systems for sample tracking. Each of these has its own export format, its own metadata conventions, and its own idiosyncrasies.
The ingestion step is where all of this is brought into a unified environment. In practice, this means building pipelines that can receive data from each source, validate that it meets expected quality and format standards on arrival, register it in a tracking system with appropriate metadata, and route it to the correct processing environment. This is not a one-time setup. Pipelines need maintenance as platforms are updated, as new assay types are added, and as data volumes grow.
The merging step, particularly in multiomics, is where the integration happens. Linking genomic data to transcriptomic data to proteomic data for the same subject at the same time point requires a robust sample identifier system. This sounds straightforward but is one of the most common failure points in practice. Sample identifiers assigned by clinical staff, by laboratory staff, and by bioinformatics staff are not always identical. Reconciling them requires careful tracking from the moment a sample is collected, not after the fact.
Cleaning
Raw multiomics data contains a substantial amount that is not biologically meaningful and needs to be removed or corrected before analysis. In genomics, this includes sequencing reads of low quality, reads that map ambiguously to the genome, and known sequencing artefacts. In RNA-seq, it includes ribosomal RNA contamination, reads from degraded samples, and adapter sequences. In proteomics, it includes contaminants, decoy peptide hits, and identifications below confidence thresholds. In metabolomics, it includes signals from solvents, reagents, and environmental contamination.
Beyond contaminant removal, cleaning in multiomics also means handling missing values systematically. Proteins that are below the detection limit of the mass spectrometer are not necessarily absent. They may simply not have been detected in a particular sample. Whether to impute these values, exclude the feature, or exclude the sample depends on the proportion of missing values and the likely mechanism of missingness. Each choice has downstream consequences for statistical power and for the biological conclusions that can be drawn.
The principle in my CPMAI notes is direct: garbage in, garbage out. In a multiomics clinical trial, the stakes of applying this principle are high. The data is not easily reproducible. A Phase 3 clinical trial represents years of work and significant investment. If the data preparation is flawed, the analysis is flawed, and the conclusions that reach regulatory submissions or publications may be unreliable.
Normalisation and transformation
Normalisation is the multiomics-specific expression of the transformation step described in general data preparation frameworks. The goal is to remove systematic technical variation so that the biological signal can be interpreted accurately.
In RNA-seq, this means accounting for differences in sequencing depth between samples, which would otherwise make a gene appear more expressed in a deeply sequenced sample simply because more reads were generated, not because expression is genuinely higher. In proteomics, it means accounting for differences in total protein loading between samples. In metabolomics, it means accounting for differences in sample concentration and instrument drift over the course of a run.
Batch effects deserve particular attention in clinical trial settings. When samples are processed in multiple analytical batches over the course of a trial, systematic differences between batches can easily exceed the biological differences between treatment groups. If batch effects are not identified and addressed in data preparation, they become confounders in the analysis. In a regulatory context, an unaddressed batch effect is not just a statistical problem. It is a question about the reliability of the data that supports a clinical claim.
Several methods exist for batch effect correction, including ComBat, which was originally developed for genomics and has been adapted for other omics types, and various normalisation approaches specific to each platform. The choice of method should be documented, justified, and validated as part of the data preparation record.
Selection and splitting
Not all features in a multiomics dataset contribute useful information to the analysis. The human genome contains roughly 20,000 protein-coding genes. A typical RNA-seq experiment measures expression for most of them. Many will have low expression across all samples, or expression that varies only due to technical noise rather than biological state. Including them in a machine learning model or a statistical analysis adds noise and computational burden without adding signal.
Feature selection in multiomics involves a combination of technical filters, which remove features that are too noisy to be reliable, and analytical filters, which select features most relevant to the biological question of interest. This is the attribute pruning concept from data preparation methodology, applied to molecular features rather than database columns.
The splitting of data into training, validation, and test sets for machine learning applications requires particular care in clinical trial multiomics settings. The dataset is often smaller than you would like and the features are high dimensional. Standard random splitting may result in imbalanced representation of clinical covariates such as disease severity, treatment arm, or collection site. Stratified splitting that preserves these distributions across subsets is generally more appropriate.
Augmentation
Data augmentation in multiomics is less straightforward than in image or text domains, where rotating, flipping, or paraphrasing are natural transformations. Biological data cannot be arbitrarily transformed without changing its meaning.
What augmentation looks like in this context is more often the integration of external reference datasets to provide additional context or to increase the effective sample size. Public repositories such as GEO, ArrayExpress, PRIDE, and the Human Protein Atlas contain large volumes of relevant data that can supplement internal trial data where appropriate. The challenge is that differences in platform, protocol, and patient population between internal and external data require careful harmonisation before they can be combined.
Synthetic data generation, using statistical models or generative approaches trained on real multiomics data, is an area of active research rather than routine practice. It holds promise for privacy-preserving data sharing and for augmenting rare subgroup representation, but requires careful validation to ensure that the synthetic data faithfully represents the distributional properties of the real data.
Pipelines: the infrastructure that makes it reproducible
One of the most important investments in any multiomics project is building data processing pipelines that are automated, documented, and version-controlled. The alternative is manual processing, which is slow, error-prone, difficult to reproduce, and impossible to audit.
At Galderma, the move from ad hoc processing scripts to structured pipelines was one of the most significant improvements in how we managed data. A pipeline that ingests raw sequencing data, applies quality control, performs alignment, quantifies expression, and outputs a normalised count matrix with accompanying quality metrics can be run consistently across all samples, documented completely, and re-run if parameters need to change or if questions arise about the processing choices. A collection of scripts run manually in an undefined order cannot offer any of those things.
The distinction between ETL and ELT approaches is relevant here. In clinical trial multiomics, where data volumes are large and the transformation steps are computationally intensive, an ELT approach that loads raw data into a central repository and transforms it there using scalable compute is often more practical than traditional ETL, which transforms data before loading. This is particularly true when different analyses may require different transformations of the same raw data, making it valuable to retain the unprocessed input.
Governance in a regulated research environment
In a pharmaceutical clinical trial, data governance is not optional or aspirational. It is a regulatory requirement. The data that supports a regulatory submission must be traceable, auditable, and reproducible. Every processing step must be documented. Access to the data must be controlled and logged. Changes to the data must be tracked with a clear record of what changed, when, and why.
Beyond regulatory requirements, governance in multiomics clinical trials also has an ethical dimension. The data contains genetic and health information from patients who gave their informed consent for specific uses of their data. Enforcing the boundaries of that consent, ensuring that data is not used for purposes beyond what participants agreed to, and protecting the privacy of individuals whose samples contributed to the dataset are responsibilities that sit within the governance framework.
Data minimisation is a principle worth applying explicitly in this context. Collect and retain the data required for the scientific and regulatory purposes of the trial. Do not accumulate data beyond what you can manage responsibly and what you have consent to use. In multiomics work where datasets are already large and complex, the temptation to generate every possible measurement can result in data management burdens that compromise quality across the board.
The 80 percent reality
There is a figure often cited in data science: roughly 80 percent of the work in any data-intensive project is data preparation, and only 20 percent is the analysis or modelling that the project was nominally designed to do. I would not dispute that figure in general. In multiomics clinical research, I think it understates the case.
The investment in data preparation in a multiomics clinical trial setting is not inefficiency or poor planning. It is the appropriate allocation of effort given the complexity of the data and the importance of the conclusions that will be drawn from it. A model or analysis built on poorly prepared multiomics data will be unreliable in ways that may not be immediately obvious, and the cost of discovering that unreliability after the fact, particularly in a regulatory context, is far greater than the cost of getting the preparation right at the start.
What I took from my time at Galderma, and what studying data preparation methodology has since helped me articulate more precisely, is that the discipline of data preparation is not a technical afterthought. It is where rigour lives. The researchers who do it well understand not just the tools and pipelines involved, but the biology of the data, the limitations of the platforms that generated it, the regulatory context it operates in, and the downstream consequences of every preparation decision they make.
A closing thought
There is a temptation, as AI tools become more capable, to imagine that data preparation will eventually be automated away. Large language models can assist with cleaning tasks. Vision models can flag quality issues in gel images or histology slides. Generative models may eventually help with imputation and augmentation in controlled settings. These are real capabilities worth taking seriously.
But the judgment required to make good data preparation decisions in multiomics research is not purely technical. It requires understanding what the data represents biologically, what the experimental design intended, what the regulatory requirements demand, and what a downstream analysis will need. That combination of scientific, operational, and contextual knowledge is not something that gets automated. It gets developed through experience, and it is genuinely hard to acquire without doing the work.
That is why I think the data steward and data custodian and data owner roles deserve more recognition than they typically receive in research organisations. They are not support functions. They are the roles that determine whether the scientific outputs of a study can be trusted.
This post draws on my experience managing multiomics data at Galderma and on data preparation principles from the PMI CPMAI certification programme. It is written for researchers and data practitioners working at the intersection of clinical research and data science.