digital · health · family: the model was never trained on this
missing data that isn't random
Standard ML imputation methods fail when missing data has systematic structure across populations
Problem statement
When datasets from multiple sources are combined — electronic health records linked with genomic and imaging data, multi-omics data from different assays, or multiple studies that each measured a different set of variables — the missing data patterns have systematic structure. Entire variable blocks are absent for certain subpopulations or data sources. Established methods for missing data, such as multiple imputation, do not usually take the structure of the missing data into account, and the paper says commonly used methods such as multiple imputation or tree-based imputation may be inappropriate or add high uncertainty when missingness is structured; it notes that structured missingness can be "missing at random" (MAR) or not, and that Rubin's classical categories do not fully capture it. As a result, ML models trained on combined heterogeneous data can be biased — performing well on the parts of the data that are complete and poorly on the subpopulations whose data is least complete — and in some cases data from historically under-served groups is missing entirely or nearly so.
Why this matters
The paper says linking data from different sources is becoming increasingly common. It draws its examples of structured missingness from health (linked clinical, genomic and imaging data), geospatial climate analysis (sensors that resolve different variables at different resolutions), surveys, and socio-economic policy, where training data is often an amalgamation of separate resources, and it expects the problem to become more acute with ever-larger models and federated learning. The paper's worked example is the Flatiron Health–Foundation Medicine Clinico-Genomic Database, which links electronic health records for roughly 100,000 patients from about 280 US cancer clinics to genomic profiling of about 600 genes: one patient's assay may cover 400 genes and another's 150, and markers such as PSA or CA125 are only measured for some cancers, so any pan-cancer model must cope with whole blocks of missing data. If structured missingness is not properly handled, such datasets can produce models that perform well on some subsets of the data and poorly on others, and may perpetuate or worsen existing inequalities.
What’s been tried and why it hasn’t worked
Data that is missing not at random is typically handled cautiously and case by case — manual exploration and correction of missingness patterns, or sensitivity analyses that vary the imputation or exclusion choices. Deep generative imputers (the paper cites a GAN-based method) have theoretical guarantees for data missing completely at random and empirical results for sparsely MAR and MNAR data, but are not yet able to deal with structured missingness in all its forms. The existing structured-missingness literature re-purposes standard tools for specific settings — k-nearest-neighbour imputation of block-missing omics data, multitask learning for missing genotype blocks, multilevel and blocked multiple imputation — and the methods are typically bespoke, with unclear generalisability. Network and higher-order geometric approaches to the structure of missingness have not yet been extensively explored. Benchmarking is itself stuck: the standard way to test an imputation method is to remove values from a complete dataset one variable at a time, which does not reproduce multivariate missingness patterns, so benchmarking structured-missingness tools requires a prior understanding of the structure those tools are meant to reveal.
What would unlock progress
The paper calls for new ways of benchmarking that reproduce multivariate missingness patterns and, as a start on the ethical side, standardised processes for documenting missingness in datasets. It calls for new causal inference methods that can cope with structured missingness, and says that involving experts with domain knowledge and social-scientific training is vital for mitigating bias. As possible routes it names sparse and collaborative representation (borrowed from recommender systems), synthetic-data generation treated as an extreme case of imputation, and designing data collection to anticipate missingness in the first place.
Entry points for student teams
A student team could construct synthetic datasets with controlled block-missingness patterns, benchmark standard imputation methods (MICE, tree-based imputation, a GAN-based imputer such as GAIN) against them, and measure downstream prediction fairness across subgroups defined by missingness pattern. This requires only standard ML tooling and publicly available datasets. Alternatively, teams could analyze the UK Biobank's or MIMIC-III's missingness structure (neither is an open download; each requires registering and being approved for access), document the patterns, and test whether imputation method choice changes clinical prediction model performance for different patient subgroups. Relevant disciplines: statistics, machine learning, data science, biomedical informatics.
Genes
Structural cousins: stuck the same way, in other fields
Sources
Mitra, R., McGough, S.F. et al., "Learning from data with structured missingness," Nature Machine Intelligence, 5, 13–23, 2023, accessed 2026-02-20 go to source ↗
Working notes
Notes made while writing and checking this brief, shown as written. They use working shorthand.
Related briefs: `DIGITAL-scientific-ai-data-scarcity` (broader data scarcity, not specifically structured missingness); `mathematics-ai-uncertainty-quantification-science` (UQ methods, tangentially related); `health-pulse-oximeter-skin-tone-bias` (hardware bias, not data bias). This brief bridges the equity-technical divide: structured missingness is a technical data problem whose consequences are inequitable prediction quality across populations. Source-bias note: the Nature Machine Intelligence framing emphasizes theoretical/methodological gaps; the actual binding constraints are data documentation practices and institutional willingness to share missingness metadata.
Reconciliation 2026-09-30: The whole write-up was read against its one source. The publisher page (https://www.nature.com/articles/s42256-022-00596-z, fetched 2026-09-30) shows only the abstract behind a paywall, so the full text was read from the authors' accepted manuscript at the University of Glasgow repository (https://eprints.gla.ac.uk/321319/5/321319.pdf, fetched 2026-09-30, full text, 21 pages; the same manuscript is also at the Warwick, UCL and arXiv repositories), and the citation was checked against the Crossref record for doi 10.1038/s42256-022-00596-z. The repository copy warns that it may differ from the published version. Changes: (1) The text said structured missingness "systematically violates" the missing-at-random assumption. The paper says the opposite: structured missingness is associated with both MAR and MNAR but "is not exclusively associated with either", and gives an age-based example that is MAR; its complaint is that standard methods "do not usually take into account the structure of the missing data". The sentence now says that. "Mean imputation" and "deep learning imputers" as MAR-assuming methods are not in the paper; the sentence now keeps to what the paper says of multiple imputation and tree-based imputation (they may be inappropriate or introduce high uncertainty). (2) The text said the UK Biobank has imaging data for ~25% of participants and genomic data for ~80%, and that All of Us and Genomics England face the problem. Neither the UK Biobank, All of Us nor those percentages appear in the paper (Genomics England appears only as an author affiliation). These were removed and replaced by the paper's own worked example, the Flatiron Health–Foundation Medicine Clinico-Genomic Database (about 100,000 patients, about 280 US cancer clinics, about 600 genes, 400- versus 150-gene assays, PSA and CA125). (3) The examples of data sources were adjusted to the paper's: "different hospitals" and "survey data from different countries" are not in the paper, which gives linked clinical, genomic and imaging data, omics experiments, and multiple studies with different measurement sets; "satellite + ground sensors" became sensors that resolve different variables at different resolutions, and "administrative records + surveys" became surveys and socio-economic policy data amalgamated from separate resources; "accelerating" became the paper's "increasingly common", and federated learning is now given as the paper gives it, as a trend that will make the problem more acute. (4) The claim that models "appear accurate on average but systematically fail for underrepresented groups" and that the worst-served populations are "typically" the already underserved was softened to the paper's wording: models that perform well on some subsets and poorly on others, with data from historically under-served groups "in some cases" missing entirely or nearly so, and the risk that structured missingness may perpetuate or exacerbate inequalities. (The paper does say, of class imbalance in biomedical datasets, that it can lead to algorithms that systematically fail for certain patient groups; "appear accurate on average" and the unqualified "will" are what the paper does not say.) (5) The whole "What's Been Tried" section rested on claims not in the paper: selection, pattern-mixture and shared-parameter models and their parametric assumptions and variable limits; VAEs; MIWAE and its inability to represent block structure; graph methods "only tested on synthetic data"; and a sentence that no method has been validated at biobank scale. The section now reports what the paper says: MNAR data is handled cautiously and case by case; a GAN-based imputer has guarantees for MCAR and empirical results for sparse MAR and MNAR but cannot yet handle structured missingness in all its forms; existing structured-missingness methods (k-nearest-neighbour block imputation, multitask genotype imputation, multilevel and blocked multiple imputation) are bespoke with unclear generalisability; geometric approaches have not yet been extensively explored; and benchmarking is entangled with understanding the missingness itself. (6) In "What Would Unlock Progress", the "benchmark suite of real-world datasets", the directed-acyclic-graph parenthetical and "separate embeddings per missingness pattern" are not in the paper; the section now names what the paper names: new benchmarking methods, standardised documentation of missingness, causal methods that cope with structured missingness, domain and social-science expertise for mitigating bias, and, as possible routes, sparse and collaborative representation, synthetic-data generation, and design of experiments. (7) Entry points: the method list (missForest, MIWAE) was changed to the methods the paper names (tree-based imputation, GAIN), and UK Biobank and MIMIC-III, which the paper does not mention, are now labelled as not open downloads (an access label checked on 2026-09-30 against the PhysioNet MIMIC-III page and an Internet Archive copy of the UK Biobank apply-for-access page, not a claim from the paper); the first entry point still needs no special facility. Confirmed in the source and left as written: the Source line (Mitra, McGough et al., Nature Machine Intelligence, volume 5, pages 13–23, 2023, per Crossref); block-structured missingness across subpopulations and data sources; the push toward combining heterogeneous data, including federated learning; that standard methods do not sufficiently address structured missingness; the display title and the long title. Not checked: the related-brief notes and the source-bias opinion in Source Notes, which are the write-up's own commentary rather than claims from the paper. Genome Tags untouched. A second reader went over these repairs against the same manuscript (Glasgow and arXiv copies) on the same day; all seven stand, and items 1, 3, 4, 6 and 7 are described here in their final wording after the second reader tightened sentences that still said more than the paper does.