health · family: the model was never trained on this
side effects only some people report
US pharmacovigilance systems are structurally blind to differential safety signals across populations
Problem statement
The US post-market surveillance system for drugs and medical devices — anchored by FDA's MedWatch adverse event reporting system — was not designed to detect whether products harm some populations more than others. The NASEM report says significant advances are needed in the ability to collect, characterize, and analyze data on social determinants of health, geographic variables, and social vulnerability. The postmarket mechanisms — Sentinel, FAERS, and payer claims databases among them — are, in the report's words, "reasonably well designed to assess problems with the safety of drugs and devices" but "do not currently extend to how drugs or devices might reinforce or exacerbate inequities and injustices in the real world," and an equity concern revealed only in analyses at the population level "cannot be addressed in systems designed to report on individual occurrences." Differential problems — like pulse oximeters being less accurate on darker skin, or clinical algorithms that would deprioritize Black patients — are the examples the two reports give; scientists raised concerns about the pulse oximeter for years, to little effect, before the problem was recognized during the COVID-19 pandemic.
Why this matters
Health inequities in drug and device safety are not hypothetical. Scientists raised concerns about pulse oximeter accuracy in people with darker skin for years, to little effect; studies then confirmed that the devices are approximately three times more likely to miss low blood oxygen levels in Black than in White patients. Several simulation studies reported that using the Sequential Organ Failure Assessment (SOFA) score to predict who should be prioritized for intensive care resources during COVID-19 shortages would deprioritize Black patients by up to 15 percent even though race is not part of the calculation: renal function is used in the score, and patients with chronic kidney disease (who are disproportionately Black) fare better than patients with acute kidney failure caused by COVID-19 despite equivalent scores. The NASEM report found that passive surveillance is undermined by voluntary reporting of data that is "frequently inaccurate, untimely, unverified, and/or biased," and that safety reporting often comes from doctors and hospitals, which may not be best positioned to identify equity considerations and metrics. An assessment of 220 publicly available university licenses found that the language on access in the "Nine Points" guidance — recommended contract clauses for university technology licensing, on issues that include making medical technologies accessible to developing countries — "has not been widely adopted."
What’s been tried and why it hasn’t worked
Patients and patient advocacy groups can voluntarily report adverse events through MedWatch, the FDA's public-facing safety reporting program, which the NASEM report says was developed to assess safety concerns, not equity concerns in real-world use; the report adds that safety reporting often comes from doctors and hospitals, which may not be best positioned to identify equity considerations, and that passive surveillance is undermined by voluntary reporting of frequently inaccurate, untimely, unverified, or biased data. Active surveillance is, in the report's words, "expensive, slow, and often narrow in scope"; the report lists Sentinel, FAERS, and payer claims databases among the postmarket data mechanisms, and notes that an equity concern revealed only at the population level cannot be addressed by systems built to report individual occurrences. Not all post-market studies requested by FDA are completed, and the report gives increased use and enforcement of Phase IV post-approval trials, to confirm safety and efficacy across the broad population of users, as one example of an incentive that could push innovation toward equity. In the report's judgment these mechanisms, "however flawed, are reasonably well designed to assess problems with the safety of drugs and devices," but do not currently extend to how a product might reinforce or exacerbate inequities in the real world.
What would unlock progress
Pharmacovigilance data infrastructure that captures social determinants of health, geographic variables, and social vulnerability alongside clinical adverse events — either through expanded reporting fields or through linkage of multiple real-world data sources; the NASEM report says combining multiple sources of real-world data can offer additional insights about equity, and names EHRs, administrative claims, and clinical registries as sources that could be integrated for device surveillance. Algorithmic approaches that can detect differential safety signals in existing claims and EHR databases despite the absence of explicit equity variables, using proxy measures and geographic correlates. Mandatory disaggregated reporting requirements for post-market surveillance that go beyond basic demographic categories to capture the variables needed for equity-relevant analysis.
Entry points for student teams
A student team could analyze existing FAERS (FDA Adverse Event Reporting System) data for a specific drug class to determine what equity-relevant signals could be extracted from current data fields and what additional variables would be needed to detect differential harm. Alternatively, teams could prototype an algorithmic approach to infer population-level differential safety signals from proxy variables (zip code, insurance type, facility characteristics) in publicly available claims data or hospital discharge databases. Relevant disciplines: health informatics, epidemiology, data science, bioethics, public health.
Genes
Structural cousins: stuck the same way, in other fields
Sources
National Academies of Sciences, Engineering, and Medicine and National Academy of Medicine, "Toward Equitable Innovation in Health and Medicine: A Framework," 2023, National Academies of Sciences, Engineering, and Medicine, "Ending Unequal Treatment: Strategies to Achieve Equitable Health Care and Optimal Health for All," 2024, (Summary) and (Chapter 5, clinical algorithms); accessed 2026-02-20 and 2026-09-30 go to source 1 ↗ go to source 2 ↗ go to source 3 ↗
Working notes
Notes made while writing and checking this brief, shown as written. They use working shorthand.
Distinct from `health-device-real-world-evidence-gap` (which covers general lack of real-world evidence for device performance — a data quantity problem) and `health-pulse-oximeter-skin-tone-bias` (which covers one specific device's calibration bias). This brief addresses the structural inability of the entire pharmacovigilance system to detect differential harm across populations — a system-level design flaw in the surveillance infrastructure, not a device-specific or data-quantity problem. Related to `digital-structured-missingness-ml-bias` (which covers how missing data in ML training creates biased models — a parallel problem in a different domain). Source-bias note: NASEM frames this as requiring "equitable innovation" across the product lifecycle; the binding constraints are genuinely data-structural (surveillance infrastructure doesn't collect the right variables) and design-related (passive reporting yields data the report calls biased), not primarily requiring new institutional coordination.
Reconciliation 2026-09-30: Fact repair after reading the whole write-up against both sources. Both were fetched on 2026-09-30 and read in full from the National Academies Press online reader: "Toward Equitable Innovation in Health and Medicine: A Framework" (2023; the catalog link above redirects to the publication page at nationalacademies.org, and the chapters were read at nap.nationalacademies.org/read/27184/chapter/1 through /12) and "Ending Unequal Treatment" (2024; the Summary at the link above, and every chapter at nap.nationalacademies.org/read/27820/chapter/1 through /15). The pharmacovigilance material is in Chapter 3 of the 2023 report (the postmarket section, around pages 75–77) and the pulse oximeter account is its Box 2-1; the Summary of the 2024 report, the only page the Source line pointed to, contains none of the claims, and the SOFA passage is in its Chapter 5 (pages 200–201), so that chapter's address was added to the Source line, and the 2023 report's co-author, the National Academy of Medicine, was added as the record gives it. (1) The text said MedWatch reports do not capture social determinants, geographic variables, social vulnerability indices, "or granular demographic data beyond basic race/ethnicity categories," and that Sentinel uses insurance claims that capture billing codes but not social variables. The report says only that significant advances are needed in the ability to collect and analyze data on social determinants of health, geographic variables and social vulnerability, lists Sentinel, FAERS and payer claims databases among the postmarket mechanisms, and says those mechanisms are "reasonably well designed to assess problems with the safety of drugs and devices" but "do not currently extend to" how products might reinforce inequities; the paragraph now says that, attributes the data-collection point to the report rather than stating flatly what the systems fail to capture, and the demographic-categories clause was removed. (2) The text said differential problems "can persist for years or decades before detection." Box 2-1 says scientists raised concerns about the pulse oximeter "for years, to little effect," and Chapter 4 speaks of "the recognition during the COVID-19 pandemic" of the inaccuracy; the sentence now says that about the pulse oximeter only, and no longer says how long the clinical-algorithm problem went unaddressed, which neither report states. (3) The text said pulse oximeters were biased "for decades before systematic research documented the problem." The report cites a 2005 study among the concerns raised for years, and says studies confirmed the devices are approximately three times more likely to miss low blood oxygen in Black than in White patients; the sentence now says that. (4) The text said the SOFA score "deprioritized Black patients by approximately 15% because serum creatinine — a biomarker correlated with race — was used as a severity indicator." The 2024 report says simulation studies reported the score "would deprioritize Black patients by up to 15 percent despite race not being used in the calculation," because renal function is in the score and patients with chronic kidney disease, who are disproportionately Black, fare better than patients with acute kidney failure from COVID-19 despite equivalent scores. Serum creatinine and a biomarker correlated with race are not in the source; the sentence now follows the report, including that these are simulations of using the score rather than a record of its use. (5) The claim that passive reporting is "skewed toward acute events in populations most likely to report (English-speaking, health-literate, connected to care)" is in neither source and was removed; the sentence now gives the report's statement that safety reporting often comes from doctors and hospitals, which may not be best positioned to identify equity considerations. (6) The licensing sentence said a review of 220 "exclusive licensing agreements" found access language "not widely adopted," limiting distribution of publicly funded innovations. The report says an assessment of 220 publicly available university licenses found the "Nine Points" language on access "has not been widely adopted"; the quoted words "not widely adopted" do not appear in that form, "exclusive" is not in the source, and the consequence clause is not either. The sentence now quotes the report, and describes the "Nine Points" guidance as the report does: recommended clauses on issues that range from limiting licensing exclusivity to making medical technologies accessible to developing countries. (7) "MedWatch voluntary reporting is the backbone of US pharmacovigilance" and "producing data biased toward populations with healthcare access and health literacy" are not in the source; the sentence now gives what the report says: that patients and patient advocacy groups can report voluntarily through MedWatch, that safety reporting often comes from doctors and hospitals, and that passive surveillance is undermined by voluntary reporting of biased data. (8) The text said Sentinel enables "faster signal detection than voluntary reporting." The report says active surveillance is "expensive, slow, and often narrow in scope"; the sentence now quotes that. (9) The text said FDA-mandated post-market studies "typically use the same homogeneous populations as pre-market trials." The report says not all requested postmarket studies are completed, and lists increased use and enforcement of Phase IV trials across the broad population of users as an example of a push incentive, not as a recommendation; the homogeneous-populations remark in the report is about the recruitment of trial populations before approval. The sentence now says what the report says. (10) The sentence that the infrastructure "was designed in an era when 'safety' meant" average harm is not in the source; it now gives the report's judgment that the mechanisms are reasonably well designed for safety but do not currently extend to how products might reinforce inequities. (11) The list of social determinants to capture (housing, income, occupation, environmental exposures) is not in the source; it now uses the report's list and its statement that combining multiple sources of real-world data can offer additional insights about equity. (12) In the Source Notes, "passive reporting inherently underrepresents vulnerable populations" became "passive reporting yields data the report calls biased." Confirmed in the sources and left as written: that MedWatch was not designed to assess equity concerns; the quotation "frequently inaccurate, untimely, unverified, and/or biased"; the count of 220 licenses; pulse oximeters and clinical algorithms as the report's examples; and FAERS as a named postmarket data system. Not checked, because they are the write-up's own proposals rather than sourced claims: the algorithmic proxy-variable approach and mandatory disaggregated reporting in the unlock section. Still in the write-up without a source behind it: the opening description of the surveillance system as "anchored by" MedWatch, which is the write-up's own framing (the report names MedWatch as one of several regulator-supported systems). A second reader went over these repairs against both reports on the same day and tightened the wording of several. No title changed. Genome tags untouched.