Skip to content

Search problems, genes, families, sources

Find a problem Families All problems Notation & genes About

Loading…

A living collection from the Altringer Eagle Lab at Dartmouth’s Thayer School of Engineering. Problems come from expert input and from what experts publish: research agendas, standards roadmaps, agency reports. The Problem Genome system drafts each write-up, and its sources are checked before it enters the collection. It can make mistakes. It exists to put problem definition at the center of innovation. Problem definitions here can be cited, and so can improvements to them. Teams can reach domain knowledge early and contribute what they learn.

How write-ups are checked Take the data Bring us a problem Teach with the collection

Genome › Failure mode

Failure mode · gene 11 of 14

built on the wrong data

72 problems carry this gene · failure:unrepresentative-data

The right quantity was measured on the wrong sample. A model trained on calm seas fails during storms. A sensor validated with clean spiked samples fails on naturally contaminated ones. A medical device calibrated mostly on lighter skin misreads darker-skinned patients. The tool inherits the blind spots of its data, and those blind spots tend to fall where the tool is needed most.

Related genes

measured a stand-inproxy-metricoptimized or certified against a stand-in that doesn't track the outcome it's supposed to protect · added August 2026 — under review for its first hundred uses ignored real-world conditionsignored-contextthe solution assumed conditions, such as power, weather, skills, or institutions, that the real setting doesn't provide can't measure what mattersdatathe data needed to understand or improve the situation is missing, inaccessible, or too poor to use the burden falls unevenlyequitythe problem or its solutions hit marginalized communities hardest, so fairness is a core design constraint

How to recognize it

Compare the data used to build or validate the tool against the population or conditions it has to serve. If the measured quantity is right but the sample doesn't represent real use, this is the gene. If the quantity itself is the wrong stand-in for the outcome, that's 'measured a stand-in' instead.

Our system re-reads samples blind to test genes like this one. Genes it disagrees on are drawn dashed.

72 problems in 17 fields. Filter them, then share the link.

field: the built worldpollinator meadow or deer blind?tested in the wrong worldhigh field: the built worldthe pole that whips oncetested in the wrong worldhigh field: society & learningthe armband says you're finetested in the wrong worldmedium field: health & medicinetwelve times too many patientstested in the wrong worldhigh field: food & the living worldalive, uncounted, and certified safemeasured the wrong thinghigh field: food & the living worldaprons as a proxy for salmonellameasured the wrong thingmedium field: food & the living worldthe house that didn't burntested in the wrong worldmedium field: society & learningon top of paskettitested in the wrong worldmedium field: food & the living worldtwo numbers for the same fieldtested in the wrong worldhigh field: digital & spaceten thousand satellites, no traffic controltested in the wrong worldmedium field: digital & spacetwo hundred years measuring g, still uncertaintested in the wrong worldmedium field: food & the living worldfishing boats that go darktested in the wrong worldmedium field: making & materialssimulated decades, real surprisestested in the wrong worldhigh field: health & medicinemalnutrition measured by the wrong bodymeasured the wrong thinghigh field: health & medicinethe experiment worked oncetested in the wrong worldhigh field: health & medicineperfect on competition day, blind in the clinictested in the wrong worldmedium field: water & energysame gas, four diagnosesthe rules got in the waymedium field: water & energyfifteen billion dollars of sludgetested in the wrong worldmedium field: society & learningconfident in the sim, lost with the patienttested in the wrong worldhigh field: digital & spacethe satellite can't see the settlements that need ittested in the wrong worldmedium field: digital & spacethe scoreboard is corruptedtested in the wrong worldhigh field: digital & spacethe code rots before the paper is citedtested in the wrong worldhigh field: the built worldthe most critical measurement, taken too latetested in the wrong worldhigh field: the built worldtwo engineers, two verdicts, one bridgetested in the wrong worldhigh field: making & materialsthe catalyst dies on its own scheduletested in the wrong worldhigh field: food & the living worldsame soil, three hundred percent aparttested in the wrong worldhigh field: food & the living worlddesertification measured by the wrong maptested in the wrong worldhigh field: making & materialsevery truckload of fly ash is a surprisetested in the wrong worldmedium field: the built worldasphalt graded for yesterday's weathertested in the wrong worldmedium field: society & learningnobody agrees how many people are in the camptested in the wrong worldmedium field: health & medicinethe gold standard is guessingmeasured the wrong thingmedium field: health & medicinehalf the antibodies miss their targettested in the wrong worldmedium field: health & medicinehalf the cancer studies don't repeatbuilt without the people who'd live with itmedium field: health & medicinewrong cells, published anywaybuilt without the people who'd live with itmedium field: water & energytested in tokyo, sold in lagosmeasured the wrong thingmedium field: society & learningthe algorithm marks down how you talktested in the wrong worldhigh field: digital & spaceis it biology or just a different labtested in the wrong worldmedium field: the built worldmixed rubble, no way to sort ittested in the wrong worldmedium field: making & materialslab brine cooperates, field brine doesn'ttested in the wrong worldhigh field: health & medicineone antivenom for a thousand snakestested in the wrong worldhigh field: health & medicinesepsis outruns the doctortested in the wrong worldhigh field: health & medicinethe mouse said it was safetested in the wrong worldhigh field: food & the living worldthe only resistance gene is failingtested in the wrong worldhigh field: food & the living worldthe payout comes when it shouldn'ttested in the wrong worldhigh field: food & the living worldsick land with no diagnosistested in the wrong worldhigh field: food & the living worldseventy-seven biosensors, one tested on real foodtested in the wrong worldhigh field: food & the living worldeighty percent of foods with no footprint datatested in the wrong worldhigh field: digital & spacemissing data that isn't randomtested in the wrong worldhigh field: the built worldcatches every fall except a real onetested in the wrong worldhigh field: food & the living worldthe sensors that grade themselvestested in the wrong worldhigh field: health & medicinethe oximeter lies about darker skinbuilt without the people who'd live with ithigh field: health & medicineevery lab, a different answerthe rules got in the wayhigh field: health & medicineside effects that arrive in twenty yearsbuilt without the people who'd live with ithigh field: health & medicineapproved without proofbuilt without the people who'd live with ithigh field: digital & spacesome solar storms kill, and we can't tell whichtested in the wrong worldhigh field: food & the living worldnobody knows how much water the snow holdstested in the wrong worldhigh field: food & the living worldthe ice model and the climate model won't mergetested in the wrong worldhigh field: digital & spaceten million alerts before dawntested in the wrong worldhigh field: food & the living worldfive billion tonnes of carbon, unmeasuredtested in the wrong worldhigh field: food & the living worldtwice the atmosphere's carbon, thawing unmodeledtested in the wrong worldhigh field: food & the living worldbillions in monitoring, forecasts still failtested in the wrong worldhigh field: water & energytwo million miles of pipe, no failure maptested in the wrong worldmedium field: health & medicineapproved, available, unusedthe rules got in the waymedium field: digital & spaceexplain the defense, expose the weaknesstested in the wrong worldhigh field: water & energybus batteries die on the route, not in the labtested in the wrong worldmedium field: health & medicinethe app was prescribed, the outcome unknownthe rules got in the waymedium field: food & the living worldcarbon sunk in the ocean, uncountedtested in the wrong worldmedium field: water & energythe savings were never realmeasured the wrong thinghigh field: food & the living worldonce it sinks, the algorithm loses sighttested in the wrong worldhigh field: food & the living worldsatellites see the spill but not its depthtested in the wrong worldhigh field: food & the living worldaccurate until the storm arrivestested in the wrong worldhigh field: the built worldknock out power, lose water, lose everythingtested in the wrong worldhigh