A problem family — different problems, same issues

the model was never trained on this

29 problems across 15 fields

These 29 models predict ocean storms, building energy use, tornado damage, permafrost thaw, and bioreactor behavior. They all fail at the same point: they were trained on data that doesn't represent the conditions where they are used. The failure arrives by five routes. Events are too rare to observe. Variables are too hidden to measure. Databases are too scattered to combine. Data is too abundant to process. Or the training data leaves out the populations the model claims to serve, which makes the problem one of who was counted.

The model was trained on data that doesn't represent the conditions where it matters most, and nobody checked whose experience was missing.

Ways this family shows up

Rarity. Models trained on common conditions fail on rare or extreme events: compound hazards, tornadoes, wildfires
Invisibility. Critical variables were never measured in the training data: carbon decomposition, how organisms adapt, carbon moving through living systems
Fragmentation. The needed data exists but is held separately across organizations, disciplines, or ecosystems
Volume. The data exists but there is more of it than anyone can process, as with particle collider collisions
Who was counted. Training data leaves out populations or conditions the model claims to serve: dialects, disadvantaged groups, other climates

The 29 problems in this family

will they adapt or collapse, nobody knowsit fell between the crackshigh

also in the answer is in a field you’ve never heard of, the theory hasn’t been invented yet

two engineers, two verdicts, one bridgetested in the wrong worldhigh

ten times the data, same computing budgetbuilt without the people who'd live with ithigh

the satellite can't see the settlements that need ittested in the wrong worldmedium

big ai, small datait fell between the cracksmedium

also in the answer is in a field you’ve never heard of

missing data that isn't randomtested in the wrong worldhigh

the algorithm marks down how you talktested in the wrong worldhigh

the savings were never realmeasured the wrong thinghigh

the building has no twintested in the wrong worldhigh

also in designed for a world that doesn’t exist here

disasters that multiply, models that don'tit fell between the crackshigh

also in the answer is in a field you’ve never heard of

could be centuries, could be decadestested in the wrong worldhigh

twice the atmosphere's carbon, thawing unmodeledtested in the wrong worldhigh

food needs water, neither system talksit fell between the cracksmedium

also in the data exists but can’t talk to itself

a hundred formats, no translationbuilt without the people who'd live with ithigh

also in the data exists but can’t talk to itself

perfect on competition day, blind in the clinictested in the wrong worldmedium

side effects only some people reportbuilt without the people who'd live with itmedium

faxes to approve a scanit fell between the cracksmedium

knock out power, lose water, lose everythingtested in the wrong worldhigh

materials rated for a climate that's gonethe rules got in the wayhigh

also in the regulator demands evidence that cannot exist

asphalt graded for yesterday's weathertested in the wrong worldmedium

building codes that ignore the tornado's spinthe rules got in the wayhigh

also in the regulator demands evidence that cannot exist

the microbe thrives in the flask, dies in the tanktested in the wrong worldhigh

also in making one is easy. making a million is the problem, the chemistry itself changes at scale

accurate until the storm arrivestested in the wrong worldhigh

fishing boats that go darktested in the wrong worldmedium

satellites see the spill but not its depthtested in the wrong worldhigh

once it sinks, the algorithm loses sighttested in the wrong worldhigh

big enough to kill, too small to tracktested in the wrong worldhigh

also in it worked in the lab

two million miles of pipe, no failure maptested in the wrong worldmedium

fire models stop where buildings startit fell between the crackshigh

also in the answer is in a field you’ve never heard of

Confidence: high = from a primary expert source (Tier 1) · medium = from an analyst, conference, or community source (Tier 2 or 3) · needs sourcing = flagged until we find a better source · the dot's size is its tier; a dashed dot needs sourcing

Loading…