Confident Predictions, No Error Bars

Neural networks and machine learning models used for scientific prediction produce point estimates without calibrated uncertainty bounds. No mathematical framework provides rigorous, computationally tractable uncertainty quantification (UQ) for deep learning in scientific applications. Bayesian neural networks are theoretically principled but computationally prohibitive for large models. Ensemble methods provide empirical uncertainty estimates but lack theoretical guarantees. Conformal prediction offers distribution-free coverage guarantees but assumes exchangeability — violated by virtually all scientific datasets (time series, spatial data, experimental sequences). Scientists cannot trust AI predictions without knowing when the model is confident and when it's guessing.

AI is being deployed across every scientific domain — molecular dynamics, weather prediction, materials discovery, genomics, particle physics — but without reliable uncertainty bounds, scientists cannot distinguish confident predictions from unreliable extrapolations. This leads to wasted experimental resources (pursuing AI-predicted candidates that are actually uncertain), missed discoveries (dismissing uncertain but correct predictions), and safety risks (deploying AI models in engineering without understanding failure modes). The NSF AI+MPS white paper identifies rigorous UQ as a foundational mathematical challenge for AI-for-science. A rigorous UQ framework would accelerate adoption across the $500+ billion global R&D enterprise.

Monte Carlo dropout provides cheap uncertainty estimates but is not theoretically grounded — dropout uncertainty doesn't correspond to any coherent probability model. Deep ensembles (training multiple independent models) provide better-calibrated uncertainty but at 5–10× computational cost, and there's no theory for how many ensemble members are enough. Bayesian neural networks with variational inference approximate the posterior but the approximation quality is unknown for any given architecture/dataset. Physics-informed neural networks (PINNs) incorporate physical constraints but don't propagate those constraints into uncertainty estimates. The fundamental mathematical challenge is that deep learning generalization theory is incomplete — we don't understand why neural networks generalize at all, let alone how to bound their prediction uncertainty.

New mathematical theory connecting neural network architecture, training data properties, and prediction uncertainty — potentially drawing from statistical learning theory, information geometry, or optimal transport. Computationally tractable methods that provide guaranteed coverage for non-i.i.d. scientific data (extending conformal prediction beyond exchangeability). Hybrid approaches that combine physics-based models (which have well-understood uncertainty propagation) with neural network components (which capture complex patterns) in a framework where total uncertainty is rigorously quantified.

A student team could benchmark existing UQ methods (MC dropout, deep ensembles, conformal prediction) on a specific scientific prediction task (e.g., molecular property prediction from the QM9 dataset), measuring calibration, sharpness, and computational cost — generating the kind of systematic comparison the field needs. Alternatively, a team could implement and evaluate a conformal prediction wrapper for a pre-trained scientific ML model, testing how well coverage guarantees hold when exchangeability assumptions are violated. Relevant skills: statistics, machine learning, computational science, probability theory.