digital · space · family: the model was never trained on this
ten times the data, same computing budget
The high-luminosity LHC will produce ten times the collision data, and current computing cannot scale to it
Problem statement
The High-Luminosity Large Hadron Collider (HL-LHC), scheduled to begin operations at CERN in 2029, will increase the LHC's collision rate by a factor of 5-7, producing approximately 200 simultaneous proton-proton collisions per beam crossing (vs. ~40 currently) and generating raw data at ~5 TB/second. After triggering and reconstruction, the experiments (ATLAS, CMS) will enlarge their data samples by about an order of magnitude over the HL-LHC's lifetime compared with the LHC's baseline programme (CERN HL-LHC project page). The Worldwide LHC Computing Grid (WLCG), which currently processes LHC data using ~1 million CPU cores across 170 sites in 42 countries, cannot scale to HL-LHC requirements through incremental hardware upgrades alone. Moore's Law improvements will provide only a factor of ~3-5× by 2029, leaving a factor of ~10-20× computing gap that must be closed through algorithmic innovation, heterogeneous computing (GPUs, FPGAs), and fundamentally new approaches to event reconstruction and analysis.
Why this matters
The HL-LHC is a $5+ billion international investment (including detector upgrades) designed to collect 10× more data than the LHC has accumulated to date, enabling measurements of the Higgs boson's self-coupling, searches for new particles beyond the Standard Model, and precision tests of fundamental symmetries. The P5 report included HL-LHC completion and exploitation in its highest-priority recommendation, alongside other projects already under way. If the computing challenge is not solved, the experiments will be forced to either discard data (reducing physics reach) or delay analysis (missing discovery opportunities). The computing gap is not a future problem — detector upgrade designs are being finalized now, and their data volumes are fixed by physics and engineering choices. The computing model must be ready when the detectors turn on.
What’s been tried and why it hasn’t worked
The current WLCG model distributes computing across a tiered hierarchy of data centers, with CERN as Tier-0 and national laboratories as Tier-1. This model was designed for LHC Run 1 (2010-2013) and has been incrementally scaled. Event reconstruction algorithms (track finding, calorimeter clustering, particle identification) use iterative combinatorial methods whose computational cost scales superlinearly with the number of simultaneous collisions — the 5× increase in pile-up produces a 10-20× increase in reconstruction time per event. Machine learning has been applied to specific subtasks (jet tagging, b-tagging, anomaly detection) with significant speedups, but full ML-based reconstruction pipelines remain research prototypes. GPU porting of reconstruction code has shown 10-100× speedups for specific algorithms, but the full reconstruction chain includes hundreds of algorithms written over 20+ years in C++, and converting this codebase to heterogeneous computing architectures is a massive software engineering effort. Commercial cloud computing could supplement the Grid, but network bandwidth costs for transferring petabytes of data and the unpredictable pricing of cloud resources create economic uncertainty.
What would unlock progress
End-to-end event reconstruction pipelines that run natively on GPUs or other accelerators, rather than porting individual algorithms piecemeal. Graph neural networks for track reconstruction that scale linearly (rather than combinatorially) with pile-up, reducing the dominant computational bottleneck. "Analysis facilities" that co-locate computing with data (columnar data formats, query-based analysis) rather than distributing copies of event data to analyst laptops. Reduced precision computing (using float16 or integer arithmetic where full float64 is unnecessary) for specific reconstruction steps. Physics-informed data reduction at the trigger level — using ML-based triggers that make more sophisticated real-time decisions about which events to keep, reducing the data volume that must be reconstructed offline.
Entry points for student teams
A student team could implement and benchmark a graph neural network for particle track reconstruction using public CMS or ATLAS simulated event samples (available via CERN Open Data), comparing computational cost and physics performance against traditional combinatorial algorithms. Alternatively, a team could prototype a GPU-accelerated implementation of a specific reconstruction algorithm (vertex finding, jet clustering, or calorimeter energy calibration) and measure the speedup achievable on consumer GPU hardware. Relevant disciplines: computer science, high-performance computing, machine learning, particle physics.
Genes
Structural cousins: stuck the same way, in other fields
Sources
"Exploring the Quantum Universe: Pathways to Innovation and Discovery in Particle Physics" (P5 Report), Particle Physics Project Prioritization Panel, 2023. accessed 2026-02-16. Also: CERN Yellow Reports on HL-LHC computing; CMS and ATLAS Phase-2 Technical Design Reports.; CERN, "The HL-LHC project," High-Luminosity LHC project website, accessed 2026-09-29; Campana, S., Marshall, Z., Di Girolamo, A., Schellman, H., Laycock, P., et al., "HEP computing collaborations for the challenges of the next decade," arXiv:2203.07237, 2022, accessed 2026-09-29 go to source 1 ↗ go to source 2 ↗ go to source 3 ↗
Working notes
Notes made while writing and checking this brief, shown as written. They use working shorthand.
- The P5 report included HL-LHC completion and exploitation in its highest-priority recommendation, alongside other projects already under way. It calls collecting, managing, and analyzing the large datasets of particle physics "a key challenge."
- The `failure:not-attempted` tag reflects that no experiment has operated at HL-LHC pile-up levels (200 simultaneous collisions). The reconstruction algorithms and computing model for this regime are untested.
- The `failure:ignored-context` tag captures the "technical debt" problem: reconstruction software written over 20+ years was designed for lower pile-up and sequential CPU execution. Adapting it to heterogeneous architectures is a larger challenge than the original algorithm development.
- The `temporal:window` tag reflects the fixed HL-LHC schedule: detector upgrades are being installed during Long Shutdown 3 (2026-2029), and the computing model must be ready when data-taking begins.
- The `domain:space` secondary tag reflects that HEP computing challenges are closely related to astrophysical data processing (SKA, Rubin Observatory) — the techniques developed for HL-LHC directly benefit adjacent fields.
- Cross-domain connection: shares the data-rate-exceeds-processing-capacity structure with digital-astronomical-transient-alert-processing (next-generation instruments that overwhelm current data pipelines) and the legacy-software-adaptation challenge with infrastructure-scada-legacy-ai-detection.
- Public data: CERN Open Data (opendata.cern.ch) provides real and simulated LHC data for student projects.
- Note reconciled 2026-08-20: a note above argues for `failure:not-attempted`; the genome now carries `failure:ignored-context` after a taxonomy revision. The original note is kept verbatim as the tagging rationale of record.
Reconciliation 2026-09-29: A site review read this write-up against the P5 report, fetched in full (166 pages) on 2026-09-29 from https://www.osti.gov/servlets/purl/2368847/, which is where the cited DOI leads. The citation record was fetched the same day from https://www.osti.gov/api/v1/records/2368847. (1) The text said the P5 report "identified HL-LHC completion and exploitation as its highest-priority near-term recommendation." The report's Recommendation 1 is "As the highest priority independent of the budget scenarios, complete construction projects and support operations of ongoing experiments and research to enable maximum science." The HL-LHC is one of several projects listed under it, with DUNE, PIP-II and the Rubin Observatory, and the report says that list is not ranked. The sentence in "Why This Matters" and the first source note now say the HL-LHC is included in the highest-priority recommendation alongside other projects. (2) The first source note said the report was "explicitly noting computing challenges as a risk to the physics program." The report does not describe computing as a risk. In section 6.7.1 it says "Collecting, managing, and analyzing these datasets is a key challenge." It says this of the large datasets of particle physics in general and gives ATLAS and CMS at the LHC as its example. The note now quotes that. Confirmed against the report and left as written: the title and year on the Source line; the 2029 start of the HL-LHC, as the report stated it in 2023; ATLAS and CMS as the experiments; the Higgs self-coupling measurement and the searches for new particles as goals; anomaly detection and graph neural networks as machine-learning methods used in particle physics. The "10× more data" figure is close to the report, which says each experiment will produce about 180 million Higgs bosons, "a factor of 10 more than what is projected for the current LHC run," and also gives "a factor of 20 enhancement of the LHC Run 2 luminosity." The CERN Open Data portal (https://opendata.cern.ch/, fetched 2026-09-29) is public and carries ATLAS and CMS data. NOT checked: the Source line also names CERN Yellow Reports and the CMS and ATLAS Phase-2 Technical Design Reports without a title or address, so they could not be identified or opened. The following figures are not in the P5 report and may rest on those documents, so they are left as written: the "50×" in the title and the "Fifty Times the Collisions" short title; the factor of 5-7 in collision rate; 200 collisions per crossing against about 40; 5 TB per second; 1 exabyte per experiment and 50-100× more data; 1 million CPU cores at 170 sites in 42 countries; the 3-5× gain from Moore's Law and the 10-20× computing gap; the $5+ billion cost; the 10-20× rise in reconstruction time; GPU speedups of 10-100×; the 20+ years of C++ code; and the 2026-2029 dates for Long Shutdown 3. Readers should note that the short title speaks of collisions while the body applies its 50-100× figure to stored data. Later the same day the editor asked for the "50×" in the titles to be checked. CERN's HL-LHC project page, fetched on 2026-09-29, says the upgrade "aims to achieve instantaneous luminosities of a factor of 5 to 7.5 larger than the LHC nominal value, corresponding to increased particle collisions, enabling the experiments to enlarge their data sample by one order of magnitude during the twelve years of HiLumi LHC operation compared with the LHC baseline programme." A 2022 Snowmass paper by the computing coordinators of the LHC experiments (Campana et al., arXiv:2203.07237, fetched the same day) says the challenge is "how to achieve a scale of computing and data management that is one order of magnitude greater than that of today, while maintaining a reasonable cost envelope." No source read gives 50×. The main title, the short title and the sentence in the Problem Statement that said "50-100× more" now say ten times, and both pages were added to the Source line. Genes untouched.