infrastructure · digital · family: it worked in the lab
the robot gets loston the job site
Construction robots lose spatial awareness in real jobsite conditions
Problem statement
Construction robots that work reliably in labs and controlled demos fail on real construction sites because their perception systems (LiDAR, cameras, depth sensors) degrade under conditions ubiquitous on active sites: airborne dust, rain, concrete splatter, vibration, and an environment that changes shape daily as work progresses. Dust clouds create phantom obstacles in LiDAR point clouds. Rain and mud coat camera lenses. The reference points that localization algorithms depend on — walls, columns, floors — don't exist yet or have moved since the last scan. Unlike factory robots operating in static, controlled environments, construction robots must navigate terrain that is uneven, unpredictable, and actively evolving.
Why this matters
Construction faces a severe labor shortage — the US industry needs an estimated 500,000+ additional workers per year. Robots could address this gap for repetitive tasks (bricklaying, rebar tying, concrete finishing, site inspection), but current deployment rates remain near zero outside demos. The perception robustness gap is the primary technical barrier preventing autonomous operation on real sites.
What’s been tried and why it hasn’t worked
Indoor robots (warehouse, hospital) use static maps and known reference points; these approaches fail when the environment changes daily. SLAM algorithms struggle when mapped features are transient (scaffolding, material stockpiles, temporary walls). Multi-sensor fusion (LiDAR + IMU + camera) helps with individual sensor degradation but still fails under simultaneous multi-modal interference (dust + vibration + changing geometry). Most published construction robot research validates in controlled or simulated environments, providing little insight into real-world robustness. Autonomous mining vehicles achieve comparable robustness underground, but their perception stacks are proprietary and not adapted for above-ground construction's faster-changing geometry.
What would unlock progress
Perception systems purpose-designed for degraded-sensor, dynamic environments — drawing on military/defense SLAM research for GPS-denied, smoke-filled environments and adapting it to construction-specific conditions. Key needs: self-cleaning sensor housings, dust-penetrating radar augmentation for LiDAR, temporal map management that distinguishes permanent structure from transient objects, and traversability assessment that handles ambiguous surfaces (wet concrete, gravel piles, puddles).
Entry points for student teams
Real-site data collection is the obvious first instinct and the wrong place to start: putting a robot on an active jobsite takes contractor sponsorship, site safety credentialing, and a sensor-equipped platform that runs well into five figures, so that door belongs to a construction-management program with a standing industry partner or to a corporate research lab, not to an unaffiliated student team. The facility-free door is to benchmark existing SLAM stacks against construction-site data that has already been collected and released: the Hilti-Oxford dataset (https://hilti-challenge.com/dataset-2022.html) provides calibrated lidar, visual, and inertial sequences recorded on real construction sites with millimeter-level ground truth and difficulty labels for ground-level, multilevel, and stairwell runs, and ConSLAM (https://github.com/mac137/ConSLAM) adds periodically rescanned construction-site sequences from a handheld RGB-camera-plus-Velodyne-plus-IMU rig with survey-grade reference scans. A team can quantify exactly where published algorithms break on featureless partially built rooms and on revisits where the geometry has changed since the last pass — the temporal-map-management failure this brief describes — using nothing but the public data and an evaluation harness. A second team could characterize sensor degradation on a bench instead of a site: mount one lidar and one camera on a fixed rig, degrade them under controlled and measurable conditions (aerosolized particulate, water spray on the lens, shaker-table vibration), fit a degradation model to the resulting point clouds and images, then inject that model as augmentation into the public construction sequences to estimate how much localization error each condition actually causes. Robotics, computer vision, and construction management skills would be most relevant.
Genome — every gene is a door
Structural cousins — same reason stuck, other fields
Sources
McKinsey, "The Impact and Opportunities of Automation in Construction," 2024; McKinsey, "Humanoid Robots in the Construction Industry: A Future Vision," 2024; arXiv, "Robotics Under Construction: Challenges on Job Sites," 2025.
verification notes (working record)
The collection team’s own sourcing notes for this brief, kept verbatim:
Related to `digital-humanoid-robot-bipedal-stability-safety` (which covers general humanoid stability, not construction-specific perception) and `infrastructure-construction-fall-detection-sim-to-real` (which covers wearable fall detection, not robot navigation). The construction-specific perception problem is distinct from both because the primary challenge is environmental degradation of the entire sensor suite, not a single-modality problem.
Reconciliation 2026-08-21: The entry-point triage flagged this brief as a single-door-fails case, and that flag is confirmed. Both prior suggestions — deploying a mobile robot on an active site, and building a real-site perception benchmark — required the same thing, physical access to a working jobsite, which carries contractor sponsorship, safety credentialing, and a five-figure sensor platform; the section therefore offered no door at all to a team without an industry partner. The replacement keeps the brief's actual research question (perception degradation and temporal map management under changing geometry) and gives it two reachable doors: benchmarking published SLAM stacks on already-released construction-site data, and bench characterization of sensor degradation fed back into that data as augmentation. The jobsite path is retained as an explicit access line naming what it takes and who owns it, rather than being padded out with a filler suggestion. Two datasets were verified by fetch on 2026-08-21 and are public with no application gate: the Hilti-Oxford SLAM Challenge dataset, https://hilti-challenge.com/dataset-2022.html, whose page states the sequences were "collected on construction sites as well as on the famous Sheldonian Theatre in Oxford," provides "accurate ground truth, at millimeter level," and uses a platform of calibrated "visual, lidar, and inertial sensors" (downloadable rosbags of 10–34 GB per sequence, with the evaluation server still accepting submissions); and ConSLAM, https://github.com/mac137/ConSLAM, whose README confirms an RGB-camera, Velodyne, and IMU rig with Leica reference scans, a direct dataset download link, and an ECCV 2022 Workshops paper. The README's own warning that the sequence-1 bag file is faulty is worth passing to any team that uses it.