Monitoring the condition of bridges, roads, tunnels, and other critical infrastructure is essential for ensuring public safety and preventing costly failures. However, developing reliable AI solutions for this task has been limited by the scarcity of high-quality annotated data.
To address this challenge, researchers developed Cracks in the Foundation (CiF), the largest and most detailed civil infrastructure segmentation dataset available. The dataset contains approximately 150,000 high-resolution images and around 250,000 fine-grained annotations, carefully curated over five years in collaboration with civil engineering experts.
More details about the methodology and experimental evaluation can be found in the paper “Cracks in the Foundation: A Civil Infrastructure Dataset to Challenge Vision Foundation Models” (arXiv:2605.18413).
When Foundation Models Meet the Real World
Although modern Vision Foundation Models (FMs) and Vision-Language Models (VLMs) have achieved remarkable performance on many computer vision tasks, the CiF benchmark demonstrates that these models still face significant challenges in real-world infrastructure inspection.
Unlike the internet images commonly used to train foundation models, infrastructure defects often appear on nearly textureless surfaces and require precise recognition of subtle shapes and fine structural details. The study shows that even state-of-the-art zero-shot models struggle to generalize effectively in these conditions.
Revealing the Limits of Current AI Systems
One of the most striking findings is that even specialized models trained specifically for this domain reach only about 25% mean Average Precision (mAP) on the benchmark. These results demonstrate that dense visual understanding of civil infrastructure remains far from solved, despite the rapid progress of modern AI systems.
By exposing these limitations, CiF establishes infrastructure inspection as an important open challenge for the computer vision community and provides a valuable benchmark for future research.
Towards More Reliable and Sustainable AI
The study highlights that achieving reliable AI is not simply a matter of increasing model size. Instead, progress depends on developing better datasets, domain-specific learning strategies, and evaluation methodologies that reflect real-world deployment scenarios.
This perspective aligns closely with the objectives of SustainML, where improving AI efficiency, robustness, and practical applicability are fundamental to building intelligent systems that deliver strong performance while making responsible use of computational resources.
Looking Ahead
As AI becomes increasingly important for infrastructure maintenance, environmental monitoring, and industrial inspection, benchmarks such as CiF will play a key role in driving future innovation. By identifying the limitations of current foundation models, this research provides a roadmap for developing the next generation of computer vision systems that are both more reliable and more sustainable in real-world applications.
This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070408.