A Multidimensional Sensor Integration Methodology for Road Surface Condition Monitoring Using AI-Based Vision and Vibration Systems
Academic Article in Scopus
This work presents a multi-sensor system for offline road surface assessment, combining vision- and vibration-based AI classification models with precise localization. The vision subsystem integrates camera imagery with LiDAR data for road segmentation and panoramic region-of-interest (ROI) placement, through a You Only Look Once (YOLO) classification model to categorize road segments into five surface condition classes. In parallel, the vibration subsystem applies a 1D-Convolutional Neural Network (CNN) to inertial signals captured while the vehicle is in motion, identifying five types of physical surface irregularities. Predictions from both subsystems are spatially integrated using Real-Time Kinematic Global Navigation Satellite System (RTK-GNSS) and an Inertial Navigation System (INS), producing centimeter-level alignment and map-based visualization of the classified road segments. Independent evaluation of each modality revealed complementary strengths: the vibration model, originally developed for a different context, generalized well to new environments for several classes, while the vision-based system provided robust forward-looking classification, even under challenging lighting. The vehicle-deployed system was validated across diverse road and illumination conditions. The proposed pipeline supports large-scale, precise, and geo-referenced road inspection by combining sensor fusion for vision with multimodal spatial integration, laying the foundation for data-driven infrastructure maintenance strategies.
