Liu, Shengshu
(2025)
Methodologies for holistic and objective SLAM benchmarking and mapping accuracy enhancement.
PhD thesis, University of Nottingham.
Abstract
Simultaneous Localization and Mapping (SLAM), commonly referred to as concurrent mapping and localization, is a key area of study in robotics. It focuses on the dual task of building a map of an environment while simultaneously determining the robot location within it—a process that blends mapping and localization. The accuracy of localization and mapping in a SLAM system is typically measured using a SLAM benchmark. SLAM benchmarking plays a pivotal role in advancing the field by providing a common ground for performance evaluation, fostering collaboration, and promoting the development of reliable and efficient SLAM algorithms with real-world applicability. It serves as a foundation for driving progress and innovation in the broader field of robotics and computer vision.
A key challenge in SLAM benchmarking is ensuring a holistic and objective evaluation of SLAM system performance, and the goal is to achieve unbiased assessment results that accurately reflect a SLAM system’s true overall capabilities. To achieve such goal, a SLAM benchmark should assess localization and mapping together as a whole within a unified global framework. Despite this, most existing studies focus solely on evaluating localization performance while neglecting mapping accuracy, primarily due to the difficulty of obtaining a high-precision environment map which can serve as a reliable ground truth reference. Even among studies that assess both aspects, localization and mapping are often evaluated separately as unrelated tasks, leading to potentially biased assessment results. Thus, the development of a holistic and objective SLAM benchmark that produces unbiased performance measures of both localization and mapping is necessary.
The challenge of acquiring high-precision maps extends beyond SLAM benchmarking and affects various robotics applications, including navigation, path planning, and obstacle avoidance, where precise mapping is critical. Current methods often rely on complex hardware setups and labour-intensive manual processes, making high-precision map generation both costly and time-consuming. This limitation has led to an additional focus in this PhD research: devising a method for enhancing mapping accuracy with the potential to generate high-precision maps more efficiently. These maps could serve as credible ground truth references for SLAM benchmarking while also being applicable to broader robotic applications. The ultimate goal is to achieve this in a simpler, more resource-efficient, and scalable manner.
This PhD project introduces innovative approaches to tackle the previously mentioned dual challenges of creating a holistic and objective SLAM benchmark while enhancing mapping accuracy. The proposed novel SLAM benchmarking method transforms all localization and mapping data into a unified global coordinate frame, where localization and mapping errors are systematically measured. Its holism lies in evaluating these two components together as a whole rather than separately, while its objectivity stems from recognizing their interdependence by maintaining the original spatial relationships among all reference frames throughout the transformation process. By leveraging the benchmark results as feedback, mapping accuracy is improved through an optimization process that minimizes localization errors. This is achieved by first optimizing the alignment between the estimated trajectory and the ground truth trajectory and then applying the resulting transformation to the estimated map to enhance its accuracy.
The optimization that minimizes the localization error is achieved through a newly proposed point cloud registration technique called Centre Point Registration-Iterative Closest Point (CPR-ICP). This enhanced variant of the Iterative Closest Point (ICP) algorithm begins by aligning two point clouds using their centroids and least-square planes, followed by the classic ICP method to further reduce discrepancies between them.
The proposed methodologies were rigorously validated through both simulation-based and real-world experiments. Results from both types of experiments demonstrated the effectiveness of the proposed SLAM benchmarking framework in accurately reflecting a SLAM system’s true performance. Furthermore, the experiments confirmed the feasibility of using benchmark feedback to improve mapping accuracy. Statistical analyses consistently showed that the CPR-ICP method outperformed the classic ICP approach in enhancing mapping accuracy. Additionally, the results also revealed a correlation between the performances of both methods in improving mapping accuracy and the size of the scene.
Therefore, it can be concluded that the proposed SLAM benchmarking method surpasses existing approaches in accurately representing the genuine overall performance of a SLAM system. Additionally, the proposed mapping accuracy enhancement method offers an efficient way to generate high-precision maps that have the potential to serve as reliable ground truth references for SLAM benchmarking and be effectively utilized in various robotic applications.
Actions (Archive Staff Only)
 |
Edit View |
Tools
Tools
