Semantic Multi-Modal SLAM System in Complex Dynamic Environment

Opportunity

The multi-modal semantic SLAM technology combines advanced localisation, mapping and environmental understanding. It significantly enhances navigation and obstacle identification in autonomous vehicles and robotics. This technology also benefits augmenting virtual and augmented reality, smart infrastructure, industrial automation, healthcare, search and rescue operations, agriculture and retail by improving efficiency, safety and user experiences.

Technology

This SLAM integrates data from sensors such as LiDAR and cameras, using advanced machine learning and AI algorithms to create a detailed understanding of the environment. It features real-time data processing, enabling dynamic adaptation to changing environments. Additionally, the system includes semantic analysis for object identification and categorisation, along with computer vision techniques for enhanced interaction. This combination ensures efficient navigation and interaction in complex environments, marking a significant advancement in robotic perception and navigation technologies.

Figure 1: AGV equipped with semantic multi-modal SLAM in a highly complex and dynamic environment. Left image is the AGV platform and localisation hardware. Right image is the mapping result with target object highlighted with multi-modal segmentation. 

Applications & Advantages

Main application areas include autonomous vehicles, smart infrastructure, industrial automation, healthcare, search and rescue missions, agriculture and retail.

Advantages:

• Supports the creation of both static and semantic dynamic maps.
• Customises target tracking and computation based on scenario complexity.
• Provides real-time semantic recognition at 10 Hz and improved accuracy compared to traditional deep-learning approaches.
• Efficiently addresses motion blur and border effect issues.