Research Focus

The objective of this project is to develop an AI-powered robot manipulation system that simplifies control inputs through natural language interfaces, such as text and voice, and realize a more cost-effective and scalable robot control. The system will also ensure the execution precision by leveraging multimodal object detection and reinforcement learning-based control optimization. The system aims to enhance human-robot interaction in complex environments by making robot control more intuitive and accessible without sacrificing execution performance.

Natural Language Processing for Control Order Interpretation

human robot interaction

This task involves developing a natural language processing (NLP) model to interpret user commands given via text or voice.

The model will need to recognize various input patterns and translate them into precise robot control instructions. This includes parsing commands, understanding contextual language cues, and mapping them to specific robotic tasks. 

The goal is to simplify robot operation, enabling intuitive interaction without the need for complex manual controls, especially in human-robot collaboration environments.

Multimodal Environment Detection and 3D Environment Construction

object sensing

This task focuses on building a comprehensive 3D representation of the robot’s environment by integrating data from multiple sensors, such as cameras, lidar, and depth sensors. 

The system will need to detect objects, obstacles, and dynamic elements in the environment to create a real-time, adaptive 3D map. Advanced AI algorithms will be used to fuse multimodal data, enabling accurate environmental awareness.

Forward-kinematics-aware Reinforcement Learning for Control Optimization

experiment robot arm

The task focuses on designing a reinforcement learning algorithm that can harness the calculation of forward kinematics for complex action manipulation optimization. 

The robot will learn to optimize its actions based on accurate kinematic predictions, improving its precision and efficiency in manipulating objects.

This approach ensures that the robot can perform complex tasks while minimizing errors, leading to more effective and reliable control strategies in dynamic environments.