Recognition of Blind Point Objects by Intelligent Human-Machine Interface and Natural Dialogue with Drivers

[Erica Zhu Feilong Jiangli 0]

Nowadays, more and more driver assistance systems equipped with cameras and sensors can detect objects more effectively than drivers. Moreover, speech recognition technology has already been equipped with vehicle navigation system. Traditional driver assistance system can detect objects (vehicles, pedestrians, etc.) and then inform the driver. However, sometimes drivers will be annoyed by such vague and one-way notification, which will lead to the driver taking the wrong road. In response to such restrictions, Mitsubishi Electric has developed new technologies for intelligent notification and natural navigation functions, which are expected to help make driving safer and more convenient.

After identifying the direction the driver is facing, the technology uses intelligent notification to alert the driver about objects beyond the driver's sight. In addition, the HMI Natural Navigation System can respond in a natural dialogue when the driver verbally asks questions about the driving route, without pressing a button or saying an activation word.

Intelligent notification function combines Mitsubishi Motor Maisart image recognition technology and vehicle video camera information to identify potential dangerous objects, such as cars, humans and other moving objects, located outside the driver's line of sight. The object facing the driver is identified by the driver monitoring system (DMS). The system monitors the driver by camera. By comparing the direction of the moving object and the direction of the driver, the blind spot of the driver is estimated. On this basis, the driver is warned by the monitor and the alarm. Tests of Mitsubishi Motor show that the system is particularly effective in detecting objects and attracting driver's attention.

Natural navigation uses DMS and a series of microphones (multiple microphones that record sound simultaneously) to detect when the driver opens his mouth and to understand the driver's voice cues. It is worth noting that the system can not only recognize voice, but also distinguish the questions that the pilot wants to ask about the navigation system and other irrelevant conversations between the crew. Drivers can talk to the system in a highly natural way to provide and receive information.（Edit by Chongqing Feilong Jiangli）