CN114523905A - System and method for displaying detection and track prediction of targets around vehicle - Google Patents

Abstract

A system for displaying object detection and trajectory prediction in the vicinity of a vehicle, said system comprising at least: a detection module having at least one sensor for detecting and locating objects around a vehicle; a prediction module capable of predicting based on an object around a vehicle; a display module capable of displaying data related to objects around a vehicle and a prediction result of the prediction module, wherein information presented by the display module is capable of being autonomously selected by a driver. And a method for displaying objects around a vehicle, the method comprising at least the steps of: detecting targets around a vehicle and positioning the targets; predicting the target based on the detection and location of the target; the prediction result is displayed to the driver, wherein the displayed information can be autonomously selected by the driver.

Description

A system and method for displaying target detection and trajectory prediction around a vehicle

technical field

The present invention provides a system and method for displaying target detection results around the vehicle and predicting the target trajectory, especially on the HUD, which can not only provide the driver with information on the type, orientation and distance of the target around the vehicle, but also provide the driver with information about the target type, orientation and distance around the vehicle. Cue the future trajectory of the target. The system and method according to the present invention can present the processed information to the driver, while ignoring irrelevant or unimportant environmental information, so as to better assist the driver's judgment in different driving scenarios.

Background technique

In recent years, with the development of vehicle intelligence and traffic intelligence, people's requirements for driving safety performance are getting higher and higher, and more and more vehicles have begun to be equipped with more intelligent assisted driving systems, such as forward collision warning systems, Automatic emergency braking, blind spot detection system, panoramic parking assist system and panoramic driving recorder, etc.

However, there are still some shortcomings in the display of objects around the vehicle in the prior art, for example, it needs to be based on V2X information, which requires the assistance of other surrounding vehicles or road infrastructure. In addition, in the prior art, the blind spots for driving are usually displayed instead of comprehensive environmental information around the vehicle. In some existing technologies, even if the comprehensive environmental information around the vehicle is displayed, the trajectory prediction of the moving target cannot be performed; or the trajectory prediction of the moving target is performed, but the displayed information is too complicated, which is difficult for the driver's attention. larger dispersion.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention proposes a system and method for displaying target detection and trajectory prediction around a vehicle. The method according to the present invention can not only provide the driver with the information of the target type, orientation and distance around the vehicle, but also can prompt the driver of the future trajectory of the target. The system and method according to the present invention can present the processed information to the driver, while ignoring irrelevant or unimportant environmental information, so as to better assist the driver's judgment in different driving scenarios.

According to a first aspect of the present invention, a system for displaying target detection and trajectory prediction around a vehicle is proposed, wherein the system at least has:

a detection module, the detection module has at least one sensor for detecting and locating objects around the vehicle;

a prediction module capable of making predictions based on objects around the vehicle;

A display module, which can display the relevant data of the target around the vehicle and the prediction result of the prediction module, wherein the information presented by the display module can be selected by the driver autonomously.

Within the framework of the present invention, the targets around the vehicle include at least other vehicles, obstacles, pedestrians, non-motor vehicles, etc. around the vehicle, wherein the obstacles include traffic facilities on or beside the road, green belts, urban infrastructure, etc. .

Within the framework of the present invention, the prediction module can be integrated in a vehicle control system, which means a control device or a group of multiple control devices configured as suitable, for example comprising an application specific integrated circuit, one or more processors , Non-transitory memory that stores instructions. The vehicle control system is, for example, an electronic control unit (ECU), also commonly referred to as "trip computer", "vehicle computer" and the like. It is also envisaged that the prediction module can be a separate module dedicated to prediction calculations.

In the framework of the present invention, "autonomously selected by the driver" means that the driver can choose the information presented on the display module according to his own needs, for example, the driver can choose according to his needs to approach a vehicle with a relative speed that exceeds a certain critical speed. Other targets are highlighted, which includes at least two cases, one is that the target is approaching at a relatively high speed, and the other is that the target is stationary, but the vehicle equipped with the system according to the invention is moving towards the target at a relatively high speed. Drivers can also choose not to display some targets according to their own needs, such as green belts on both sides of the road, trash cans, lane lines, etc. can be presented according to the needs of the driver. This selection should be made before driving the vehicle.

Through the driver's autonomous choice, the presented information can be individually set, so that the presented content is not too complicated, and it can also avoid ignoring the information that is critical to safety, which is helpful for the driver to quickly judge and respond accordingly. operation to improve the safety of the vehicle.

According to a preferred embodiment of the present invention, the at least one sensor includes at least one of the following sensors: a vehicle-mounted camera, a millimeter-wave radar, an ultrasonic radar, and a lidar. Within the framework of the present invention, it is preferable to arrange a plurality of sensors on the front and rear of the vehicle body and on both sides of the vehicle body, for example, one millimeter-wave radar and one camera are arranged on the front and rear of the vehicle body, and two millimeter-wave radars and one camera are arranged on each side of the vehicle body. . Naturally, it is also conceivable to provide other types and numbers of sensors according to the design requirements of the vehicle.

According to a preferred embodiment of the present invention, the detection module can obtain at least the type, size and movement speed of the target based on the detection result of at least one sensor. For example, after receiving the detection signal of the sensor, the detection module analyzes and processes the detection signal, and classifies the target according to the contour and/or the speed of movement, for example, attributing the target to a passenger car, a commercial vehicle, a transportation facility , pedestrians, non-motorized vehicles, urban infrastructure and other categories. Furthermore, within the framework of the present invention, the detection module is preferably also able to acquire vehicle driving data via the steering wheel steering angle sensor and the wheel speed sensor.

According to a preferred embodiment of the present invention, the prediction result includes at least the movement trajectory and/or the expected speed of the target. Within the framework of the present invention, the display module can display the relevant data and prediction results of the targets around the vehicle, and the driver can also choose the display style according to his own needs, such as whether to display the predicted trajectory, and choose the time length and color of the predicted trajectory to display. , select the number of environmental perception results and the type of target. The driver can make selections through a control panel provided on the display module, and the control panel may be, for example, a button provided on the vehicle central control or a control interface provided on a touch screen of the vehicle central control. It is also conceivable that the control panel can be provided on a mobile device connected to the vehicle, in particular on an application of the mobile device. The connection between the mobile device and the vehicle can be realized in a wired manner and a wireless manner, for example, a connection manner common to those skilled in the art, such as cable, Bluetooth, Zigbee (Zigbee), and WiFi. The mobile device may be a tablet computer, a mobile phone, a notebook computer, and the like. The connection of the mobile device to the vehicle can also be made indirectly, eg via a remote server, thereby enabling remote modification of the display settings.

According to the present invention, preferably, the display module may further have a sounding module, which is used to sound an alarm to the driver in addition to a visual reminder in an emergency.

Within the framework of the present invention, for example, the driver can choose to present the target approaching at a speed exceeding the first relative speed in red, and the target approaching at a speed exceeding the second relative speed but lower than the first relative speed in yellow, for example, according to his own needs, Objects that are stationary or stationary relative to the host vehicle are shown in green. The first relative speed and the second relative speed can be fixed, but it is also conceivable that the first relative speed and the second relative speed are variable, in particular depending on the speed of the vehicle. It is also conceivable to make different settings for different types of targets, for example, setting stationary targets (such as guardrails, green belts) to always appear green, and moving targets (such as pedestrians, non-motor vehicles, other vehicles) ) are set to appear different colors depending on their relative speed. It can also be set in combination with the location of the target, for example, the target behind the driver, behind and away from the driver is always displayed in green, the target directly in front of the driver and close to the target is displayed in red, and the target in front of or to the side of the driver is displayed as red. Displayed in yellow.

According to a preferred embodiment of the present invention, the system further has a fusion module, which performs data fusion on the detection results of the sensors through a fusion algorithm, so that the target information and road information are fused. The fusion module can acquire the required data from the detection module, such as the detected target type and speed, and then fuse the acquired data with map data or road data to calculate the environment around the driver's position. The map data can be stored in the ECU or in the memory of the fusion module itself, and the fusion module can be integrated in the ECU or a separately set module dedicated to fusion operation.

According to a preferred embodiment of the present invention, the prediction module performs prediction based on the data fusion result of the fusion module, and sends the fusion result and the prediction result to the display module. As a result, the environment around the driver's location can be more accurately presented.

Within the framework of the present invention, the display module is preferably a HUD (Head Up Display) device. HUD device, also known as head-up display or head-up display, is used to project information to the driver's eyes, and is usually designed to present the vehicle's driving information on the vehicle's front window (or windshield), so that the driver can See driving information head-up. Such driving information is usually presented on the dashboard of the vehicle or on the central control display of the vehicle in the prior art. By using the HUD unit, the driver can focus on the front without having to turn his attention to the dashboard while driving.

According to a preferred embodiment of the present invention, the display module, especially the HUD device, can present environmental information around the vehicle and related data of the target in a bird's-eye view. Presenting in the form of a bird's-eye view can make the environmental information around the vehicle more intuitive, which is conducive to the driver's quick judgment and corresponding operations.

According to a second aspect of the present invention, a method for displaying objects around a vehicle is proposed, the method comprising at least the following steps:

detecting objects around the vehicle, and locating the objects;

predicting the target based on the detection and localization of the target;

The prediction results are displayed to the driver, wherein the displayed information can be selected autonomously by the driver.

According to the method of the present invention, preferably, there is a data fusion step after the detection step, in the data fusion step, the information sum of the target is fused with road information or map information, and in the prediction step based on The fusion result is predicted, and the fusion result and the prediction result are displayed in the display step.

The method according to the invention can be carried out by means of the system according to the invention.

Within the framework of the present invention, according to specific design requirements, the order and function of each module can also be adjusted accordingly, as long as the target detection and trajectory prediction results around the vehicle can be finally given.

Within the framework of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

Description of drawings

The system and method for displaying object detection and trajectory prediction around a vehicle according to the present invention are schematically described below with embodiments.

FIG. 1 shows a system for displaying target detection and trajectory prediction around a vehicle according to a preferred embodiment of the present invention;

FIG. 2 shows the environmental information around the vehicle and the corresponding predicted trajectory presented in the form of a bird's eye view, especially on the HUD device.

Detailed ways

FIG. 1 shows a system for displaying object detection and trajectory prediction around a vehicle according to a preferred embodiment of the present invention. In this preferred embodiment, the system according to the invention has a detection module, a fusion module, a prediction module and a display module.

In particular, the detection module has a plurality of on-board cameras and/or millimeter-wave radars and/or ultrasonic radars and/or lidars, for example, one millimeter-wave radar and one camera are arranged on the front and rear of the vehicle body, and two millimeter-wave radars are arranged on each side of the vehicle body. Wave radar and a camera. The millimeter-wave radar on the vehicle body can detect the speed of the target around the vehicle, and the camera can detect the image data of the target around the vehicle, so the detection module can obtain other vehicles, obstacles, pedestrians, non-motor vehicles, etc. around the vehicle. information, and at least the type, size and movement speed of the target can be obtained based on this information. For example, after receiving the detection signal of the sensor, the detection module analyzes and processes the detection signal, and classifies the target according to the contour and/or the movement speed. Within the framework of the present invention, the detection module is preferably also able to acquire vehicle driving data via the steering wheel steering angle sensor and the wheel speed sensor.

Next, the fusion module performs data fusion on the detection results of the sensors through a fusion algorithm, so that the information of the target and the road information are fused. The fusion module can acquire information such as target type and speed from the detection module, and then fuse the acquired information with map information or road information to calculate the environmental conditions around the driver's position.

After the fusion module performs fusion operation, the prediction module performs prediction based on the data fusion result of the fusion module, and sends the fusion result and the prediction result to the display module. As a result, the environment around the driver's location can be more accurately presented.

The display module can display the relevant data and prediction results of the targets around the vehicle. The driver can choose the display style according to his own needs, such as whether to display the predicted trajectory, choose the time length and color of the predicted trajectory, choose the number of environmental perception results and type of target, etc. The driver can make selections through a control panel provided on the display module. The control panel may also not be provided on the display module, for example, it is provided on the central control of the vehicle or on the mobile device connected to the vehicle.

In the system shown in FIG. 1 , for example, the driver can choose to present different targets in different colors according to their own needs, for example, present a more dangerous target in red, and present a certain dangerous target in red. In yellow, objects that are stationary or relatively stationary relative to the vehicle are displayed in green. The "risk" refers in particular to the possibility of a collision with the vehicle.

It is also conceivable that the driver can make different settings for different types of targets, for example, set the stationary target (such as guardrail, green belt) to always appear green, and the moving target (such as pedestrian, non-motor vehicle) , other vehicles) are set to appear different colors based on their relative speed.

Figure 2 shows the environmental information around the vehicle and the corresponding predicted trajectory presented in the form of a bird's-eye view, especially on the HUD device.

In Figure 2, the "target vehicle" in the middle lane represents the vehicle where the driver is. It can be seen from Figure 2 that around the driver's position, there is a large truck on the left, and there is a road ahead on the left that wants to cross the road. For pedestrians, there is a non-motorized vehicle driving from left to right in front, a passenger car in the front right, a pedestrian on the right who wants to cross the road, and a non-motorized vehicle away from the driver in the rear right. In Figure 2, each target has its predicted trajectory next to it (shown as a number of broken lines). It is conceivable that the predicted trajectory is displayed in different colors to indicate its different speed of movement; or its speed of movement is shown numerically beside (eg above or below) the predicted trajectory. Here, "moving speed" can mean absolute moving speed or relative moving speed, absolute moving speed refers to the speed of the target relative to the ground, and relative moving speed refers to the speed of the target relative to the target vehicle. In the case of expressing different moving speeds by color, for example, green means moving away, red means approaching, and the speed of moving away or approaching can also be expressed according to the depth of the color; in the case of expressing moving speed by numbers, such as You can use a "+" before the number to indicate closeness, and a "-" before the number to indicate distance.

In the example shown in FIG. 2 , for example, a non-motor vehicle with a rear right side away from the driver can be displayed in green, but the non-motor vehicle can also be omitted for clarity, since it is obvious that the non-motor vehicle has small risk. For example, the pedestrians on the right side of the driver and the pedestrians in front of the left can choose to be displayed in yellow, and the non-motor vehicles directly in front of the driver can be displayed in red, because the pedestrians on the right side of the driver and the pedestrians in front of the left have a higher risk. while the non-motor vehicle directly in front of the driver has the highest risk.

In the example shown in FIG. 2 , the motor vehicles on the left and right of the driver can be displayed differently in particular depending on their relative speeds, for example, for a passenger car in front of the right, where the speed of the passenger car is lower than the speed of the target vehicle In the case of the vehicle speed (relatively close), it may be displayed in yellow, for example, and if the speed of the passenger car is higher than the speed of the target vehicle (relatively far away), it may be displayed in green, for example. It can also be displayed in combination with the driving behavior of the target vehicle. For example, for a passenger car in the front right, if the relative position of the passenger car is close to the target vehicle and the target vehicle turns on the right-turn indicator, the passenger car in the front right will be displayed. is red. Similarly, for a large truck on the left, a similar setting can be used, for example, the relative position of the large truck is close to the target vehicle and the target vehicle turns on the left turn indicator, the large truck is displayed in red; or, for example, if the large truck is displayed in red; The large truck and the target vehicle remain relatively stationary and the distance is less than 10 meters. When the target vehicle turns on the left turn indicator, the large truck is displayed in red, and when the target vehicle keeps going straight, the large truck is displayed in yellow.

Of course, it is also conceivable to display different colors according to the type of the target, for example: the pedestrian is displayed in green, and the pedestrian is displayed in light green, green and dark green according to the degree of danger; the non-motor vehicle is displayed in yellow, according to its danger Light yellow, yellow and dark yellow are shown in the degree of danger; motor vehicles are shown in red, and pink, red and dark red according to their degree of danger. The selected color can also be personalized according to the driver's own needs.

The above selection should be made before driving the vehicle, and the selected parameters are stored by the display module so that repeated selections are not required. Within the framework of the present invention, those skilled in the art can select the displayed information according to the concept of the present invention, so that the targets with higher risk can be displayed with higher priority, and the targets with lower risk can be displayed with higher priority. Display at a lower priority or not at all without creative effort.

It should be understood that the above-illustrated embodiments of the present invention are exemplary and non-limiting, and are used to illustrate typical implementations of the air quality information level indicating system according to the present invention. More precisely, in addition to the above-described embodiments, numerous variants are also possible, which are meaningful to those skilled in the art, resulting from the combination of the individual features of the invention.

Claims (10)

1. A system for displaying target detection and trajectory prediction around a vehicle, wherein the system at least has: a detection module, the detection module has at least one sensor for detecting and locating objects around the vehicle; a prediction module capable of making predictions based on objects around the vehicle; A display module, which can display the relevant data of the target around the vehicle and the prediction result of the prediction module, wherein the information presented by the display module can be selected by the driver autonomously. 2. The system of claim 1, wherein the at least one sensor comprises at least one of the following sensors: vehicle camera, millimeter wave radar, ultrasonic radar, lidar. 3. The system according to any one of the preceding claims, wherein the detection module is capable of obtaining at least the type, size and movement speed of the target based on the detection result of the at least one sensor. 4. The system according to any one of the preceding claims, wherein the prediction result comprises at least the movement trajectory and/or the expected speed of the target. 5. The system according to any one of the preceding claims, wherein the system further has a fusion module, which fuses the detection results of the sensor through a fusion algorithm, so that the information of the target is fused. and integrated with road information. 6. The system according to claim 5, wherein the prediction module performs prediction based on the data fusion result of the fusion module, and sends the fusion result and the prediction result to the display module. 7. The system of any preceding claim, wherein the display module is a HUD device. 8. The system according to any one of the preceding claims, wherein the display module is capable of presenting environmental information around the vehicle and relevant data of the target in a bird's eye view. 9. A method for displaying objects around a vehicle, wherein the method comprises at least the steps of: detecting objects around the vehicle, and locating the objects; predicting the target based on the detection and localization of the target; The prediction results are displayed to the driver, wherein the displayed information can be selected autonomously by the driver. 10. The method according to claim 9, wherein there is a data fusion step after the detection step, in the data fusion step, the information sum of the target and the road information are fused, and in the prediction step based on the road information. The fusion result is predicted, and the fusion result and the prediction result are displayed in the display step.

Images