Disclosure of Invention
The invention provides a trackless equipment safety pre-warning method and system based on multi-source information fusion, which aims to (1) realize real-time interactive pre-warning of vehicles and personnel, ensure that the vehicles and the personnel can send and receive effective warning signals, (2) provide dead zone identification capability for the vehicles and eliminate pre-warning blind spots, (3) reduce the operation amount of an identification device, improve the timeliness and stability of identification, and (4) realize linkage of pre-warning and vehicle control and improve safety.
The technical scheme of the invention is as follows:
A trackless equipment safety precaution method based on multisource information fusion includes the steps:
The method comprises the steps that 1, a vehicle and an operator report positions to a central control system in real time, and the central control system judges whether track intersection exists between the vehicle and the operator in real time according to the positions of the vehicle and the operator;
step 2, if the intersection exists between the vehicle and the operator, starting a safety early warning identification device arranged on the vehicle, starting a stereoscopic perception system on the operator, and then continuously executing the step 3;
Step 3, the environmental safety monitoring system judges whether the possibility of collision between the operator and the vehicle and the distance between the operator and the vehicle exist or not through a safety early warning recognition device, and a monitoring result is obtained;
step 4, the central control system judges the early warning level according to the result of the environmental safety monitoring system, if the early warning level is low, the alarm device on the vehicle and the stereoscopic perception system on the operator body send out low-frequency alarm, and then the step 3 is skipped;
And 5, predicting whether the vehicle still collides with the operator under the condition that the driver takes the brake after receiving the alarm, if the predicted result considers that the vehicle cannot collide, both an alarm device on the vehicle and a three-dimensional sensing system on the operator send out high-frequency alarm, then returning to the step 3, carrying out environmental safety monitoring again and judging the early warning level, and if the predicted result considers that the vehicle collides, the central control system sends an active brake signal to a control system of the vehicle to control the vehicle to actively brake until the danger is relieved, and unlocking the vehicle and returning to the step 1.
In the step 1, whether a track intersection exists between a vehicle and an operator is judged according to an electronic fence, wherein the electronic fence is a virtual interference range which is set by a central control system around the real-time position of the vehicle and/or the operator, and when judging, if the intersection exists between the electronic fence of the vehicle and the operator or the intersection exists between the electronic fence and the position of the other party, the two are considered to exist.
In the step 2, the central control system finds that the track between the vehicle and the operator has an intersection, then the clean protection device on the safety early-warning identification device is closed before the safety early-warning identification device is started, the central control system detects that the track between the vehicle and the operator does not have the intersection, then whether the clean protection device on the safety early-warning identification device is started or not is detected, if not, the clean protection device is started, and after the clean protection device is determined to be started, the step 1 is returned.
As a further improvement of the trackless equipment safety precaution method based on multi-source information fusion, the specific steps of the step 3 are as follows:
step 3.1, starting a safety early warning recognition device by the environment safety detection system at a high frame rate;
Step 3.2, the safety early warning recognition device evaluates the noise of the acquired environmental information, if the noise is too high, the step 3.3 is executed after the environmental information noise reduction algorithm is started, otherwise, the step 3.3 is directly executed;
step 3.3, the safety early warning recognition device monitors surrounding obstacles in real time;
Step 3.4, the safety early warning recognition device judges whether to switch the frame rate according to the current acquired data, if so, the safety early warning recognition device jumps to step 3.3 after switching, otherwise, the safety early warning recognition device executes step 3.5;
step 3.5, the safety early warning recognition device obtains the distance between the monitored obstacle target and the vehicle, and gives a monitoring result;
and 3.6, waiting for a period of time, if a closing request is received, closing the environment safety monitoring system, otherwise, jumping to the step 3.3, and continuing to monitor.
In step 3.5, firstly, calculating a space which the vehicle needs to pass through according to the current speed, load and position of a steering wheel, namely a monitoring space, when the vehicle continues to run according to the current state, then, capturing and identifying whether an operator enters the monitoring space in real time by a camera in the safety early-warning identification device, if the operator enters the monitoring space, acquiring the position and the movement speed of the operator relative to the vehicle by an image shot by the camera, further acquiring a scanning result of a radar, and taking the distance between an obstacle corresponding to the position and the vehicle in the scanning result as the distance between the operator and the vehicle.
As a further improvement of the trackless equipment safety early warning method based on multi-source information fusion, after an image of an operator entering a monitoring space is acquired through a camera, the image is displayed on a vehicle-mounted display screen, different audible and visual alarm states are set according to the distance between the operator and a vehicle, and meanwhile, alarm information with different colors is provided on the vehicle-mounted display screen around the operator.
In step 4, if the vehicle or the operator needs to send out alarm signals of a plurality of different grades at the same time, the highest grade is used for alarming.
In step 5, if the predicted result is considered to collide, the central control system monitors the current braking state of the vehicle after sending an active braking signal to the control system of the vehicle, if the driver has braked or temporarily cancels the active braking function, the step returns to step 3, otherwise, the vehicle is controlled to actively brake.
The trackless equipment safety early warning system based on the early warning method comprises the central control system, and further comprises a vehicle-mounted host computer, a vehicle-mounted display screen, a vehicle-mounted positioning device, a vehicle-mounted alarm device and the safety early warning recognition device which are arranged on each vehicle;
The vehicle-mounted host is in communication connection with the central control system, and the vehicle-mounted display screen, the vehicle-mounted positioning device, the vehicle-mounted alarm device and the safety early warning recognition device are respectively in communication connection with the vehicle-mounted host;
The early warning system also comprises a stereoscopic perception system which is arranged on an operator and is in communication connection with the central control system, wherein the stereoscopic perception system comprises a human body positioning device;
the vehicle-mounted alarm device and the stereoscopic perception system are provided with an audible and visual alarm module and a vibration alarm module.
The safety early warning recognition device comprises four groups of cameras and ultrasonic radars which are arranged at the left front, the right front, the left rear and the right rear of the vehicle;
The vehicle-mounted alarm device comprises an in-vehicle alarm and an out-of-vehicle alarm.
Compared with the prior art, the intelligent early warning system has the advantages that (1) when collision possibility exists, alarm signals of different grades including vibration and sound and light are synchronously sent to a vehicle and an operator, so that the driver and the operator can realize danger and dangerous degree at the same time, corresponding measures are quickly taken, real-time interactive early warning of the vehicle and the operator is finally realized, effectiveness of warning is ensured, on the basis of an electronic fence, the possibility of collision between the operator and the vehicle and the distance are further judged through the safety early warning recognition device, comprehensive safety recognition early warning capability is provided for the vehicle, an early warning blind spot is eliminated, early warning accuracy is improved, 3) the safety early warning recognition device and a three-dimensional sensing system are started only after the central control system finds that an intersection exists between the vehicle and the operator according to the electronic fence, the operation amount of the vehicle safety early warning recognition device is reduced, the timeliness and stability of recognition are improved, decision on whether to actively intervene in braking after the vehicle and the operator are finally realized, the safety early warning is improved according to actual conditions, the linkage of the safety warning is improved, the safety warning recognition device can be further judged, the safety warning device is not influenced by the automatic time is not used, the safety warning device is fully monitored, the safety environment pollution is reduced, the safety is not has been monitored, and the safety environment pollution is fully is reduced, and the safety is not has been monitored, and the safety is fully has been monitored, and the safety environment has been fully has been reduced.
Detailed Description
The technical scheme of the invention is described in detail below with reference to the accompanying drawings:
as shown in fig. 3, the present embodiment provides a control system, which includes a central control system 1 running on a server, and further includes a vehicle-mounted host 2, a vehicle-mounted display screen 3, a vehicle-mounted positioning device, a vehicle-mounted alarm device and the safety pre-warning recognition device, which are mounted on each vehicle.
The central control system 1 can remotely manage various vehicle-mounted terminals, implement functions of remote upgrading, remote fault removal and the like, can call or check the remote working state of equipment in real time, and the platform system supports comprehensive butt joint and uploading of various data.
The vehicle-mounted host machine 2 integrates functions of four-way blind area algorithm early warning intelligent function, audio and video monitoring and the like, CAN receive and process GPS or high-precision positioning data and CAN bus information, has functions of video and audio recording, driving information recording, wireless data uploading, on-site printing and the like, and has stronger information processing capability. The vehicle-mounted host machine 2 can accurately detect various alarms based on a deep learning algorithm, and timely carry out video and voice reminding, so that a driver is alert to danger possibly occurring in advance, and the driving safety is effectively improved.
The in-vehicle host 2 is in communication connection with the central control system 1 in a wireless manner. The vehicle-mounted display screen 3, the vehicle-mounted positioning device, the vehicle-mounted alarm device and the safety pre-warning recognition device are respectively in communication connection with the vehicle-mounted host machine 2. The vehicle-mounted positioning device acquires the positioning of the vehicle in a GPS mode, a Beidou mode and the like and reports the positioning in real time.
The safety early warning recognition device comprises four groups of fisheye cameras 6 (can be monocular cameras and binocular cameras and can also be provided with infrared functions) and ultrasonic radars 7 (can be laser radars and millimeter wave radars) which are arranged at the left front, the right front, the left rear and the right rear of the vehicle, so that blind areas in different directions can be conveniently recognized, and full coverage is realized.
The vehicle-mounted alarm device comprises an in-vehicle alarm 4 and an out-of-vehicle alarm 5. The in-car alarm 4 mainly provides an alarm for a driver, and the out-car alarm 5 provides an alarm for surrounding operators.
The early warning system also comprises a stereoscopic perception system which is worn on the body of an operator and is in communication connection with the central control system 1. The stereoscopic perception system can be in the form of a helmet or a safety back clip, comprises a human body positioning device, acquires the positioning of an operator in a GPS (global positioning system), beidou and other modes, and reports the positioning in real time.
All vehicle-mounted alarm devices and three-dimensional perception systems are provided with an audible and visual alarm module and a vibration alarm module.
As shown in fig. 1, the working process of the safety warning of the warning system is as follows:
the central control system 1 is led into a daily operation task, then a vehicle-mounted system (comprising the vehicle-mounted host machine 2, the vehicle-mounted display screen 3 and the vehicle-mounted positioning device) on the trackless equipment and a stereoscopic perception system on the body of an operator are started, then an electronic fence data packet and a trackless vehicle running track data packet are issued, and then the following steps are executed:
and step 1, reporting the position of the vehicle and the operator to the central control system 1 in real time. The central control system 1 judges whether a track intersection exists between the vehicle and the operator in real time according to the positions of the vehicle and the operator.
Further, in step 1, whether a track intersection exists between the vehicle and the operator is judged according to the electronic fence. The electronic fence refers to a virtual interference range set by the central control system 1 around the real-time position of the vehicle and/or the operator, and the size of the range is preset according to the volume of the operator and the vehicle. When judging, if the intersection exists between the vehicle and the electronic fence of the operator or the intersection exists between the electronic fence and the position of the other party, the intersection of the two tracks is considered.
And 2, if an intersection exists between a certain vehicle and a certain operator, starting a safety early warning recognition device arranged on the vehicle, starting a stereoscopic perception system on the operator, preparing early warning, and then continuing to execute the step 3. If no intersection exists, returning to the step 1 to continue monitoring.
Furthermore, the central control system 1 finds that the track between a certain vehicle and other operators has intersection, and before the safety early warning recognition device is started, the clean protection device on the safety early warning recognition device is closed, so that the cleaning work is prevented from interfering with early warning. The central control system 1 detects whether the cleaning protection device on the safety early warning identification device is started or not when finding that the track between a certain vehicle and a certain operator does not have intersection, if not, the cleaning protection device is started, and after determining that the cleaning protection device is started, the step 1 is returned. The mode can fully utilize time to clean the identification device, so that the identification device always works in an optimal state.
The cleaning protection device comprises a brush, a cleaning liquid supply module and the like which swing back and forth, and the structural principle of the cleaning protection device is the same as that of a vehicle glass cleaning device, and the cleaning protection device is not described in detail herein.
However, the presence of intersections merely represents a relatively close distance and does not represent a potential collision, and therefore the following steps are also needed for further screening.
And 3, judging whether the possibility of collision between the operator and the vehicle and the distance between the operator and the vehicle exist or not by the environment safety monitoring system through the safety early warning recognition device, and obtaining a monitoring result. The environment safety monitoring system can be operated on the vehicle-mounted host machine 2 or in the central control system 1 according to the requirement.
As shown in fig. 2:
And 3.1, starting a safety early warning and identifying device by the environment safety detection system at a high frame rate.
And 3.2, the safety early warning recognition device evaluates the noise of the acquired environmental information, if the noise is too high, the environmental information noise reduction algorithm is started, then the step 3.3 is executed, and otherwise, the step 3.3 is directly executed.
As one of alternative noise assessment and noise reduction approaches:
firstly, establishing an imaging model of an image in fog noise:
Iλ(x)=Jλ(x)t(x)+Aλ(1-t(x))
Where I is the fog pattern received by the camera, J is the original signal of the scene, t is the transmissivity of the environment, and a is the light. lambda means that these terms are all wavelength dependent and are related to the three channels RGB in the image.
The model is improved, the illumination influence and the transmissivity graph are fused, the unified parameter K is used for representing the model, and a clear image can be reconstructed from the observed fog graph through the K value:
Jλ(x)=Kλ(x)Iλ(x)-Kλ(x)+1.
and learning out characteristics of different levels by using the neural network model, recovering Kmap in the scene, and completing noise reduction processing of the image based on the clear image from the above formula to the scene.
And 3.3, the safety early warning recognition device monitors surrounding obstacles in real time in a shooting and scanning mode.
And 3.4, the safety early warning recognition device judges whether to switch the frame rate according to the currently acquired data, if the shot image is not clear enough, the human body recognition cannot be realized, the safety early warning recognition device needs to switch to the high frame rate, and if the shot image is too clear, the safety early warning recognition device can switch back to the low frame rate. And after switching, jumping to the step 3.3 to re-shoot, otherwise, not switching and executing the step 3.5.
And 3.5, the safety early warning recognition device acquires the distance between the monitored obstacle target and the vehicle, and gives a monitoring result through a fusion decision.
Specifically, firstly, according to the current speed, load and position of the steering wheel of the vehicle, the space which the vehicle needs to pass through for continuing to run according to the current state, namely the monitoring space, is calculated, wherein the monitoring space comprises an area which can be directly observed by a driver and a blind area which is difficult to observe. Then, a camera in the safety early warning identification device captures and identifies whether an operator enters the monitoring space in real time. If an operator enters, the position and the movement speed of the operator relative to the vehicle are obtained through the image shot by the camera, the scanning result of the radar is further obtained, and the distance between the obstacle corresponding to the position and the vehicle in the scanning result is taken as the distance between the operator and the vehicle.
Further, the safety early warning recognition device can realize the detection and recognition of the dynamic and static states of the personnel, provide the change monitoring of the relative displacement between all operators and trackless equipment in the dead zone, set different warning priorities according to the change of the relative displacement, and lay a foundation for adopting different warning strategies, realizing active warning and classifying early warning in the subsequent steps. The vehicle-mounted host machine 2 CAN be connected with a vehicle body CAN signal, and CAN judge the movement trend by judging the gear, the turn light, the vehicle speed and the steering wheel corner signal and combining the relative distance, the relative movement speed, the movement direction and the like of the human and the vehicle, so that a foundation is laid for obtaining the danger degree later and giving corresponding early warning signals according to a preset rule.
Further, after the image of the operator entering the monitoring space is obtained through the camera, the image can be displayed on the vehicle-mounted display screen 3, the driver is prompted to pay attention to the safety of related personnel through the image, different audible and visual alarm states are set according to the distance between the operator and the vehicle, meanwhile, warning information with different colors is provided on the vehicle-mounted display screen 3 around the operator, for example, the blue data represent personnel are located outside the early warning distance, the blue data represent personnel are located in the primary early warning distance range, the yellow data represent personnel are located in the secondary early warning distance range, the orange data represent personnel are located in the tertiary early warning distance range, the red data represent personnel are located in the quaternary early warning distance range, the risk level, the category and the azimuth are intuitively fed back, and the driver is told in a classification and grading manner.
The distance can be identified in advance according to the image and fused with the detection result of the radar. Taking the height of a normal person as an example, an identification frame with the same size can be arranged on the picture of the camera 6, the position and the distance with trackless equipment are analyzed through displaying the body parts and the change speed of the person in the camera 6 in different early warning distances, and the nearby person is accurately positioned through the feedback distance data of the ultrasonic range radar. The distance threshold value is set on the data processing interface, and alarms with different degrees are performed when different threshold values are triggered.
In this step, as one of the optional decision modes, an environmental information weight function is defined first:
Wherein F (k) is a camera information weight function, r (k i) is an infrared camera information weight function, G (m) is a radar information weight function, and J (n) is a positioning information weight function. i is the i-th device sensor signal conveying information. The collision early warning grades among vehicles, operators and vehicles can be calculated through the comprehensive weight function of the environmental information, the coordinate parameters of the safe distance range are transmitted to the central control system 1, and the comprehensive judgment is carried out to carry out the safety early warning of different grades.
It should be noted that the above decision method is not the only decision method. In actual operation, the decision result can be directly obtained by combining the preset rule through the azimuth and the distance of the operator.
And 3.6, waiting for a period of time, if a closing request is received, closing the environment safety monitoring system, otherwise, jumping to the step 3.3, and continuing to monitor.
And 4, as shown in fig. 1, the central control system 1 judges the early warning level according to the result of the environmental safety monitoring system, if the early warning level is low, the alarm device on the vehicle and the stereoscopic perception system on the operator send out low-frequency alarm, and then the step 3 is skipped, and if the early warning level is high, the step 5 is continuously executed.
And 5, predicting whether the vehicle still collides with the operator or not under the condition that the driver takes the brake after receiving the alarm. If the predicted result indicates that the vehicle cannot collide, the warning device on the vehicle and the stereoscopic perception system on the operator body all send out high-frequency warning to prompt the operator to avoid and brake the driver, and then the vehicle returns to the step 3 to monitor the environmental safety again and judge the early warning level. If the predicted result indicates that the vehicle can collide, the central control system 1 sends an active braking signal to the control system of the vehicle, monitors the current braking state of the vehicle, and jumps back to the step 3 if the driver has braked or temporarily cancels the active braking function, otherwise, controls the vehicle to actively brake until the danger is relieved, unlocks the vehicle and returns to the step 1.
Further, if a certain vehicle or operator needs to send out alarm signals of a plurality of different grades at the same time, the highest grade is used for alarming. Meanwhile, after the operator feels the alarm of the stereoscopic perception system, the operator can observe the surrounding environment, and observe which vehicle gives out the alarm frequency consistent with the alarm frequency on the body, so as to determine the vehicle which should be avoided at present.
It should be noted that, since the determination time in step 5 is short, the audible and visual alarm, the vibration alarm, and the image alarm on the in-vehicle display screen 3 occur substantially simultaneously. As a part of cooperative alarm, the high-frequency and low-frequency alarm can be further refined during alarm, and surrounding personnel can quickly sense the dangerous degree in the modes of amplifying influence, changing the volume of a loudspeaker, increasing buzzing prompt, changing the speech speed of voice prompt and the like according to the distance value. For example, when the relative distance between the person and the front end or the rear end of the car body is 5 meters to 7 meters, a first-level early warning is sent out, the high-definition screen in the cab pops up the corresponding image in the front side or the rear side blind area, when the relative distance between the person and the front end or the rear end of the car body is 3 meters to 5 meters, the relative distance is becoming smaller, a second-level early warning is sent out, the high-definition screen in the cab pops up the corresponding image in the front side or the rear side blind area, the loudspeaker rings at the same time, the audible-visual alarm sounds and blinks, when the relative distance between the person and the front end or the rear end of the car body is becoming smaller, the high-definition screen in the cab pops up the corresponding image in the front side or the rear side blind area, the loudspeaker rings up the prompt sound and blinks, the audible-visual alarm sounds at the same time, the loudspeaker sounds the audible-visual alarm sounds the sound and blinks at the time, the sound-visual alarm sounds the sound and blinks at the time, and the vehicle is prevented from being braked directly by the driver. The vehicle head and the vehicle tail are provided with the primary warning area which is 7 meters away from the vehicle body, when the distance between the personnel and the front end or the rear end of the vehicle body exceeds 7 meters, the primary warning is released, the driver is not warned in a warning manner, the relative distance is between 5 meters and 7 meters, and when the relative distance is becoming larger, the primary warning is also released, and the driver is not warned in a warning manner so as to reduce invalid warning. The alarm classification of the personnel and the vehicles in the left and right dead zones is consistent with the front end and rear end classification method, and the classification distance is smaller, and the method is specifically characterized in that when the distance between the personnel and the left or right of the vehicle body is 2 meters to 1 meter and the relative distance is becoming smaller, a first-level early warning is sent out, and a high-definition screen in a cab pops up a corresponding image in the left or right dead zone; when the relative distance between the person and the left or right side of the car body is 1 meter to 0.6 meter, a secondary early warning is sent out, a high-definition screen in the cab pops up a corresponding image in the left or right blind area, the loudspeaker rings at the same time, an audible and visual alarm sounds and flashes, when the relative distance between the person and the front or the rear end of the car body is 0.3 meter to 0.6 meter, a tertiary early warning is sent out, the high-definition screen in the cab pops up a corresponding image in the left or right blind area, the loudspeaker rings at the same time, the audible and visual alarm sounds and flashes, when the relative distance between the person and the front or the rear end of the car body is less than 0.3 meter, a four-stage early warning is sent out, the high-definition screen in the cab pops up a corresponding image in the left or right blind area, the loudspeaker rings at the same time, the audible and visual alarm sounds at the same time, the engine directly takes emergency braking to avoid hurting the person.
The photo and video of the risk point early warning and the information of the related early warning can be stored in a local storage or uploading central control system 1, the blind area range or the early warning distance is increased or decreased through automatic data processing or manual processing, and the alarm is further set in a refined mode aiming at unused scenes such as reversing, straight running, turning and the like, so that dangerous false alarms are reduced, and the effectiveness of the alarm is improved.