CN108803668B - Intelligent inspection unmanned aerial vehicle nacelle system for static target monitoring - Google Patents
Intelligent inspection unmanned aerial vehicle nacelle system for static target monitoring Download PDFInfo
- Publication number
- CN108803668B CN108803668B CN201810651606.XA CN201810651606A CN108803668B CN 108803668 B CN108803668 B CN 108803668B CN 201810651606 A CN201810651606 A CN 201810651606A CN 108803668 B CN108803668 B CN 108803668B
- Authority
- CN
- China
- Prior art keywords
- target
- module
- pod
- monitoring
- unmanned aerial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 52
- 230000003068 static effect Effects 0.000 title claims abstract description 29
- 238000007689 inspection Methods 0.000 title claims description 39
- 230000007547 defect Effects 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims abstract description 7
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000013135 deep learning Methods 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 3
- 230000001133 acceleration Effects 0.000 claims description 2
- 238000003672 processing method Methods 0.000 claims description 2
- 238000000034 method Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000036544 posture Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 239000004020 conductor Substances 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000012212 insulator Substances 0.000 description 2
- 125000006850 spacer group Chemical group 0.000 description 2
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 1
- 206010063659 Aversion Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0094—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/12—Target-seeking control
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D3/00—Control of position or direction
- G05D3/12—Control of position or direction using feedback
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Closed-Circuit Television Systems (AREA)
- Image Analysis (AREA)
Abstract
The utility model relates to an unmanned air vehicle technical field especially relates to an unmanned aerial vehicle nacelle system is patrolled and examined to intelligence of static target monitoring, this system can control unmanned aerial vehicle and nacelle according to the planning course and the initial position and posture parameter of presetting the task of patrolling and examining and catch the target, through degree of depth learning algorithm and training model identification, fix a position the target, and control the nacelle and adjust cloud platform gesture and camera focus gradually, lock, enlarge the target and snapshot monitoring image, be used for artifical or intelligent identification part defect, thereby realize intelligent seizure, lock, snapshot part target, improve system operating efficiency, reduce personnel's skill requirement.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicles, in particular to an intelligent inspection unmanned aerial vehicle pod system for static target monitoring.
Background
The industries such as power grids, railways, water conservancy, oil and gas pipelines and the like all have a large number of equipment facilities erected on the earth surface, and because the equipment facilities are exposed in the natural environment for a long time, the equipment facilities not only have self quality and process life, but also bear normal operation load and weather influence, so that the defects and risks such as pollution damage, displacement deformation and the like are inevitable, and the operation safety and benefit of the system are directly influenced.
Therefore, a manual line inspection system is established in each industry, workers inspect along the line to observe the state of surface equipment facilities, detect part defects in time, guarantee maintenance and guarantee normal operation of the system.
With the development of unmanned aerial vehicle technology in recent years, a high-performance and low-cost small and medium-sized unmanned aerial vehicle is introduced in line inspection attempts, missing detection and error detection possibly caused by the limitation of inspection personnel observation point positions, skill experience and responsibility consciousness are controlled through field snapshot component high-definition images and background professional manual or intelligent film reading, and inspection quality and efficiency are improved.
General unmanned aerial vehicle that patrols and examines needs the on-spot manual control unmanned aerial vehicle of professional flight control hand, selects the point of shooing, locking target and snapshot photo, then improves on-the-spot personnel skill experience requirement by a wide margin, and current patrolling and examining personnel are difficult to competence, influence personnel's adjustment and human cost, and then on-the-spot manual control snapshot is difficult to guarantee high efficiency, all-round monitoring part state, still need to continue to promote unmanned aerial vehicle system's technical performance that patrols and examines.
Disclosure of Invention
The application provides an unmanned aerial vehicle nacelle system is patrolled and examined to static target monitoring's intelligence to part target is caught, is locked, is taken a candid photograph to intelligence, improves system operation efficiency, reduces personnel's technical ability requirement.
In order to solve the technical problem, the application provides the following technical scheme:
the application provides an unmanned aerial vehicle nacelle system is patrolled and examined to static target monitoring's intelligence to part target is caught, is locked, is taken a candid photograph to intelligence, improves system operation efficiency, reduces personnel's technical ability requirement.
In order to solve the technical problem, the application provides the following technical scheme:
an intelligent routing inspection unmanned aerial vehicle pod system for static target monitoring, comprising: the system comprises a ground station inspection task module, an airborne flight control module, a nacelle module, an identification positioning module and a locking control module; the ground station inspection task module presets a planning route corresponding to an inspection task, an unmanned aerial vehicle and an initial pose parameter of the nacelle module, sends the planning route and the initial pose parameter to the airborne flight control module, controls the unmanned aerial vehicle to hover and the attitude of the nacelle module, and acquires an initial image of a monitoring target; the pod module comprises a sensor and a multi-degree-of-freedom cradle head, executes instructions of the airborne flight control module and the locking control module, controls and adjusts the attitude of the multi-degree-of-freedom cradle head and the focal length of a lens of the sensor, sequentially enters an initial attitude, a locking attitude and a capturing attitude, and obtains photos or video images of a target and defects and risks of the target, wherein the photos or video images comprise initial, locking and monitoring images; the identification positioning module is matched with an embedded computing platform transplanted with a deep learning algorithm, intelligently identifies the monitoring target in the initial image acquired by the pod module, calculates the target pixel coordinate, extracts the target characteristic image and sends the target characteristic image to the locking control module; the locking control module is used for matching a real-time target locking image acquired by the pod module by taking the target pixel coordinate resolved by the identification and positioning module and the extracted target characteristic image as templates, resolving the real-time pixel coordinate and the picture proportion of the target, and controlling and adjusting the cloud platform posture and the camera focal length of the pod module step by step to lock and amplify the target and capture the monitoring image of the target.
The intelligent inspection unmanned aerial vehicle pod system for static object monitoring is characterized in that the pod module comprises a sensor, a lens and a lens, wherein the sensor is one or more of visible light, infrared light, ultraviolet light and multispectral sensors.
The intelligent inspection unmanned aerial vehicle pod system for monitoring the static targets, wherein preferably, under the unmanned aerial vehicle pose and pod attitude precision, in an initial image obtained by the pod module executing the instruction of the onboard flight control module, the length or area of the monitored targets is positioned in the pod visual field and the picture accounts for 10-20%, and the identification and positioning module identifies and captures the monitored targets, calculates the target pixel coordinates and extracts the target characteristic image; in a monitoring image obtained by the pod module executing the locking control module instruction, the length or the area of a monitoring target is positioned in the field of view of the pod and is drawn by 40-60%.
The intelligent inspection unmanned aerial vehicle pod system for static target monitoring is characterized in that the identification and positioning module identifies and captures a monitored target through a deep learning algorithm and by using a target training model and an instruction of the airborne flight control module.
The intelligent inspection drone pod system for static target monitoring as described above, wherein preferably the lockout control module comprises: an outer loop PID controller and an inner loop PID controller; the position coordinates of the central point of the target pixel and the position coordinates of the central point of the frame are used as input signals of the outer ring PID controller, and the expected movement speed of the target in the frame is obtained through calculation of the outer ring PID controller; and the expected movement speed and the movement speed of the current target pixel center point are used as input signals of the inner ring PID controller, and the adjustment information of the pod module holder is obtained through calculation of the inner ring PID controller.
The intelligent inspection drone pod system for static object monitoring as described above, wherein preferably the input signal to the inner loop PID controller further comprises: the expected moving speed of the central point of the target pixel at the current moment and the expected moving speed of the central point of the target pixel at the last moment.
The intelligent inspection unmanned aerial vehicle pod system for static target monitoring is characterized in that the locking control module is used for sending a zoom command to the pod module when the time that the center position of a target pixel continuously reaches the threshold value of the center of a picture reaches a preset value, and the pod module changes the focal length of a lens according to the zoom information.
The intelligent patrol unmanned aerial vehicle pod system for static target monitoring comprises a locking control module, a pod module and a locking control module, wherein the locking control module is used for sending a snapshot monitoring image command to the pod module when the target frame occupation ratio reaches a preset threshold range and the target pixel center position continues to reach a preset value in the time of the frame center threshold.
The intelligent inspection unmanned aerial vehicle pod system for static object monitoring as described above, wherein the preset threshold range is preferably 40% to 60%.
Compared with the background technology, the intelligent inspection unmanned aerial vehicle pod system for monitoring the static targets can control the unmanned aerial vehicle and the pod to capture the targets according to the planned route and the initial pose parameters of the preset inspection task, identify and position the targets through a depth learning algorithm and a training model, control the pod to gradually adjust the posture of a holder and the focal distance of a camera, lock and amplify the targets and capture monitoring images for manually or intelligently identifying the defects of components, so that the intelligent capturing, locking and capturing of the component targets are realized, the operating efficiency of the system is improved, and the requirement of personnel skills is lowered.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present invention, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a schematic diagram of a smart inspection unmanned aerial vehicle pod system for static target monitoring provided by an embodiment of the present application;
FIG. 2 is a schematic diagram of a lockout control module provided by an embodiment of the present application;
fig. 3 is an optical zoom diagram of a camera provided in an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative only and should not be construed as limiting the invention.
The application provides an unmanned aerial vehicle nacelle system is patrolled and examined to static target monitoring's intelligence, it patrols and examines as the example to use the electric wire netting, a part target for regularly patrolling and examining on the power transmission line and the shaft tower, mainly include the conductor spacer, the stockbridge damper, the voltage-sharing shielding ring, insulators etc., this unmanned aerial vehicle nacelle system is patrolled and examined to static target monitoring's intelligence, unmanned aerial vehicle chooses for use electronic many rotor type, nacelle module apolegamy high definition zooms digital camera, in order to provide intelligence seizure, discernment, locking and snapshot conductor spacer, the stockbridge damper, the voltage-sharing shielding ring, the function of parts such as insulators, a pollution damage that is used for backstage manual work or intelligent identification part probably to appear, defects and risks such as aversion deformation.
As shown in fig. 1, the intelligent routing inspection unmanned aerial vehicle pod system for static target monitoring provided by the present application includes: the ground station inspection task module 11, the onboard flight control module 12, the pod module 13, the identification positioning module 14 and the locking control module 15.
The ground station inspection task module 11 supports technicians in the field to design and preset planning routes corresponding to all inspection tasks and initial pose parameters of the unmanned aerial vehicle and the pod module 13, supports task managers to select the inspection task, sends corresponding planning route information and the initial pose parameters of the pod module 13 to the airborne flight control module 12, controls the unmanned aerial vehicle to hover and preset the inspection initial pose on site, and controls the pod module 13 to enter the preset initial pose to acquire an initial image of a monitored target.
The pod module 13 is used for selecting a high-definition zooming digital camera and a multi-degree-of-freedom cradle head, executing instructions of the airborne flight control module 12 and the locking control module 15, controlling and adjusting the attitude (the roll angle and the pitch angle of the pod) and the focal length of a lens, sequentially entering the initial attitude, the locking attitude and the capturing attitude, acquiring photos or video images of the target and defects and risks thereof, including initial, locking and monitoring images, sending the initial image to the identification positioning module 14, and sending the locking image to the locking control module 15.
The identification positioning module 14 is configured with an embedded computing platform which is provided with a GPU, a CPU, a memory and a flash memory and runs an operating system, a deep learning algorithm is transplanted, the intelligent identification pod module 13 acquires a monitoring target in an initial image, the target pixel coordinate is calculated, a target characteristic image is extracted, and the target characteristic image is sent to the locking control module 15.
And the locking control module 15 is used for carrying out template matching on the real-time target locking image acquired by the pod module 13 by taking the target pixel coordinate resolved by the identification and positioning module 14 and the extracted target characteristic image as templates, resolving the real-time pixel coordinate and the picture proportion of the target, and controlling and adjusting the cradle head posture and the camera focal length of the pod module 13 step by step to lock and amplify the target and capture the monitored image of the target.
When the pod module 13 is in the initial and locked postures, the camera is set to be in a video mode, and the initial and locked images are video frame extraction images; when the camera is in a snapshot posture, the camera is set to be in a shooting mode, and the monitoring image is a high-definition photo.
The deep learning algorithm transplanted by the recognition and positioning module 14 is a branch of machine learning, the operation generally depends on a GPU, the program is based on a CUDA framework, a model is formed by training and learning to summarize a statistical rule of a large amount of target sample data, the target model is used for target recognition, and the target model in the application is obtained by screening training samples for monitoring targets by professionals and is completed in a special training system.
The airborne flight control module 12 is a core system of the whole flight process of the unmanned aerial vehicle, such as finishing takeoff, air flight, task execution, return recovery and the like, generally comprises three parts, namely a sensor, an airborne computer and a servo action device, and realizes three main functions, namely unmanned aerial vehicle attitude stabilization and control, unmanned aerial vehicle task device management and emergency control.
Because in the process of capturing, positioning and locking a snapshot target, the unmanned aerial vehicle hovers at a fixed point and has position drift and attitude swing, and in addition, the pod has response delay and adjustment precision, the target can shake or even deviate in the camera view field, therefore, the following technical measures are designed in the application:
and designing and configuring the unmanned aerial vehicle and pod initial pose parameters according to the length or area frame ratio of 10-20% of the monitored target in the initial image acquired by the pod module 13 so as to ensure that the monitored target is positioned in the visual field of the camera and the resolution of the monitored target supports the identification and positioning module 14 to identify and capture the monitored target, solve the target pixel coordinate and extract the target characteristic image under the unmanned aerial vehicle pose and pod pose precision.
A locking control module is designed and configured to capture a preset value according to the fact that the length or the area frame ratio of a monitored target in a monitoring image acquired by the pod module 13 is 40-60%, so that the monitored target is located in the center of the camera view field under the unmanned aerial vehicle pose and pod pose accuracy, and the resolution of the monitored target supports manual or intelligent target defect identification.
In the process of locking and amplifying the target, the locking control module 15 is configured to automatically adjust the attitude of the pod module 13 so as to ensure that the monitored target is positioned in the center of the camera view field under the unmanned aerial vehicle attitude and the pod attitude precision.
As shown in fig. 2, the lock control module 15 includes: an outer ring PID controller 151 and an inner ring PID controller 152, where the target pixel coordinate calculated by the identification and positioning module 14 and the target real-time pixel coordinate obtained by the lock control board 15 in the above embodiments can be both used as feedback signals, in this application, it is preferable that the position coordinate of the center point of the target pixel is used as a feedback signal, the position coordinate of the center point of the frame is the position coordinate of the center point of the photographed video image, the position coordinate of the center point of the target pixel and the position coordinate of the center point of the frame are used as input signals of the outer ring PID controller 151, and it may be that the difference between the horizontal direction and the vertical direction is obtained from the position coordinate of the center point of the target pixel and the position coordinate of the center point of the frame, and then the expected movement speed of the target in the frame is obtained by calculation of the outer ring PID controller 151; the expected movement speed and the movement speed of the current target pixel center point are used as input signals of an inner ring PID controller, wherein the movement speed of the target pixel center point is the real-time movement speed of a target, and can be obtained through calculation of position coordinates of the current moment and the previous moment, or can be obtained through direct testing of components such as a sensor, adjustment information of the pod holder is obtained through calculation of an inner ring PID controller 152, the adjustment information of the pod holder comprises a pitch axis PWM signal and a course axis PWM signal, and the pod module 13 adjusts the attitude of the pod holder in real time according to the adjustment information of the pod holder.
In order to prevent the large operation of the actuator due to the large deviation of the signals input by the outer loop PID controller 151 and the inner loop PID controller 152, the deviation signal processing may be performed on the input signals, specifically as follows:
when the deviation of the input signal is larger than a set threshold value, an inverse parabolic signal processing method is adopted, and the threshold value calculation formula is as follows:
wherein linear _ dist is a linear interval threshold, and acc _ max is that the target object is inAnd limiting the maximum acceleration of the moving picture, wherein p is a scaling factor.
When the absolute value of the deviation between the desired moving speed of the target in the frame and the moving speed of the center point of the target pixel is within the linear interval threshold, the input of the inner loop PID controller 152 is the original deviation amount, that is: when the absolute value of the deviation between the desired moving speed of the target in the frame and the moving speed of the center point of the target pixel is greater than the threshold of the linear section, the input of the inner loop PID controller 152 is the value of the original deviation amount after inverse parabolic transformation, and the transformation formula is:
where PID _ input is the input to the inner loop PID controller 152 and error _ rate is the deviation of the desired and actual movement speeds.
Because the vibration and swing in the pose control process of the unmanned aerial vehicle affect the effect of target locking control, the inner loop PID controller 152 adds a feedforward processing link to quickly respond to the change of external disturbance, please refer to fig. 2 continuously, that is, the expected speed of feedforward is added to the input signal, the expected speed of feedforward is calculated by the expected moving speed of the center point of the target pixel at the current moment and the expected moving speed of the center point of the target pixel at the last moment, and specifically, the following steps are included:
the deviation between the expected moving speed of the central point of the target pixel at the current moment and the expected moving speed of the central point of the target pixel at the last moment is subjected to inverse parabolic processing and then proportional operation is carried out, and an expected speed signal for feedforward is calculated, namely:
wherein, PID _ input _ forward is as follows: the desired velocity signal, target _ rate (t), of the feed forward is: the desired speed at the current moment; after adding the desired rate of feed forward in the input signal to inner loop PID controller 152, the final input signal to inner loop PID controller 152 is:
wherein input _ PIDinner_ringIs the final input signal to the inner loop PID controller 152Number; the inner ring PID input signal directly obtains the PWM control signal of the motor after PID operation, and the calculation formula is as follows:
wherein, PWM (n) is a PWM control signal of the motor, Kp is a proportional gain, Ki is an integral gain, and Kd is a differential gain; the PWM control signal directly acts on the brushless motor of the holder, and the motor rotates to change the position coordinates of the target in the picture to form negative feedback.
On the basis of the above embodiment, in order to ensure that the target length or area occupies a proper proportion in the frame, when the time that the target pixel center position continues at the frame center threshold reaches a preset value, the lock control module 15 sends a zoom instruction to the pod module 13, and the pod module 13 changes the camera focal length according to the zoom information; as shown in fig. 3, changing the focal length of the camera can realize zooming by means of an optical lens structure, and an object to be photographed is enlarged and reduced by moving a lens of the camera, so that when an imaging plane moves in a horizontal direction, an angle of view and the focal length change; when the length or area frame ratio of the target pixel reaches a preset threshold, the locking control module 15 sends a photographing instruction to the pod module 13, and the pod module 13 controls the camera to photograph according to the photographing instruction.
The lock control module 15 sends zoom commands to the pod module 13, which may implement different zoom factors according to the target frame ratio, for example: the frame ratio of the target pixel length or area is 10-20%, the optical zoom is performed by 3 times, the optical zoom is performed by 20-40%, and the optical zoom is performed by 1 time when the ratio is 40-60%.
In order to ensure the quality of the shot, namely the definition of the picture, preferably, after the time that the center position of the target pixel is continuously within the frame center threshold and the time that the proportion of the area of the target pixel in the frame area is continuously within the preset threshold reach preset values, the locking control module 15 sends a shooting instruction to the pod module 13 to execute continuous shooting actions, and the locking control module 15 finishes the one-time polling shooting task.
Because the intelligent inspection unmanned aerial vehicle nacelle system of static target monitoring of this application can whole intelligent flight and catch, lock, take a candid photograph the part target, improve system work efficiency by a wide margin, reduce personnel's technical skill requirement, patrol and examine with current manual work and control unmanned aerial vehicle of flying to control and have apparent technological and economic advantage in the aspect of monitoring quality, efficiency of patrolling and examining and operation safety etc..
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (8)
1. An intelligent routing inspection unmanned aerial vehicle pod system for static target monitoring, comprising:
the system comprises a ground station inspection task module, an airborne flight control module, a nacelle module, an identification positioning module and a locking control module;
the ground station inspection task module presets a planning route corresponding to an inspection task, an unmanned aerial vehicle and an initial pose parameter of the nacelle module, sends the planning route and the initial pose parameter to the airborne flight control module, controls the unmanned aerial vehicle to hover and the attitude of the nacelle module, and acquires an initial image of a monitoring target;
the pod module comprises a sensor and a multi-degree-of-freedom cradle head, executes instructions of the airborne flight control module and the locking control module, controls and adjusts the attitude of the multi-degree-of-freedom cradle head and the focal length of a lens of the sensor, sequentially enters an initial attitude, a locking attitude and a capturing attitude, and obtains photos or video images of a target and defects and risks of the target, wherein the photos or video images comprise initial, locking and monitoring images;
the identification positioning module is matched with an embedded computing platform transplanted with a deep learning algorithm, intelligently identifies the monitoring target in the initial image acquired by the pod module, calculates the target pixel coordinate, extracts the target characteristic image and sends the target characteristic image to the locking control module;
the locking control module is used for matching a real-time target locking image acquired by the pod module by taking the target pixel coordinate resolved by the identification and positioning module and the extracted target characteristic image as templates, resolving the real-time pixel coordinate and the picture proportion of the target, and controlling and adjusting the cloud deck attitude and the camera focal length of the pod module step by step to lock and amplify the target and capture a monitoring image of the target;
the lock control module includes: an outer loop PID controller and an inner loop PID controller; the position coordinates of the central point of the target pixel and the position coordinates of the central point of the frame are used as input signals of the outer ring PID controller, and the expected movement speed of the target in the frame is obtained through calculation of the outer ring PID controller; the expected movement speed and the movement speed of the current target pixel center point are used as input signals of the inner ring PID controller, and the adjustment information of the pod module holder is obtained through calculation of the inner ring PID controller;
the offset signal processing may be specifically performed on the input signal as follows:
when the deviation of the input signal is larger than a set threshold value, an inverse parabolic signal processing method is adopted, and the threshold value calculation formula is as follows:
wherein linear _ dist is a linear interval threshold, acc _ max is the maximum acceleration limit of the target object moving in the frame, and p is a scale factor;
when the target is in the pictureWhen the absolute value of the deviation between the expected movement speed and the movement speed of the central point of the target pixel is within the linear interval threshold, the input of the inner ring P ID controller is the original deviation amount, namely: and when the absolute value of the deviation between the expected movement speed of the target in the frame and the movement speed of the center point of the target pixel is greater than the threshold value of a linear interval, the input of the inner loop PID controller is the value of the original deviation amount after inverse parabolic transformation, and the transformation formula is as follows:
wherein PID _ input is the input to the inner loop PID controller, error _ rate is the deviation of the desired and actual movement speeds;
adding a feedforward expected speed in the input signal, wherein the feedforward expected speed is calculated by the current time target pixel central point expected moving speed and the last time target pixel central point expected moving speed, and the specific steps are as follows:
the deviation between the expected moving speed of the central point of the target pixel at the current moment and the expected moving speed of the central point of the target pixel at the last moment is subjected to inverse parabolic processing and then proportional operation is carried out, and an expected speed signal for feedforward is calculated, namely:
wherein, PID _ input _ forward is as follows: the desired velocity signal, target _ rate (t), of the feed forward is: the desired speed at the current moment; after adding the desired rate of feed forward in the input signal to the inner loop PID controller, the final input signal to the inner loop PID controller is: input _ PIDinner_ringPID _ input + PID _ input _ forward, wherein input _ PIDinner_ringIs the final input signal of the inner loop PID controller; the inner ring PID input signal directly obtains the PWM control signal of the motor after PID operation, and the calculation formula is as follows:
wherein, PWM (n) is a PWM control signal of the motor, Kp is a proportional gain, Ki is an integral gain, and Kd is a differential gain; the PWM control signal will directly actAnd a brushless motor of the holder is used, and the motor rotates to change the position coordinates of the target on the picture to form negative feedback.
2. The intelligent inspection drone pod system for static object monitoring of claim 1 wherein the sensors of the pod module are one or more of visible, infrared, ultraviolet, and multispectral sensors with fixed focus or zoom lenses.
3. The intelligent inspection unmanned aerial vehicle pod system for static object monitoring according to claim 1, wherein under the unmanned aerial vehicle pose and pod pose accuracy, in an initial image obtained by the pod module executing the onboard flight control module instruction, the length or area of a monitored object is positioned in the pod visual field and is drawn by 10-20%, and the identification and positioning module identifies and captures the monitored object, calculates the pixel coordinate of the object and extracts the characteristic image of the object; in a monitoring image obtained by the pod module executing the locking control module instruction, the length or the area of a monitoring target is positioned in the field of view of the pod and is drawn by 40-60%.
4. The intelligent inspection unmanned aerial vehicle pod system for static object monitoring according to claim 1, wherein the recognition and positioning module recognizes and captures the monitored object through a deep learning algorithm and utilizing an object training model and the onboard flight control module instructions.
5. The intelligent inspection drone pod system for static object monitoring of claim 1 wherein the input signals to the inner loop PID controller further include: the expected moving speed of the central point of the target pixel at the current moment and the expected moving speed of the central point of the target pixel at the last moment.
6. The intelligent inspection unmanned aerial vehicle pod system for static target monitoring according to claim 1, wherein the lock control module sends a zoom command to the pod module when a target pixel center position reaches a preset value for a duration of a frame center threshold, and the pod module changes a lens focal length according to zoom information.
7. The intelligent inspection unmanned aerial vehicle pod system for static target monitoring according to claim 1, wherein the locking control module sends a snapshot monitoring image instruction to the pod module when the target frame occupancy reaches a preset threshold range and the target pixel center position continues to reach a preset value within the frame center threshold time.
8. The intelligent inspection drone pod system for static object monitoring of claim 7 wherein the preset threshold range is 40% to 60%.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810651606.XA CN108803668B (en) | 2018-06-22 | 2018-06-22 | Intelligent inspection unmanned aerial vehicle nacelle system for static target monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810651606.XA CN108803668B (en) | 2018-06-22 | 2018-06-22 | Intelligent inspection unmanned aerial vehicle nacelle system for static target monitoring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108803668A CN108803668A (en) | 2018-11-13 |
| CN108803668B true CN108803668B (en) | 2021-08-24 |
Family
ID=64084560
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810651606.XA Active CN108803668B (en) | 2018-06-22 | 2018-06-22 | Intelligent inspection unmanned aerial vehicle nacelle system for static target monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108803668B (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109765939A (en) * | 2018-12-21 | 2019-05-17 | 中国科学院自动化研究所南京人工智能芯片创新研究院 | Cloud platform control method, device and the storage medium of unmanned plane |
| CN109977924A (en) * | 2019-04-15 | 2019-07-05 | 北京麦飞科技有限公司 | For real time image processing and system on the unmanned plane machine of crops |
| CN110312071B (en) * | 2019-04-30 | 2021-02-02 | 中国联合网络通信集团有限公司 | Control method of unmanned aerial vehicle and unmanned aerial vehicle |
| CN110633629A (en) * | 2019-08-02 | 2019-12-31 | 广东电网有限责任公司清远供电局 | Power grid inspection method, device, equipment and storage medium based on artificial intelligence |
| CN110351483B (en) * | 2019-08-05 | 2024-09-17 | 深圳职业技术大学 | Control method of multi-lens self-adaptive zooming unmanned aerial vehicle monitoring platform |
| CN110602484B (en) * | 2019-08-29 | 2021-07-27 | 海南电网有限责任公司海口供电局 | Online checking method for shooting quality of power transmission line equipment |
| CN110825110A (en) * | 2019-11-13 | 2020-02-21 | 昆明能讯科技有限责任公司 | Acquisition flight method for power line visible light point cloud resolving photo |
| CN110879602B (en) * | 2019-12-06 | 2023-04-28 | 安阳全丰航空植保科技股份有限公司 | Unmanned aerial vehicle control law parameter adjusting method and system based on deep learning |
| CN112585946A (en) * | 2020-03-27 | 2021-03-30 | 深圳市大疆创新科技有限公司 | Image shooting method, image shooting device, movable platform and storage medium |
| CN111669501B (en) * | 2020-06-18 | 2021-10-22 | 南方电网数字电网研究院有限公司 | Shooting method and device based on unmanned aerial vehicle, computer equipment and medium |
| CN111860461B (en) * | 2020-08-05 | 2024-02-27 | 西安应用光学研究所 | Autonomous zooming method for built-in optical sensor of photoelectric pod |
| CN112363176B (en) * | 2020-11-06 | 2023-08-04 | 日立楼宇技术（广州）有限公司 | Elevator hoistway inspection and modeling method and device and inspection and modeling system |
| WO2022126375A1 (en) * | 2020-12-15 | 2022-06-23 | 深圳市大疆创新科技有限公司 | Zooming method, imaging device, gimbal and movable platform |
| CN113415433B (en) * | 2021-07-30 | 2022-11-29 | 成都纵横大鹏无人机科技有限公司 | Pod attitude correction method and device based on three-dimensional scene model and unmanned aerial vehicle |
| CN114511551A (en) * | 2022-02-22 | 2022-05-17 | 金华高等研究院(金华理工学院筹建工作领导小组办公室) | Ground damage identification system based on machine vision |
| CN114564049B (en) * | 2022-03-21 | 2024-06-14 | 厦门大学 | Unmanned aerial vehicle wide area search device and method based on deep learning |
| CN114743111A (en) * | 2022-04-12 | 2022-07-12 | 国网山东省电力公司经济技术研究院 | Power transmission and transformation construction hidden danger inspection method and system based on unmanned aerial vehicle AI double shooting |
| CN116301016A (en) * | 2022-12-02 | 2023-06-23 | 深圳市布雷斯塔科技有限公司 | A clear photographing method of UAV for automatic power inspection |
| CN116543619B (en) * | 2023-07-04 | 2023-08-29 | 中国科学院长春光学精密机械与物理研究所 | UAV photoelectric pod simulation training system |
| CN117750204B (en) * | 2023-11-14 | 2024-09-24 | 东南大学 | Method and device for visual same-speed tracking and shooting of moving targets on a conveyor line |
| CN120406557B (en) * | 2025-07-07 | 2025-09-26 | 北京凌云光子技术有限公司 | Intelligent nacelle control method and system for unmanned aerial vehicle |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0232415A (en) * | 1988-07-21 | 1990-02-02 | Fujitsu Ltd | Displaying controller for detecting movement of pointing device |
| CN101811578A (en) * | 2010-04-23 | 2010-08-25 | 福建省电力有限公司福州电业局 | Special photoelectric nacelle of power patrol unmanned helicopter |
| CN103149939A (en) * | 2013-02-26 | 2013-06-12 | 北京航空航天大学 | Dynamic target tracking and positioning method of unmanned plane based on vision |
| CN105578101A (en) * | 2015-12-23 | 2016-05-11 | 中国南方电网有限责任公司超高压输电公司检修试验中心 | Helicopter photoelectric pod storage system and implementation method thereof |
| CN106005455A (en) * | 2016-08-08 | 2016-10-12 | 北京宇鹰科技有限公司 | Two-axis pod system based on geographic coordinate system pointing control |
| CN205920414U (en) * | 2016-07-29 | 2017-02-01 | 苏州天地衡遥感科技有限公司 | Machine carries optoelectronic pod platform |
| CN106502259A (en) * | 2016-11-21 | 2017-03-15 | 国网山东省电力公司电力科学研究院 | Electric inspection process low profile photovoltaic gondola control device, gondola, unmanned plane and method |
| CN106647814A (en) * | 2016-12-01 | 2017-05-10 | 华中科技大学 | System and method of unmanned aerial vehicle visual sense assistant position and flight control based on two-dimensional landmark identification |
| CN106774436A (en) * | 2017-02-27 | 2017-05-31 | 南京航空航天大学 | The control system and method for the rotor wing unmanned aerial vehicle tenacious tracking target of view-based access control model |
| CN107203184A (en) * | 2017-06-20 | 2017-09-26 | 南京理工大学 | The dynamic control method of straight line steering wheel Electric Loading System |
| CN107544531A (en) * | 2017-09-27 | 2018-01-05 | 成都纵横自动化技术有限公司 | Line data-logging method, apparatus and unmanned plane |
| US9886871B1 (en) * | 2011-12-27 | 2018-02-06 | PEAR Sports LLC | Fitness and wellness system with dynamically adjusting guidance |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
| CN108107920A (en) * | 2017-12-19 | 2018-06-01 | 天津工业大学 | A kind of microminiature twin shaft vision stablizes holder target detection tracing system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102183955A (en) * | 2011-03-09 | 2011-09-14 | 南京航空航天大学 | Transmission line inspection system based on multi-rotor unmanned aircraft |
| CN104796672B (en) * | 2015-05-09 | 2018-04-20 | 合肥工业大学 | A kind of unmanned plane is met an urgent need monitoring head device |
| CN204992418U (en) * | 2015-09-22 | 2016-01-20 | 南方电网科学研究院有限责任公司 | An automatic inspection device for transmission line defects of unmanned aerial vehicles |
| CN105959181B (en) * | 2016-06-17 | 2019-07-12 | 国网山东省电力公司电力科学研究院 | It is a kind of for examining the system and method for detecting power patrol unmanned machine |
| CN106741008B (en) * | 2016-12-29 | 2020-11-06 | 北京交通大学 | Railway line foreign matter identification method and system |
-
2018
- 2018-06-22 CN CN201810651606.XA patent/CN108803668B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0232415A (en) * | 1988-07-21 | 1990-02-02 | Fujitsu Ltd | Displaying controller for detecting movement of pointing device |
| CN101811578A (en) * | 2010-04-23 | 2010-08-25 | 福建省电力有限公司福州电业局 | Special photoelectric nacelle of power patrol unmanned helicopter |
| US9886871B1 (en) * | 2011-12-27 | 2018-02-06 | PEAR Sports LLC | Fitness and wellness system with dynamically adjusting guidance |
| CN103149939A (en) * | 2013-02-26 | 2013-06-12 | 北京航空航天大学 | Dynamic target tracking and positioning method of unmanned plane based on vision |
| CN105578101A (en) * | 2015-12-23 | 2016-05-11 | 中国南方电网有限责任公司超高压输电公司检修试验中心 | Helicopter photoelectric pod storage system and implementation method thereof |
| CN205920414U (en) * | 2016-07-29 | 2017-02-01 | 苏州天地衡遥感科技有限公司 | Machine carries optoelectronic pod platform |
| CN106005455A (en) * | 2016-08-08 | 2016-10-12 | 北京宇鹰科技有限公司 | Two-axis pod system based on geographic coordinate system pointing control |
| CN106502259A (en) * | 2016-11-21 | 2017-03-15 | 国网山东省电力公司电力科学研究院 | Electric inspection process low profile photovoltaic gondola control device, gondola, unmanned plane and method |
| CN106647814A (en) * | 2016-12-01 | 2017-05-10 | 华中科技大学 | System and method of unmanned aerial vehicle visual sense assistant position and flight control based on two-dimensional landmark identification |
| CN106774436A (en) * | 2017-02-27 | 2017-05-31 | 南京航空航天大学 | The control system and method for the rotor wing unmanned aerial vehicle tenacious tracking target of view-based access control model |
| CN107203184A (en) * | 2017-06-20 | 2017-09-26 | 南京理工大学 | The dynamic control method of straight line steering wheel Electric Loading System |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
| CN107544531A (en) * | 2017-09-27 | 2018-01-05 | 成都纵横自动化技术有限公司 | Line data-logging method, apparatus and unmanned plane |
| CN108107920A (en) * | 2017-12-19 | 2018-06-01 | 天津工业大学 | A kind of microminiature twin shaft vision stablizes holder target detection tracing system |
Non-Patent Citations (3)
| Title |
|---|
| Research on stabilizing and tracking control system of tracking and sighting pod;Shanzhong Liu;《Journal of Control Theory and Applications》;20111228;第10卷;第107-112页 * |
| The Control System Design of the Pod Based on Fuzzy-PID;Bao Gui Wang;《Applied Mechanics and Materials》;20140430;第615卷;第52-57页 * |
| 基于PID的电力巡检光电吊舱稳定平台控制系统设计;王柯;《控制工程》;20150731;第135-140页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108803668A (en) | 2018-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108803668B (en) | Intelligent inspection unmanned aerial vehicle nacelle system for static target monitoring | |
| CN111272148B (en) | Adaptive imaging quality optimization method for autonomous inspection of transmission lines by unmanned aerial vehicle | |
| CN112164015B (en) | Monocular vision autonomous inspection image acquisition method, device and power inspection UAV | |
| CN112585554B (en) | UAV inspection method, device and UAV | |
| CN104865971B (en) | The control method and unmanned plane of a kind of polling transmission line unmanned plane | |
| CN107729808B (en) | An image intelligent acquisition system and method for unmanned aerial vehicle inspection of transmission lines | |
| CN106468918B (en) | Standardized data acquisition method and system for line inspection | |
| CN112904877A (en) | Automatic fan blade inspection system and method based on unmanned aerial vehicle | |
| CN108229587B (en) | Autonomous transmission tower scanning method based on hovering state of aircraft | |
| Gu et al. | Autonomous wind turbine inspection using a quadrotor | |
| CN112711267B (en) | Unmanned aerial vehicle autonomous inspection method based on RTK high-precision positioning and machine vision fusion | |
| CN102809969A (en) | Unmanned aerial vehicle control system and method | |
| Vidal et al. | UAV vision aided positioning system for location and landing | |
| Ohta et al. | Image acquisition of power line transmission towers using UAV and deep learning technique for insulators localization and recognition | |
| US20230366775A1 (en) | Method, aerial vehicle and system for detecting a feature of an object with a first and a second resolution | |
| JP2024129048A (en) | Inspection Systems | |
| CN113273173A (en) | Inspection method and device for movable platform, movable platform and storage medium | |
| Xi et al. | A vision-based inspection strategy for large-scale photovoltaic farms using an autonomous UAV | |
| CN114967731A (en) | An automatic search method for field personnel based on UAV | |
| CN116310891A (en) | A cloud-edge collaborative transmission line defect intelligent detection system and method | |
| CN109765931B (en) | Near-infrared video automatic navigation method suitable for breakwater inspection unmanned aerial vehicle | |
| CN114326771A (en) | Unmanned aerial vehicle shooting route generation method and system based on image recognition | |
| CN112585945A (en) | Focusing method, device and equipment | |
| CN112257514B (en) | Infrared vision intelligent detection shooting method for equipment fault inspection | |
| CN119759075A (en) | Distribution network line drone inspection auxiliary photography method and device based on improved YOLOV5 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20191121 Address after: 510507 floor 9-11, A4, No. 181, science Avenue, Science City, Luogang District, Guangzhou City, Guangdong Province Applicant after: Guangzhou Bureau of Extra High Voltage Transmission Company of China Southern Power Grid Co., Ltd. Address before: 101399 silicon Shield Technology Park, 10 South Street, Shunyi District military camp, Beijing Applicant before: Aerospace Scene (Beijing) Technology Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |