CN100565557C - Infrared Human Target Tracking System Based on Particle Dynamic Sampling Model - Google Patents

Infrared Human Target Tracking System Based on Particle Dynamic Sampling Model Download PDF

Info

Publication number
CN100565557C
CN100565557C CNB2008100698107A CN200810069810A CN100565557C CN 100565557 C CN100565557 C CN 100565557C CN B2008100698107 A CNB2008100698107 A CN B2008100698107A CN 200810069810 A CN200810069810 A CN 200810069810A CN 100565557 C CN100565557 C CN 100565557C
Authority
CN
China
Prior art keywords
target
particle
tracking
particles
human body
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CNB2008100698107A
Other languages
Chinese (zh)
Other versions
CN101303726A (en
Inventor
郭永彩
云廷进
高潮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University
Original Assignee
Chongqing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University filed Critical Chongqing University
Priority to CNB2008100698107A priority Critical patent/CN100565557C/en
Publication of CN101303726A publication Critical patent/CN101303726A/en
Application granted granted Critical
Publication of CN100565557C publication Critical patent/CN100565557C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The invention provides a kind of system for tracking infrared human body target based on corpuscle dynamic sampling model, this system comprises thermal infrared imager, image collecting device, central processing unit and video display devices; Image collecting device is gathered the video data stream of thermal infrared imager and is given central processing unit, and central processing unit is formed the consecutive image sequence with the infrared video data and image sequence is followed the tracks of.Tracking is realized by random particles sampling and migration control, is object with each human body target, and the observation particle according to each target produces the stochastic sampling particle again, and according to the judgment rule of the invalid sampling particle of setting, removes unnecessary particle; The target of following the tracks of in the pond is carried out condition judgement; Tracking results after central processing unit is handled is sent into video display devices and is shown; Adopt the OO technology complex target state of human body is encapsulated, overcome the conventional target tracking and can't realize reliable tracking problem human body target in the infrared image sequence.

Description

System for tracking infrared human body target based on corpuscle dynamic sampling model
Technical field
The present invention relates to the tracker of human body target, be specifically related to a kind of tracking infrared human body target technology based on corpuscle dynamic sampling model and object-oriented method.
Background technology
Compare with visual light imaging, the advantage of infrared imaging is self-evident: do not need secondary light source, almost can carry out all weather operations under any meteorological condition and environment.Along with the particularly development of the infrared CCD manufacturing technology of non-refrigeration focal surface and the decline of price of infrared device manufacturing technology, the application of infrared imaging system is expanded rapidly.
Human body target is the key element of the most active and most worthy in the scene, and therefore the field that the tracking technique of complicated non-rigid body targets such as human body target in the infrared image is used is very extensive.Main application comprises: various aspects such as infrared target system, infrared navigation system, infrared life saving system, frontier defense warning system, intelligent monitor system, automobile DAS (Driver Assistant System) at night, fire control and disaster rescue and public security, night patrol law enforcement backup system, Intelligent Human-Machine Interface system, Intelligent Robots Navigation System.Yet, because human body is non-rigid body, attitude complicated and changeable, motion is subjective random strong, do not have fixedly the characteristics of motion and can follow, even make under the visible light condition, the human body target in the scene is followed the tracks of also very difficult.The tracking of the human body target in the infrared video also will be subjected to the restriction of the defective of infrared image own simultaneously: infrared image is a gray level image, image quality is poor, and resolution is low, objective fuzzy, grain details is few, makes that the human body tracking algorithm in some visible lights can't be used for infrared tracking.
Sum up and adopt the main difficulty of infrared imagery technique, comprise the following aspects human body target tracking:
(1) human body is non-rigid body target, has attitude complicacy and polytrope
As everyone knows, rigid body such as human body target and automobile, aircraft destination properties is different: for the rigid body target, self shape of target can not change, when target changes with respect to the observer visual angle, be attitude when changing, can determine this variation by a definite affine maps equation, the clarification of objective point, still keep its stability as edge angle point, texture characteristic points, can come target is followed the tracks of with these characteristic areas.Optical flow method as classics is followed the tracks of, and by separating optical flow equation, determines the movement locus of target.For human body target, situation is many with regard to complexity.At first, the human body target attitude has diversity, can't portray all human body targets with fixing template, and the description of unique point and characteristic area does not have unitarity; Secondly, human body target is except participating in based on the change that also is accompanied by own attitude the motion of surrounding environment, even same target, the size of current time and previous moment target, pattern, attitude may be inequality fully, therefore, also can't be under a lot of situations in the different moment to same target, the method for use characteristic point coupling is determined the mapping of target front and back state, makes traditional tracking optical flow method based on the rigid body target be restricted the pacing items that does not possess realization.
(2) motion of human body target is subjective random
Because the height of human body target is subjective random, target may can change movement velocity, direction of motion, forms of motion and motion path at any time at any time under the unforeseen situation of observer, can not set up a suitable exercise model and summarize people's this behavior, in most cases just, the motion of human body target does not have any rule and can follow, and can not use the motion model with stationary power system to portray the characteristics of motion of human body.This just makes some based on trackings that the characteristics of motion is supposed, can not follow the tracks of human body target effectively as kalman filter method.
(3) defective of infrared imaging self
Because infrared image is a gray level image, aimless colouring information, the tracking based on color characteristics such as colour of skin clothing of widespread use can't use in the visible images; Because the human body target grain details is less in the infrared image, the local grain of target is expressed does not have good identifiability, some non-rigid body method for tracking target that calculate by the texture similarity in the visible images method for tracking target, as traditional Mean Shift tracking, carry out similarity analysis by former and later two regional Bhattacharyya coefficients, with the track window port area of Bhattacharyya coefficient maximum as matching area as matching area, also inapplicable in the tracking infrared human body target process, this is because the grey level histogram feature of different target is all very approaching, the characteristic quantity that is used to judge the difference between the different target is less, and the possibility that the Bhattacharyya coefficient of use grey level histogram carries out similar coupling appearance mistake coupling is very big.
(4) the not reliability of human body target detection algorithm
Because human body target changes the detection algorithm that causes unreliable except self attitude is complicated, the human body target image is owing to be subjected to the interference of background, incomplete and broken phenomenon often appear in target image, lost the essential characteristic of human body target fully, even detection algorithm is absolutely reliable, also can't identify human body target this moment, and therefore complete tracking scheme based on detection algorithm is infeasible.
Tracking to the infrared human body target all is to be based upon on the basis that the state of target is supposed both at home and abroad: move with uniform velocity as hypothetical target, the attitude of hypothetical target is that erectility (pedestrian), supposition video camera are static etc., use Kalman filtering, particle filter or Mean Shift method are carried out the tracking based on status predication method and similarity analysis, and the application scenario is very limited and tracking results is undesirable.
Summary of the invention
At existing tracking infrared human body target technology above shortcomings, the purpose of this invention is to provide a kind ofly based on corpuscle dynamic sampling model and object-oriented method, improve to follow the tracks of the system for tracking infrared human body target of reliability.
The object of the present invention is achieved like this: based on the system for tracking infrared human body target of corpuscle dynamic sampling model, this system comprises thermal infrared imager, image collecting device, central processing unit and video display devices; Image collecting device is gathered the video data stream of thermal infrared imager and is given central processing unit, and central processing unit is formed the consecutive image sequence with the infrared video data stream and image sequence is handled; Native system is controlled the tracking that realizes target by random particles sampling and migration, with each human body target is object, and the observation particle collection according to each target produces the stochastic sampling particle again, and, remove unnecessary particle according to the judgment rule of the invalid sampling particle of setting; The target of following the tracks of in the pond is carried out condition judgement, realize the human body target in the image sequence is accurately followed the tracks of; Tracking results is sent into video display devices and is shown, this central processing unit realizes that the step of human body target tracking is as follows:
(1) target detection: monitoring video data stream, judge whether to call the static human algorithm of target detection, catch the target that enters the visual field, and check target and follow the tracks of the matching state of the tracking sign in the pond, if do not need to call the static human algorithm of target detection, then change step (10) over to;
(2) by calling the static human object detection method human body target is detected;
(3) detect current time and whether comprise fresh target, if do not find that fresh target then changes step (10) over to;
(4) target following initialization: utilize the dynamic particle modeling technique of human body target that human body target is carried out tracking initiation, this technology dispenses random particles and samples in detected human body target correspondence image regional extent, move the modeling of finishing target by particle Mean Shift, produce observation particle collection, finish the target following initialization;
(5) add the tracking sign: add to follow the tracks of identifying to the fresh target after the initialization, and preserve the fresh target characteristic attribute;
(6) fresh target joins and follows the tracks of the pond, and the state machine of fresh target is set;
(7) following the tracks of pond internal object status checking upgrades: follow the tracks of constantly at next, with each human body target is object, observation particle according to each target, again produce the stochastic sampling particle, and, utilize the elimination method of invalid particle to remove invalid tracking particle according to the judgment rule of the invalid sampling particle of setting; Dbjective state is judged: the determination methods that adopts target state machine is carried out state and is judged following the tracks of target in the pond, and the main state of target has that target is normal, target is hidden, target disappears and target is blocked adhesion mutually;
(8) obtain the target current state;
(9) carry out video according to current state and show, change step (1) again over to;
Whether be empty, if follow the tracks of Chi Weikong, change step (9) over to if (10) detecting the tracking pond;
(11) each target of following the tracks of in the pond is followed the tracks of, change step (7) over to;
The elimination method of the invalid particle in the described step (7) is to determine effective particle minimal gray threshold value according to the pixel grey scale distribution of previous moment observation particle position correspondence, current time random particles position is rejected for the particle that gray-scale value is lower than this threshold value, guarantee that most of particle that adopts all drops on the target.
The determination methods of the target state machine in the described step (7) comprises:
The judgement that target is hidden and target disappears: observe the decision of particle position correspondence image gray difference constantly according to front and back; When gray difference during greater than setting threshold, target is considered to change over to hidden state; When changing hidden state near visual field border, continuous certain time of hidden state or target think that target disappears;
The judgement of target occlusion adhesion: the condition flag that target is blocked by background is to hide; Just definite target is blocked fully mutually when two or more trace labellings drop on the same target.
Target is judged normally: be default setting;
Compared to existing technology, the present invention has following advantage:
(1) the present invention has used for reference the thought of traditional particle filter algorithm, use the random particles Sampling techniques to obtain the model of target, but it is different with traditional particle filter tracking algorithm based on prediction, the present invention is not the coupling tracking that realizes target by the posteriority similarity relation of object module, but realizes target following by the actual migration process of control sampling particle.Experimental results show that, the present invention only adopts minority to adopt particle can realize reliable tracking to target, calculated amount is little, tracking can be carried out in real time, having avoided traditional particle filter algorithm is to avoid particle to degenerate and particle exhausts the posteriority distribution probability that adopts a large amount of sampling particles to calculate target, makes that the calculated amount of system is overweight and real-time is poor.
(2) take random sampling technique dynamically to set up object module, can between global follow state and local tracking mode, change automatically.When being lost the complete resemblance of target by partial occlusion, target still can reliably follow the tracks of.
(3) tracking is operated at each target, takes Object-oriented Technique to encapsulate the essence of outstanding tracking problem; The method that is proposed and the characteristics of motion of target are irrelevant, have solved because target travel subjective random makes the problem based on traditional track algorithm inefficacy of characteristics of motion hypothesis such as Kalman filtering; It is irrelevant whether the method that is proposed and the motion state of target and background change, and can reliably follow the tracks of target under the situation arbitrarily, solved the problem that can't be used for dynamic background and static pedestrian is followed the tracks of based on the motion segmentation tracking target.
(4) introduce the finite state machine technology, solved changeablely or block to hide and cause the target detection failure because of targeted attitude, based target detects the unreliable problem of track algorithm, has improved the robustness of tracker.
(5) this method can be widely used in fields such as military affairs, safety, intelligent monitoring, driver assistance, fire control and disaster rescue, man-machine interface and intelligent robot navigation, and potential economic worth and social value are very big.
Description of drawings
Fig. 1 is the main process flow diagram of the inventive method operational process;
Fig. 2 is an enforcement illustration of setting up trace model;
Wherein, (a) target image, (b) state model (c) measures model, (d) track center;
Fig. 3 is the change procedure that adopts trace labelling in the target occlusion process of the present invention;
Fig. 4 judges the method whether target occlusion takes place according to the position relation of trace labelling;
Wherein, (a) do not block, (b) block;
Fig. 5 is the matching relationship of target and trace labelling.
Embodiment
The present invention is from OO angle, detect human body target after, the method that adopts the convergence state of target area dynamic random sampling particle migration that target is observed has realized the tracking to human body target in the infrared video.
Referring to Fig. 1, the present invention is based on the system for tracking infrared human body target of corpuscle dynamic sampling model, this system comprises thermal infrared imager, image collecting device, central processing unit and video display devices; Image collecting device is gathered the video data stream of thermal infrared imager and is given central processing unit, and central processing unit is formed the consecutive image sequence with the infrared video data stream and image sequence is handled; Native system is controlled the tracking that realizes target by random particles sampling and migration, with each human body target is object, and the observation particle collection according to each target produces the stochastic sampling particle again, and, remove unnecessary particle according to the judgment rule of the invalid sampling particle of setting; The target of following the tracks of in the pond is carried out condition judgement, realize the human body target in the image sequence is reliably followed the tracks of; Tracking results is sent into video display devices and is shown, this central processing unit realizes that the step of human body target tracking is as follows:
(1) target detection: monitoring video data stream, judge whether to call the static human algorithm of target detection, catch the target that enters the visual field, and check target and follow the tracks of the matching state of the tracking sign in the pond, if do not need to call the static human algorithm of target detection, then change step (10) over to;
(2) by the static human object detection method human body target is detected;
(3) detect current time and whether comprise fresh target, if do not find that fresh target then changes step (10) over to;
(4) target following initialization: utilize the dynamic particle modeling technique of human body target that human body target is carried out tracking initiation, this technology dispenses random particles and samples in detected human body target correspondence image regional extent, move the modeling of finishing target by particle Mean Shift, produce observation particle collection, finish the target following initialization;
(5) add the tracking sign: add to follow the tracks of identifying to the fresh target after the initialization, promptly add ID number to fresh target, and preservation fresh target characteristic attribute;
(6) fresh target joins and follows the tracks of the pond, and the state machine of fresh target is set;
(7) following the tracks of pond internal object status checking upgrades: follow the tracks of (this tracking time minimum interval constantly at next, it can be the time of a two field picture, it also can be the time of setting), with each human body target is object, observation particle according to each target, again produce the stochastic sampling particle, and, utilize the elimination method of invalid particle to remove invalid tracking particle according to the judgment rule of the invalid sampling particle of setting; Dbjective state is judged: the determination methods that adopts target state machine is carried out state and is judged following the tracks of target in the pond, and the main state of target has that target is normal, target is hidden, target disappears and target is blocked adhesion mutually;
(8) obtain the target current state;
(9) carry out video according to the target current state and show, change step (1) again over to;
Whether be empty, if follow the tracks of Chi Weikong, change step (9) over to if (10) detecting the tracking pond;
(11) each target of following the tracks of in the pond is followed the tracks of, change step (7) over to;
The elimination method of the invalid particle in the described step (7) is the pixel grey scale distribution according to previous moment observation particle position correspondence, to the grey scale pixel value of all observation particle correspondences of target correspondence according to ordering from high to low, choose the mean value of the higher front m gray-scale value of gray scale by setting number of particles or definite number percent and determine effective particle minimal gray threshold value (wherein, m is a natural number), may be to overcome picture noise to the adverse effect of selection of threshold, the particle that current time random particles position corresponding grey scale value is lower than this threshold value is rejected, and guarantees that most of particle that adopts all drops on the target; For guaranteeing carrying out smoothly of follow-up tracking work, when the tracking particle of all new generations did not satisfy the threshold value qualifications, the number lower limit of setting effective particle collection was at least 1, chose effective particle according to its corresponding gradation of image value is descending.
The determination methods of the target state machine in the described step (7) comprises: the judgement that target is hidden and target disappears: observe the decision of particle position correspondence image gray difference constantly according to front and back; When gray difference during greater than setting threshold, target is considered to change over to hidden state; When changing hidden state near visual field border, continuous certain time of hidden state (can the value of setting up on their own, also can be fixed according to the sampling frame number, or set according to own needs) or target think that target disappears;
The judgement of target occlusion adhesion: the condition flag that target is blocked by background is to hide; Just definite target is blocked fully mutually when two or more trace labellings drop on the same target.
Target is judged normally: be default setting;
The static human object detection method that the present invention relates to adopts prior art, once delivers paper with regard to the method as the inventor.Dispensing random particles is prior art, promptly is the Monte Carlo sampling method.Mean Shift is the meaning that average shifts, and is a kind of algorithm well known in the art.Described tracking pond is an object container, and size is equal to the target number in the current scene.State machine is the generic noun in this area, promptly is the residing state of target.
Below the tracking infrared human body target technology that the present invention is based on corpuscle dynamic sampling model is elaborated:
1, the dynamic particle sampler modeling technique of human body target:
Trace model is divided into two parts: state model and measurement model.The position of zone in image of supposing the initial target place is S=I[ulx, uly; Lrx, lry], for less target, each pixel of target area is placed a particle; For bigger target, dispense M random particles in the target area by certain probability density, be equivalent to the sampling of sampling of the intensity profile of target, sample number is M, reduces calculated amount.
Note sampling particle assembly { P i = ( x p i , y p i ) , x p i ∈ [ ulx , lrx ] , x p i ∈ [ uly , lry ] } i = 1 M , The gray-scale value that each particle is used its picture position, place uses the Uniform kernel function as eigenwert, carries out Mean Shift convergence and analyzes.Particle is at the diaxon Mean Shift motion vector that makes progress
m x ( x p i , y p i ) = Σ x = x p i - w x p i + w Σ y = y p i - h y p i + h xI ( x , y ) Σ x = x p i - w x p i + w Σ y = y p i - h y p i + h I ( x , y ) - x p i m y ( x p i , y p i ) = Σ x = x p i - w x p i + w Σ y = y p i - h y p i + h yI ( x , y ) Σ x = x p i - w x p i + w Σ y = y p i - h y p i + h I ( x , y ) - y p i
W, h are respectively the bandwidth of used kernel function in the formula, and (x y) is image (x, y) coordinate position pixel corresponding gray, (x to I p i, y p i) be the coordinate position of i particle of particle set, m x(x p i, y p i) and m y(x p i, y p i) be respectively particle at the diaxon Mean Shift motion vector that makes progress.Why adopting Uniform nuclear, is because can quicken the execution of Mean Shift algorithm by integral image for bigger target image.Work as m x(x p i, y p i), m y(x p i, y p i) during less than setting threshold, record convergence position.Because the brightness ratio background luminance of human body target wants high in the infrared image, by Mean Shift convergence of algorithm characteristic as can be known, all particles all near the local maximum migration of grey level probability density function, are promptly moved by the higher position of human body target brightness of background in image.The final convergence position of each particle is by the intensity profile of target and selected bandwidth function decision.
The final convergence location sets of supposing all particles is { P j = ( x p j , y p j ) } j = 1 N , The local maximum position that is the gray scale density function of target can be expressed with N particle.When choosing rational bandwidth function, particle is finally restrained the do not place one's entire reliance upon appearance profile of target of position, use this state model, has extraordinary robustness for non-rigid body targets such as human bodies, when the target part was blocked by background, state model was still effective, just the portrayal of whole object state was converted into portrayal to the target local state, for tracking, just follow the tracks of and be converted into local the tracking by whole object.
For the measurement of dbjective state by realizing to all particles of expressing dbjective state by the relative space position cluster analysis.Use the coordinate of complex vector located each particle of expression { P j = x p j + i * y p j } j = 1 N , Use its position coordinates as eigenwert, all particles are carried out the non-supervision cluster of Mean Shift by relative space position, the Mean Shift vector m (P of two direction of principal axis j) be:
m ( P j ) = Σ P = P j - h P ∈ { P j } P j + h P * g ( | | P - P j h | | 2 ) g ( | | P - P j h | | 2 ) - P j
G (.) is the expression-form of kernel function in the formula, and h is the radius of kernel function.As m (P j) when being zero, cluster is finished.Suppose convergence of all categories position after the cluster and the number of particles { (v that comprises i, w i) I=1 L, then the measuring value of particle position is { ( x i , y i ) = ( real ( v i ) , imag ( v i ) ) } i = 1 L , The weight coefficient w of each particle weight measured value iDetermine by the number of particles normalization that its corresponding classification comprised:
w i = w i / Σ i = 1 L w i
L is all observation populations in the formula.Use all weighted mean values that measure the particle coordinate positions as movement locus point coordinate position:
Trajctory(x，y)＝(real(v i*w i)，imag(v i*w i))
Referring to Fig. 2 (a) and (b), (c) with (d) for setting up the example of trace model.
2, the generation control method of random particles
To measure particle P j = x j + i × y j ( i = - 1 ) Be example, the propagation particle collection that is produced is:
{PropagateParticles}＝{Δd x*random(-1，1)+(x j+ΔV x)
+i(Δd y*random(-1，1)+(y j+ΔV y))}repeat(M)
Δ d in the formula x, Δ d yBe the sample area range of control, determined the size of detection window,, and use the size of target to carry out loose constraint by the distance decision between current measurement particle and its most contiguous measurement particle.Δ V x, Δ V yEstimate that for the single step displacement of target on two direction of principal axis M is for producing the number of particle.
The number M that propagates particle measures the propagation coefficient decision of particle by this.On the basis of hypothetical target motion continuity, before and after can thinking in two two field pictures skew of the weighting center of target not too large.Therefore, give the bigger weight coefficient of particle that previous frame measures particle weighting center.Reliable method is directly will measure particle weighting center as a new measurement particle and give bigger propagation coefficient, and remaining weight coefficient distributes according to the inverse of the distance at itself and weighting center.Suppose that measuring the weighting center position coordinates is P c, measure particle P i, then the allocation rule of weight coefficient can be:
ws c = C ( 0 < C &le; 1 ) ws i = 1 - C d ( P i , P c ) &Sigma; P j &NotEqual; P c 1 / d ( P j , P c )
D (.) is a distance function in the formula, ws cBe center sampling particle weight coefficient, ws iFor all the other measure i sampling particle P in the particle iWeight coefficient.Generally speaking, the C value can be arranged in 0.3～0.7 scope, when target hour, we do not need even to consider that remaining measures particle, and whole coefficient weights are all distributed to the measurement particle that is in the weighting center, i.e. C=1.
3, the elimination method of invalid particle
Image pixel gray-scale value to observation particle correspondence sorts according to descending, the gray-scale value array that sorts is chosen front m particle corresponding grey scale value by the minimum threshold of the certain percentage of total number of particles or setting to be averaged as basic threshold value, in all random particles the respective pixel gray-scale value greater than this value or therewith the particle that differs in certain positive and negative percentage error scope of value be effective particle, remaining is invalid particle, (wherein, m is a natural number).
For making the smooth execution of algorithm, the minimal amount of particle must guarantee, is used for the judgement of subsequent algorithm target state machine.
4, the definition of objective attribute target attribute, method of operating and related news incident:
4.1 human body target attribute:
4.1.1 sign: the unique identification of target in total system
4.1.2 fundamentals such as the size that target is carved at a time, position, shape, feature description vector, measurement particle information, movement locus;
4.1.3 target is carved residing state machine at a time.
4.1.4 case pointer: record object is from the state that occurs to the whole tracking cycle that disappears, for the objective attribute target attribute maintenance provides pointer.
4.2 the method for operating of human body attribute:
Method of operating mainly realizes the setting of objective attribute target attribute and update method are mainly comprised:
4.2.1 target following state-maintenance: finish parameter initialization, work such as the modification of objective attribute target attribute;
4.2.2 methods such as target following state demonstration: the content of the tracking results that decision will show and display mode.
4.3 message event:
The human body target classification need respond two incidents:
4.3.1 objective attribute target attribute update event:, safeguard dbjective state according to following the tracks of the attribute update event that trigger in the pond;
4.3.2 tracking results presented event: according to following the tracks of the tracking results presented event that trigger in the pond, the current tracking mode of target is shown.
4.4 follow the tracks of the container of pond as all targets in the current scene (target that comprises the state of being hidden), its attribute definition is:
4.4.1 follow the tracks of identification list (Tracker List): the tracking identification number of each target
4.4.2 follow the tracks of sign vernier (Tracker ID Cursor): the number of record all targets since beginning from tracking;
4.4.3 frame counter (Frame Counter): record is when the information of pre-treatment image.
4.4.4 state event record variable: the target following sign of phenomenon takes place to hide and block in record.
4.5 method of operating definition:
Because main tracing process realizes in following the tracks of the pond class, so method of operating definition more complicated:
4.5.1 following the tracks of tabulation safeguards: be responsible for following the tracks of the foundation and the maintenance work in pond.
4.5.2 dbjective state machine check: the state machine to target checks, judges incidents such as target is hidden, blocked, disappearance;
4.5.3 state machine is handled: the disposal route that takes place when the state machine event update of target.
4.5.4 target detection: the target in the current frame image is detected again.
4.6 message event definition:
4.6.1 state update event:, notify corresponding object entity to carry out state and upgrade according to the result of status checking.
4.6.2 tracking results presented event: the notice corresponding object is carried out tracking results and is shown.
5, the judgement of target state machine:
Judgement for hidden state, gradation of image value decision according to observation particle position correspondence, comparative approach is identical with aforementioned invalid particle elimination method, adjacent moment effective particle threshold separately before and after selecting by fixed percentage, by separately effectively the gray difference of particle collection whether surpass threshold limit value and judge whether target is hidden.
Adopt a kind of very simple method for the judgement of shelter target.At first learn about when taken place to block, as shown in Figure 3;
Figure (C) takes as and has taken place to block when two trace labellings all point to same target.When target will be blocked, can judge according to following straightforward procedure: referring to Fig. 4 (a) and (b).
The image-region that two trace labelling Tracker A and Tracker B are surrounded carries out binaryzation, and the result after the binaryzation is carried out the connected region mark, if two Tracker belong to same connected region, thinks that then target has taken place to block.
6, object matching inspection
When target occur to be hidden, need proofread and correct tracking results when occlusion state or target disappear.For the target that disappears, directly it is rejected in following the tracks of pool list, when hiding with occlusion state, then need the result who calls the combining target detection to proofread and correct for target.When here need to solve two problems: (1) is the invocation target detection algorithm; (2) how to proofread and correct tracking results by the target detection result.
For first problem, design two kinds of strategies:
(1) regularly triggers: when Continuous Tracking continues the regular hour, trigger the target detection incident, emerging target is proofreaied and correct and found to tracking results.
(2) initiatively trigger: initiatively trigger according to following the tracks of the state machine of target and the relation decision of the position between the Tracker in the Tracker tabulation in the pond.In Tracker tabulation, exist position distance between vanishing target or certain the several Tracker less than setting threshold, in the time of may existing target to block mutually, should trigger the target detection incident, tracking results is proofreaied and correct.
After the invocation target detection method, at first detected target area and Tracker position are complementary according to the space length criterion, Tracker and target are one-to-one relationships during normal condition.When according to coupling back, locus when if the Tracker number also has distance between unnecessary and the Tracker greater than the minimum safe distance set, then may be since some human body target by due to background or other human body target partial occlusion, follow the tracks of and still can normally continue, only need use testing result to upgrade to the essential information of the target that matches and get final product, all the other can upgrade voluntarily according to tracking results.More complicated be situation or two kinds of situations that the target of being mated by Tracker or target mated by a plurality of Tracker simultaneously to occur not to take place simultaneously, analyze this unbalanced coupling appears and possible reason as shown in Figure 4.
For the target of not mated, determine its ownership by seeing the position that whether has vanishing target and shelter target and target to occur around it by Tracker.When target from visual field border enter or other targets we could conclude immediately that this target is the target that newly enters scene when all normally following the tracks of, whether must finish under other situations is that vanishing target reappears or shelter target separates and could determine after these are checked that the Tracker of target belongs to.
In when, Tracker transfer phenomena or target having taken place after two targets intersect to be separated having recovered from being hidden into, history feature amount by the target of preserving is mated, be to improve the accuracy rate of coupling, select for use the features such as size, locus, feature description vector of target to carry out joint decision.
Fig. 5 shows the matching relationship of target and trace labelling.
The present invention innovates mainly and shows:
(a) the dynamic particle modeling technique of human body target
Set up the trace model of target by dynamic sampling model,, finish dynamic modeling target by special Mean Shift method control particle migratory direction.
(b) elimination method of invalid particle
Because random particles is based on an observation particle generation of target constantly, the random particles of therefore unavoidable some new generation is gone to outside the target, the present invention determines effective particle minimal gray threshold value according to the pixel grey scale distribution of previous moment observation particle position correspondence, current time random particles position is rejected for the particle that gray-scale value is lower than this threshold value, guarantee that most of particle that adopts all drops on the target.
(c) determination methods of target state machine
Vanishing target and disappearance target are judged: observe the decision of particle position correspondence image gray difference constantly according to front and back.When gray difference during greater than setting threshold, target is considered to change over to hidden state.When changing hidden state near visual field border, continuous certain time of hidden state or target think that target disappears.
The judgement of shelter target: blocking mutually between the shelter target feeling the pulse with the finger-tip mark here.The condition flag that target is blocked by background is to hide.Because each trace labelling is for a target, thus target when blocking mutually each target for the tracking sign still exist.Just definite target was blocked fully mutually when algorithm of the present invention had only two or more trace labellings to drop on the same target.
The present invention is based on OO thought, is basic object with each human body in the scene; Set up a container---follow the tracks of the pond, be used for holding all human body targets of current scene (comprising the target of being blocked fully) by background; Introduce the finite state machine technology, realize being in the reliable tracking of various complicated state human body targets in the scene.The present invention is based on the basic thought of particle filter, different with traditional particle filter algorithm, the prediction of not being correlated with does not need the calculating of similar function, but finishes reliable tracking to target by the particle transition process.

Claims (1)

1、一种基于粒子动态采样模型的红外人体目标跟踪系统，该系统包括红外热像仪、图像采集装置、中央处理器和视频显示装置；图像采集装置采集红外热像仪的视频数据流并送给中央处理器，中央处理器将红外视频数据流组成连续的图像序列并对图像序列进行处理；其特征在于本系统通过随机粒子采样和迁移控制实现对目标的跟踪，以各个人体目标为对象，根据各目标的观测粒子，重新产生随机采样粒子，并且根据设定的无效采样粒子的判断规则，去除多余的粒子；对对象容器内的目标进行状态判定，实现对图像序列中的人体目标进行准确跟踪；跟踪结果送入视频显示装置进行显示，所述中央处理器实现人体目标跟踪的步骤如下：1. An infrared human body target tracking system based on a particle dynamic sampling model, the system includes an infrared thermal imager, an image acquisition device, a central processing unit, and a video display device; the image acquisition device collects the video data stream of the infrared thermal imager and sends it to To the central processor, the central processor forms the infrared video data stream into a continuous image sequence and processes the image sequence; it is characterized in that the system realizes the tracking of the target through random particle sampling and migration control, and takes each human target as the object, According to the observed particles of each target, random sampling particles are regenerated, and redundant particles are removed according to the set judgment rules for invalid sampling particles; the status of the targets in the object container is judged, and the human body targets in the image sequence are accurately determined. Tracking; the tracking result is sent to the video display device for display, and the steps for the central processing unit to realize the tracking of the human body target are as follows: (1)目标检测：监测视频数据流，判断是否需要调用静态人体目标检测算法，捕捉进入视场的目标，并检查目标和对象容器内的跟踪标识的匹配状况，若不需要调用静态人体目标检测算法，则转入步骤(10)；(1) Target detection: monitor the video data stream, judge whether it is necessary to call the static human target detection algorithm, capture the target entering the field of view, and check the matching status of the target and the tracking mark in the object container, if it is not necessary to call the static human target detection Algorithm, then turn to step (10); (2)通过静态人体目标检测方法对人体目标进行检测；(2) detect the human target by the static human target detection method; (3)检测当前时刻是否包含新目标，若未发现新目标则转入步骤(10)；(3) Detect whether the current moment includes a new target, if no new target is found, then proceed to step (10); (4)目标跟踪初始化：利用人体目标的动态粒子建模技术对人体目标进行跟踪初始化；(4) Target tracking initialization: use the dynamic particle modeling technology of the human target to track and initialize the human target; (5)添加跟踪标识：给初始化后的新目标添加跟踪标识，并保存新目标特征属性；(5) Add a tracking mark: add a tracking mark to the new target after initialization, and save the new target characteristic attribute; (6)新目标加入到对象容器，设置新目标的状态机；(6) The new target is added to the object container, and the state machine of the new target is set; (7)对象容器内目标状态检查更新：在下一跟踪时刻，以各个人体目标为对象，根据各目标的观测粒子，重新产生随机采样粒子，并且根据设定的无效采样粒子的判断规则，利用无效粒子的剔除方法去除无效的跟踪粒子；目标状态判断：采用目标状态机的判断方法对对象容器内的目标进行状态判断，目标主要的状态有目标正常、目标隐藏、目标消失和目标互相遮挡粘连；(7) Check and update the state of objects in the object container: At the next tracking moment, each human body object is used as an object, and random sampling particles are regenerated according to the observed particles of each object, and the invalid sampling particles are used according to the set judgment rules for invalid sampling particles. Particle elimination method removes invalid tracking particles; Target state judgment: Use the target state machine judgment method to judge the state of the target in the object container. The main states of the target include target normal, target hidden, target disappearing and targets blocking each other; (8)获取目标当前状态；(8) Obtain the current state of the target; (9)根据目标当前状态进行视频显示，再转入步骤(1)；(9) Carry out video display according to the current state of the target, and then proceed to step (1); (10)检测对象容器是否为空，若对象容器为空，转入步骤(9)；(10) Whether the detection object container is empty, if the object container is empty, proceed to step (9); (11)对对象容器内的各目标进行跟踪，转入步骤(7)；(11) track each target in the object container, and proceed to step (7); 所述步骤(4)中的动态粒子建模技术是在检测到的人体目标对应图像区域范围内布撒随机粒子进行采样，通过粒子均值转移迁移完成对目标的建模，产生观测粒子集；The dynamic particle modeling technique in the described step (4) is to spread random particles within the range of the corresponding image area of the detected human target for sampling, complete the modeling of the target through particle mean value transfer and migration, and generate an observation particle set; 所述步骤(7)中的无效粒子的剔除方法是根据前一时刻观测粒子位置对应的像素灰度分布状态，对目标对应的所有观测粒子对应的像素灰度值按照由高到低排序，按设定粒子数目或确定的百分比选取灰度较高的前面m个灰度值的平均值确定有效粒子最小灰度阈值，以克服图像噪声可能对阈值选取的不利影响，将当前时刻随机粒子位置对应灰度值低于这一阈值的粒子剔除，保证大部分采用粒子都落在目标上；当所有新产生的跟踪粒子都不满足阈值限定条件时，设定有效粒子集的数目下限为1，按照其对应的图像灰度值由大到小选取有效粒子；The method for removing invalid particles in the step (7) is to sort the pixel gray values corresponding to all observed particles corresponding to the target according to the gray scale distribution state of the pixels corresponding to the observed particle positions at the previous moment, and sort them according to Set the number of particles or a certain percentage to select the average value of the first m gray values with higher gray levels to determine the minimum gray threshold of effective particles, so as to overcome the possible adverse effects of image noise on threshold selection, and correspond to random particle positions at the current moment Particles whose gray value is lower than this threshold are eliminated to ensure that most of the adopted particles fall on the target; when all newly generated tracking particles do not meet the threshold limit conditions, the lower limit of the number of effective particle sets is set to 1, according to The corresponding image gray value is selected from large to small effective particles; 所述步骤(7)中的目标状态机的判断方法包括：The judging method of the target state machine in the described step (7) comprises: 目标隐藏和目标消失的判断：根据前后时刻观测粒子位置对应图像灰度差异决定；当有效粒子集的灰度平均值差异大于设定阈值时，目标被认为转入隐藏状态；隐藏状态持续超过设定时间或目标在视场边界附近转入隐藏状态时认为目标已经消失；Judgment of target hiding and target disappearance: It is determined according to the gray level difference of the image corresponding to the observed particle position at the time before and after; when the average gray level difference of the effective particle set is greater than the set threshold, the target is considered to be in the hidden state; the hidden state continues to exceed the set threshold. It is considered that the target has disappeared when the fixed time or when the target turns into a hidden state near the boundary of the field of view; 目标遮挡粘连的判断：目标被背景遮挡的状况标记为隐藏；当两个或多个跟踪标记落在同一个目标上时才确定目标进入互相遮挡状态；Judgment of target occlusion and adhesion: the target is marked as hidden when it is occluded by the background; when two or more tracking marks fall on the same target, it is determined that the target enters the state of mutual occlusion; 目标正常的判断：为缺省状态。Judgment that the target is normal: it is the default state.
CNB2008100698107A 2008-06-06 2008-06-06 Infrared Human Target Tracking System Based on Particle Dynamic Sampling Model Expired - Fee Related CN100565557C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB2008100698107A CN100565557C (en) 2008-06-06 2008-06-06 Infrared Human Target Tracking System Based on Particle Dynamic Sampling Model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB2008100698107A CN100565557C (en) 2008-06-06 2008-06-06 Infrared Human Target Tracking System Based on Particle Dynamic Sampling Model

Publications (2)

Publication Number Publication Date
CN101303726A CN101303726A (en) 2008-11-12
CN100565557C true CN100565557C (en) 2009-12-02

Family

ID=40113625

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2008100698107A Expired - Fee Related CN100565557C (en) 2008-06-06 2008-06-06 Infrared Human Target Tracking System Based on Particle Dynamic Sampling Model

Country Status (1)

Country Link
CN (1) CN100565557C (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI628624B (en) * 2017-11-30 2018-07-01 國家中山科學研究院 Improved thermal image feature extraction method

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101908150B (en) * 2010-06-25 2012-05-30 北京交通大学 Human body detection method
CN102867214B (en) * 2012-07-26 2015-03-18 福建天晴数码有限公司 Counting management method for people within area range
CN104866821B (en) * 2015-05-04 2018-09-14 南京大学 Video object tracking based on machine learning
CN106774327B (en) * 2016-12-23 2019-09-27 中新智擎科技有限公司 A kind of robot path planning method and device
CN107707810B (en) * 2017-08-21 2020-08-28 广州紫川电子科技有限公司 Thermal infrared imager-based heat source tracking method, device and system
CN108664912B (en) * 2018-05-04 2022-12-20 北京学之途网络科技有限公司 Information processing method and device, computer storage medium and terminal
CN110349217A (en) * 2019-07-19 2019-10-18 四川长虹电器股份有限公司 A kind of target candidate location estimation method and its device based on depth image
CN110490904B (en) * 2019-08-12 2022-11-11 中国科学院光电技术研究所 Weak and small target detection and tracking method
CN112489085B (en) * 2020-12-11 2025-04-04 赵华 Target tracking method, target tracking device, electronic device and storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
K-均值聚类中心分析法实现红外人体目标分割. 云廷进等.光电工程，第35卷第3期. 2008 *
基于均值漂移和粒子滤波的红外目标跟踪. 魏坤等.光电子.激光，第19卷第2期. 2008 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI628624B (en) * 2017-11-30 2018-07-01 國家中山科學研究院 Improved thermal image feature extraction method

Also Published As

Publication number Publication date
CN101303726A (en) 2008-11-12

Similar Documents

Publication Publication Date Title
CN100565557C (en) Infrared Human Target Tracking System Based on Particle Dynamic Sampling Model
CN110765964B (en) Detection method of abnormal behavior in elevator car based on computer vision
Yuan Video-based smoke detection with histogram sequence of LBP and LBPV pyramids
CN104281853B (en) A kind of Activity recognition method based on 3D convolutional neural networks
CN112560741A (en) Safety wearing detection method based on human body key points
CN110232380A (en) Fire night scenes restored method based on Mask R-CNN neural network
CN108921039A (en) The forest fire detection method of depth convolution model based on more size convolution kernels
CN106203274A (en) Pedestrian&#39;s real-time detecting system and method in a kind of video monitoring
CN106686377B (en) A kind of video emphasis area determination method based on deep-neural-network
CN108052865A (en) A kind of flame detecting method based on convolutional neural networks and support vector machines
CN106778595A (en) The detection method of abnormal behaviour in crowd based on gauss hybrid models
CN108376406A (en) A kind of Dynamic Recurrent modeling and fusion tracking method for channel blockage differentiation
CN113065431B (en) Human body violation prediction method based on hidden Markov model and recurrent neural network
CN116665016B (en) Single-frame infrared dim target detection method based on improved YOLOv5
CN114299106B (en) A high-altitude object throwing warning system and method based on visual sensing and trajectory prediction
CN108563977A (en) A kind of the pedestrian&#39;s method for early warning and system of expressway entrance and exit
CN109086803A (en) A kind of haze visibility detection system and method based on deep learning and the personalized factor
CN110197121A (en) Moving target detecting method, moving object detection module and monitoring system based on DirectShow
CN109214331A (en) A kind of traffic haze visibility detecting method based on image spectrum
CN113378638B (en) Method for identifying abnormal behavior of turbine operator based on human body joint point detection and D-GRU network
CN109242019A (en) A kind of water surface optics Small object quickly detects and tracking
Yimin et al. Abnormal behavior detection based on optical flow trajectory of human joint points
CN111626109A (en) Fall-down behavior analysis and detection method based on double-current convolutional neural network
CN108182410A (en) A kind of joint objective zone location and the tumble recognizer of depth characteristic study
CN110097571A (en) The vehicle collision prediction technique of quick high accuracy

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Assignee: Hubei Jiuzhiyang Infrared System Co., Ltd.

Assignor: Chongqing University

Contract record no.: 2010420000142

Denomination of invention: System for tracking infrared human body target based on corpuscle dynamic sampling model

Granted publication date: 20091202

License type: Exclusive License

Open date: 20081112

Record date: 20100903

CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20091202

Termination date: 20150606

EXPY Termination of patent right or utility model