CN112097769B - Homing pigeon brain-hippocampus-imitated unmanned aerial vehicle simultaneous positioning and mapping navigation system and method - Google Patents

Abstract

The invention discloses a simultaneous positioning and mapping navigation system and method for an unmanned aerial vehicle imitating the brain-hippocampus of a carrier pigeon. The pose cell module, the head orientation cell module, the grid cell module and the experience map module; the method of the invention simulates the characteristics of two independent navigation mechanisms in the hippocampus of pigeons, and based on the continuous attractor neural network model, two types of brain-like navigation are established respectively. The path integral algorithm, combined with modules such as local view cells, pose cells and experience maps, realizes redundant and robust positioning and navigation, which makes up for the shortcomings of the original RatSLAM algorithm that does not have redundant navigation, and can assist UAVs to achieve navigation And positioning, improve the level of UAV intelligence.

Description

Simultaneous positioning and mapping navigation system and method of unmanned aerial vehicle imitating carrier pigeon brain-hippocampus

technical field

The invention relates to a simultaneous positioning and mapping navigation system and method for an unmanned aerial vehicle imitating the brain-hippocampus of a carrier pigeon, belonging to the field of autonomous navigation of unmanned aerial vehicles.

Background technique

With the advent of the information age, navigation technology plays an increasingly important role in the new battlefield environment. The important problem to be solved by intelligent robots represented by UAVs is autonomous navigation, that is, UAVs can sense and learn through their own sensors in a dynamic and unstructured environment, so as to achieve goal-oriented obstacle avoidance and autonomous movement. process. In December 2007, the U.S. Department of Defense issued the "Roadmap for Unmanned Aerial Vehicle Systems 2007-2032", which pointed out that as the definition of the autonomous control level of UAVs continues to be clarified, its autonomous navigation technology is being combined with artificial intelligence, and will be used in certain areas. Ability to perceive and make decisions about the environment.

At present, UAVs have systematically developed navigation methods such as inertial navigation and satellite navigation, but their intelligence level is still relatively low compared with the navigation capabilities of organisms themselves. The satellite navigation system is vulnerable to malicious interference and damage, and the navigation signal may be rejected or weakened in the case of high-rise buildings, mountain bottoms, and canyons. Higher cost and larger volume. Simultaneous localization and mapping (SLAM) is one of the main methods for autonomous navigation and positioning of UAVs in unstructured or unknown complex environments, but the existing UAV SLAM systems have the following outstanding problems: Relying on environmental perception sensors such as high-precision and expensive lidar, it is necessary to establish an accurate physical model of the world and UAV; it is greatly affected by the environment, and the level of autonomous intelligence is low, which cannot well meet the requirements of UAVs for navigation systems, and needs to develop Autonomous and intelligent navigation.

Over hundreds of millions of years of evolution, many animals have evolved the bizarre ability to perceive information about the outside world and use it for navigation. Unlike humans who rely on high-tech and sophisticated equipment, creatures in nature can achieve precise navigation through their unique physiological structure and rely only on environmental information, so as to achieve behaviors such as foraging, homing and long-distance migration. The worms, with only 302 neurons, were still able to find food based on the concentration of smells through olfactory signals. Other nervous systems that are slightly more complex, like bees, have more and more sophisticated path-planning strategies. Birds and mammals rely on more powerful and complex neural networks to survive in complex environments. The key to solving the problem of autonomous intelligent navigation is to study how higher animals abstract complex environmental information into cognitive information in the brain to form an understanding of the environment, so as to reproduce the intelligence displayed by higher animals in complex environments.

Animals can still locate and navigate accurately and quickly without having high-precision sensory organs and high-resolution maps. This highly intelligent navigation ability relies on the cognitive map generated by the brain's hippocampus that can represent the spatial relationship of environmental things , the formation of cognitive maps is closely related to the presence of spatial cells in the hippocampus of the brain. The firing activity of spatial cells in the hippocampus is thought to form an expression of an internal map of the environment, the so-called cognitive map. With the successive discovery of several key navigation cells, such as place cells, head-oriented cells, grid cells and border cells, which are related to cognitive navigation in the hippocampus of mammalian brains such as rats, it has provided insights into the mechanism of animal brain navigation. possible. Inspired by these mechanisms, brain-like navigation technology based on artificial intelligence has been greatly developed in recent years, and a navigation method to simulate the neural process of rodent brain navigation, represented by RatSLAM, has been proposed. At present, there are two main aspects of robot navigation research based on the cognitive mechanism of the rodent hippocampus environment. The mobile robot is equipped with a rat brain nervous system; the other is a robot real-time localization and map construction (Simultaneous Localization And Mapping, SLAM) based on hippocampal neurobehavior. The former focuses on simulating the navigation behavior of animals from the behavioral level to realize the cognitive navigation of robots; the latter focuses more on the simulation of the hippocampal structure, strictly imitating the information transmission mechanism between various regions in the hippocampus, and realizing the role of various spatial cells. . Relatively speaking, the former is more practical, suitable for existing navigation applications, and there is already RatSLAM for projects.

Compared with rodents, the navigation system of carrier pigeons has more advantages and is more suitable for the needs of intelligent navigation scenarios of drones. Studies have shown that the hippocampus of carrier pigeons plays a key role in the learning of landmark navigation in the natural environment, and its navigation characteristics have two apparently independent mechanisms: one is to directly provide homing guidance through memorized visual landmarks, called "piloting" Mechanism, autonomous positioning and navigation can be achieved only by relying on vision; the second is to recall the compass direction through familiar landmarks and homing, which is called "mosaic map". Therefore, the hippocampal structure of carrier pigeons combines compass orientation and landmark guidance to form a dual control system with simultaneous or oscillatory switching. The navigation of carrier pigeons in familiar areas is a joint navigation strategy that combines compass and landmark information. If the rodent-inspired RatSLAM and other mammal-based bionic SLAM methods are directly used to simulate the hippocampal navigation mechanism of carrier pigeons, there is insufficient physiological basis to accurately reflect the physiological phenomenon and cognitive function of the hippocampal structure of carrier pigeons in navigation. Implementation process. At the same time, the intelligent navigation and homing mechanism of pigeons is very consistent with the technical requirements of UAV visual autonomous intelligent navigation, especially if the intelligent mechanism emerging in the process of pigeon navigation and homing can be applied to the autonomous navigation of UAVs, it will undoubtedly have broad applications. application prospects.

Aiming at the shortcomings of the existing brain-like neural network system navigation system, the invention is inspired by the navigation mechanism of the hippocampus of the carrier pigeon, and proposes an unmanned aerial vehicle SLAM navigation system and method imitating the brain-hippocampus of the carrier pigeon. The SLAM navigation method proposed by the present invention simulates the navigation mechanism of the carrier pigeon brain-hippocampus neural network from the perspective of bionics, integrates two independent path integration technologies based on the attractor neural network, and cooperates with local view correction to realize redundant navigation mapping and Positioning can effectively improve the autonomy of UAVs.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a simultaneous positioning and mapping navigation system and method for an unmanned aerial vehicle imitating the brain-hippocampus of a homing pigeon, aiming at simulating the cognitive navigation mechanism of the hippocampus of the homing pigeon brain, and establishing an unmanned aerial vehicle imitating the brain-hippocampus of a homing pigeon. The navigation system is specifically implemented to simulate the characteristics of the homing pigeon hippocampus with two positioning and navigation mechanisms. Two continuous attractor neural network models are established to implement two path integral algorithms respectively, and the topology is established with local view correction to achieve robustness and redundancy. Navigation realizes redundant navigation and improves the intelligence level of UAVs.

The present invention is a simultaneous positioning and mapping navigation system for unmanned aerial vehicles imitating the pigeon brain-hippocampus, hereinafter referred to as the SLAM navigation system, and the system framework is shown in FIG.

The SLAM navigation system consists of 7 modules, namely: 1) image acquisition and preprocessing module, 2) local view cells module, 3) odometer module, 4) pose cell module, 5) Head-facing cell module, 6) Grid cell module, and 7) Experience map module. According to different functions, the system can be divided into two independent brain-like navigation subsystems. The difference is based on different attractor network models. The mileage information and landmark perception are integrated to form a consistent representation of the environment. The image acquisition and preprocessing module, the local field of view cell module, the odometer module, the pose cell module, and the experience map module together constitute the first carrier pigeon hippocampus recognition module. The cognitive navigation subsystem, the image acquisition and preprocessing module, the local field of view cell module, the pose cell module, the head orientation cell module, the grid cell module, and the experience map module together constitute the second pigeon hippocampal cognitive navigation subsystem .

1) Image acquisition and preprocessing module, including image acquisition and image preprocessing. For image acquisition, a visual sensor needs to be used to collect real-time images; then image preprocessing converts color images into grayscale images, and performs image enhancement and image segmentation processing. The segmented images are used in the visual odometry module and the partial view module. calculate.

2) The Local view cells (LV) module is an expandable cell array, and each LV represents a different visual scene in the environment. The LV simulates a snapshot of the current environment seen by the animal, and when a new visual scene is seen, a new LV is created and associated with the original pixel data and pose cells (PCs) in that scene; and Provides an activated effect when a known area (loopback) is encountered. The LV module section is shown in Fig. 1, and the extracted features-scanline intensity profiles of different images are shown in Fig. 2a,b,c.

3) The odometer module, including two independent odometer modules, one is a visual odometer based on the Scanline intensity profile, which roughly determines the motion of the camera by comparing the difference between consecutive images, as shown in Figure 3 . The other is the visual odometry module based on feature points, which provides motion information for the head facing cell module and the grid cell module, which is more accurate. The visual odometry process based on feature points is shown in the lower left corner of Figure 1.

4) Pose cells (PC) module, PC is implemented as a three-dimensional (3D) competitive attractor neural network that converges to a stable activation pattern on its cells through local activation and global inhibition. Competing attractor neural network units can have many configurations, but typically each unit excites cells that are close to itself and inhibits cells further away, which results in a clump of activity that eventually dominates, called an activity packet. Activities near the activity pack will tend to be activated, and activities injected away from it will be inhibited. If enough activity is injected, new packets can "win" while old packets disappear. The PC module part shown in Figure 1 and the PC discharge effect and spatial distribution diagram shown in Figure 4a and b.

5) Head direction cells (HD) module, when the animal faces a specific direction, the HD will produce the maximum discharge, simulating the HD preference direction mechanism, each cell has a different preference direction, and the joint response of the HD completes the dynamic estimation For horizontal encoding, a continuous head orientation signal is produced when all the HD information is integrated. HD needs to accept external motion signals. Since the traditional RatSLAM core motion sensing information comes from a rough visual odometry, it is greatly affected by the external environment, and there are problems of low reliability and poor environmental adaptability. The present invention uses the visual odometry motion estimation algorithm based on feature points to estimate the direction, and the HD schematic diagram of the discharge characteristics of the HD and four preferred directions is shown in FIG. 5 .

6) Grid cells (GC) module, GC shows obvious regular discharge response to the position of animals in two-dimensional space, which is considered to be the neural basis of dead position reckoning. To simulate the neural response of the GC, a 2D continuous attractor neural network is used, the network pattern will be driven by the velocity input to flow, and the displacement of the animal can be represented by the flow, so that dead reckoning can be achieved based on the motion information. If the motion information of the UAV is estimated from the vision system, the speed vector and heading of the UAV can be calculated, and then used to drive the periodic network to generate a bitmap stream, thereby simulating the GC path integration mechanism from a biological point of view . Figure 6 shows two consecutive lattice patterns of the GC, with corresponding active neurons marked by red circles, from which we can see that the lattice patterns flow to the left. The model of the GC formation mechanism uses the path integral of the speed-adjusted input signal to characterize the periodic grid field, that is, the module simulates the GC discharge to realize the mileage calculation, so the module receives the speed information of the UAV. The GC needs to receive the velocity estimated by the visual odometry motion estimation algorithm based on feature points, and at the same time receive the direction input of the HD to jointly drive the GC to form a motion flow.

7) Experience map (EP) module, EP simulates the cognitive map in the pigeon brain, which can organize the information flow from PC, LV and self-motion estimation into a set of related spatial experiences to construct a topological map. A single experience in the EP is defined by the connection of the active mode in the PC and the active mode in the LV. In EP, every experience has an associated location and orientation. Creating a new node will also create a connection to a previously active node, and the connection encapsulates the attitude transition between nodes based on mileage information; when the drone re-enters a known area, the LV, as previously experienced, will be The empirical position and orientation stored in the EP is aligned with the transformation information of the current position to perform map correction, as shown in the EP module in Figure 1.

A method for simultaneous positioning and map building and navigation of an unmanned aerial vehicle imitating the pigeon brain-hippocampus. The implementation steps are as follows:

Step 1: Establish and initialize the 3D pose cells (Pose cells, PC) continuous attractor neural network dynamics model.

The PC is a 3D continuous attractor network (CAN) collection of cells, connected by activating and inhibitory connections, with characteristics similar to the navigation neurons found in mammals, namely Grid cells. A PC is an abstract cell that updates its activity by replacing activity packs to encode positions in a large-scale environment for UAV navigation tasks. The PC network dynamics are such that the steady state is a cluster of individual activated units, known as active packets. The centroid of this activity bag encodes the UAV's best internal estimate of its current pose. This dynamic behavior is achieved through local activation, global inhibitory connectivity, and the activation weight matrix ε _{a, b, c} is described as The following formula:

where k _p , k _d are the position and direction variance constants; a, b, c represent the distance between the activation unit and the origin of the x', y', θ' joint coordinate system, respectively. The weight matrix for suppression is the same, but the variance constant and suppression range are different.

Step 2: Establish and initialize the 2D grid cells (Grid cells, GC) continuous attractor neural network dynamics model.

GCs display a pronounced regular firing response to the animal's position in 2D space, and if coupled with a velocity input, can drive the GC's continuous attractor network model to propagate its network state along the manifold, perform precise path integration, and thus encode the animal position in space. The present invention adopts the following continuous attractor model to model the dynamic characteristics of Grid cells

where f is a nonlinear function, the synaptic activation of neuron i is s _i , W _ij is the synaptic weight matrix from neuron j to neuron i, τ is the time constant of the neural response, and the feedforward input B i of neuron _i .

Step 3: Image acquisition and image preprocessing.

The real-time image is acquired by the camera, and the image is grayscaled. Then, an image enhancement algorithm based on Laplacian is used to enhance the image contrast. The Laplace operator is defined as:

The Laplacian operator can enhance the image, and the final sharpening formula is:

where g is the output, f is the original image, and c is the coefficient.

Finally, the image is cropped into 3 different regions A, B, C, as shown in Figure 3, which are used to calculate odometry rotation, translation, and image matching, respectively.

Step 4: Visual odometry motion estimation based on the Scanline intensity profile.

The odometer is implemented based on the scan line intensity profile. The scan line intensity profile is formed by summing the intensity values of the image pixel columns and then normalizing the obtained row vector to form a one-dimensional vector or profile, as shown in the LV module in Figure 2, Profiles obtained for different images. The profile is used to estimate rotational and forward speeds between images used for odometry, and to compare the current image with previously seen images to perform local view calibration.

The rotational velocity is then estimated by determining the relative horizontal offset of the two consecutive scanline distributions such that the sum of the absolute differences between the two profiles is minimized, the translational velocity is estimated by multiplying the minimum difference by a scaling factor, and is limited to within the maximum value to prevent spurious results due to large changes in lighting. The comparison of the current image with the previously seen image is performed by calculating the mean absolute difference between the intensity profiles of the two scan lines, as in equation (5):

where I ^j and I ^k are the scanline intensity profiles to be compared, s is the profile offset, and w is the image width. The value of s that makes f(s) the minimum value is the relative movement of successive images I ^j and I ^k :

Where the offset ρ ensures sufficient overlap between the profiles, then the angle and velocity are related to s and f(s) respectively, and the calculation formula is:

Δθ=σs _m (7)

v=min[v _cal f(s _m ,I ^j ,I ^j-1 ),v _max ] (8)

Step 5: Get the direction of the head direction cells (HD).

This step is synchronized with the fourth step, and the external self-motion information can be obtained through the speed and direction sensors. HD is a neuron involved in an animal's directional heading perception relative to its environment, each cell has only one maximum firing direction, and can fire anywhere as long as the animal's head is pointed in the appropriate direction. Unlike a compass, HD does not rely on Earth's magnetic field, but on landmarks and ego-motion signals. The HD discharges regardless of whether the animal is moving or stationary, and is independent of the animal's ongoing behavior. HD is thought to represent the neural substrate for the directional heading that an organism perceives in the environment, enabling precise navigation.

A velocity modulation signal was generated using the HD, i.e. the firing rate was proportional to the current head orientation and the velocity of the animal's movement. The preferred direction of the i-th HD can be expressed as an angular offset θ _i with respect to the main orientation θ ₀ , then the adjustment kernel for a given HD is:

and the animal's instantaneous angular velocity w(t), then the head orientation signal is:

Among them, m is the number of HDs, and d _i (t) is the head orientation signal of the i-th cell whose preferential orientation is θ _i in HD at time t.

Step 6: Path integration based on grid cell (GC) flow.

Receive the speed and direction information in step 5, update the activation response of the cell according to the GC neural network dynamics model in step 2, and calculate the relative motion information.

The activation pattern of the GC will be driven by the velocity input, while the motion displacement can be calculated from the flow of activated cells, which means that dead reckoning can be achieved based on the motion information. For mobile UAV, according to the motion information of UAV estimated from step 5, such as translation velocity and angular velocity, the velocity vector v and heading direction θ of UAV can be calculated, and then use them to drive the periodic network to generate Bitmap stream. On the basis of estimating the position of the UAV based on the flow, the visual odometry based on the GC flow can be realized. Figure 6 shows two typical continuous lattice patterns, two groups of typical active neurons are marked by red circles, from which lattice pattern flow and direction can be found. The weight matrix W _ij in formula (2) is

in

where l, a, γ, β are the relevant simulation parameters. The feedforward input is:

是指向θ_j的单位向量，v是速度向量，A是包络函数，调节神经元输入的强度，如果l,α非0，则速度与网络动力学耦合，并驱动形成模式流。in

is the unit vector pointing to θ _j , v is the velocity vector, A is the envelope function that regulates the strength of the neuron input, if l, α are non-zero, the velocity is coupled with the network dynamics and drives the formation of pattern flow.

Step 7: Local view cells (LV) generate a local view template.

LV extracts image information and generates view templates. In this step, the area A in FIG. 3 is used as the sub-image, and then the scanning line intensity profile of the sub-image is also calculated, and the generated one-dimensional vector is the image template. At the same time, pass the activity information to the pose cells and create a connection, and perform step 8.

During LV calculation, the current contour (ie, the intensity contour of the scan line in the A region) is compared with the template using the mean absolute intensity difference function in formula (5). The comparison is made over a small range of pixel offsets ψ, as in Eq. (14)

Which represents the current profile of I ^j , which is all stored profiles. Comparing the difference with a threshold determines whether the current view is sufficiently similar to an existing scene, or is a new scene.

Step 8: Pose cells (PC) are activated and updated.

The PC adopts the dynamics model established in step 1 to represent the three-degree-of-freedom pose of the UAV, which is updated and calibrated by attractor dynamics, path integral information and LV information. The PCs are arranged in three dimensions, with two dimensions (x', y') representing the manifold at the absolute position (x, y) and one dimension θ' representing the manifold on the absolute head orientation θ. The motion estimation in step 4 is used to drive the path integral in the attitude unit, the partial view template in step 7 triggers the LV associated with the PC through association learning, and the activation effect of the PC is updated according to the dynamic model in step 1. The change in activity of the PC due to local excitation is given by:

where n _x' , n _y' , n _θ' are the dimensions of the PC matrix in (x', y', θ') space.

Step 9: Calculate relative motion and create an experience map (EP) to prepare for topological connections.

EP is a fine-grained topology map composed of many individual experiences e connected by transformations, and its role is to organize information flow from PC, LV and self-motion into a set of related spatial experiences. A single experience in the EP is defined by the connection of the activity pattern _Pi in the PC and the activity pattern _Vi in the LV, while being located at the position _pi . So it can be defined as a triple:

e _i ={P ⁱ ,V ⁱ , ^pi } (16)

The first experience is created at an arbitrary starting point, and subsequent experiences are built on the basis of the first experience. When one or both of _Pi or _Vi changes, a new experience is formed and linked to the previous experience through a transformation that encodes distances found from self-motion cues. New experiences and transformations will continue to form as drones move into new areas.

In this step, the self-motion information of step 4 and step 6 and the partial view template of step 7 are obtained, and the relative motion is calculated and accumulated. Then to judge whether the current experience is a new scene or an existing scene, it is necessary to compare the current P _i and V _i with all the experiences, and pass the scoring standard S of formula (17):

S=μ _p |P ⁱ -P|+μ _v |V ⁱ -V| (17)

where μ _p , μ _v are the weights of pose code and local view code. If it is new experience, go to step ten; otherwise, go to step eleven.

Step 10: Create a new experience and update the experience map (EP).

When the score of formula (17) exceeds the maximum threshold, it is considered a new scene, and the cumulative relative motion increment is greater than the threshold, a new experience is created, and the maximum threshold is set by experience. By performing pose-related transformation from ego-motion information, the pose change from odometry measurements is t _ij :

t _ij = {Δp ^ij } (18)

Δp ^ij is the pose change obtained according to the odometer, and t _ij establishes the connection between the previous experience e _i and the new experience e _j , then the new experience is:

e _j = {P _j , V _j , p ⁱ +Δp ^ij } (19)

This way a new map is formed by linking to previous experience through the transformation of the code.

Step 11: Perform local view calibration, determine loopback, filter out false information, and perform map correction. If the local view is determined to be an already encountered scene, the active LV injects activity into the PC associated with that scene through the learned connections, with the injected activity from the LV filtered by the network's attractor dynamics. If the activity of the LV is injected into the inactive area of the PC network, there is a contradiction, the global suppression connection tends to suppress the new packet, and return to step ten; otherwise, the PC and LV active positions are the same, the conversion of the active position and direction with the EP storage Information alignment to perform map correction.

Since the odometry motion estimation based on image matching inevitably has accumulated errors, when the loop closure occurs, the accumulated transformation position changes make the experience at the loopback less likely to match the previous same position. Therefore, in order to achieve a match, the location of all experiences needs to be updated, i.e. a map correction:

where α0 is the correction rate constant, _{Nf is} the number of connections from experience _ei to other experiences, and _Nt is the number of connections from other experiences to experience _ei .

Step 12: Repeat steps 3 to 11, output two independent visual odometry pose estimates in real time; pose cell activation map and experience map to achieve simultaneous positioning and navigation.

The present invention is a simultaneous positioning and mapping navigation system and method for an unmanned aerial vehicle imitating the brain-hippocampus of a carrier pigeon. Two brain-like navigation path integration algorithms are established respectively, and combined with modules such as local view cells, pose cells and experience maps, to achieve redundant and robust positioning and navigation, which makes up for the shortcomings of the original RatSLAM algorithm that does not have redundant navigation. It can assist UAVs to achieve navigation and positioning, and improve the intelligence level of UAVs.

Description of drawings

Figure 1. Structure diagram of the UAV SLAM navigation system imitating the pigeon brain-hippocampus

Figure 2a, b, c partial view generation template schematic diagram

Figure 3 Image preprocessing block rendering

Figure 4a,b 3D pose cell activation and spatial distribution effect diagram

Figure 5a and b are schematic diagrams of cell discharge and preference direction

Figure 6a,b 2D grid cell flow diagrams of periodic networks driven by motion information

Figure 7 Flow chart of the UAV SLAM navigation system imitating the pigeon brain-hippocampus

Figure 8a,b,c,d Visual odometry motion estimation results, pose cell activation maps and experience maps

The labels and symbols in the figure are explained as follows:

LV1, LV2, LV3: partial view templates

Visual Odometry1: The first visual odometry module

Visual Odometry2: Second Visual Odometry Module

Bundle Adjustment: beam adjustment method

x', y', θ': three-dimensional coordinate system, x', y' represents displacement, θ' represents direction

E, N, W, S: The head is facing the cell's preferred direction, corresponding to (0°, 90°, 180°, 270°)

Detailed ways

Referring to Figures 1 to 8, the following is a specific example of a SLAM navigation method for unmanned aerial vehicles imitating the brain-hippocampus of a carrier pigeon to verify the effectiveness of the proposed system and the method thereof, and the data set used is Karlsruhe, Germany The KITTI dataset co-founded by the Polytechnic Institute and Toyota American Institute of Technology. The UAV SLAM navigation system and method imitating the pigeon brain-hippocampus, the implementation process is shown in Figure 7, and the specific steps of the method are as follows:

Step 1: Establish and initialize the 3D pose cells (PC) continuous attractor neural network dynamics model.

The dynamical behavior of PC network dynamics is achieved through local excitatory, global inhibitory connectivity, described by Equation (1). In the present invention, the activated position and direction variance constant is 1, and the activation range is 7, that is, the activation parameters are as follows:

In the same way, the parameter value of suppressing the connection is:

Step 2: Establish and initialize the 2D grid cells (GC) continuous attractor neural network dynamics model.

GCs display a pronounced regular firing response to the animal's position in 2D space, and if coupled with a velocity input, can drive the GC's continuous attractor network model to propagate its network state along the manifold, perform precise path integration, and thus encode the animal position in space. The present invention adopts the continuous attractor model such as formula (2) to model the dynamic characteristics of grid cells, and f adopts a nonlinear function

τ is the time constant of the neural response, which is 0.5ms in the simulation.

Step 3: Image acquisition and image preprocessing.

The real-time image is acquired by the camera, and the image is grayscaled. Then, an image enhancement algorithm based on Laplacian is used to enhance the image contrast. The Laplacian operator can be used to enhance the image, and the final sharpening formula is as described in Equation (4). Finally, the image is cropped into 3 different regions A, B, and C, as shown in Figure 3, which are used to calculate odometry rotation, translation, and image matching, respectively.

Step 4: Motion estimation based on visual odometry based on scanline intensity profile.

According to equations (5)-(6), the minimum value of the difference and the corresponding displacement are obtained by taking the minimum value of the average absolute difference between the scanning line intensity profiles of consecutive images; then the solution is solved according to equations (7)-(8). Angle and speed changes.

Step 5: Get the direction of the head direction cells (HD).

Acquire continuous images, use the visual odometry estimation algorithm based on feature points to obtain pose estimation, and then solve the direction and velocity information. Passed to HD and GC respectively. The present invention assumes that the entire neural network has N=128 ² neurons, and the preference direction of each neuron i is (0°, 90°, 180°, 270°), as shown in Figure 5a and b, used to determine its Outputs the direction the weight is moving, and determines the velocity input it receives. The preferred direction is specified by _θi , and each 2×2 block contains one neuron for each preferred direction, as in Eq. (9). Then, according to the direction information obtained by the odometer, it is substituted into equation (10) to solve the head orientation signal of HD.

Step 6: Path integration based on grid cell flow.

其中λ_net＝13为网格的周期，l为输出权重的位移。本次仿真采用具有周期边界条件的网络，因此包络函数处处为1：For mobile drones, the speed vector v and heading direction θ of the drone can be calculated according to the motion information of the drone estimated from step 5, such as translational velocity and angular velocity; the speed and direction information of step 5 can be received. , calculate the weight matrix as described in step 2, and update the activation response of the cells in combination with formulas (11)-(13). They are then used to drive periodic networks to generate bitmap streams. On the basis of estimating the position of the UAV based on the flow, the visual odometry based on the GC flow can be realized. Where l, a, γ, β are the relevant simulation parameters, the values are l=2, a=1, γ=1.05β,

where _λnet = 13 is the period of the grid, and l is the displacement of the output weight. This simulation uses a network with periodic boundary conditions, so the envelope function is 1 everywhere:

A(x)=1 (24)

Step 7: Local view cells (LV) generate a local view template.

The image information is extracted, the scanline intensity profile of the sub-image is calculated, and the view template is generated. At the same time, this activity information is passed to the pose cells and connections are created.

Step 8: Pose cells (PC) activation and update.

The pose cells adopt the dynamics model established in step 1 to represent the three-degree-of-freedom pose of the UAV, and are updated and calibrated by attractor dynamics, path integration and LV processes. The motion estimation in step 4 is used to drive the path integral in the attitude unit, which can form a redundant navigation mechanism, which is one of the main innovations in simulating the pigeon brain-hippocampus, and then the two are fused to drive the path integral in the attitude unit. Combined with the partial view template in step 7, the LV associated with the PC is triggered through association learning, and the activation effect of the PC is updated according to the dynamic model in step 1. The activity change of PC due to local excitation is given by Eq. (15).

Step 9: Calculate 3D relative motion and create an experience map (EP) to prepare for topological connections.

In this step, continuous self-motion information and partial view information are acquired, and relative motion is calculated and accumulated. Then to judge whether the current experience is a new scene or an existing scene, it is necessary to compare the current P _i and V _i with all the experiences, and obtain the score S by formula (17), and if the new experience is new, perform step 10; Otherwise, go to step eleven.

Step 10: Create a new experience and update the experience map (EP).

When the score of equation (17) exceeds the maximum threshold, a new scene of the equation is identified, and the cumulative relative motion increment is greater than the threshold, then an experience is created. By performing pose correlation transformation from self-motion information, the pose change from odometer measurement is t _ij , which is linked to previous experience through coded transformation according to equation (19) to form a new map.

Step 11: Perform local view calibration, determine loopback, filter out false information, and perform map correction.

If the local view is determined to be an already encountered scene, the active LV injects activity into the PC associated with that scene through the learned connections, with the injected activity from the LV filtered by the network's attractor dynamics. If the activity of the LV is injected into the inactive area of the PC network, there is a contradiction, the global suppression connection tends to suppress the new packet, and return to step ten; otherwise, the PC and LV active positions are the same, the conversion of the active position and direction with the EP storage Information alignment to perform map correction.

Since the odometry motion estimation based on image matching inevitably has accumulated errors, when the loop closure occurs, the accumulated transformation position changes make the experience at the loopback less likely to match the previous same position. Therefore, in order to achieve matching, the locations of all experiences need to be updated, and loose map correction is implemented according to equation (20). where α ₀ is the correction rate constant, N _f is the number of connections from experience e _i to other experiences, and N _t is the number of connections from other experiences to experience e _i . In the simulation, alpha was set to 0.5 (larger values may cause map instability).

Step 12: Repeat steps 3 to 11, output two independent visual odometry pose estimation trajectories in real time as shown in Figure 8a, b, which can realize redundant navigation: when a navigation system fails (such as magnetic field interference), it can be Switch to another navigation system. The output pose cell activation map is shown in Figure 8c, and the experience map is shown in Figure 8d, enabling simultaneous localization and navigation. Through the comparison of the odometer motion estimation and the experience map, it can be seen that there is an error in the position estimates of the two odometer estimates, and after the partial view calibration, it can be correctly judged that the drone has returned to the known area, and the experience position in the experience map can be correctly determined. Aligning with the stored translation information between orientation and experience to perform map correction, a consistent cognitive map is obtained.

Claims (2)

1. An unmanned aerial vehicle of imitative homing pigeon brain-hippocampus simultaneously fixes a position and builds picture navigation, hereinafter is called SLAM navigation for short, its characterized in that:

the SLAM navigation system consists of 7 modules which are respectively as follows: 1) the system comprises an image acquisition and preprocessing module, a 2) local view field cell module, a 3) odometer module, a 4) position posture cell module, a 5) head orientation cell module, a 6) grid cell module and a 7) experience map module, and is divided into two independent brain-like navigation subsystems according to different functions, wherein the image acquisition and preprocessing module, the local view field cell module, the odometer module, the position posture cell module and the experience map module jointly form a first homing pigeon brain-hippocampus cognitive navigation subsystem, and the image acquisition and preprocessing module, the local view field cell module, the position posture cell module, the head orientation cell module, the grid cell module and the experience map module jointly form a second homing pigeon brain-hippocampus cognitive navigation subsystem;

the image acquisition and preprocessing module comprises image acquisition and image preprocessing; the image acquisition needs to adopt a visual sensor to acquire a real-time image; then, image preprocessing is carried out to convert the color image into a gray image, image enhancement and image blocking processing are carried out, and the blocked image is used for calculation of a visual odometer module and a local view module;

the local view field cell module is an expandable cell array, and each local view field cell represents different visual scenes in the environment; the local visual field cell simulates a current environment snapshot seen by an animal, and when a new visual scene is seen, a new local visual field cell is created and is associated with original pixel data and a pose cell in the scene; and provide activation when a known region or loop is encountered;

wherein the odometer module comprises two independent odometer modules, one is a visual odometer based on the scan line intensity profile, and the motion of the camera is roughly determined by comparing the difference of successive images; the other is a visual odometer module based on the characteristic points, which provides motion information for the head-facing cell module and the grid cell module;

the position and posture cell module is used for realizing a three-dimensional competitive attractor neural network, and converging a stable activation mode on the unit of the position and posture cell module through local activation and global inhibition; there are many configurations of competing attractor neural network elements, but each element will excite cells close to itself and suppress cells further away, which results in a cluster of events that eventually dominates, called the activity package; activities near the activity package tend to be activated, activities injected away from it are suppressed; if enough activity is injected, the new packet "wins" and the old packet disappears;

5) a head-oriented cell module for estimating a direction by a feature point-based visual odometry motion estimation algorithm;

6) the grid cell module simulates grid cell discharge to realize mileage calculation, so that the module receives speed information of the unmanned aerial vehicle; the grid cells need to receive the speed estimated by the visual odometer motion estimation algorithm based on the feature points, and meanwhile, the receiving head inputs the speed in the direction towards the cells to jointly drive the grid cells to form a motion flow;

7) the experience map module simulates a cognitive map in the brain of the carrier pigeon, organizes information streams from the attitude cells, the local view field and the motion estimation into a group of related spatial experience, and constructs a topological map; a single experience in the experience map is defined by the concatenation of the activity pattern in the pose cell and the activity pattern in the local field of view; in the experience map, each experience has an associated position and direction; creating a new node that will also create connections to previously active nodes, the connections encapsulating gesture transitions between nodes based on mileage information; when the drone returns to the known area again, the local field of view is the same as the previous experience, and map correction will be performed by aligning the empirical position and orientation stored in the empirical map with the conversion information of the current position.

2. An unmanned aerial vehicle simultaneous positioning and mapping navigation method imitating homing pigeon brain-hippocampus is characterized in that: the method comprises the following steps:

the method comprises the following steps: establishing a 3D pose cell continuous attractor neural network dynamic model and initializing;

the pose cell is called PC below, is a 3D continuous attractor network cell set, is connected through activating and inhibiting connection, and is characterized by being similar to a navigation neuron found in mammals, namely a grid cell; the PC is an abstract cell that updates its activities by replacing activity packages to encode the location in the large-scale environment of the drone navigation task; PC network dynamics such that steady state is single shotA live cell cluster, referred to as a live package; the centroid of the activity packet is the best internal estimation of the coded unmanned aerial vehicle on the current attitude of the coded unmanned aerial vehicle, the dynamic behavior is realized through local activation and global inhibition connectivity, and the activation weight matrix epsilon_a,b,cDescribed as follows:

wherein k is_p,k_dIs the position and direction variance constant; a, b and c respectively represent the distances between the activation unit and the origin of the x ', y ' and theta ' combined coordinate system; the weight matrix of the suppression is the same, but the variance constant and the suppression range are different;

step two: establishing a 2D grid cell continuous attractor neural network dynamic model and initializing;

modeling grid cell dynamics using a continuous attractor model

Where f is a non-linear function and the synaptic activation of neuron i is s_i，W_ijIs a synaptic weight matrix from neuron j to neuron i, tau is the time constant of the neural response, and the feedforward input B of neuron i_i；

Step three: image acquisition and image preprocessing;

acquiring a real-time image through a camera, and carrying out gray processing on the image; then, enhancing the image contrast by adopting an image enhancement algorithm based on a Laplace operator; the laplacian is defined as:

the laplacian operator can enhance the image, and the obtained sharpening formula is as follows:

where g is the output, f is the original image, c is the coefficient;

finally, the image is cropped into 3 different regions A, B, C for computing odometer rotation, translation, and image matching, respectively;

step four: visual odometry motion estimation based on scan line intensity profiles;

the odometer is implemented based on a scan line intensity profile that is normalized by summing the intensity values of the columns of pixels of the image, then forming a one-dimensional vector, i.e., profile, that is used to estimate the rotation and advance speed between the images for the odometer and to compare the current image with the previously seen images to perform local view calibration;

then estimating the rotation speed by determining the relative horizontal offset of the two consecutive scanning line distributions to minimize the sum of absolute differences between the two profiles, estimating the translation speed by multiplying the minimum difference by a scaling factor and limiting it within a maximum value to prevent spurious results due to large variations in illumination; the comparison of the current image with the previously seen image is performed by calculating the average absolute difference between the two scan line intensity profiles, as in equation (5):

wherein I^jAnd I^kIs the scan line intensity profile to be compared, s is the profile offset, w is the image width; the continuous image I is the s value for making f(s) take the minimum value^jAnd I^kRelative movement:

where the offset p ensures sufficient overlap between the profiles, the angle and velocity are related to s and f(s), respectively, as calculated by:

Δθ＝σs_m (7)

v＝min[v_calf(s_m,I^j,I^j-1),v_max] (8)

step five: the head is directed towards the cell, hereinafter referred to as HD acquisition direction;

the step four is synchronous, and external self-movement information is obtained through a speed sensor and a direction sensor;

generating a speed adjustment signal using HD, i.e. the discharge rate is proportional to the current head orientation and the speed of the animal movement; the preferential direction of the ith HD is represented as a direction relative to the principal direction theta₀Angular offset theta of_iThen, given the adjustment kernel for HD:

and the instantaneous angular velocity w (t) of the animal, then the head orientation signal is:

wherein m is the number of HD, d_i(t) is the preferential orientation in HD of θ_iThe head orientation signal of the ith cell at time t;

step six: path integration based on the grid cell flow;

receiving the speed and direction information of the step five, updating the activation response of the cells according to the grid cell neural network dynamic model of the step two, and calculating the relative motion information;

the activation mode of the grid cells is driven by speed input, the movement displacement is calculated by the flow of the activated cells, and dead reckoning is realized according to the movement information; for mobile drones, based on the motion information of the drone estimated from step five, i.e.Translating the speed and the angular speed, calculating a speed vector v and a heading direction theta of the unmanned aerial vehicle, and then driving a periodic network to generate a dot-matrix stream by using the speed vector v and the heading direction theta; on the basis of flow-based unmanned aerial vehicle position estimation, visual mileage measurement based on grid cell flow is realized; weight matrix W in equation (2)_ijIs composed of

Wherein

Wherein l, a, gamma and beta are relevant simulation parameters; the feed forward inputs are:

wherein

Is directed at theta_jV is a velocity vector, A_b(. h) is an envelope function that adjusts the strength of the neuron input, if l, α is not 0, then the velocity couples with the network dynamics and drives the formation of the modal flow;

step seven: local visual field cells are called LV generation local view template for short hereinafter;

the LV extracts image information to generate a view template; the step adopts the area A as a sub-image, then calculates the scanning line intensity profile of the sub-image, and generates a one-dimensional vector, namely an image template; meanwhile, the activity information is transmitted to the pose cells and connection is established, and the step eight is executed;

when the LV is calculated, the average absolute intensity difference function in the formula (5) is adopted to compare the current contour, namely the scanning line intensity contour of the area A, with the template; the comparison is made over a small range of pixel offsets psi, as in equation (14)

Wherein is represented by^jThe current profile is all stored profiles; comparing the difference value with a threshold value to determine whether the current view is sufficiently similar to the existing scene or a new scene;

step eight: the pose cells are called PC activation and updating for short;

the PC represents the three-degree-of-freedom pose of the unmanned aerial vehicle by adopting the dynamics model established in the first step, and the three-degree-of-freedom pose is updated and calibrated by the attractor dynamics, the path integral information and the LV information; the PCs are arranged in a three-dimensional manner, the two dimensions (x ', y ') representing the manifold at the absolute position (x, y), and the one dimension θ ' representing the manifold at the absolute head orientation θ; the motion estimation in the step four is used for driving path integration in the attitude unit, the local view template in the step seven triggers LV related to the PC through related learning, and the activation effect of the PC is updated according to the dynamic model in the step one; the change in activity of the PC due to local excitation is given by:

wherein n is_x',n_y',n_θ'Is the dimension of the PC matrix in (x ', y ', θ ') space;

step nine: calculating relative motion, and creating an experience map to prepare for topological connection;

the experience map, hereinafter referred to as EP, is a fine-grained topological graph formed by a plurality of individual experiences e through conversion connection, and the experience map, the LV and the motion information flow are organized into a group of related spatial experiences; a single experience in EP is derived from the activity pattern P in the PC_iAnd activity pattern V in LV_iIs simultaneously located at position p_i(ii) a Thus defined as a triple:

e_i＝{Pⁱ,Vⁱ,pⁱ} (16)

the first experience is created at an arbitrary starting point, and the subsequent experiences are established on the basis of the first experience; when P is presentⁱOr VⁱWhen one or both of them change, a new experience is formed and linked to the previous experience by a conversion encoding the distances found from the self-movement cues; as the drone enters a new area, new experiences and transitions continue to form;

acquiring the self-movement information of the fourth step and the self-movement information of the sixth step and the local view template of the seventh step, and calculating and accumulating relative movement; then, whether the current experience is a new scene or an existing scene is judged, and the current P needs to be determined_iAnd V_iCompared to all experience, by scoring criterion S of formula (17):

S＝μ_p|Pⁱ-P|+μ_v|Vⁱ-V| (17)

wherein mu_p,μ_vIs the weight of the pose code and the local view code; if the experience is new, executing a step ten; otherwise, executing step eleven;

step ten: creating new experience and updating an experience map;

when the score of the formula (17) exceeds a maximum threshold value, a new scene is determined, and the accumulated relative movement increment is larger than the threshold value, new experience is created, wherein the maximum threshold value is set through experience; by performing pose correlation transformation on the slave self-movement information, the pose change from the odometer measurement is t_ij：

t_ij＝{Δp^ij} (18)

Δp^ijIs based on the pose change, t, obtained by the odometer_ijEstablishes a previous experience e_iAnd new experience e_jThe new experience is:

e_j＝{P_j,V_j,pⁱ+Δp^ij} (19)

thus, a new map is formed by linking to the previous experience through the conversion of the codes;

step eleven: executing local view calibration, judging a loop, filtering false information, and executing map correction; if it is determined that the local view is a scene that has been encountered, the active LV injects activity into the PC associated with the scene through the learned connections, filtering the injected activity from the LV by the attractor dynamics of the network; if the activity of the LV is injected into the inactive area of the PC network, contradiction occurs, the global suppression connection tends to suppress the new packet, and the step ten is returned; otherwise, the PC and the LV activity positions are the same, and the activity positions and the direction are aligned with the conversion information stored in the EP to execute map correction;

since the odometer motion estimation based on image matching inevitably has accumulated errors, when a loop occurs, all empirical positions need to be updated in order to achieve matching, i.e. map correction:

wherein alpha is₀Is the constant of the correction rate, N_fIs derived from experience e_iNumber of connections to other experiences, N_tFrom other experiences to experience e_iThe number of connections of (a);

step twelve: repeating the third step to the eleventh step, and outputting two independent visual odometer pose estimations in real time; and the pose cell activation map and the experience map realize simultaneous positioning and navigation.

Images