CN115337009A - Gait recognition and prediction method based on full-connection and cyclic neural network - Google Patents

Abstract

The invention discloses a gait recognition and prediction method based on a fully connected and cyclic neural network, which is applied to the field of exoskeleton robots, and aims at the fact that the exoskeleton motor drive cannot be controlled in real time according to the user's active action intention in the prior art The problem of torque; the present invention uses fully connected and cyclic neural network to identify and predict the gait of the lower extremity exoskeleton, and designs a fast gait recognition network based on a fully connected network and a gait prediction network based on a cyclic neural network. Based on a large amount of data, the rapid recognition and effective prediction of human gait can be realized, which is used to compensate the predictive control algorithm of the lower extremity exoskeleton robot, so that the exoskeleton can provide better assistance when the human body walks, so as to achieve the lower extremity exoskeleton The auxiliary function of the robot to hemiplegia patients effectively improves the stability and reliability of the lower extremity exoskeleton robot when assisting patients to walk.

Description

A Gait Recognition and Prediction Method Based on Fully Connected and Recurrent Neural Networks

technical field

The invention belongs to the field of exoskeleton robots, in particular to a lower limb gait recognition and prediction technology.

Background technique

As a human-machine integrated device that combines human intelligence and mechanical power, the exoskeleton can make the powerful power provided by the machine be used by the human body through the simple control of the operator, so that the operator can complete tasks that cannot be completed by itself. As an auxiliary walking device, the lower extremity exoskeleton couples the mechanical structure of the exoskeleton with the human legs. Through human control and external energy supply, the operator who is inconvenient or unable to walk can walk independently. Moreover, different gaits and paces can be designed to adapt to patients with different disabilities and improve the therapeutic effect. The exoskeleton is mainly composed of the following parts: (1) The mechanical structure part. Due to its weight-bearing function requirements, the weight-bearing exoskeleton mostly adopts the hip+knee+ankle structure, while the rehabilitation exoskeleton is mostly used for patients and needs to reduce joint activities, so the hip+knee structure is often used. Most of the mechanical structures are light in weight, high in strength, and anti-fatigue materials, such as aluminum alloy, titanium alloy, nanomaterials, etc.; (2) power system. The power system of the exoskeleton mainly provides a source of power for the power of the exoskeleton, and the way of providing power can be hydraulic pressure, motor, pneumatic, etc.; (3) sensor system. The sensor system of the exoskeleton is mainly used to obtain various signals in the process of human-computer interaction to judge the human body's gait or exercise intention; (4) control system. Usually, software such as Matlab/Simulink is used to implement the proposed control algorithm and related methods, and then downloaded to the corresponding hardware controller.

Human gait recognition and prediction are mainly applied to the design of gait predictive control algorithm. For example, the lower extremity exoskeleton robot is guided by the real-time data of the joints obtained by the sensor in the human-computer interaction test, and the deep learning method is used to classify and Predict gait, assist the exoskeleton control algorithm, and provide an appropriate reference range for the design of the control algorithm.

With the popularization and promotion of exoskeleton robots in daily life, the traditional control strategy and the control method of passive walking of the patient do not consider the wearer's movement intention, which reduces the user's initiative, but based on gait recognition and prediction The control method can control the torque driven by the exoskeleton motor in real time according to the user's active action intention, so as to realize the drive of the exoskeleton joint motor to effectively follow the wearer's movement intention and make a corresponding response.

Contents of the invention

In order to solve the above technical problems, the present invention proposes a lower extremity gait recognition and prediction method based on fully connected and recurrent neural networks, which can effectively reduce control delay, follow the wearer's movement intention, and improve the comfort of exoskeleton wearing.

The technical solution adopted in the present invention is: a method for recognizing and predicting gait outside the lower limbs based on fully connected and recurrent neural networks, including:

S1, collect plantar pressure, joint angle and angular velocity of knee joint, joint angle and angular velocity of hip joint;

S2. Perform gait division and data calibration on the data collected in step S1, so as to obtain a gait labeling data set;

S3. Construct a lower limb gait recognition network;

S4. Construct a lower limb gait prediction network;

S5, using the data set in step S2 to train the lower limb gait recognition network and the lower limb gait prediction network;

S6. The lower limb exoskeleton performs real-time gait recognition and gait prediction according to the trained lower limb gait recognition network and lower limb gait prediction network, so as to achieve the effect of assisting walking.

Beneficial effects of the present invention: the method of the present invention trains the fully connected network and the cyclic neural network model through the collected data set, and then uses the network output as an auxiliary application in the predictive control algorithm of the lower extremity exoskeleton, so that the lower extremity exoskeleton robot can assist the patient. When the patient is actively walking, it can provide real-time assisting effects according to the patient's intention, which changes the previous limitation that patients can only walk passively according to the preset trajectory, and solves the problems of difficulty in gait fitting and predictive control algorithm design. , which improves the safety of the lower extremity exoskeleton robot, effectively reduces the human-computer interaction force, and improves the assisting effect.

Description of drawings

Fig. 1 is the model training schematic diagram of the method of the present invention provided by the embodiment of the present invention;

Fig. 2 is the gait recognition prediction schematic diagram of the method of the present invention provided by the embodiment of the present invention;

Fig. 3 is a block diagram of the gait acquisition system provided by the embodiment of the present invention;

Fig. 4 is the physical figure of the gait acquisition system provided by the embodiment of the present invention;

5 is a schematic diagram of the basic structure of a fully connected neural network provided by an embodiment of the present invention;

FIG. 6 is a schematic diagram of the basic structure of the cyclic neural network provided by the embodiment of the present invention;

Fig. 7 is a physical diagram of an exoskeleton device assisted by the method of the present invention provided by the embodiment of the present invention.

Detailed ways

In order to facilitate those skilled in the art to understand the technical content of the present invention, the content of the present invention will be further explained below in conjunction with the accompanying drawings.

The method of the present invention comprises: the human body gait data set collection process that is used for fully connected and recurrent neural network model training and testing; The network model that is used for gait fast recognition and effective prediction is set up; The method of the present invention specifically comprises the following three part:

1. Data set construction based on lightweight acquisition system

1.1. Design of the data acquisition system for the lower extremity exoskeleton experimental platform. The key data that the data acquisition system of the lower extremity exoskeleton experimental platform needs to measure are: the size of the plantar force, the joint angle and angular velocity of the knee joint and hip joint. The system structure block diagram of the acquisition system is shown in Figure 3. The main control selects STM32F407VET6; the plantar pressure sensor selects the ARIZON-Model1021 membrane box sensor, and cooperates with the transmitter based on AD620 to realize 6-channel real-time acquisition of plantar pressure; the IMU selects Chaonuclear Electronics HI226 six-axis attitude sensor realizes the measurement of knee joint angle, angular velocity and angular acceleration.

Acquire plantar pressure data during walking, specifically through three pressure sensors set at key points on the soles of the feet to collect contact force data between the soles of the feet and the ground during walking.

Obtain the joint angular velocity and angular acceleration of the hip joint and knee joint during walking, specifically through the multi-axis acceleration sensors installed on the left and right calves, left and right thighs, and waist to obtain data such as lower limb posture, joint angular velocity and angular acceleration.

In the design of the acquisition system, since multiple and various sensors are involved, it is necessary to pay attention to the synchronization of sampling time.

1.2. Gait data collection. The collection process of the human gait data set is as follows: through the construction of the human lower limb feature acquisition device based on the IMU and the plantar pressure sensor as shown in Figure 3, the key points of the joint angle, angular velocity and sole of the human lower limbs can be realized under different experimental sites and different walking states. Acquisition of point pressure.

The experimental sites for data collection include: flat cement road, flat asphalt road, flat grass, flat brick road, flat up (down) slope asphalt road, undulating grass, undulating dirt road and stairs.

The walking states for data collection include: up and down stairs, up and down slopes, slow walking and fast walking.

1.3. Gait division standard and data calibration. Analyze the gait and gait classification standards and definitions, and divide and calibrate the data set according to the data obtained by the acquisition device and the four-point principle of walking gait, and make a standard walking gait data set. In the present invention, a gait cycle is defined from the time when the left heel hits the ground to the next time the left heel hits the ground, and within a gait cycle, it is subdivided into the support phase of the left foot in front, the swing of the left foot supporting the right foot off the ground There are four gait phases, the support phase with the right foot in front, and the swing phase with the right foot supporting the left foot off the ground. According to the classification standard, the sensor data curve is drawn by MATLAB and combined with the video recording of the data collection process for labeling to obtain a reliable lower limb gait labeling dataset.

2. Construction of a fast recognition network for lower extremity gait

2.1. Network characteristics and construction. A fully connected neural network is actually a plurality of neurons connected according to certain rules. As shown in Figure 4, it is a fully connected (FC) neural network.

As shown in Figure 4, ω _ji represents the weight coefficient between neuron i and neuron j, _xi represents the i-th component of the input, and y _i represents the i-th output of the network, and the network transfer is written in the following matrix form:

f(x)=simoid(x)

是某层的输入向量，

是某层的输出向量。Where f is the activation function, the sigmoid activation function is used in the fully connected neural network, W is the weight matrix of a certain layer,

is the input vector of a certain layer,

is the output vector of a certain layer.

Under the premise of fully considering the characteristics of the fast classification network, the number of network layers and the number of neurons in each layer are designed.

2.2. Gait recognition network training data preprocessing. Perform data preprocessing for the gait recognition network, and format the original data according to the network input and output requirements. In order to realize fast gait recognition, the training data of the gait recognition network training samples are the knee joint hip joint angle and angular acceleration data collected by the sensor at two sampling times t-1 and t, and the pressure data of the plantar pressure sensor. The label data is the gait label at time t. Only a small amount of sensor data features are used for classification, which reduces the computational complexity of the network and effectively improves the network prediction speed.

3. Construction of lower limb gait prediction network

3.1. Analysis and construction of network characteristics. Recurrent Neural Network (RNN) is a neural network for processing sequence data. Compared with the general neural network, it can handle the data of sequence change. In the gait prediction task, the connection between gait sequences can be fully considered, and the accuracy and rationality of the prediction can be improved.

As shown in Figure 5, a simple recurrent neural network consists of an input layer, a hidden layer, and an output layer. The function expression of the network without considering the bias can be written as:

It can be seen from the calculation formula of the output layer O _t that the output layer of the simple RNN is a fully connected layer, and each node is connected to the hidden layer, where S _t is the state of the hidden layer of the recurrent neural network, V is the weight matrix of the output layer, g is the activation function for the output, regardless of bias. It can also be seen from the calculation formula of the hidden layer S _t that the state in the hidden layer is the sum of the product of X _t and U and the product of S _t-1 and W, where X _t represents the input at the current moment, U represents the weight of this input, S _t-1 represents the state of the last hidden layer, W represents the weight matrix of the last S _t-1 value as the input of this time, and f is the activation function of the hidden layer. Therefore, the hidden layer formula can be brought into the output layer expression multiple times, and the following results are obtained:

O _t ＝g(V·f(U·X _t +Wf(U·X _t-1 +W·f(U·X _t-2 …))))

The output value O _t of the cyclic neural network is affected by the previous input values X _t , X _t-1 , X _t-2 ..., so the cyclic neural network can look forward to any number of input values, and then extract feature.

Considering the characteristics of the above-mentioned recurrent neural network, in order to realize effective gait prediction and ensure the calculation speed at the same time, the overall network structure used in the present invention is shown in Figure 6. First, the fast classification network is used as the feature extraction layer to extract features from the sensor data , the result obtained by the feature extraction layer is used as the input value of the cyclic neural network, that is, the above-mentioned X _t , X _t-1 , X _t-2 , and then the feature extraction layer is connected with the cyclic neural network layer, making full use of the cyclic neural network for The memory characteristics of the time series are used for step prediction; on the input data, the gait prediction neural network uses the knee joint and hip joint angle, angular acceleration data and the pressure data of the plantar pressure sensor collected by the sensor from tk to t time as input, using The gait classification results at time t+k are used as labels for training, and the number of network input data is adjusted according to the size of the prediction time interval k to achieve effective gait prediction.

3.2. Design and predict network training data preprocessing. Perform data preprocessing for the gait prediction network, and format the original data according to the network input and output requirements. In order to achieve effective gait prediction, the training data is collected from the sensor at time t-k to t, the angle and angular acceleration data of the knee joint and hip joint and the pressure data of the plantar pressure sensor, and the label data of the training data is the gait at time t+k Label, which can adjust the size and format of the input data according to the size of the prediction interval.

4. Assisted walking based on gait recognition and prediction network

After the gait recognition and prediction network training is completed, and the user wears the exoskeleton shown in Figure 7, the system can obtain the wearer's plantar pressure and the angle, angular velocity and angular acceleration of the knee joint and hip joint in real time. After these data pass through the gait recognition network, the current gait phase of the wearer can be identified in real time. Therefore, the use of the gait recognition network can quickly provide the current gait reference and limit for the control algorithm to ensure that the exoskeleton does not It will cause loss of control and cause harm to the human body. At the same time, these data can predict the angle of the knee joint and hip joint of the human body at the subsequent time after passing through the gait prediction network, so as to design the control rate according to the predicted joint angle.

The designed and used network of the present invention uses MATLAB to build and train, and after the training is completed, C language codes can be automatically generated and cross-platform applications can be realized. The network of the present invention uses the angle and angular acceleration of the knee joint and the hip joint and the pressure of the plantar pressure sensor as the network input, and the network output is the real-time gait recognition result and the gait prediction result.

Those skilled in the art will appreciate that the embodiments described here are to help readers understand the principles of the present invention, and it should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Various modifications and variations of the present invention will occur to those skilled in the art. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of the claims of the present invention.

Claims (4)

1. a kind of gait recognition and prediction method outside the lower limbs based on full connection and recurrent neural network, it is characterized in that, comprising: S1, collect plantar pressure, joint angle and angular velocity of knee joint, joint angle and angular velocity of hip joint; S2. Perform gait division and data calibration on the data collected in step S1, so as to obtain a gait labeling data set; S3. Construct a lower limb gait recognition network; S4. Construct a lower limb gait prediction network; S5, using the data set in step S2 to train the lower limb gait recognition network and the lower limb gait prediction network; S6. The lower limb exoskeleton performs real-time gait recognition and gait prediction according to the trained lower limb gait recognition network and lower limb gait prediction network, so as to achieve the effect of assisting walking. 2. A kind of gait recognition and prediction method based on full connection and cyclic neural network for lower extremities according to claim 1, characterized in that, the gait division in step S2 is specifically: starting from the grounding of the left heel to the next A left heel strike is defined as a gait cycle, which is subdivided into a left foot forward stance phase, a left foot support right foot swing phase, a right foot forward stance phase, and a right foot stance phase within a gait cycle The swing of the left foot off the ground corresponds to these four gaits. 3. a kind of lower extremity gait recognition and prediction method based on fully connected and recurrent neural network according to claim 2, is characterized in that, during the training of lower limb gait recognition network in step S5, the training data that adopts is t- At two sampling times 1 and t, the sensor collects the knee joint hip joint angle and angular acceleration data and the pressure data of the plantar pressure sensor, and the label data of the training data is the gait label at time t. 4. a kind of lower extremity gait recognition and prediction method based on fully connected and recurrent neural network according to claim 3, is characterized in that, during the training of lower extremity gait prediction network in step S5, the training data that adopts is t-k to The sensor at time t collects the knee joint hip joint angle and angular acceleration data and the pressure data of the plantar pressure sensor, and the label data of the training data is the gait label at time t+k.

Images