CN109766844B - A mobile terminal identity identification and memory method based on brooch device - Google Patents

Abstract

The invention proposes a mobile terminal identity identification and memory method based on a brooch device, which can facilitate users to know the identity information of the counterparty in real time. The method mainly includes the following steps: Step 1: The user downloads and installs the smart companion memory client on the mobile terminal; Step 2: The user starts the smart companion memory client, and transmits the face image collected by the brooch device to the mobile terminal via Bluetooth. Mobile terminal; Step 3: The mobile terminal transmits the received face image collected by the brooch device to the server, and uses the trained preset neural network to classify and recognize the face image collected by the brooch device. The method can quickly identify the identity information of the counterparty by using a simple brooch device and a mobile terminal, display the identity identification and memory results of the counterparty in real time, and automatically prompt the user of the counterparty's name and related information, thereby greatly reducing the probability of misidentification or calling the wrong name.

Description

A mobile terminal identity identification and memory method based on brooch device

technical field

The present invention relates to the technical field of electronic equipment, in particular to a method for identifying and remembering the identity of a mobile terminal based on a brooch device.

Background technique

In the era of big data, the explosive growth of data. The amount of data that people need to deal with is increasing day by day. Business people have a lot of business and a wide range of communication. This makes it difficult for them to accurately remember the names and related information of everyone in business communication. Business people often recognize the wrong person or call the wrong name in communication, causing unnecessary embarrassment and misunderstanding, and leaving a bad impression on the other party. This will directly affect the subsequent communication, the establishment and maintenance of interpersonal relationships. Therefore, how to assist business people to quickly and accurately identify the identities of communication objects in the process of communication has become an increasingly urgent need of today's business people.

People's memory gradually deteriorates with age. For middle-aged and elderly people, it is also quite difficult to accurately remember relatives and friends around them. Therefore, middle-aged and elderly people also need a tool to help them accurately identify the identities of relatives and friends.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a mobile terminal identification and memory method based on a brooch device. The brooch device is used to collect a face image and send it to a mobile terminal through bluetooth, and then use a neural network to classify and identify the face, and display the identity identification on the mobile terminal. It can memorize the results and automatically prompt the user, which is convenient for the user to know the identity information of the other party in real time, thus greatly reducing the probability of identifying the wrong person or calling the wrong name.

In order to achieve the above purpose of the invention, the present invention proposes a method for identifying and remembering the identity of a mobile terminal based on a brooch device, the method comprising the following steps:

S1) The user downloads and installs the smart companion memory client on the mobile terminal.

S2) The user starts the smart companion memory client: if the user starts the client for the first time, it is necessary to perform user registration on the client first, import several groups of face images and the relevant contact information corresponding to these face images in advance, and store the preset The imported groups of face images and the relevant contact information corresponding to these face images are saved to the server-side database, and used as a training set to train a preset neural network on the server side. After the preset neural network training is completed, the client starts the brooch device. And receive the face image collected by the brooch device, and finally turn on the Bluetooth of the mobile terminal to pair the mobile terminal with the brooch device, and transmit the face image collected by the brooch device to the mobile terminal through the Bluetooth of the mobile terminal; For the client, the user needs to log in to the client first, then the client starts the brooch device, receives the face image collected by the brooch device, and finally turns on the Bluetooth of the mobile terminal, so that the mobile terminal is paired with the brooch device, and the data collected by the brooch device is The face image is transmitted to the mobile terminal through the Bluetooth of the mobile terminal.

Wherein, the collection of the face image by the brooch device is realized by using the photographing button on the brooch device to take pictures.

S3) The mobile terminal transmits the received face image collected by the brooch device to the server, and uses the preset neural network trained in step S2) on the server to classify and recognize the face image collected by the brooch, and finally communicate with the server-side database. Match the relevant contact information corresponding to the pre-imported face image in the mobile terminal. If the matching is successful, the matching success information will be displayed on the mobile terminal interface; if the matching fails, the matching failure information will be displayed on the mobile terminal interface and the failed brooches will be saved. The face image collected by the device and the contact information corresponding to the face image are sent to the unmatched library in the server-side database. After saving, the contact information corresponding to the face image can be added to the server-side contact information library.

In addition, the smart companion memory client also includes adding, deleting, modifying and checking the contact information in the contact information database, viewing the history record page of the successfully matched contacts, the saved face image and the corresponding face image. Contact information display page and client setting page, etc.

The mobile terminal is a smart phone, a tablet computer, or the like.

The invention provides a mobile terminal identity identification and memory method based on a brooch device. By using a simple brooch device and a mobile terminal, the identity information of the other party can be quickly identified, and the intelligent companion memory client in the mobile terminal can display the other party in real time on the mobile terminal. Identity identification and memory results, automatically prompt the user with the name and related information of the other party, so that the user can know the identity information of the other party in real time, so as to carry out more in-depth and active communication and maintain good interpersonal relationships, thereby greatly reducing the number of people who recognize the wrong person or call the wrong name. probability.

Description of drawings

FIG. 1 is a flow chart of a method for identifying and remembering an identity of a mobile terminal according to the present invention.

FIG. 2 is a flowchart of a detailed operation of the mobile terminal for matching face images according to the present invention.

FIG. 3 is a flow chart of the detailed operation of training a face image with a neural network algorithm on the server side of the mobile terminal of the present invention.

FIG. 4 is the user registration and login interface of the smart companion memory client according to the present invention.

FIG. 5 is a display page of successful matching of the smart companion memory client according to the present invention.

FIG. 6 is a display page for matching failure of the smart companion memory client according to the present invention.

FIG. 7 is a smart companion memory client unmatched library page of the present invention.

FIG. 8 is a page for adding a new contact to the user of the smart companion memory client according to the present invention.

FIG. 9 is a history page of the smart companion memory client of the present invention to view the contacts that have been matched successfully.

FIG. 10 is a display page of related information of the saved contact face image in the smart companion memory client of the present invention.

FIG. 11 is the setting page of the smart companion memory client of the present invention.

FIG. 12 is a schematic diagram of the neural network adopted in the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer and clearer, the technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

The present invention proposes a method for identifying and remembering the identity of a mobile terminal based on a brooch device, the flowchart of which is shown in Figure 1, and the method includes the following steps:

S1) The user downloads and installs the smart companion memory client on the mobile terminal.

S2) The user starts the smart companion memory client: the registration and login interface is shown in Figure 4. If the user starts the client for the first time, click the [Register] button to register the user and import several groups of face images and the corresponding face images in advance. The relevant contact information, the pre-imported several groups of face images and the relevant contact information corresponding to these face images are saved to the server-side database, and as a training set, back-propagation (Back Propagation, BP) is trained on the server side as a training set. ) neural network, after the BP neural network training is completed, the client starts the brooch device, receives the face image collected by the brooch device, and finally turns on the Bluetooth of the mobile terminal, so that the mobile terminal is paired with the brooch device, and the face collected by the brooch device The image is transmitted to the mobile terminal through the Bluetooth of the mobile terminal; if the user does not start the client for the first time, the user must first enter the account password and click the [Login] button to log in to the client, and then the client starts the brooch device and receives the people collected by the brooch device. face image, and finally turn on the bluetooth of the mobile terminal to pair the mobile terminal with the brooch device, and transmit the face image collected by the brooch device to the mobile terminal through the bluetooth of the mobile terminal. Wherein, the collection of the face image by the brooch device is realized by using the photographing button on the brooch device to take pictures.

The training set includes several groups of samples, wherein each group of samples includes several attributes, some of which are related to the face image, and the remaining part of the attributes are related to the relevant contact information corresponding to the face image.

S3) The mobile terminal transmits the received face image collected by the brooch device to the server side, and uses the BP neural network trained in step S2) on the server side to classify and recognize the face image collected by the brooch device, and finally compares it with the server-side database. The relevant contact information corresponding to the pre-imported face image is matched, and the detailed operation steps of the mobile terminal intelligent companion memory client to match the face image are shown in Figure 2:

If the matching is successful, the matching success information is displayed on the mobile terminal interface. As shown in Figure 5, the displayed image is the current face image collected by the brooch device, and the displayed information is the user's description of the relevant contact information corresponding to the current face image. For example, "a certain place", etc., the user can also click the [Replace] icon in the client to choose whether to replace the pre-imported face image in the server-side database with the current face image collected by the brooch device with the same correlation as the current face image. If the user chooses to replace the face image of the contact information, the current face image collected by the brooch device will be saved to the server-side database, and the replaced pre-imported face image will be deleted. The current face image and the contact information corresponding to the replaced pre-imported face image are used as a new set of samples in the training set for the subsequent training of the BP neural network. If the user chooses not to replace, the current data collected by the brooch device will be saved. The face image is sent to the history page in the client;

If the matching fails, display the matching failure information on the mobile terminal interface and save the current face image collected by the brooch device that failed to match and the contact information corresponding to the current face image collected by the brooch device to the unmatched library in the server-side database. As shown in Figure 6, the user can enter the unmatched library to view the face images that fail to match, and add and delete contact information for the face images that fail to match, as shown in Figure 7, specifically: the user clicks The [Add] icon adds contact information that does not match the face image in the library, such as name, position, etc., as shown in Figure 8; the user clicks the [Save] icon to save the current face image collected by the brooch device and the corresponding contact information To the server-side database, as a new set of samples in the training set for the subsequent training of the BP neural network; the user clicks the [Delete] icon, and the face images in the unmatched database will be deleted; if the user wants to view the deleted face image The image can be viewed by entering the history page in the client, as shown in Figure 9.

Further, before using the pre-imported groups of face images and the relevant contact information corresponding to these face images as the training set, it is necessary to pre-process the pre-imported groups of face images, and the pre-processing method includes the following steps:

a1) First, denoise a pre-imported face image;

a2) Perform geometric correction processing on the denoised face image in step a1), which is to achieve position calibration and eliminate the influence of face scale change and rotation;

a3) Since the face recognition rate is greatly affected by the lighting conditions and the average gray value of each face image is different, the gray scale value of the face image that has undergone geometric correction processing in step a2) should also be normalized operation;

a4) Judging whether there is a human face in the face image after step a3) grayscale amplitude normalization operation: if there is, enter step a41); if not, enter step a42);

a41) Perform key point detection, alignment and calibration operations on the face image after the grayscale amplitude normalization operation in step a3), and use the principal component analysis (PCA) algorithm to convert it into a 1×20-dimensional feature column vector. It is sent to the BP neural network as a group of sample input together with the relevant contact information corresponding to the face image, trains the BP neural network, and enters step a5);

a42) reminding the user that this face image is an invalid image on the mobile terminal interface, and needs to re-import a new pre-imported face image, and then enters step a1);

a5) Import the next pre-imported face image and enter step a1) until the preprocessing of all pre-imported face images is completed.

Further, the method of using the training set to train the BP neural network is shown in Figure 3, which specifically includes the following steps:

b1) Initialize the BP neural network: randomly assign a non-zero value to each weight and bias value in the BP neural network, and then enter step b2);

b2) Input a set of samples for BP neural network learning, and forward calculate the input value and output value of each layer of neurons in the BP neural network;

b3) Determine whether the actual output of the final output layer of the BP neural network processed in step b2) is consistent with the expected output: if they are consistent, enter step b31), if they are inconsistent, enter b32);

b31) Input the next group of samples for BP neural network learning, and then enter step b4);

b32) Calculate the local gradient value of each layer of the BP neural network according to the back-propagation algorithm, and then enter step b33);

b33) Amend each weight and bias value in the BP neural network according to the calculated local gradient value of each layer of the BP neural network, and then proceed to step b2).

b4) Determine whether the BP neural network has learned all the samples: if so, enter step b5); if not, enter step b2);

b5) end the training;

The 3-layer BP neural network is used here because it can realize any non-curve mapping, and the neural network with more layers is not used because compared with the 3-layer neural network, when the back-propagation algorithm is used to update, there are more layers. The neural network of is prone to getting trapped in local minima of the weights and bias values, as shown in Figure 12. The back propagation neural network (BP) network has 20 neurons in the input layer, the number of neurons in the hidden layer is calculated as shown in Equation 1.7, and the number of neurons in the output layer depends on the relevant contact information in the server-side database The number of categories, that is, the number of attributes of related contact information.

To reduce the error rate, we use the improved Sigmoid function as the activation function of the BP neural network, as shown in Equation 1.1. The activation function used by the BP neural network is shown in formula (1.1),

where χ=w ₁ x ₂ +w ₂ x ₂ +w ₃ x ₃ +...+w _d x _d +b, b is the bias value, and the mth group of samples is represented as (X _m , Y _m ), Among them, m=1, 2, 3,...,q,q is the total number of samples, X _m =(x ₁ ,x ₂ ,...,x _d ) represents the face images related to the mth group of samples attribute, Y _m represents the attribute related to the relevant contact information corresponding to the face image in the mth group of samples, x _p (p=1,2,3,...,d) is the mth group of samples in _Xm The value of the p-th attribute of , d is the number of attributes of the sample in X _m , that is, the number of neurons in the input layer of the BP neural network, d=20, w _p (p=1,2,3,. ..,d) is the weight value corresponding to x _p .

The error function of the BP neural network when the sample is (X _m , Y _m ) is shown in formula (1.2):

为第m个样本在第i个输出神经元的实际输出，

表示第m个样本在第i个输出神经元的期望输出。Among them, n is the number of attributes of the sample in Y _m , that is, the number of output neurons of the BP neural network, n=1024,

is the actual output of the i-th output neuron for the m-th sample,

represents the expected output of the mth sample at the ith output neuron.

For each group of samples, the correction rules for each weight and bias value in the BP neural network are shown in formulas (1.3)-(1.4), (1.5) and (1.6):

均表示从BP神经网络第l-1层中的第j个神经元指向第l层的第i个神经元的权重值，

均表示从BP神经网络第l-1层中的第j个神经元指向第l层的第i个神经元权重调整值的计算，E为误差函数，对于每m组样本，采用式(1.2)所示的方法计算误差函数，

为计算E对

偏导数，同理，对于每一组样本，

为第l层的第i个神经元的偏置值，

第l层的第i个神经元偏置调整值的计算，

为计算E对

偏导数，ρ为学习率，取值为0～1，N为隐含层神经元的个数，

表示向上取整，d为BP神经网络输入层神经元的个数，n为BP神经网络输出层神经元的个数，常数a的取值范围是1～10，由于输入特征较多，此处a＝10，BP神经网络的层数为3层，包括位于第1层的输入层、位于第2层的隐含层和位于第3层的输出层，即l＝2,3。where, for each set of samples,

Both represent the weight value from the j-th neuron in the l-1 layer of the BP neural network to the i-th neuron in the l-th layer,

Both represent the calculation of the weight adjustment value from the j-th neuron in the l-1 layer of the BP neural network to the i-th neuron in the l-th layer, E is the error function, and for each m group of samples, formula (1.2) The method shown computes the error function,

For calculating E pair

Partial derivative, in the same way, for each set of samples,

is the bias value of the i-th neuron in the l-th layer,

The calculation of the bias adjustment value of the i-th neuron in the l-th layer,

For calculating E pair

Partial derivative, ρ is the learning rate, ranging from 0 to 1, N is the number of neurons in the hidden layer,

Represents rounded up, d is the number of neurons in the input layer of the BP neural network, n is the number of neurons in the output layer of the BP neural network, and the value range of the constant a is 1 to 10. Due to the large number of input features, here a=10, the number of layers of the BP neural network is 3 layers, including the input layer located in the first layer, the hidden layer located in the second layer and the output layer located in the third layer, that is, l=2,3.

As shown in FIG. 10, in the embodiment of the present invention, click the [Contact] icon to view the group display of the contact information corresponding to the saved face image and the contact information corresponding to the saved face image. For example, this implementation In the method, the contact information corresponding to the saved face images is grouped according to the shooting location, such as Shanghai, Chengdu, Wuhan, etc. In addition, users can also group in other ways according to their own needs.

As shown in Figure 11, in the example of the present invention, click the [Settings] icon to view the client setting information, such as the location information when the face image is classified and recognized, the specific time when the face image is classified and recognized, and the specific identity of the face image. information, etc.

Click the [Bluetooth] icon to view the Bluetooth matching information.

Click the [Logout] icon to log out of the current account. The next time you log in, you need to log in again.

In addition, the smart companion memory client also includes a history page for viewing successfully matched contacts.

The above-mentioned mobile terminals are smart phones, tablet computers, and the like.

The invention provides a mobile terminal identity identification and memory method based on a brooch device. By using a simple brooch device to collect a face image and sending it to a mobile terminal through bluetooth, the face is classified and recognized by a neural network, and the other party is displayed on the mobile terminal in real time. Identity identification and memory results, automatically prompt the user with the name and related information of the other party, so that the user can know the identity information of the other party in real time, so as to carry out more in-depth and active communication and maintain good interpersonal relationships, thereby greatly reducing the number of people who recognize the wrong person or call the wrong name. probability.

The specific embodiments of the present invention are described above, but only for illustration, and the relevant interface diagrams are schematic diagrams. Various modifications may be made to these embodiments in practice without departing from the principles and spirit of the present invention. Where equivalent replacements or equivalent replacements are adopted, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are included in the protection list.

Claims (6)

1. A mobile terminal identity authentication and memory method based on a brooch device is characterized by comprising the following steps:

s1) a user downloads and installs an intelligent companion memory client on a mobile terminal;

s2) the user starts the intelligent partner memory client: if a user starts the client for the first time, the user is required to register at the client, a plurality of groups of face images and related contact information corresponding to the face images are led in advance, the led-in groups of face images and the related contact information corresponding to the face images are stored in a database at a server, a preset neural network is trained at the server as a training set, the client starts a brooch device after the preset neural network training is finished, the face images collected by the brooch device are received, finally, bluetooth of the mobile terminal is opened, the mobile terminal is matched with the brooch device, and the face images collected by the brooch device are transmitted to the mobile terminal through the Bluetooth of the mobile terminal; if the user does not start the client for the first time, the user needs to log in the client firstly, then the client starts the brooch device, receives the face image collected by the brooch device, finally opens the Bluetooth of the mobile terminal to enable the mobile terminal to be matched with the brooch device, and transmits the face image collected by the brooch device to the mobile terminal through the Bluetooth of the mobile terminal;

s3) the mobile terminal transmits the received face image collected by the brooch equipment to the server, and the server classifies and identifies the face image collected by the brooch equipment by using the preset neural network trained in the step S2), and finally matches the face image with the related contact information corresponding to the face image imported in advance in the database of the server: if the matching is successful, displaying matching success information on a mobile terminal interface, wherein the display image is a current face image acquired by the brooch device, and the display information is description of relevant contact information corresponding to the current face image by the user; if the matching fails, displaying matching failure information on a mobile terminal interface, and storing a current face image acquired by the brooch device with the matching failure and contact information corresponding to the current face image acquired by the brooch device to an unmatched library in a server-side database;

the preset neural network is a back propagation BP neural network, and the method for training the BP neural network by using the training set specifically comprises the following steps:

b1 Initializing the BP neural network: randomly assigning a non-zero value to each weight value and bias value in the BP neural network, and then entering the step b 2);

b2 Input a group of samples for preset neural network learning, and forward-computing input values and output values of neurons in each layer of the preset neural network;

b3 Judging whether the actual output of the final output layer of the preset neural network processed by the step b 2) is consistent with the expected output: if they match, the procedure goes to step b 31), and if they do not match, the procedure goes to step b 32);

b31 Input the next set of samples to preset neural network learning, then go to step b 4);

b32 Calculating a local gradient value of each layer of the preset neural network according to a back propagation algorithm, and then entering the step b 33);

b33 Correcting each weight value and bias value in the preset neural network according to the calculated local gradient value of each layer of the preset neural network, and then entering the step b 2);

b4 ) determine whether the preset neural network has learned all samples: if yes, entering step b 5); if not, entering the step b 2);

b5 Finish training;

the activation function of the BP neural network is a modified Sigmoid function expressed as

Wherein χ = w ₁ x ₂ +w ₂ x ₂ +w ₃ x ₃ +…+w _d x _d + b, b are offset values, and the m-th group of samples is denoted as (X) _m ，Y _m ) Wherein m = l,2, 3.., q, q is the total number of samples, X _m ＝(x ₁ ，x ₂ ，...，x _d ) Representing attributes associated with the face image in the mth group of samples, Y _m Representing an attribute, x, associated with the associated contact information corresponding to the face image in the mth group of samples _p P =1,2, 3.. D is the m-th group of samples at X _m D is the value of the sample at X _m I.e. the number of input layer neurons of the BP neural network, d =20 _p P =1,2, 3.. D is x _p A corresponding weight value;

for each group of samples, each weight and bias value modification rule in the BP neural network is as follows:

the number of hidden layer neurons is calculated as

Meaning that the rounding is up, wherein,

represents a weight value pointing from the jth neuron in layer l-1 of the BP neural network to the ith neuron in layer l,

representing the calculation of the weight adjustment value of the ith neuron pointing from the jth neuron in the l-1 layer of the BP neural network to the ith neuron, E is an error function, and for the mth group of samples, the error function

n is the attribute number of the sample in Ym, namely the number of the output neurons of the BP neural network, n =1024,

the actual output at the ith output neuron for the mth sample,

representing the expected output of the mth sample at the ith output neuron,

to calculate E pairs

The partial derivatives, in the same way,

is a firstThe bias value of the ith neuron of layer l,

the calculation of the ith neuron bias adjustment value at the l-th layer,

to calculate E pairs

The partial derivative, rho is the learning rate, the value is 0-1, d is the number of neurons in the input layer of the BP neural network, the value range of the constant a is 1-10, the number of layers of the BP neural network is 3, l =2,3, the input layer is positioned at the 1 st layer, the hidden layer is positioned at the 2 nd layer, and the output layer is positioned at the 3 rd layer.

2. The method for authenticating and memorizing the identity of the mobile terminal according to claim 1, wherein in the step S2), before the pre-imported groups of facial images and the related contact information corresponding to the facial images are taken as the training set, the pre-imported groups of facial images need to be pre-processed, and the pre-processing method comprises the following steps:

a1 Firstly, denoising a pre-imported human face image;

a2 Performing geometric correction processing on the face image subjected to denoising processing in the step a 1);

a3 Carrying out gray scale amplitude normalization operation on the face image subjected to the geometric correction processing in the step a 2);

a4 Judging whether a face exists in the face image subjected to the gray scale amplitude normalization operation of the step a 3): if yes, entering step a 41); if not, entering step a 42);

a41 Carrying out key point detection and alignment calibration on the face image subjected to the gray scale amplitude normalization operation in the step a 3), converting the face image into a characteristic column vector with 1 x 20 dimensions by utilizing a Principal Component Analysis (PCA) algorithm, sending the characteristic column vector and related contact information corresponding to the face image to a Back Propagation (BP) neural network together to be used as a group of samples for input, training the BP neural network, and entering the step a 5);

a42 Prompting the user that the face image is an invalid image on a mobile terminal interface, and re-importing a new pre-imported face image, and then entering the step a 1);

a5 Importing the next pre-imported face image and entering the step a 1) until all pre-imported face images are preprocessed.

3. The method for authenticating and memorizing the identity of a mobile terminal according to claim 2, wherein the step S2) of capturing the face image by the brooch device is implemented by taking a picture by using a picture button on the brooch device.

4. The mobile terminal identity authentication and memorizing method according to claim 3, wherein in the step S3), when the matching is successful, the user can further select whether to replace the face image with the same related contact information as the current face image, which is previously imported in the server-side database, with the current face image acquired by the brooch device, and if the user selects the replacement, the current face image acquired by the brooch device is saved in the server-side database, and the replaced previously imported face image is deleted at the same time, and the server side uses the current face image acquired by the replaced brooch device and the contact information corresponding to the replaced previously imported face image as a new set of samples in a training set for subsequent training of the BP neural network; and if the user selects not to replace the face image, storing the current face image acquired by the brooch device to a history page in the client.

5. The method for authenticating and memorizing the identity of the mobile terminal according to claim 3, wherein in the step S3), when the matching fails, the user can enter the unmatched library to check the face image with the failed matching, and perform the operation of adding and deleting the contact information to the face image with the failed matching; the user can also store the current face image and the corresponding contact information acquired by the brooch equipment to a server-side database to be used as a group of new samples in a training set for subsequent training of the BP neural network; and face images in the unmatched library can be deleted; and can view the deleted face image.

6. The method as claimed in claim 4 or 5, wherein the constant a is a =10.

Images