CN108806670B - Audio recognition method, device and storage medium - Google Patents

Abstract

The invention discloses a speech recognition method, a device and a storage medium, which belong to the technical field of speech processing. The method includes: collecting target speech to be recognized, and acquiring acoustic features of the target speech. The target recognition model is invoked, the acoustic features are input into the target recognition model, and the behavioral intent label corresponding to the target speech is output, and the target recognition model is used to identify the behavioral intent corresponding to the speech according to the acoustic characteristics of any speech. In the embodiment of the present invention, whether it is standard speech or non-standard speech, the corresponding behavioral intent can be identified through the target recognition model based on the acoustic features, which enhances the applicability of speech recognition.

Description

Speech recognition method, device and storage medium

technical field

Embodiments of the present invention relate to the technical field of speech processing, and in particular, to a speech recognition method, device, and storage medium.

Background technique

At present, speech recognition technology has been widely used. For example, in the process of using the terminal, the user may use the voice recognition technology to control the terminal, for example, control the terminal to turn on the camera, and the like.

In the related art, after the terminal collects the voice input by the user, it sends the voice to a voice conversion server, and the voice conversion server can convert the voice into text, and then sends the converted text to the terminal. After receiving the text, the terminal can send the text to a semantic recognition server, and the semantic recognition server performs semantic recognition on the text and feeds back the recognition result to the terminal. In this way, the terminal can perform a corresponding operation based on the identification result.

However, in the above implementation process, only standard speech can be recognized, that is, only Mandarin can be recognized, and the applicability of speech recognition is poor.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a speech recognition method, device and storage medium, which can solve the problem of poor applicability of speech recognition in the related art. The technical solution is as follows:

In a first aspect, a speech recognition method is provided, the method comprising:

Collect the target speech to be recognized;

obtaining the acoustic features of the target speech;

Invoke the target recognition model, input the acoustic features into the target recognition model, and output the behavioral intent label corresponding to the target voice, and the target recognition model is used to identify the corresponding voice according to the acoustic features of any voice. behavioral intent.

Optionally, before invoking the target recognition model, the method further includes:

Acquiring the acoustic features of at least one voice training sample and the behavioral intent label corresponding to each voice training sample;

Based on the acoustic feature of the at least one speech training sample and the behavioral intent label corresponding to each speech training sample, the recognition model to be trained is trained to obtain the target recognition model.

Optionally, obtain the behavioral intent label corresponding to each speech training sample, including:

Get at least one voice;

determining a behavior operation corresponding to each voice in the at least one voice;

Generate a behavioral intent label corresponding to each behavioral operation;

The at least one speech is determined as the at least one speech training sample, and each generated behavioral intent label is determined as a behavioral intent label of the corresponding speech training sample.

Optionally, before the acquiring at least one voice, the method further includes:

According to the voiceprint feature of each voice, query whether the at least one voice comes from a target user, and the target user refers to a user associated with the first terminal;

When the at least one voice comes from the target user, the operation of acquiring the at least one voice is performed.

Optionally, according to the voiceprint feature of each voice, query whether the at least one voice is from the target user, including:

determining the difference value between the voiceprint feature of each voice and the preset voiceprint feature;

When the difference between the voiceprint feature of each voice and the preset voiceprint feature is less than a preset threshold, it is determined that the at least one voice comes from the target user.

Optionally, after the recognition model to be trained is trained based on the acoustic feature of the at least one speech training sample and the behavioral intent label corresponding to each speech training sample, and the target recognition model is obtained, the method further includes:

The target recognition model is shared with a second terminal, where the second terminal refers to a terminal associated with the first terminal.

In a second aspect, a speech recognition device is provided, the device comprising:

The acquisition module is used to collect the target voice to be recognized;

a first acquisition module, used for acquiring the acoustic features of the target speech;

The calling module is used to call the target recognition model, input the acoustic features into the target recognition model, and output the behavioral intent label corresponding to the target voice, and the target recognition model is used to identify according to the acoustic features of any voice The behavioral intent corresponding to the voice.

Optionally, the device further includes:

The second acquisition module is used to acquire the acoustic feature of at least one voice training sample and the behavioral intent label corresponding to each voice training sample;

A training module, configured to train the recognition model to be trained based on the acoustic feature of the at least one speech training sample and the behavioral intent label corresponding to each speech training sample to obtain the target recognition model.

Optionally, the second obtaining module is used for:

Get at least one voice;

determining a behavior operation corresponding to each voice in the at least one voice;

Generate a behavioral intent label corresponding to each behavioral operation;

The at least one speech is determined as the at least one speech training sample, and each generated behavioral intent label is determined as a behavioral intent label of the corresponding speech training sample.

Optionally, the second obtaining module is also used for:

According to the voiceprint feature of each voice, query whether the at least one voice comes from a target user, and the target user refers to a user associated with the first terminal;

When the at least one voice comes from the target user, the operation of acquiring the at least one voice is performed.

Optionally, the second obtaining module is also used for:

determining the difference value between the voiceprint feature of each voice and the preset voiceprint feature;

When the difference between the voiceprint feature of each voice and the preset voiceprint feature is less than a preset threshold, it is determined that the at least one voice comes from the target user.

Optionally, the device further includes:

A sharing module, configured to share the target recognition model with a second terminal, where the second terminal refers to a terminal that is associated with the first terminal.

In a third aspect, a computer-readable storage medium is provided, where instructions are stored on the computer-readable storage medium, and when the instructions are executed by a processor, the speech recognition method described in the first aspect above is implemented.

In a fourth aspect, there is provided a computer program product containing instructions that, when run on a computer, cause the computer to execute the speech recognition method described in the first aspect above.

In a fifth aspect, a computing device is provided, comprising:

one or more processors;

memory for storing one or more programs,

When the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the speech recognition method described in the first aspect above.

The beneficial effects brought by the technical solutions provided in the embodiments of the present invention are:

The target speech to be recognized is collected, and the acoustic features of the target speech are acquired. Call the target recognition model. Since the target recognition model can identify the behavioral intent corresponding to the speech according to the acoustic features of any speech, after inputting the acquired acoustic features into the target recognition model, the behavior corresponding to the target speech can be output Intent label. In the embodiment of the present invention, whether it is standard speech or non-standard speech, the corresponding behavioral intent can be identified through the target recognition model based on the acoustic features, which enhances the applicability of speech recognition.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a flowchart of a speech recognition method according to an exemplary embodiment;

2 is a flowchart of a speech recognition method according to another exemplary embodiment;

3 is a schematic structural diagram of a speech recognition apparatus according to an exemplary embodiment;

4 is a schematic structural diagram of a speech recognition apparatus according to another exemplary embodiment;

5 is a schematic structural diagram of a speech recognition apparatus according to another exemplary embodiment;

FIG. 6 is a schematic structural diagram of a terminal 600 according to another exemplary embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

Before describing in detail the speech recognition method provided by the embodiments of the present invention, the application scenarios and implementation environments involved in the embodiments of the present invention are briefly introduced.

First, the application scenarios involved in the embodiments of the present invention are briefly introduced.

In some application scenarios, in order to improve the convenience of operation, there is a need to control the terminal through voice, for example, the application scenarios include but are not limited to environments such as home and vehicle. In the process of controlling the terminal by voice, in order to know the behavior intention corresponding to the voice, voice recognition needs to be performed. At present, in the process of speech recognition, it is necessary to convert the speech to be recognized into text, and then perform semantic recognition on the converted text. However, in the above implementation process, only text conversion and semantic recognition can be performed on standard speech, and non-standard speech (such as dialect) cannot be recognized, resulting in poor applicability of speech recognition.

To this end, the embodiment of the present invention provides a speech recognition method, which can recognize the corresponding behavioral intent through the target recognition model based on the acoustic features of the speech. Non-standard speech can realize speech recognition, which increases the applicability of speech recognition. For the specific implementation process, please refer to the embodiment shown in FIG. 1 or FIG. 2 below.

Next, the implementation environment involved in the embodiments of the present invention is briefly introduced.

The speech recognition method provided by the embodiment of the present invention may be performed by a first terminal, and the first terminal may be configured with a speech collection device, for example, the speech collection device may be a microphone array, etc., for speech collection. In some embodiments, the first terminal may be any electronic product that can interact with the user in one or more ways, such as a keyboard, a touch panel, a touch screen, a remote control, a voice interaction or a handwriting device, for example PC, mobile phone, smart phone, PDA, wearable device, handheld computer PPC, wearable device, tablet computer, smart car machine, smart TV, smart speaker, etc. In practical applications, when the first terminal is an electronic product that can interact with the user by voice, a client capable of recognizing, parsing, understanding, processing and responding to the user's natural language command and outputting the response result can be mounted/installed on the first terminal. The terminal (which can be in the form of an APP), or the client can only perform speech recognition on the natural language command input by the user, but requires a corresponding server to parse, understand, process and respond to the user's natural language command. And return the response result to the client for output.

Further, the first terminal may be connected to at least one second terminal, and in a possible implementation manner, the at least one second terminal and the first terminal may both belong to the same user.

Here, the second terminal may be any electronic product that can perform human-computer interaction with the user in one or more ways such as a keyboard, a touch panel, a touch screen, a remote control, a voice interaction or a handwriting device, such as a PC, Mobile phones, smart phones, PDAs, wearable devices, PDAs, PPCs, wearable devices, tablet PCs, smart car machines, smart TVs, smart speakers, etc. In practical applications, when the second terminal is an electronic product that can interact with the user by voice, a client capable of recognizing, parsing, understanding, processing and responding to the user's natural language command and outputting the response result can be mounted/installed on the second terminal. The terminal (which can be in the form of an APP), or the client can only perform speech recognition on the natural language command input by the user, but requires a corresponding server to parse, understand, process and respond to the user's natural language command. And return the response result to the client for output.

Here, both the first terminal and the second terminal include an electronic device that can automatically perform numerical calculation and information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs) ), programmable gate array (FPGA), digital processor (DSP), embedded devices, etc.

After introducing the application scenarios and implementation environments involved in the embodiments of the present invention, the speech recognition method provided by the embodiments of the present invention will be described in detail next with reference to the accompanying drawings.

Please refer to FIG. 1. FIG. 1 is a flowchart of a speech recognition method according to an exemplary embodiment. The speech recognition method can be executed by the above-mentioned first terminal, and the speech recognition method can include the following implementation steps:

Step 101: Collect the target speech to be recognized.

Step 102: Acquire the acoustic features of the target speech.

Step 103: call the target recognition model, input the acoustic feature into the target recognition model, output the behavioral intent label corresponding to the target voice, and the target recognition model is used to identify the behavioral intent corresponding to the voice according to the acoustic feature of any voice .

Further, before calling the target recognition model, acquire the acoustic feature of at least one voice training sample and the behavioral intent label corresponding to each voice training sample, and based on the acoustic feature of the at least one voice training sample and the behavior corresponding to each voice training sample Intent label, train the recognition model to be trained, and obtain the target recognition model.

Wherein, the above-mentioned implementation process of acquiring the behavioral intent label corresponding to each speech training sample may include: acquiring at least one speech; determining the behavioral operation corresponding to each speech in the at least one speech; generating the behavioral intent label corresponding to each behavioral operation; The at least one speech is determined as the at least one speech training sample, and each generated behavioral intent label is determined as a behavioral intent label of the corresponding speech training sample.

Further, before acquiring at least one voice, according to the voiceprint feature of each voice, query whether the at least one voice comes from a target user, and the target user refers to a user associated with the first terminal; when the at least one voice When the voices all come from the target user, the operation of acquiring at least one voice is performed.

In a possible implementation manner, according to the voiceprint feature of each voice, the implementation of querying whether the at least one voice comes from the target user may include: determining the difference between the voiceprint feature of each voice and a preset voiceprint feature When the difference between the voiceprint feature of each voice and the preset voiceprint feature is less than a preset threshold, it is determined that the at least one voice comes from the target user.

In some embodiments, the recognition model to be trained is trained based on the acoustic feature of the at least one speech training sample and the behavioral intent label corresponding to each speech training sample, and after the target recognition model is obtained, the second The terminal shares the target recognition model, and the second terminal refers to a terminal associated with the first terminal.

In the embodiment of the present invention, the target speech to be recognized is collected, and the acoustic features of the target speech are acquired. Call the target recognition model. Since the target recognition model can identify the behavioral intent corresponding to the speech according to the acoustic features of any speech, after inputting the acquired acoustic features into the target recognition model, the behavior corresponding to the target speech can be output Intent label. In the embodiment of the present invention, whether it is standard speech or non-standard speech, the corresponding behavioral intent can be identified through the target recognition model based on the acoustic features, which enhances the applicability of speech recognition.

FIG. 2 is a flowchart of a speech recognition method according to another exemplary embodiment. This embodiment is illustrated by applying the speech recognition method to the above-mentioned first terminal. The speech recognition method may include the following implementations step:

Step 201: Collect the target speech to be recognized.

When the user wants to control the first terminal by voice, he can speak directly to a voice collecting device such as a microphone array in the first terminal. Correspondingly, the first terminal can collect the words spoken by the user through the voice collecting device, that is, collect the target voice to be recognized.

Further, the first terminal can collect the target voice to be recognized when receiving a voice recognition instruction, and the voice recognition instruction can be triggered by a user, and the user can be triggered by a specified operation, and the specified operation can include a click operation, a sliding operation etc., which is not limited in this embodiment of the present invention.

For example, a voice recognition option may be provided in the first terminal, and when the user wants to control the first terminal by voice, the user may click on the voice recognition option to trigger the voice recognition instruction. After receiving the voice recognition instruction, the first terminal collects the target voice to be recognized.

Step 202: Acquire the acoustic features of the target speech.

In order to facilitate subsequent speech recognition of the target speech, the first terminal acquires the acoustic feature of the target speech. Wherein, the acoustic feature can be used to describe at least one of loudness, pitch, frequency and timbre of the target speech.

Step 203: call the target recognition model, input the acoustic feature into the target recognition model, output the behavioral intent label corresponding to the target voice, and the target recognition model is used to identify the behavioral intent corresponding to the voice according to the acoustic feature of any voice .

Since the target recognition model can be used to identify the behavioral intention corresponding to the speech according to the acoustic characteristics of any speech, the first terminal can output the corresponding target speech after inputting the acoustic characteristics of the target speech into the target recognition model. behavioral intent label.

In some embodiments, the behavioral intent tag can be a behavioral intent sequence, in other words, the behavioral intent tag can be used to describe a series of behavioral actions, for example, the behavioral intent tag can be used to describe “turn on the camera, then Log in to WeChat, and play music later.” This series of behaviors and actions, etc.

Further, after determining the behavior intent label corresponding to the target speech, the first terminal may execute the behavior operation corresponding to the behavior intent label. For example, if the behavior intent tag is the activation camera tag, the first terminal may activate the installed camera application. In this way, the purpose of using the target voice to control the first terminal is achieved.

Further, before calling the target recognition model, model training needs to be performed to obtain the above-mentioned target recognition model. In a possible implementation, the training process may include: acquiring the acoustic features of at least one voice training sample and corresponding to each voice training sample. Based on the acoustic feature of the at least one speech training sample and the behavioral intent label corresponding to each speech training sample, the recognition model to be trained is trained to obtain the target recognition model.

That is, the first terminal acquires the acoustic feature of the at least one voice training sample and the behavioral intent label corresponding to each voice training sample, and compares the acquired acoustic feature of the at least one voice training sample with the corresponding value of each voice training sample. The behavioral intent label is input into the recognition model to be trained, and the recognition model is subjected to deep learning and training, thereby obtaining a target recognition model capable of recognizing the behavioral intent of any speech based on the acoustic features of the speech.

In some embodiments, the recognition model to be trained may be a CNN (Convolutional Neural Networks, convolutional neural network) model, which is not limited in this embodiment of the present invention.

Wherein, the above-mentioned implementation process of acquiring a behavior intent label corresponding to each speech training sample may include: acquiring at least one speech, and determining a behavior operation corresponding to each speech in the at least one speech. A behavioral intent label corresponding to each behavioral operation is generated, the at least one speech is determined as the at least one speech training sample, and each generated behavioral intent label is determined as a behavioral intent label of the corresponding speech training sample.

In a possible implementation manner, the behavior operation corresponding to each voice is performed when an operation instruction triggered for each voice is received. In some embodiments, the user may input at least one voice to the first terminal, and after inputting each voice in the at least one voice, a behavior operation may be triggered for each voice by manual triggering. For example, the user can say "start Baidu map" to the first terminal in dialect, and then the user can manually start Baidu map. At this time, the first terminal may collect each voice in the at least one voice, and determine the corresponding behavior operation, for example, determine the behavior operation of starting the Baidu map. After that, the first terminal generates a behavior intent label of the determined behavior operation, for example, the behavior intent label may be "start Baidu map label". The first terminal determines each acquired speech as each speech training sample in the at least one speech training sample, and determines each generated behavior label as a behavior intention label of the corresponding speech training sample. In this way, the first terminal can train the recognition model to be trained based on the determined at least one speech training sample and the behavior intention label corresponding to each speech training sample.

Of course, the above is only described by taking an example that the behavior operation corresponding to each voice is performed when an operation instruction triggered for each voice is received. In another embodiment, the behavior operation corresponding to each voice may also be performed when the standard control voice entered for each voice is collected. Wherein, the standard control voice usually refers to Mandarin. In some embodiments, since the first terminal can recognize Mandarin, the user inputs at least one voice to the first terminal, and, after inputting each voice in the at least one voice, can control the voice for each voice through a standard control voice. A voice-triggered behavioral action. For example, the user can say "start Baidu map" to the first terminal in dialect, and then the user can control the first terminal to start Baidu map in Mandarin. At this time, the first terminal may collect each voice in the at least one voice, and determine the corresponding behavior operation, for example, determine the behavior operation of starting the Baidu map. After that, the first terminal generates a behavior intent label of the determined behavior operation, for example, the behavior intent label may be "start Baidu map label". The first terminal determines each acquired speech as each speech training sample in the at least one speech training sample, and determines each generated behavior label as a behavior intention label of the corresponding speech training sample.

Further, before acquiring the at least one voice, the source of the at least one voice can also be limited, and the corresponding processing process includes: according to the voiceprint feature of each voice, inquiring whether the at least one voice comes from the target user, the target user The user refers to a user with an associated relationship with the first terminal, and when the at least one voice comes from the target user, the operation of acquiring the at least one voice is performed.

In some embodiments, such as in a multi-person scenario, the first terminal may only perform training based on the voice of a target user associated with itself. For example, the target user may be the owner of the first terminal. In this case, before acquiring the at least one voice, the first terminal needs to determine whether the at least one voice is from the target user. The first terminal may query whether the at least one voice comes from the target user according to the voiceprint feature of each voice.

In a possible implementation manner, the above-mentioned implementation of querying whether the at least one voice comes from the target user according to the voiceprint feature of each voice may include: determining the voiceprint feature and the preset voiceprint feature of each voice When the difference between the voiceprint feature of each voice and the preset voiceprint feature is less than a preset threshold, it is determined that the at least one voice comes from the target user.

The preset voiceprint feature may be preselected and stored in the first terminal, and the preset voiceprint feature may be the voiceprint feature of the target user. In this way, after acquiring the voiceprint feature of each voice, the first terminal can compare the acquired voiceprint feature of each voice with the pre-stored preset voiceprint feature, and determine the voiceprint feature of each voice and the preset voiceprint feature. Set the difference value between voiceprint features.

In some embodiments, the acquired voiceprint feature of each voice may be pattern-matched with a pre-stored preset voiceprint feature to determine the difference between the voiceprint feature of each voice and the preset voiceprint feature value. The method for pattern matching may include: probability statistics method, artificial neural network method, etc., which are not limited in this embodiment of the present application.

When the difference value is less than the preset threshold, it means that the difference between the voiceprint feature of a certain voice being compared and the preset voiceprint feature is not big, and at this time, it can be determined that the voice being compared is from the target user. Conversely, if the difference value is greater than the preset threshold, it means that the difference between the voiceprint feature of a certain voice being compared and the preset voiceprint feature is relatively large. Therefore, it can be determined that the compared voice does not come from the voiceprint feature. Target users. In this way, by comparing each voiceprint feature with a preset threshold, it can be determined whether the at least one voice comes from the target user.

The preset threshold may be set by the user according to actual needs, or may be set by default by the first terminal, which is not limited in this embodiment of the present invention.

Further, the above is only an example of querying whether the at least one voice comes from the target user according to the voiceprint feature of each voice. In another embodiment, the at least one voice can be queried according to other information. The target user, for example, can also determine whether the at least one voice comes from the target user in combination with the sound source position of each voice.

In a possible implementation manner, the first terminal may determine, according to a voiceprint feature of any voice in the collected at least one voice, whether the any voice comes from the target user. When it is determined that the any voice is from the target user, then determine whether the sound source position of the other voices in the at least one voice is the same as the sound source position of the any voice, that is, determine whether the at least one voice comes from the same direction. When the sound source positions of the at least one voice are all the same, it is determined that the at least one voice comes from the same user, that is, they all come from the target user. Otherwise, it may be determined that the at least one voice does not all come from the same target user. The determination of the sound source position of each speech may be determined based on parameters such as the received intensity of the collected speech, which is not limited in this embodiment of the present invention.

It should be noted that, when the above at least one voice does not all come from the target user, the voice that does not come from the target user in the at least one voice may be deleted, and then the deleted at least one voice is obtained.

So far, the speech recognition method involved in the embodiment of the present invention has been implemented. Further, since the terminals used by the user may include multiple terminals, in order to enable other terminals to perform speech recognition in the same way as the first terminal, after the first terminal trains the recognition model to be trained to obtain the target recognition model, The target recognition model can be shared. For specific implementation, please refer to the following step 204.

Step 204: Share the target recognition model with a second terminal, where the second terminal refers to a terminal that is associated with the first terminal.

In a possible implementation manner, the association relationship refers to belonging to the same user as the first terminal, or the association relationship refers to having a connection relationship with the first terminal, and further alternatively, the association relationship refers to having a connection relationship with the first terminal The first terminal is in the same environment and so on.

Next, description will be given by taking an example that the second terminal and the first terminal belong to the same user, for example, both the second terminal and the first terminal belong to the above-mentioned target user. That is to say, in order to enable the target user to use voice to control the second terminal when using the second terminal, the first terminal can send the target recognition model to itself after training to obtain the target recognition model. A second terminal belonging to the same user. Correspondingly, after receiving the target recognition model shared by the first terminal, the second terminal saves the target recognition model locally.

Of course, the first terminal can also receive the trained target recognition model sent by the second terminal, that is to say, the second terminal can also perform model training according to the above implementation manner, and share the trained target recognition model with the target recognition model. first terminal. The first terminal receives the trained target recognition model shared by the second terminal, and further, when the target recognition model is stored in the first terminal, if the latest target recognition model shared by the second terminal is received, the The first terminal may delete the originally stored target recognition model, and save the latest target recognition model shared by the second terminal.

It is worth mentioning that the above-mentioned first terminal shares the target recognition model obtained after training with the second terminal, so that the second terminal saves the same target recognition model as the first terminal. In this way, the second terminal can directly use the target recognition model. The recognition model performs speech recognition, avoiding the need for self-training, and reducing the number of training times for the second terminal.

In the embodiment of the present invention, the target speech to be recognized is collected, and the acoustic features of the target speech are acquired. Call the target recognition model. Since the target recognition model can identify the behavioral intent corresponding to the speech according to the acoustic features of any speech, after inputting the acquired acoustic features into the target recognition model, the behavior corresponding to the target speech can be output Intent label. In the embodiment of the present invention, whether it is standard speech or non-standard speech, the corresponding behavioral intent can be identified through the target recognition model based on the acoustic features, which enhances the applicability of speech recognition.

FIG. 3 is a schematic structural diagram of a speech recognition apparatus according to an exemplary embodiment, and the speech recognition apparatus may be implemented by software, hardware, or a combination of the two. The speech recognition device may include:

The collection module 301 is used to collect the target speech to be recognized;

a first acquisition module 302, configured to acquire the acoustic features of the target speech;

The calling module 303 is used to call the target recognition model, input the acoustic features into the target recognition model, and output the behavioral intent label corresponding to the target voice, and the target recognition model is used for the acoustic features of any voice according to the Identify the behavioral intent corresponding to the speech.

Optionally, please refer to FIG. 4, the device further includes:

The second acquisition module 304 is used to acquire the acoustic feature of at least one speech training sample and the behavioral intent label corresponding to each speech training sample;

The training module 305 is configured to train the recognition model to be trained based on the acoustic feature of the at least one speech training sample and the behavioral intent label corresponding to each speech training sample to obtain the target recognition model.

Optionally, the second obtaining module 304 is used for:

Get at least one voice;

determining a behavior operation corresponding to each voice in the at least one voice;

Generate a behavioral intent label corresponding to each behavioral operation;

The at least one speech is determined as the at least one speech training sample, and each generated behavioral intent label is determined as a behavioral intent label of the corresponding speech training sample.

Optionally, the second obtaining module 304 is further configured to:

According to the voiceprint feature of each voice, query whether the at least one voice comes from a target user, and the target user refers to a user associated with the first terminal;

When the at least one voice comes from the target user, the operation of acquiring the at least one voice is performed.

Optionally, the second obtaining module 304 is further configured to:

determining the difference value between the voiceprint feature of each voice and the preset voiceprint feature;

When the difference between the voiceprint feature of each voice and the preset voiceprint feature is less than a preset threshold, it is determined that the at least one voice comes from the target user.

Optionally, please refer to FIG. 5, the apparatus further includes:

The sharing module 306 is configured to share the target recognition model with a second terminal, where the second terminal refers to a terminal that is associated with the first terminal.

In the embodiment of the present invention, the target speech to be recognized is collected, and the acoustic features of the target speech are acquired. Call the target recognition model. Since the target recognition model can identify the behavioral intent corresponding to the speech according to the acoustic features of any speech, after inputting the acquired acoustic features into the target recognition model, the behavior corresponding to the target speech can be output Intent label. In the embodiment of the present invention, whether it is standard speech or non-standard speech, the corresponding behavioral intent can be identified through the target recognition model based on the acoustic features, which enhances the applicability of speech recognition.

It should be noted that: when the speech recognition device provided in the above embodiment implements the speech recognition method, only the division of the above functional modules is used for illustration. , that is, dividing the internal structure of the device into different functional modules to complete all or part of the functions described above. In addition, the speech recognition device and the speech recognition method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.

FIG. 6 shows a structural block diagram of a terminal 600 provided by an exemplary embodiment of the present invention. The terminal 600 can be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, the standard audio layer of the moving picture experts compression), MP4 (Moving Picture Experts Group Audio Layer IV, the standard audio layer of the moving picture experts compression) 4) Player, laptop or desktop computer. Terminal 600 may also be called user equipment, portable terminal, laptop terminal, desktop terminal, and the like by other names.

Generally, the terminal 600 includes: a processor 601 and a memory 602 .

The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 601 may adopt at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, programmable logic array) accomplish. The processor 601 may also include a main processor and a coprocessor. The main processor is a processor used to process data in a wake-up state, also called a CPU (Central Processing Unit, central processing unit); A low-power processor for processing data in a standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor 601 may further include an AI (Artificial Intelligence, artificial intelligence) processor, where the AI processor is used to process computing operations related to machine learning.

Memory 602 may include one or more computer-readable storage media, which may be non-transitory. Memory 602 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 601 to implement the speech recognition provided by the method embodiments in this application. method.

In some embodiments, the terminal 600 may optionally further include: a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602 and the peripheral device interface 603 may be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 603 through a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 604 , a touch display screen 605 , a camera 606 , an audio circuit 607 , a positioning component 608 and a power supply 609 .

The peripheral device interface 603 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 601 and the memory 602 . In some embodiments, processor 601, memory 602, and peripherals interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one of processor 601, memory 602, and peripherals interface 603 or The two can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The radio frequency circuit 604 is used for receiving and transmitting RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals. The radio frequency circuit 604 communicates with the communication network and other communication devices via electromagnetic signals. The radio frequency circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 604 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 604 may communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: World Wide Web, Metropolitan Area Network, Intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), wireless local area network and/or WiFi (Wireless Fidelity, Wireless Fidelity) network. In some embodiments, the radio frequency circuit 604 may further include a circuit related to NFC (Near Field Communication, short-range wireless communication), which is not limited in this application.

The display screen 605 is used to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 605 is a touch display screen, the display screen 605 also has the ability to acquire touch signals on or above the surface of the display screen 605 . The touch signal may be input to the processor 601 as a control signal for processing. At this time, the display screen 605 may also be used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards. In some embodiments, there may be one display screen 605, which is provided on the front panel of the terminal 600; in other embodiments, there may be at least two display screens 605, which are respectively arranged on different surfaces of the terminal 600 or in a folded design; In still other embodiments, the display screen 605 may be a flexible display screen, which is disposed on a curved surface or a folding surface of the terminal 600 . Even, the display screen 605 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen. The display screen 605 can be made of materials such as LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, organic light emitting diode).

The camera assembly 606 is used to capture images or video. Optionally, the camera assembly 606 includes a front camera and a rear camera. Usually, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth-of-field camera to realize the background blur function, the main camera It is integrated with the wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting functions or other integrated shooting functions. In some embodiments, camera assembly 606 may also include a flash. The flash can be a single color temperature flash or a dual color temperature flash. Dual color temperature flash refers to the combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.

Audio circuitry 607 may include a microphone and speakers. The microphone is used to collect the sound waves of the user and the environment, convert the sound waves into electrical signals and input them to the processor 601 for processing, or to the radio frequency circuit 604 to realize voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively disposed in different parts of the terminal 600 . The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 601 or the radio frequency circuit 604 into sound waves. The loudspeaker can be a traditional thin-film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for distance measurement and other purposes. In some embodiments, the audio circuit 607 may also include a headphone jack.

The positioning component 608 is used to locate the current geographic location of the terminal 600 to implement navigation or LBS (Location Based Service, location-based service). The positioning component 608 may be a positioning component based on the GPS (Global Positioning System, global positioning system) of the United States, the Beidou system of China or the Galileo system of Russia.

The power supply 609 is used to power various components in the terminal 600 . The power source 609 may be alternating current, direct current, disposable batteries or rechargeable batteries. When the power source 609 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. Wired rechargeable batteries are batteries that are charged through wired lines, and wireless rechargeable batteries are batteries that are charged through wireless coils. The rechargeable battery can also be used to support fast charging technology.

In some embodiments, terminal 600 also includes one or more sensors 610 . The one or more sensors 610 include, but are not limited to, an acceleration sensor 611 , a gyro sensor 612 , a pressure sensor 613 , a fingerprint sensor 614 , an optical sensor 615 and a proximity sensor 616 .

The acceleration sensor 611 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 600 . For example, the acceleration sensor 611 can be used to detect the components of the gravitational acceleration on the three coordinate axes. The processor 601 can control the touch display screen 605 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 611 . The acceleration sensor 611 can also be used for game or user movement data collection.

The gyroscope sensor 612 can detect the body direction and rotation angle of the terminal 600 , and the gyroscope sensor 612 can cooperate with the acceleration sensor 611 to collect 3D actions of the user on the terminal 600 . The processor 601 can implement the following functions according to the data collected by the gyro sensor 612 : motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

The pressure sensor 613 may be disposed on the side frame of the terminal 600 and/or the lower layer of the touch display screen 605 . When the pressure sensor 613 is disposed on the side frame of the terminal 600, the user's holding signal of the terminal 600 can be detected, and the processor 601 can perform left and right hand identification or quick operation according to the holding signal collected by the pressure sensor 613. When the pressure sensor 613 is disposed on the lower layer of the touch display screen 605 , the processor 601 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 605 . The operability controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

The fingerprint sensor 614 is used to collect the user's fingerprint, and the processor 601 identifies the user's identity according to the fingerprint collected by the fingerprint sensor 614, or the fingerprint sensor 614 identifies the user's identity according to the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 601 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 614 may be provided on the front, back or side of the terminal 600 . When the terminal 600 is provided with physical buttons or a manufacturer's logo, the fingerprint sensor 614 may be integrated with the physical buttons or the manufacturer's logo.

Optical sensor 615 is used to collect ambient light intensity. In one embodiment, the processor 601 may control the display brightness of the touch display screen 605 according to the ambient light intensity collected by the optical sensor 615 . Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 605 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 605 is decreased. In another embodiment, the processor 601 may also dynamically adjust the shooting parameters of the camera assembly 606 according to the ambient light intensity collected by the optical sensor 615 .

A proximity sensor 616 , also called a distance sensor, is usually provided on the front panel of the terminal 600 . The proximity sensor 616 is used to collect the distance between the user and the front of the terminal 600 . In one embodiment, when the proximity sensor 616 detects that the distance between the user and the front of the terminal 600 gradually decreases, the processor 601 controls the touch display screen 605 to switch from the bright screen state to the off screen state; when the proximity sensor 616 detects When the distance between the user and the front of the terminal 600 gradually increases, the processor 601 controls the touch display screen 605 to switch from the screen-off state to the screen-on state.

Those skilled in the art can understand that the structure shown in FIG. 6 does not constitute a limitation on the terminal 600, and may include more or less components than the one shown, or combine some components, or adopt different component arrangements.

The embodiment of the present application further provides a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can execute the above embodiment shown in FIG. 1 or FIG. 2 Provided speech recognition methods.

The embodiment of the present application also provides a computer program product including instructions, which, when running on a computer, enables the computer to execute the speech recognition method provided by the embodiment shown in FIG. 1 or FIG. 2 .

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, etc.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (6)

1. a kind of audio recognition method is applied in first terminal, which is characterized in that the described method includes:

Acquire target voice to be identified；

Obtain the acoustic feature of the target voice；

The acoustic feature is input in the Model of Target Recognition by invocation target identification model, exports the target voice Corresponding behavior intention labels, the Model of Target Recognition are used to identify that the voice is corresponding according to the acoustic feature of any voice Behavior be intended to；

Wherein, before the invocation target identification model, this method further include:

Obtain the acoustic feature and the corresponding behavior intention labels of each voice training sample of at least one voice training sample；

The corresponding behavior meaning of acoustic feature and each voice training sample based at least one voice training sample Icon label are treated trained identification model and are trained, obtain the Model of Target Recognition；

Wherein, the corresponding behavior intention labels of each voice training sample are obtained, comprising:

Obtain at least one voice；

Determine the corresponding behavior operation of each voice at least one described voice, wherein the corresponding behavior behaviour of each voice Make include it is following any one of at least: the corresponding behavior operation of-each voice is being received for each speech trigger It is performed when operational order；The corresponding behavior operation of each voice, which can also be, to be collected for each Speech Record It is performed when the standard control voice entered；

It generates each behavior and operates corresponding behavior intention labels；

At least one described voice is determined as at least one described voice training sample, and each behavior of generation is intended to Label is determined as the behavior intention labels of corresponding voice training sample；

Before described at least one voice of acquisition, further includes:

According to the vocal print feature of each voice, inquire whether at least one described voice is all from target user；

When at least one described voice is all from the target user, the operation for obtaining at least one voice is executed；

The acoustic feature and the corresponding row of each voice training sample based at least one voice training sample It for intention labels, treats trained identification model and is trained, after obtaining the Model of Target Recognition, further includes:

Share the Model of Target Recognition to second terminal, the second terminal, which refers to, has incidence relation with the first terminal Terminal.

2. the method as described in claim 1, which is characterized in that the vocal print feature according to each voice inquires institute State whether at least one voice is all from target user, comprising:

Determine the difference value between the vocal print feature and default vocal print feature of each voice；

When the difference value between the vocal print feature of each voice and the default vocal print feature is respectively less than preset threshold, really At least one fixed described voice is all from the target user.

3. a kind of speech recognition equipment, it is applied in first terminal, which is characterized in that described device includes:

Acquisition module, for acquiring target voice to be identified；

First obtains module, for obtaining the acoustic feature of the target voice；

Calling module is used for invocation target identification model, the acoustic feature is input in the Model of Target Recognition, exports The corresponding behavior intention labels of the target voice, the Model of Target Recognition are used to be identified according to the acoustic feature of any voice The corresponding behavior of the voice is intended to；

Described device further include:

Second obtains module, and the acoustic feature and each voice training sample for obtaining at least one voice training sample are corresponding Behavior intention labels；

Training module, for acoustic feature and each voice training sample based at least one voice training sample Corresponding behavior intention labels are treated trained identification model and are trained, obtain the Model of Target Recognition；

The second acquisition module is used for:

Obtain at least one voice；

Determine the corresponding behavior operation of each voice at least one described voice；

It generates each behavior and operates corresponding behavior intention labels；

At least one described voice is determined as at least one described voice training sample, and each behavior of generation is intended to Label is determined as the behavior intention labels of corresponding voice training sample；

The second acquisition module is also used to:

According to the vocal print feature of each voice, inquire whether at least one described voice is all from target user, the mesh Mark user refers to the user for having incidence relation with the first terminal；

When at least one described voice is all from the target user, the operation for obtaining at least one voice is executed；

Described device further include:

Sharing module, for sharing the Model of Target Recognition to second terminal, the second terminal refers to described first eventually Hold the terminal with incidence relation.

4. device as claimed in claim 3, which is characterized in that the second acquisition module is also used to:

Determine the difference value between the vocal print feature and default vocal print feature of each voice；

When the difference value between the vocal print feature of each voice and the default vocal print feature is respectively less than preset threshold, really At least one fixed described voice is all from the target user.

5. a kind of computer readable storage medium, instruction is stored on the computer readable storage medium, which is characterized in that institute It states and realizes method of any of claims 1-2 when instruction is executed by processor.

6. a kind of calculating equipment, comprising:

One or more processors；

Memory, for storing one or more programs,

When one or more of programs are executed by one or more of processors, so that one or more of processors Execute such as method of any of claims 1-2.