TWI809335B - Personalized speech recognition method and speech recognition system - Google Patents

Abstract

The disclosure provides a personalized speech recognition method and a speech recognition system. The method includes: in response to obtaining a voice input message, forwarding the voice input message to a cloud voice recognition system; receiving specific acoustic recognition information and a specific voice recognition result corresponding to the voice input message from the cloud voice recognition system; using a personalized voice recognition model dedicated to a voice recognition module to correct the specific voice recognition result as a first voice recognition results based on the specific acoustic recognition information.

Description

Personalized speech recognition method and speech recognition system

The present invention relates to a voice recognition technology, and in particular to a personalized voice recognition method and a voice recognition system.

Nowadays, speech recognition is mostly performed through a cloud speech recognition system, and the cloud speech recognition system mainly performs speech recognition by presetting the language used. However, when the user's voice message contains mixed Chinese and English, homophones and nouns in special fields, etc., the voice recognition results are often not as expected.

In this case, in order to modify the speech recognition result, the user needs to upload relevant personal data/information to the cloud speech recognition system to create a personalized speech recognition model. However, nowadays, the awareness of personal data protection is on the rise. Collecting personal data related information for the correction of voice recognition results will violate the privacy of personal data and the rationality of data utilization.

In view of this, the present invention provides a personalized speech recognition method and a speech recognition system, which can be used to solve the above technical problems.

The present invention provides a personalized voice recognition method suitable for a first voice recognition module, including: responding to obtaining a voice input message, forwarding the voice input message to a cloud voice recognition system, wherein the cloud voice recognition system responds to the voice Input a message to generate a specific acoustic recognition information and a specific voice recognition result corresponding to the voice input message; receive the specific acoustic recognition information and specific voice recognition result corresponding to the voice input message from the cloud voice recognition system; use the first voice exclusively A first humanized speech recognition model of the recognition module is used to modify the specific speech recognition result to a first speech recognition result based on the specific acoustic recognition information.

The invention provides a speech recognition system, which includes a first speech recognition module. The first voice recognition module is configured to: in response to obtaining a voice input message, forward the voice input message to a cloud voice recognition system, wherein the cloud voice recognition system generates a specific signal corresponding to the voice input message in response to the voice input message Acoustic recognition information and a specific speech recognition result; receiving specific acoustic recognition information and specific speech recognition results corresponding to the voice input message from the cloud speech recognition system; using a first personalized speech recognition model dedicated to the first speech recognition module to The specific speech recognition result is modified to a first speech recognition result based on the specific acoustic recognition information.

Please refer to FIG. 1 , which is a schematic diagram of a speech recognition system and a cloud speech recognition system according to a first embodiment of the present invention. In the embodiment of the present invention, the speech recognition system 110 may be various smart devices, voice assistants or any electronic device with a speech recognition function, but is not limited thereto.

In FIG. 1 , the speech recognition system 110 may include a speech recognition module 111 , which may record a first personalized speech recognition model dedicated to a first user. In an embodiment of the present invention, the first personalized speech recognition model is trained based on a plurality of first personal information specific to the first user, wherein the first personal information may include the first user's address book, web browsing information, At least one of social software articles and communication software conversations, but not limited thereto.

In one embodiment, after the speech recognition module 111 obtains the above-mentioned first personal information of the first user, each first personal information can be converted into a plurality of corresponding feature vectors, as the first personalized speech recognition model. training data. Afterwards, the speech recognition module 111 can train the first personalized speech recognition model based on the above training data. In this way, the first personalized speech recognition model can learn the characteristics of the first user, such as word usage habits and habitual pronunciation habits, and use them as a basis for subsequent revision of speech recognition results, but it is not limited thereto.

In addition, as shown in FIG. 1 , the voice recognition system 110 can be connected to the cloud voice recognition system 120 . In an embodiment of the present invention, the cloud speech recognition system 120 may store a general speech recognition model. However, as previously mentioned, when the user's voice information contains mixed Chinese and English, homonyms and nouns in special fields, etc., the voice recognition results provided by this general voice recognition model are often not as expected, and the voice recognition of the present invention The system 110 can improve this situation by implementing the personalized speech recognition method in FIG. 2 .

Please refer to FIG. 2 , which is a flowchart of a personalized voice recognition method according to an embodiment of the present invention. The method of this embodiment can be executed by the voice recognition system 110 in FIG. 1 , and the details of each step in FIG. 2 will be described below with the components shown in FIG. 1 .

In the embodiment of the present invention, it is assumed that the first user intends to control the voice recognition system 110 to perform a specific operation by sending a voice input message VS to the voice recognition system 110 .

In this case, in step S210 , in response to obtaining the voice input message VS, the voice recognition module 111 may forward the voice input message VS to the cloud voice recognition system 120 . Correspondingly, the cloud speech recognition system 120 (the general speech recognition model) can, for example, generate specific acoustic recognition information AR and specific speech recognition result VR corresponding to the speech input message VS in response to the speech input message VS, and can convert the specific acoustic recognition information to The AR and the specific voice recognition result VR are sent back to the voice recognition system 110 .

Correspondingly, in step S220 , the voice recognition module 111 may receive specific acoustic recognition information AR and specific voice recognition result VR corresponding to the voice input message VS from the cloud voice recognition system 120 .

After that, in step S230 , the speech recognition module 111 can use the first personalized speech recognition model dedicated to the speech recognition module to modify the specific speech recognition result VR into the speech recognition result VO1 based on the specific acoustic recognition information AR.

Moreover, after the voice recognition result VO1 is obtained, the voice recognition system 110 may also perform a specific operation corresponding to the voice recognition result VO1.

In order to make the above concepts easier to understand, several application scenarios will be added below for further explanation.

In the first application scenario, assume that the voice input message VS sent by the first user is "Call Xiaoming Wang" (Xiaoming Wang is, for example, a contact in the address book of the first user). Correspondingly, the cloud speech recognition system 120 can respond to the specific acoustic recognition information AR and the specific speech recognition result VR provided by the voice input message VS, which can be respectively "d a3 d i4 e4 nn4 h u4 a4 g e3 i3 u2 a2 ng2 x i3 a3 u3 m i2 ng2" and "Call Wang Xiaoming".

In this case, if the voice recognition module 111 performs subsequent operations directly based on the specific voice recognition result VR (ie, "call Wang Xiaoming"), it will result in the result that the correct dialing cannot be performed.

Therefore, in this embodiment, after the voice recognition module 111 receives the above-mentioned specific acoustic recognition information AR and the specific voice recognition result VR, for example, the first personalized voice recognition model can be used to convert the specific voice recognition result based on the specific acoustic recognition information AR VR (that is, "call Wang Xiaoming") is corrected to the speech recognition result VO1 of "call Wang Xiaoming".

Specifically, since the first personalized speech recognition model has been trained based on various first personal information (including contacts) of the first user, the first personalized speech recognition model can know "Wang Xiaoming" based on the specific acoustic recognition information AR The word should be the contact person "Wang Xiaoming" in the address book of the first user in essence, and the above-mentioned speech recognition result VO1 can be generated according to it.

In this way, the voice recognition system 110 can correctly perform a specific operation required by the first user (eg, dial Wang Xiaoming) based on the voice recognition result VO1 .

In the second application scenario, it is assumed that the voice input message VS sent by the first user is "starting the electric cooker", but the word "electric cooker" in it is due to the accent/pronunciation habits of the first user (for example, the habitual station language), which makes its tone closer to "order song", but the word "activation" is the correct pronunciation. Correspondingly, the cloud speech recognition system 120 can respond to the specific acoustic recognition information AR and the specific speech recognition result VR provided by the voice input message VS, which can be "d i3 e3 nn3 g er1 q i3 d o4 ng4" and "order start ".

In this case, if the voice recognition module 111 directly performs subsequent operations based on the specific voice recognition result VR (that is, "starting the song"), it may not be able to correctly perform the operation of starting the electric cooker.

Therefore, in this embodiment, after the voice recognition module 111 receives the above-mentioned specific acoustic recognition information AR and the specific voice recognition result VR, for example, the first personalized voice recognition model can be used to convert the specific voice recognition result based on the specific acoustic recognition information AR VR (that is, "starting song order") is corrected to the speech recognition result VO1 of "starting electric cooker".

Specifically, since the first user may usually use Taiwanese, his related webpage browsing information may include a large number of webpages in Taiwanese (such as video and audio webpages in Taiwanese). In this case, the first personalized speech recognition model has been trained based on the first user's various first personal information (including the above-mentioned web browsing information), so the first personalized speech recognition model can learn based on the specific acoustic recognition information AR that " The word "order a song" should be "electric pot" in essence, and the above speech recognition result VO1 can be generated accordingly.

In this way, the voice recognition system 110 can correctly perform a specific operation required by the first user (such as starting the electric cooker) based on the voice recognition result VO1 .

As can be seen from the above, the method of the present invention allows the speech recognition module to modify the specific speech recognition result to the first speech recognition result based on the first personalized speech recognition model specific to the first user. Since the first speech recognition result is closer to the pronunciation/word habits of the first user, the speech recognition module can be made to perform more correct operations accordingly. In addition, because the above-mentioned technical means do not involve the means of uploading any first personal information of the first user to the cloud speech recognition system, it can also achieve the effect of protecting personal data.

In other embodiments, the above-mentioned first speech recognition result can be further integrated with speech recognition results provided by other speech recognition modules to generate an integrated speech recognition result corresponding to a specific group.

Please refer to FIG. 3 , which is a schematic diagram of a speech recognition system and a cloud speech recognition system according to a second embodiment of the present invention. In FIG. 3 , the speech recognition system 110 may include a plurality of speech recognition modules 111 - 11N, wherein each of the speech recognition modules 111 - 11N may be dedicated to a corresponding user. For ease of understanding, the speech recognition system 110 only includes the speech recognition modules 111 and 112 for illustration below, but the present invention is not limited thereto.

In the second embodiment, the relevant details of the speech recognition module 111 can refer to the description in the first embodiment, and will not be repeated here. In addition, the speech recognition module 112 may record a second personalized speech recognition model dedicated to the second user, wherein the second user and the first user may belong to the same specific group (such as members of the same office, members of the same laboratory, members of the same team, etc.).

In an embodiment of the present invention, the second personalized speech recognition model is trained based on a plurality of second personal information specific to the second user, wherein the second personal information may include the second user's address book, web page At least one of browsing information, social software articles, and communication software conversations, but not limited thereto.

In the second embodiment, after the speech recognition module 112 obtains the above-mentioned second personal information of the second user, each second personal information can be converted into a plurality of corresponding feature vectors as the second personalized speech recognition Multiple training data for the model. Afterwards, the speech recognition module 112 can train the second personalized speech recognition model based on the above training data. In this way, the second personalized speech recognition model can learn the second user's character habits, idiomatic pronunciation habits and other characteristics, so as to serve as the basis for subsequent correction of the speech recognition results, but it is not limited thereto.

In the second embodiment, it is assumed that the first/second user belonging to the above-mentioned specific group wants to control the voice recognition system 110 to perform a specific operation by sending a voice input message VS to the voice recognition system 110 .

In this case, the voice recognition system 110 can forward the voice input message VS to the cloud voice recognition system 120 according to the previous teaching, and the cloud voice recognition system 120 can correspondingly return the specific acoustic recognition information AR corresponding to the voice input message VS and a specific voice recognition result VR.

Afterwards, the voice recognition module 111 can modify the specific voice recognition result VR into the voice recognition result VO1 by using the first humanized voice recognition model based on the teaching of the previous embodiment.

In addition, after the voice recognition module 112 obtains the specific acoustic recognition information AR and the specific voice recognition result VR corresponding to the voice input message VS, the second personalized voice recognition model dedicated to the voice recognition module 112 can be used to The recognition information AR modifies the specific voice recognition result VR into the voice recognition result VO2. That is, the second personalized speech recognition model can modify the specific speech recognition result VR to the speech recognition result VO2 based on the second user's word usage/pronunciation habit, but it is not limited thereto.

Afterwards, the speech recognition system 110 can generate an integrated speech recognition result VO corresponding to the above-mentioned specific group based on the speech recognition results VO1 and VO2.

It can be seen that the integrated speech recognition result VO generated based on the above teaching can be more in line with the word usage/pronunciation habits of the members of the above-mentioned specific group, and thus better speech recognition effect can be obtained. Moreover, since the above technical means do not involve the means of uploading the personal information of any member of a specific group to the cloud speech recognition system, it can also achieve the effect of protecting the personal information of each member.

In one embodiment, the speech recognition system 110 may integrate the speech recognition results VO1 and VO2 into an integrated speech recognition result VO corresponding to the above-mentioned specific group based on multiple weights corresponding to the first user and the second user.

For example, assume that the first user is the leader/manager of the above-mentioned specific group, and the second user is a general member of the above-mentioned specific group. In this case, the weight corresponding to the first user may be set higher than that of the second user. Therefore, when the speech recognition system 110 integrates the speech recognition results VO1 and VO2, the generated integrated speech recognition result VO may be closer to the word usage/pronunciation habits of the first user, but the present invention is not limited thereto.

In addition, in other embodiments, if the above-mentioned specific group includes N members in total, the speech recognition system 110 can be adjusted as shown in FIG. 3 . That is, the voice recognition system 110 may include voice recognition modules 111 - 11N respectively corresponding to the N members. In this case, after the voice recognition system 110 receives the specific acoustic recognition information AR and the specific voice recognition result VR, the voice recognition modules 111~11N can use the corresponding personalized voice recognition models to correct the specific voice recognition result VR into voice Identification results VO1~VON. Afterwards, the voice recognition results VO1˜VON can be integrated through the above teaching to generate an integrated voice recognition result VO. In this way, the integrated speech recognition result VO can be more in line with the word usage/pronunciation habit of the above-mentioned specific group, so that the speech recognition system 110 can more correctly perform the specific operation required by the above-mentioned specific group, but it is not limited thereto.

Please refer to FIG. 4 , which is a schematic diagram of a first humanized speech recognition model according to an embodiment of the present invention. In this embodiment, the first personalized speech recognition model 400 dedicated to the first user is, for example, a traditional language model (Back-off N-gram Language Models) or a recurrent neural network-like language model (Recurrent Neural Network Language Modeling, RNNLM), but not limited to this.

As shown in FIG. 4 , it is assumed that the first humanized speech recognition model 400 includes an input layer 401 , a hidden layer 402 and an output layer 403 . In this embodiment, after the speech recognition model 111 obtains certain personal information of the first user (such as communication software conversations, web browsing information, address book, common language and place of use, etc.) with the authorization of the first user , the speech recognition model 111 can convert the personal information into a feature vector and input it to the input layer 401 . In addition, in this embodiment, the hidden layer 402 includes, for example, 50 layers, and each layer includes, for example, 512 nodes. In addition, the output layer 403 can output a probability, and the speech recognition model 111 can use the probability to solve the relevant parameters of the first humanized speech recognition model 400 through the backward transfer algorithm.

Afterwards, the specific speech recognition result VR is corrected or re-scored through the trained first humanized speech recognition model 400 (that is, P(W) in the following formula), and the better one in the first personalized speech recognition model 400 is selected. The high probability combination is used as the final speech recognition result VO1, but it is not limited thereto.

In the above formula, O represents the voice input message, which may include a voice feature vector sequence [o1 , o2 , . . . , oT]. In addition, the specific speech recognition result VR recognized by the cloud speech recognition system 120 may correspond to a text sequence Indicates that it includes, for example, a series of words (such as [w1, w2, ... , wm]. In an embodiment of the present invention, the operation performed by the first humanized speech recognition model 400 can be understood as finding out the The word sequence of A Posteriori, MAP) probability, which is the most likely corresponding output text sequence of O .

In this embodiment, p(W|O) is, for example, the post-event probability of occurrence of the word sequence W when a voice input message O is given, and it can be further expressed as P(W), p(O|W) and p (O), where p(O|W) represents the probability density function of the acoustic model (Acoustic Model) to generate O (that is, the voice input message), which is used to estimate the similarity of the acoustic model corresponding to the voice input message O occurring in the text sequence W . In addition, P(W) represents the probability of the first humanized speech recognition model 400 generating the text sequence W, which is used to evaluate the rationality of the text sequence W in the training corpus and correct the result of text confusion in the acoustic model, so that the final first speech The identification result meets the expectation of the first user.

To sum up, the present invention has at least the following features: (1) Under the condition of protecting personal privacy, authorized personal information can be used to train a personalized speech recognition model in a local speech recognition system, and through this personalized speech The recognition model is used to correct the results of speech recognition; (2) Personalized speech recognition models can be applied to other speech recognition application scenarios; (3) Speech recognition models for individuals, specific fields, or specific groups can be integrated according to the application scenarios.

Although the present invention has been disclosed above with the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field may make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention should be defined by the scope of the appended patent application.

110:Speech recognition system 111~11N: Speech recognition module 120: Cloud Speech Recognition System 400: The first humanized speech recognition model 401: Input layer 402: hidden layer 403: output layer VO1~VON: Voice recognition result VO: Integrate speech recognition results AR: Specific Acoustic Identification Information VR: Specific Speech Recognition Results VS: Voice input message S210~S230: steps

FIG. 1 is a schematic diagram of a speech recognition system and a cloud speech recognition system according to a first embodiment of the present invention. FIG. 2 is a flow chart of a personalized speech recognition method according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a speech recognition system and a cloud speech recognition system according to a second embodiment of the present invention. FIG. 4 is a schematic diagram of a first humanized speech recognition model according to an embodiment of the present invention.

S210~S230: steps

Claims (11)

A personalized voice recognition method suitable for a first voice recognition module, comprising: in response to obtaining a voice input message, forwarding the voice input message to a cloud voice recognition system, wherein the cloud voice recognition system responds to the voice Inputting information to generate a specific acoustic recognition information and a specific voice recognition result corresponding to the voice input message; receiving the specific acoustic recognition information and the specific voice recognition result corresponding to the voice input message from the cloud voice recognition system; using A first humanized speech recognition model dedicated to the first speech recognition module to modify the specific speech recognition result to a first speech recognition result based on the specific acoustic recognition information, wherein the first humanized speech recognition model includes a corresponding The acoustic model of the speech input message, and the first humanized speech recognition model scores the specific speech recognition result and finds the word sequence with the largest posterior probability as the first speech recognition result. The method as claimed in claim 1, wherein the first personalized speech recognition model is trained based on a plurality of first personal information specific to a first user. The method according to claim 2, wherein the first personal information includes at least one of the first user's address book, web browsing information, social software articles, and communication software conversations. The method as described in claim item 2, further comprising: Each of the first personal information is converted into a plurality of corresponding feature vectors as a plurality of training data of the first personal speech recognition model; the first personal speech recognition model is trained based on the training data. The method as claimed in claim 1, further comprising performing a specific operation based on the first speech recognition result. A voice recognition system, comprising: a first voice recognition module configured to: respond to obtaining a voice input message, forward the voice input message to a cloud voice recognition system, wherein the cloud voice recognition system responds to the voice input message to generate a specific acoustic recognition information and a specific voice recognition result corresponding to the voice input message; receiving the specific acoustic recognition information and the specific voice recognition result corresponding to the voice input message from the cloud voice recognition system; Using a first humanized speech recognition model specific to the first speech recognition module to modify the specific speech recognition result to a first speech recognition result based on the specific acoustic recognition information, wherein the first humanized speech recognition model includes a corresponding An acoustic model of a message is input into the speech, and the first humanized speech recognition model scores the specific speech recognition result and finds a word sequence with the largest posterior probability as the first speech recognition result. The speech recognition system as claimed in claim 6, wherein the first personalized speech recognition model is trained based on a plurality of first personal information specific to a first user, and the first user belongs to a specific group. The speech recognition system as described in claim 7, further comprising a second speech recognition module configured to: receive the specific acoustic recognition information corresponding to the speech input message and the specific speech recognition from the cloud speech recognition system Result: using a second personalized speech recognition model specific to the second speech recognition module to correct the specific speech recognition result into a second speech recognition result based on the specific acoustic recognition information. The speech recognition system as claimed in claim 8, wherein the second personalized speech recognition model is trained based on a plurality of second personal information specific to a second user. The speech recognition system as claimed in claim 9, wherein the speech recognition system generates an integrated speech recognition result corresponding to the specific group based on the first speech recognition result and the second speech recognition result. The speech recognition system as claimed in claim 10, wherein the speech recognition system integrates the first speech recognition result and the second speech recognition result based on a plurality of weights corresponding to the first user and the second user into The integrated speech recognition result corresponding to the specific group.

Images