Published: | Updated:

HOW DOES 'TRANSLATOTRON' WORK?

Translatotron is based on a sequence-to-sequence network which takes source spectrograms, a visual representation of the soundwaves, as input and generates spectrograms of the translated content in the target language.

It also makes use of two other separately trained components: a neural vocoder that converts output spectrograms to waveforms.

Optionally, a speaker encoder that can be used to maintain the character of the source speaker’s voice in the synthesized translated speech. 

During training, the sequence-to-sequence model uses a multitask objective to predict source and target transcripts at the same time as generating target spectrograms. 

However, no transcripts or other intermediate text representations are used during inference.

Advertisement

Read articles that feature this panel

Google's Translate can now listen to a language and make it into an audio translation in the original speaker's voice

California-based Google's Translatotron converts one language into another, for example Spanish to English, without the need for a...

Most watched News videos