Meta AI has recently announced a breakthrough in generative AI for speech called Voicebox. It is a model that can create and edit high-quality audio clips in various languages and styles, without being trained for specific tasks.
Voicebox uses a novel technique called Flow Matching, which enhances the performance of diffusion models.
It can produce speech in six languages: English, French, German, Spanish, Polish and Portuguese.
It can also perform tasks such as text-to-speech synthesis, speech editing, noise reduction, cross-lingual style transfer, and diverse speech sampling.
Voicebox surpasses the current state-of-the-art models on measures of word error rate and audio similarity.
Due to the potential risks of misuse, Voicebox is not publicly available, but Meta AI has developed a classifier that can detect whether an audio clip is generated by Voicebox or not.