Nvidia has unveiled a new experimental generative AI, dubbed a "universal tool for sound manipulation."
This model, known as Foundational Generative Audio Transformer Opus 1 (or Fugatto), can understand text prompts and utilize them to generate audio or modify existing music, voice, and sound files. Developed by an international team of AI researchers, NVIDIA claims this collaboration has enhanced its "multifaceted and multilingual capabilities."
Raphaël Valle, one of the project's researchers and a manager of applied audio research at NVIDIA, stated: "We aimed to create a model that understands and generates sound in the same way humans do."
The company provided several examples of where Fugatto might prove valuable. For instance, music producers can quickly prototype songs that can be easily edited by switching styles, voices, and instruments.
Individuals will be able to use Fugatto to create language learning materials using a selected voice. Game developers will be able to produce various versions of pre-recorded sounds that adapt to changes in gameplay based on the players' choices and actions.
Moreover, researchers found that the model can carry out tasks it wasn't specifically trained to do with minimal additional tuning. For example, it can combine separate learned commands to create an angry voice with a particular accent or the sound of birds singing during a thunderstorm. The model also generates sounds that vary over time, such as the noise of approaching rain.
NVIDIA has not disclosed whether Fugatto will be made publicly available. However, this AI model is not the first generative model capable of producing sounds from text prompts. Previously, Meta released an open AI toolkit that can generate sounds from text descriptions, and Google has its own AI called MusicLM that transforms text into music.
Comments (0)
There are no comments for now