NVIDIA has taken a step towards the mass adoption of AI technologies by open-sourcing Audio2Face. This is a breakthrough for game and animation developers: the tool allows you to create realistic facial expressions of characters based on sound.
Audio2Face includes an SDK, plugins for Maya and Unreal Engine 5, and ready-made AI models. They transform speech into natural lip and face movements, including emotions in real time. The new version 3.0 is based on diffusion methods, which provides more accurate reproduction and better emotion recognition.
The technology is already being used in projects by major companies, including Codemasters, NetEase and Reallusion. According to Wojciech Pazdur from The Farm 51, thanks to Audio2Face, they were able to save hundreds of hours of manual animation while working on Chernobylite 2: Exclusion Zone.
The digital avatar market is growing rapidly. In 2024, its volume was estimated at $29.06 billion, and by 2032 it should reach $652.29 billion. Games account for about 30% of the segment, and growth in this area is 49.24% per year.
The essence of Audio2Face is in voice analysis: the system determines phonemes and intonation, and then automatically generates suitable facial expressions. This eliminates long manual work and makes emotions in games and movies even more believable.
