NVIDIA wants to make future games more realistic thanks to AI-powered game characters

Generative AI coming to games.

New Covert Protocol Tech Demo, Source: NVIDIA/Inworld

NVIDIA is collaborating with Inworld for a new tech demo called New Covert Protocol. The game is a detective style mystery that requires users to gather crucial information. It’s a simulation where gamers can interact with characters who all have their own unique personalities and stories. These characters will react in real-time to decisions made by users, which can lead to different game outcomes.

The company Inworld is working on making their own Software Development Kit integration with MetaHuman. This is based on NVIDIA ACE (Avatar Cloud Engine) technology, which as already presented by the company for a short ‘bar scene’ demo roughly 9 months ago .

Thanks to the SDK, it would be possible to incorporate three important AI technologies into games, including ACE, NVIDIA Riva Automatic Speech Recognition (ASR) and NVIDIA Audio2Face (A2F). NVIDIA has presented the SDK working in Unreal Engine 5, allowing game developers to write the story and bio for each character, creating more realistic game scenarios. Just imagine something like this is integrated into games like Grand Theft Auto 6.

New Covert Protocol Tech Demo, Source: NVIDIA/Inworld

Live translations and lip-syncing

NVIDIA is also working with game developers to enable accurate lip-syncing for bilingual languages. While the scrips can be written and voiced in one language, it can easily be converted into a multi-language version thanks to the power of generative AI.

The company has demonstrated how this works after translating Chinese to English in a game called World of Jade Dynasty. This is an Unreal Engine 5 MMO game and a sequel to Jade Dynasty.  This technology called Audio2Face does not require any external online service to work.

Digital Humans for medical market

NVIDIA is also working on establishing a generative AI assistant for medical fields. With the help of a company Hippocratic AI & UnneQ, NVIDIA is showcasing how digital humans can work in a safety-focused environment. Such assistants can coordinate tasks for patients, perform post-discharge management, and even call patients on the phone.

Source: NVIDIA