Nvidia Maxine platform uses AI to enhance video conferencing

In the current pandemic world, Work From Home (WFH) has become the new norm of society. With people spending most of their time completing projects and collaborating with their team members virtually.

Demand for video conferencing platforms has seen a sudden growth in their market value. With services like Zoom and Google Meet receiving increased popularity, it looks like the video conferencing services are likely to grow even further.

But video conferencing has its fair share of troubles.

Yep, we all experienced it. Be it attending online classes, collaborating with your colleagues, or attending an important interview. It is tough to expect high-quality video output from so many platforms out there.

But the problem is not just related to software. For software to work efficiently, it requires the support of equal quality hardware. This is something most of us don’t have. Thus we end compromising in our day to day video meetings.

But things are on the verge of change.

Nvidia announced its new videoconferencing platform for the developers. Namely, the ‘Nvidia Maxine’, the platform is claimed to be able to fix the most common problems in video calls, while improving the overall quality of the videos.

How does the new tech work?

The platform will process call in the cloud which using the company’s own GPU’s and boosts call in numerous ways using the support of Artificial Intelligence. The role of AI in Maxine will be to realign caller’s faces and gazes so that they’re always looking directly at their camera.

Maxine can also reduce bandwidth requirement for the videos “down to one-tenth of the requirements of the H.264 streaming video compression standard”. It does this by only transmitting “key facial points”. It also offers features like face re-lighting, real-time translation, and also transcription and animated avatars.

Nvidia’s general manager for media and entertainment Richard Kerris stated, “We’ve all experienced times where bandwidth has been a limitation in our conferencing we’re doing daily these days, if we apply AI to this problem we can reconstruct the difference scenes on both ends and only transmit what needs to transmit, and thereby reducing that bandwidth significantly.”

Nvidia compression feature uses an Artificial Intelligence method known as ‘generative adversarial networks’ or GANs that partially reconstruct callers’ faces in the cloud.

Nvidia in its blog posted, “Instead of streaming the entire screen of pixels, the AI software analyzes the key facial points of each person on a call and then intelligently re-animates the face in the video on the other side. this makes it possible to stream video with far less data flowing back and forth across the internet.” This technique is the same one used in many deepfakes.

The company is no doubt hoping its cloud computing and impressive AI R&D work will help it rise above its competitors in the market. The real test, however, lies in the company convincing the established video conferencing companies to adopt Nvidia’s technology.

Nvidia’s Maxine is a toolkit for third-party firms to improve their software. So far, the company has only announced its partnership with the communications firm Avaya, which will be using select features of Maxine.

Nvidia’s latest announcement shows us that the future of videoconferencing will be more Artificial Intelligence Oriented.

source

Do check out: