Sign In

Communications of the ACM

ACM TechNews

­w's Lip-Syncing Obama Demonstrates New Technique to Turn Audio Clips Into Realistic Video

View as: Print Mobile App Share:
A new system can match audio clips to video to create realistic-looking lip-synced videos.

Given an audio clip of former President Obama speaking about healthcare at a campaign event and an existing video of a weekly address in the Oval Office, for instance, a new University of Washington system can synthesize a realistic, lip-synced video of O

Credit: University of Washington

University of Washington (UW) researchers have developed a system that can take audio clips from one speech and sync them with video clips from another speech, creating realistic-looking lip-synced videos.

The researchers say the technology could be used to apply a moving face to historic audio recordings, or to improve videoconferencing.

In a demonstration of the technology, the researchers used video of former U.S. President Barack Obama from a range of appearances and combined it with audio spoken in separate instances.

A recurrent neural network, trained on many hours of Obama's weekly address footage, learned the mapping from raw audio features to mouth shapes.

"These type of results have never been shown before," says UW professor Ira Kemelmacher-Shlizerman.

The researchers note the technology also could lead to better video chat performance.

The system will be presented at the upcoming ACM SIGGRAPH 2017 conference in Los Angeles, CA.

From GeekWire
View Full Article


Abstracts Copyright © 2017 Information Inc., Bethesda, Maryland, USA


No entries found

Sign In for Full Access
» Forgot Password? » Create an ACM Web Account