Communications of the ACM
w's Lip-Syncing Obama Demonstrates New Technique to Turn Audio Clips Into Realistic Video
Given an audio clip of former President Obama speaking about healthcare at a campaign event and an existing video of a weekly address in the Oval Office, for instance, a new University of Washington system can synthesize a realistic, lip-synced video of O
Credit: University of Washington
University of Washington (UW) researchers have developed a system that can take audio clips from one speech and sync them with video clips from another speech, creating realistic-looking lip-synced videos.
The researchers say the technology could be used to apply a moving face to historic audio recordings, or to improve videoconferencing.
In a demonstration of the technology, the researchers used video of former U.S. President Barack Obama from a range of appearances and combined it with audio spoken in separate instances.
A recurrent neural network, trained on many hours of Obama's weekly address footage, learned the mapping from raw audio features to mouth shapes.
"These type of results have never been shown before," says UW professor Ira Kemelmacher-Shlizerman.
The researchers note the technology also could lead to better video chat performance.
The system will be presented at the upcoming ACM SIGGRAPH 2017 conference in Los Angeles, CA.
From GeekWire
View Full Article
Abstracts Copyright © 2017 Information Inc., Bethesda, Maryland, USA
No entries found