Sign In

Communications of the ACM

ACM TechNews

Human-Computer Music Performance System Links Music and Musical Gestures

View as: Print Mobile App Share:
Machine Orchestra

Computers, robots, and humans perform together in the Machine Orchestra, which uses captured musical gestures.

Credit: Ajay Kapur

University of Victoria researchers have developed a new method for capturing musical gestures and mapping them to sounds. The researchers' method is a hybrid of the direct acquisition method, which involves the use of sensors to create hyper-instruments, and the indirect acquisition method, which involves using a microphone and signal-processing algorithms. The Victoria team temporarily attached sensors to an instrument to capture musical gestures and a microphone to capture sound. The data was analyzed and the team developed a surrogate sensor, which determined the musical gestures based only on the analyzed sound captured from the microphone.

View a video of the Machine Orchestra performing at the Walt Disney Concert Hall.

"The main advantage of the method is that it allows large amounts of training data for machine-learning algorithms to be acquired without human annotation simply by playing an instrument enhanced with sensors," says Victoria computer scientist George Tzanetakis.

The system improves on previous methods because it does not hinder performers or their instruments when playing, and it does not require large amounts of processing and analysis.

View Full Article


Abstracts Copyright © 2010 Information Inc., Bethesda, Maryland, USA



No entries found

Sign In for Full Access
» Forgot Password? » Create an ACM Web Account