They create a camera that records the sounds of various instruments separately from their vibrations

Since it was patented by Alexander Graham Bell, the microphone has undergone a series of improvements that have helped to improve its performance.

However, despite the effectiveness of this device to record every last musical note of different instruments played by a group of musicians in a room, in the end everything will be condensed into a single recording.

It is in this instance where the ideal would be to record each musician separately playing their instrument, and then have a sound engineer take care of remixing all the material.

In that sense, there are software tools that process an audio recording and then extract the individual sounds generated by different elements within it. However, the final material does not end up being quite good compared to what would be obtained if the microphone captured the direct sound of the source.

To solve this situation, a group of researchers at the Robotics Institute in the School of Computer Science at Carnegie Mellon University devised a way to accomplish this through the use of video-camerasthereby compensating for the lack of a microphone that could differentiate sound vibrations moving in the air.

These video cameras serve as optical microphonesnamely, capture the vibrations produced by an instrumentsuch as a guitar, which are then analyzed in order to replicate the sound generated by it, being able to do this even when no sound is recorded.

As far as their operation is concerned, these cameras activate the brightness of a laser light source on the structure of a vibrating instrument, so that they can capture the dappled light pattern generated by this action. All this to 63 thousand frames per secondthus neglecting the motions present in a vibration oscillating at a speed of 20 thousand times per secondthe same range as the human ear.

One camera has a global shutter while the other has a rolling shutter. The work done by both is then processed by an algorithm which more accurately determines the motion described by laser speckle patterns vibrating at 63,000 times per second.

This is how in the end these cameras make it possible get audio from multiple sources into a single videoeither of instruments played by musicians or by loudspeakers projecting different music.