Video Linked To Time Quantized Audio Track

Quantizing in audio is nothing new. This is where an audio recording of a drum track for instance is moved around as necessary to fit into a precise, predetermined matrix.

So if you have a sloppy drummer, you quantize the recording so it moves late beats backwards and early beats forward. It's the rhythmic equivalent of autotune for singers.

The proposed idea is to have the audio track for a video quantized, but retaining its alignment to the video track. Easiest example would be to show a drummer playing sloppily, the audio/video quantize function would be activated and you would hear AND SEE a drummer playing perfectly on beat.

This effect is currently done by hand where people manually cut up videos and drop them into audio tracks to make novelty, usually humorous music videos.

So this could be done automatically or with some manual editing ti fine tweak it. You'd take a video of your aunt complaining about her bunion, click "analyze audio track" and it would move the main "beats" of her conversation to the nearest quarter note for instance giving it the appearance that she's rapping. Alternatively, you could manually look at the peaks and valleys of the audio signal over-layed on a beat matrix and you could move it by hand. Once it sounded right, you'd hit "play" and watch the video of the person wrapping.

Doesn't have to be just people talking either. Any video with sound could be made a rhythmic performance piece.

You could also add auto tune to the audio track and make any video where people were talking or even making noise a music video.