Quantizing in audio is nothing new. This is where an audio
recording of a drum track for instance is moved around as
necessary to fit into a precise, predetermined matrix.
So if you have a sloppy drummer, you quantize the
recording
so it moves late beats backwards and early beats forward.
It's
the rhythmic equivalent of autotune for singers.
The proposed idea is to have the audio track for a video
quantized, but retaining its alignment to the video track.
Easiest example would be to show a drummer playing
sloppily, the audio/video quantize function would be
activated and you would hear AND SEE a drummer playing
perfectly on beat.
This effect is currently done by hand where people
manually
cut up videos and drop them into audio tracks to make
novelty, usually humorous music videos.
So this could be done automatically or with some manual
editing ti fine tweak it. You'd take a video of your aunt
complaining about
her
bunion, click "analyze audio track" and it would move the
main "beats" of her conversation to the nearest quarter
note
for instance giving it the appearance that she's rapping.
Alternatively, you could manually look at the peaks and
valleys of the audio signal over-layed on a beat matrix and
you could move it by hand. Once it sounded right, you'd hit
"play" and watch the video of the person wrapping.
Doesn't have to be just people talking either. Any video
with
sound could be made a rhythmic performance piece.
You could also add auto tune to the audio track and make
any video where people were talking or even making noise
a music video.