How to keep audio and video (GEM) synchronised?

Question

Here is the setup:

I have a video file and an audio file with the same duration. The video file is played with [pix_film] and the audio file via [readsf~], both are then distorted by several effects, triggered by user interaction.

How can I keep the video and audio synchronised?

Answer 1

In case you want the user to change the speed of the playback (so not using [readsf~] but rather a sampler based on [vline~] or [phasor~]), then you also need to adjust the speed of playback of the video. A tutorial how to do this and also the playback with more than once instance of Pd to avoid audio dropouts can be found at https://github.com/mxa/AudioVideoPatches

Answer 2

both [pix_film] and [readsf~] will act in logical time, so they should be sychronized out of the box.

the only problem i can imagine is, that [pix_film] is playing back the movie at the wrong framerate (in which case, video and audio are still synched, but somewhat differently then what you expected). the playback framerate of Gem is fixed to the render framerate.

basically you have to possibilities to fix that:

• change Gem's framerate to the framerate of your film (this obviously won't work if you want to play multiple videos with differing framerates simultaneously)
• change the "frame seek rate" of [pix_film] to match the global framerate.

the 2nd solution can be achieved with something like this:

[t a b]
|      \   
|       \  [r counter]
|        [f]
|        |   [r incr]
|        [+ 0]
|        |
|        [t f b]
|        |     [s counter]
[pix_film]
     |
     [$3(
     |    [r gemrate]
     [/ 20]
     |
     [s incr]