FindAudioSyncScript

Latest revision as of 23:03, 19 May 2013

I would like to share this script as an approach to finding the appropriate DelayAudio value to use to correct lip sync errors in video material.

It has always been a pain trying to find the right delay offset by simply tweaking the delay and watching the results over and over, I can never quite find the right value before frustration takes hold.

The script below takes a short section of an input video which has some recognizable sharp sound and its matching visual event. For an interlaced source I Bob deinterlace it to double the video temporal resolution and also take the free ride to resize at this point. I then have a function to return the clip with a specified DelayAudio value and a subtitle identifying the delay used. I was finding that I was getting used to the delayed audio and could no longer discern the best sequence, I found adding a FadeIO helped the mind reset its zero reference between samples. I generate a series of these clips and AlignedSplice them all together. At the end of the process I then slow everything down using AssumeFPS based on reclocking the audio to 8Khz.

Sample Script

#
# Generate 11 variations of an input clip with varying audio delays
#
# Seperate into fields to double video temporal resolution
# Slow everything down to exagerate the Audio lipsync error
#
# Iterate the following values to find the optimal AudioDelay setting

Global Offset= 0.000 # Middle value of the sweep range being tested
Global Scale = 0.250 # Scale the sweep range from +/- 1 second

AVISource("Nemisis.avi", True, "YUY2") # Select the input file

# ===========

AssumeFrameBased() # Correctly describe frame/field attribute
AssumeTFF() # Correctly describe field order
Trim(6115, -30) # Define the range of frames to investigate
Bob(Height=240) # Double the FrameRate, Choose a good height
BilinearResize(320, Height()) # Choose a good width

#
# AlignedSplice each prospective clip sample, specify a range of delays
#

DoDelayAudio(-1.0) ++ \
DoDelayAudio(-0.8) ++ \
DoDelayAudio(-0.6) ++ \
DoDelayAudio(-0.4) ++ \
DoDelayAudio(-0.2) ++ \
DoDelayAudio( 0.0) ++ \
DoDelayAudio(+0.2) ++ \
DoDelayAudio(+0.4) ++ \
DoDelayAudio(+0.6) ++ \
DoDelayAudio(+0.8) ++ \
DoDelayAudio(+1.0)

#
# Slow everything down, keep Audio and Video synced
# Audio will be 8kHz, Video will be slowed to match
#

AssumeFPS(8000.0 * FrameRate() / AudioRate(), True)

#
# Adjust the Audio Delay, Add a Subtitle, Fade In/Out
#

Function DoDelayAudio(Clip Clip, Float Delay)
{
  Delay = (Delay * Scale) + Offset
  Return DelayAudio(Clip, Delay).Subtitle("Delay " + String(Delay)).FadeIO(6)
}