MVTools

Abstract

Table of contents

• I) About MVTools
• II) Function descriptions
• III) Examples
• IV) Disclaimer (don't skip that part, but we don't force you to learn it either)
• V) Revisions
• VI) Download

I) About MVTools

MVTools plugin for AviSynth 2.5 is collection of functions for estimation and compensation of objects motion in video clips. Motion compensation may be used for strong temporal denoising, advanced framerate conversions, image restoration and other tasks.

The plugin contains the motion estimation server-function MVAnalyse to find the motion vectors and several motion compensation client-functions (MVCompensate, MVMask and others) which use these vectors.

Plugin uses block-matching method of motion estimation (similar methods are used in MPEG2, MPEG4, etc). At analysis stage plugin divides frames by small blocks and try to find for every block in current frame the most similar (matching) block in second frame (previous or next). The relative shift of these blocks is motion vector. The main measure of block similarity is sum of absolute differences (SAD) of all pixels of these two blocks compared. SAD is a value which says how good the motion estimation was.

The output of MVAnalyse (server) is special clip with motion vector information in some format.

At compensation stage the plugin client functions read the motion vectors and use them to move blocks and form motion compensated frame (or realize some other full or partial motion compensation or interpolation function). Every object (block) in this (fully) compensated frame is placed in the same position as this object in current frame. So, we may (for example) use strong temporal denoising even for quite fast moving objects without producing annoying artefactes and ghosting (object's features and edges are coincide if compensation is perfect). Plugin can create compensated neighbor frames for every current frame, and denoise it by internal function. Alternatively, you can use compensated and original frames to create interleaved clip, denoise it by any external temporal filter, and select central cleaned original frames for output (see examples).

Of course, the motion estimation and compensation is not ideal and precise. In some complex cases (video with fading, ultra-fast motion, or periodic structures) the motion estimation may be completely wrong, and compensated frame will be blocky and (or) ugly. Severe difficulty is also due to objects mutual screening (occlusion) or reverse opening. Complex Avisynth scripts with many motion compensation functions may eat huge amount of memory and result in very slow processing. It is not simple but quite advanced plugin. Use it for appropriate cases only, and try tune its parameters. There are many discussions about motion compensation using at doom9 Avisynth forum. In particular see old MVTools thread, true motion thread, new MVTools thread and some other. Try read postings in addition to this documentation and ask for support there. If you really interested in motion estimation and compensation topics, you can easy find numerous scientific publications (use WWW search).

Notes: The plugin is still under development. Current version has some limitations. Only progressive YV12, YUY2 video is supported. Use color format conversion and try use (smart) bob-deinterlace for interlaced video (SeparateFields may works too with or without SelectEven/SelectOdd). Some complex scripts (MVBOB, MCBOB) use MVTools for motion compensated deinterlace.
Alternatively you can try to use Motion plugin by mg262.

II) Function descriptions

Common parameters

Filters that use motion vectors have common parameters. Those are the scene-change detection thresholds, and the mmx / isse flags. They also use one or several vectors stream, which are produced by MVAnalyse.

int thSCD1 : threshold which decides whether a block has changed between the previous frame and the current one. When a block has changed, it means that motion estimation for it isn't relevant. It occurs for example at scene changes. So it is one of the thresholds used to tweak the scene changes detection engine. Raising it will lower the number of blocks detected as changed. It may be useful for noisy or flickered video. The threshold is compared to the SAD (Sum of Absolute Differences, a value which says how bad the motion estimation was ) value. For exactly identical blocks we have SAD=0. But real blocks are always different because of objects complex movement (zoom, rotation, deformation), discrete pixels sampling, and noise. Suppose we have two compared 8x8 blocks with every pixel different by 5. It this case SAD will be 8x8x5 = 320 (block will not detected as changed for thSCD1=400). If you use 4x4 blocks, SAD will be 320/4. If you use 16x16 blocks, SAD will be 320*4. Really this parameter is scaled internally in MVTools, and you must always use reduced to block size 8x8 value. Default is 400 (since v.1.4.1).

int thSCD2 : threshold which sets how many blocks have to change for the frame to be considered as a scene change. It is ranged from 0 to 255, 0 meaning 0 %, 255 meaning 100 %. Default is 130 ( which means 51 % ).

bool isse : flags which allows to disable ISSE and MMX optimizations if set to false (for debugging). Default is true. If your processor doesn't support ISSE MMX optimizations, it will be disabled anyway ( and you won't be able to activate them )

MVAnalyse

MVAnalyse (clip, int "blksize", int "blksizeV", int "pel", int "level", int "search", int "searchparam", int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", int "lsad", int "pnew", int "plevel", bool "global", int "pzero", int "overlap", int "overlapV", string "outfile", int "sharp", int "rfilter", clip "pelclip", int "dct", int "divide", bool "mc", int "idx", int "sadx264")

Estimate motion by block-matching method and produce special output clip with motion vectors data (used by other functions).
Some hierarchical multi-level search methods are implemented (from coarse image scale to finest). Function uses zero vector and neighbors blocks vectors as a predictors for current block. Firstly difference (SAD) are estimated for predictors, then candidate vector changed by some values to some direction, SAD is estimated, and so on. The accepted new vector is the vector with minimal SAD value (with some penalty for motion coherence).

blksize : Size of a block (horizontal). It's either 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.

blksizeV : vertical size of a block. Default is equal to horizontal size. Additional options: 4 for blksize=8 and 8 or 2 for blksize=16.

pel : it is the accuracy of the motion estimation. Value can only be 1, 2 or 4. 1 means a precision to the pixel. 2 means a precision to half a pixel, 4 means a precision to quarter a pixel, produced by spatial interpolation (more accurate but slower and not always better due to big level scale step). Default is 2 since v1.4.10.

level : it is the number of coarse levels NOT used in the hierarchical analysis made while searching for motion vectors. The lower the usually better (vectors with any length can be found). It is kept variable for study's purposes mostly. Sometimes level is useful to prevent large (false) vectors (computer graphics, etc). Default : 0 (all levels are used).

search, searchparam, pelsearch : search decides the type of search at every level, and searchparam is an additional parameter (step, radius) for this search, and pelsearch is the parameter at finest (pel) level:

• search = 0 : 'OneTimeSearch'. searchparam is the step between each vectors tried ( if searchparam is superior to 1, step will be progressively refined ).
• search = 1 : 'NStepSearch'. N is set by searchparam. It's the most well known of the MV search algorithm.
• search = 2 : Logarithmic search, also named Diamond Search. searchparam is the initial step search, there again, it is refined progressively. It's the default search type ( with default searchparam = 2 and default pelsearch = pel )
• search = 3 : Exhaustive search, searchparam is the radius (square side is 2*radius+1). It is slow, but it gives the best results, SAD-wise.

isb : allows to choose between a forward search (motion from the previous frame to current one) for isb=false and a backward search (motion from following frame to the current one) for isb=true (isb stands for "IS Backward", it is implemented and named exactly as written here, do not ask :-). Default isb=false.

chroma : set to true, it allows to take chroma into account when doing the motion estimation (false - luma only). Default is true.

delta : set the frame interval between the reference frame and the current frame. By default, it's 1, which means that the motion vectors are searched between the current frame and the previous ( or next ) frame. Setting it to 2 will allow you to search mvs between the frame n and n-2 or n+2 ( depending on the isb setting ).

There are few advanced parameters which set coherence of motion vectors for so called true motion estimation. Some matched blocks from other frame may be most similar to sample blocks of current frame by intensity criterion (SAD), but not correspond to true object motion. For example, they may belong to other similar object in different corner of the frame or belong to some periodic structure. "True motion" parameters try maintain the motion field more coherent, instead of some random vectors distribution. It is especially important for partial motion compensation and interpolation. Some parameters are experimental and may be removed (replaced) in next versions after testing. Please report your conclusions.

truemotion is a preset of these parameters values. It allows easy to switch default values of all "true motion" parameters at once. Set it true for true motion search (high vector coherence), set it false to search motion vectors with best SAD. Default is true since v1.4.10. In any case you can tune each parameter individually.

lambda : set the coherence of the field of vectors. The higher, the more coherent. However, if set too high, some best motion vectors can be missed. Values around 400 - 2000 (for block size 8) are strongly recommended. Internally it is coefficient for SAD penalty of vector squared difference from predictor (neighbors), scaled by 256.
Default is 0 for truemotion=false and 1000*blksize*blksizeV/64 for truemotion=true.

lsad: SAD limit for lambda using. Local lambda is decreased (smoothly since v1.10.2) if SAD value of vector predictor (formed from neighbor blocks) is greater than the limit. It prevents bad predictors using but decreases the motion coherence. Values above 1000 (for block size=8) are recommended for true motion.
Default is 400*blksize*blksizeV/64 for truemotion=false and 1200*blksize*blksizeV/64 for truemotion=true.

pnew: relative penalty (scaled to 256) to SAD cost for new candidate vector. New candidate vector must be better will be accepted as new vector only if its SAD with penalty (SAD + SAD*pnew/256) is lower then predictor cost (old SAD). It prevent replacing of quite good predictors by new vector with a little better SAD but different length and direction.
Default is 0 for truemotion=false and 50 for truemotion=true.

plevel: penalty factor lambda level scaling mode. Value=0 - no scaling, 1 - linear, 2 - quadratic dependence from hierarchical level size. Note, that vector length is smaller at lower level.
Default is 0 for truemotion=false and 1 for truemotion=true

global: estimate global motion (at every level) and use it as an additional predictor. Only pan shift is estimated (no zoom and rotation). Use false to disable, use true to enable. Default is false for truemotion=false and true for truemotion=true.

pzero: relative penalty (scaled to 256) to SAD cost for zero (and global) vector. It prevent replacing of quite good predictor by zero vector with a little better SAD (lambda is not used for zero and global vectors). Default is equal to pnew since v1.11.

overlap: block overlap value (horizontal). Must be even and less than block size (up to blksize/2 for MVCompensate). The step between blocks for motion estimation is equal to (blksize-overlap). N blocks cover the size ((blksize-overlap)*N + overlap) on frame. Try use overlap value from blksize/4 to blksize/2. The greater overlap, the more blocks number, and the lesser the processing speed. Default value is 0.
Functions with overlap support are: MVFlow, MVFlowInter, MVFlowFps (not MVFlowFps2), MVShow, MVMask, MVCompensate, MVDeGrain1, MVDeGrain2, MVDeGrain3.

overlapv: vertical block overlap value. Default is equal to horizontal. Must be even for YV12 and less than block size.

outfile: name of file to write motion vectors data. This data may be used by some external program or may be by next MVTools versions for second pass coding, etc.
Produced binary file has a header (MVAnalysisData structure, see MVInterface.h source code), and the data sequence:
frame number, vector data (Vx, Vy, SAD) of every block, next valid frame number, this frame vector data, and so on.
Default - empty string, do not write.

sharp: subpixel interpolation method for pel=2,4.
Use 0 for soft interpolation (bilinear), 1 for bicubic interpolation (4 tap Catmull-Rom), 2 for sharper Wiener interpolation (6 tap, similar to Lanczos).
Default is 2.

rfilter: hierarchical levels reduce method. Use 0 for simple 4 pixel summation (averaging, like BilinearResize, old method), use 1 for filter 1/4, 1/2, 1/4 (like ReduceBy2) for more smooth motion estimation.
Default is 1 (since v1.11.0.2).

pelclip: upsampled source clip for using instead of internal subpixel interpolation (for pel>1).
Pixels at rows and colunms positions multiple to pel (0,2,4,... for pel=2) must be original source pixels, other pixels must be interpolated.
Example for pel=2: LanczosResize(width*2,height*2,src_left=0.25, src_top=0.25). (Recent note: it is true for luma, but is not exactly corresponded to chroma pixels positions of internal MVTools interpolation. Nevertheless vectors and motion compensation are quite similar for usual clips, same chroma would be with src_left=0.5 for YUY2 and additionally src_top=0.5 for YV12).
Other useful example is EEDI2 edge-directed resampler.
For correct results you must use (the same) pelclip for whole chain of MVTools functions which use same source clip (and idx).
Default is not defined.

dct: using of block DCT (frequency spectrum) for blocks difference (SAD) calculation. In particular it can improve motion vector estimation at luma flicker and fades.
0 - usual spatial blocks, do not use DCT;
1 - use block DCT instead of spatial data (slow for block size 8x8 and very slow for other sizes);
2 - mixed spatial and DCT data; weight is dependent on mean frame luma difference;
3 - adaptive per-block switching from spatial to equal-weighted mixed mode (experimental, a little faster).
4 - adaptive per-block switching from spatial to mixed mode with more weight of DCT (experimental, a little faster).
added in v1.9.5.3 modes: (require sadx264 0-7, uses fastest SATD function, luma only!)
5 - SATD instead of SAD for luma
6 - same as 2 only use SATD
7 - same as 3 only use SATD
8 - same as 4 only use SATD
9 - similar to 2, use SATD and weight ranges from SAD only to equal SAD & SATD
10 - similar to 3/4,use SATD weight is on SAD, only on strong luma changes
Default = 0.

divide: post-processing motion vectors by dividing every block into 4 subblocks.
0 - do not divide;
1 - divide blocks and assign the original vector to all 4 subblocks;
2 - divide blocks and assign median (with 2 neighbors) vectors to subblocks;
Default = 0. Block size and overlap values must be selected to be acceptable after internal dividing.

mc: addditionally store motion compensated frames in vector clip (used by MVDenoise). Default is false (do not store for memory saving) since v1.11.4.

idx: index of clip. It allows to internally reuse (share) the multilevel data computed for frames of the clip by several instances of the filter (or by other functions) for more fast processing and memory usage decreasing. It is especially useful for pel=2 to avoid computing the interpolation twice, when doing a forward & backward search on the same clip. If you use idx, you should always use positive values, and you should only use the same value for filters which work on the same clip ( else, the analysis won't work properly ). By default, a unique negative number is given for each filter instance (which will create its own multilevel data).

sadx264: use SAD functions from x264 codec if available for the blocksize
0 - Autodetect CPU and select best function
1 - use MMX 16x16, 16x8, 8x8, 8x4, 4x4 (8x16, 4x8 used on chroma only)
2 - use MMX 16x16, 16x8, 8x8, 8x4 (8x16, used on chroma only) aligned to 32 Byte blocks
3 - use MMX 16x16, 16x8, 8x8, 8x4 (8x16, used on chroma only) aligned to 64 Byte blocks, best for Pentium M
4 - use SSE2 16x16, 16x8 (no special alignment)
5 - use SSE2 16x16, 16x8, aligned to 64 Byte blocks (good option on Core)
6 - use SSE3 16x16, 16x8 (seems only to work faster on Pentium 4E & Core1)
7 - use SSSES3 16x16, 16x8, aligned to 64 Byte blocks (good option on Core2)
Modes 8-12 are meant for debug testing only - use dct >4 instead.
Using sadx264 > 7 means: use selected instead of SAD for any luma/chroma SAD where possible!
for SAD / SATD mix and luma only use dct
8 - use SSD mmx, works on any mode as 1
9 - use SATD mmx, works on any mode as 1
10 - use SATD SSE2, works ony any mode like 2
11 - use SATD SSSE3, works ony any mode like 2
12 - use SATD SSSE3 with PHADD on 8xY, works ony any mode like 2
Default is 0, use -1 or >=13 to use old v1.9.4 SAD algo and completely deactivate x264 SAD functions

Try use MVShow function to check estimated motion field and tune parameters.

Note: MVAnalyse (if Pel=2) detects fieldbased video flag (after SeparateFields) and automatically makes correction of motion vectors according to verticlal shift of fields with different parity. Try use AssumeFrameBased if you do not need in it.

MVCompensate

MVCompensate (clip, clip "vectors", bool "scbehavior", int "mode", int "thSAD", bool "fields", clip "pelclip", int "idx")

Do a full motion compensation of the frame. It means that the blocks pointed by the mvs in the reference frame will be moved along the vectors to reach their places in the current frame.

scbehavior ( by default true ), decide which frame will be kept on a scene change. If true, the frame is left unchanged. Else, the reference frame is copied into the current one.

mode can be either 0 or 1 (default). Mode=0 previously used the compensation made during the vectors search. Now it is processed as mode=1. Mode=1 means it recomputes compensation from vectors data ( because you may want to apply vectors to a different clip that the one on which you searched ). Old Mode=2 is now disabled.

thSAD is SAD threshold for safe (dummy) compensation. If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

fields: if set to true and pel=2, then we add appropriate vertical shifts (by halfpixel) of fields for fieldbased video. Try use it for deinterlacing (not for denoising).
Default is false.

pelclip: upsampled source clip for using instead of internal subpixel interpolation (the same in MVAnalyse).
Default is not defined.

idx works the same way as idx in MVAnalyse.

Overlapped blocks processing is implemented as window block summation (like FFT3DFilter, overlap value up to blksize/2) for blocking artefactes decreasing.

MVDenoise

MVDenoise (clip, clip mvs [,...], bool "Y", bool "U", bool "V", int "thT", int "thSAD", int "thMV")

Makes a temporal denoising with motion compensation from several vector clips mvs. Reference frames are motion compensated and then merged into the current frame.

The first threshold, thT, decides whether the pixel which comes from the previous or the following frame has to be taken into account. If this pixel differs more than thT from the pixel of the current frame, it is not used.

The second one, thSAD, decides whether the block has changed or not ( same principle as thSCD1 ). If it has changed, the pixels aren't merged with those of the previous or following frame.

thMV is the vector's length over which the block isn't used for denoising.

Finally, Y, U and V tell which planes should be denoised.

Defaults are : Y, U and V are true, thT = 10, thSAD = 200 and thMV=30.

The function is obsolete. Overlapped blocks processing is implemented as simple sequential covering by block from left to right, from top to bottom. The function uses motion compensation data stored in vector clips.

MVMask

MVMask (clip, clip "vectors", float "ml", float "gamma", int "kind", int "Ysc")

Creates mask clip from motion vectors data. Mask is defined by blocks data, but is interpolated to fit full frame size. The mask is created both on the luma and on chroma planes. Mask values may be from 0 (min) to 255 (max).

kind parameter defines kind of mask.

Mode kind=0 creates motion mask from the motion vectors length. It builds a better mask than MotionMask ( MaskTools ) because motion vectors are a lot more reliable than the algorithm of MotionMask. Mask value 0 means no motion at all ( the length of the motion vector is null ). The longer vector length, the larger mask value (saturated to 255), the scale is defined by ml.

kind=1 allows to build a mask of the SAD (sum of absolute differences) values instead of the vectors' length. It can be useful to find problem areas with bad motion estimation. (Internal factor blocksize*blocksize/4 is used for normalization of scale ml.)

kind=2 allows to build a occlusion mask (bad blocks due to rupture, tensile). Currently, some normalized sum of positive blocks motion differences is used. It can be scaled with ml.

kind=3 allows to build a mask of horizontal component of motion vector in pel units plus 128. Scaled factors are not used.

kind=4 allows to build a mask of vertical component of motion vector in pel units plus 128. Scaled factors are not used.

kind=5 - motion colormap as x,y components of motion vector shown in U, V color planes (in pel units plus 128, scaled factors are not used).

ml parameter defines the scale of motion mask. When the vector's length (or other kind value) is superior or equal to ml, the output value is saturated to 255. The lesser value results to lesser output.

gamma is used to defined the exponent of relation output to input. gamma = 1.0 implies a linear relation, whereas gamma = 2.0 gives a quadratic relation.

And finally, Ysc is the value taken by the mask on scene change

Defaults are : kind = 0, ml = 100, gamma = 1.0, and Ysc = 0.

MVSCDetection

MVSCDetection (clip, clip "vectors", int "Ysc")

Creates scene detection mask clip from motion vectors data. The mask is created both on the luma and on chroma planes. Output without scene change is 0.

Ysc is the value taken by the mask on scene change, default is 255.

MVShow

MVShow (clip, clip "vectors", int "scale", int "sil", int "tol", bool "showsad")

Shows the motion vectors.

scale allows to enlarge the motion vectors, in order for example to gain in accuracy ( when pel > 1 and scale = 1, you can't see a variation of less than one pixel ).

sil allows to see a different level of analysis ( when searching for motion vectors, a hierarchical analysis is done, and it may be interesting to see what happens at higher levels ).

tol is a tolerance threshold. If the distortion induced by the motion vector is over var>tol the vector isn't shown.

Finally, showsad allows to show the mean SAD after compensating the picture.

Defaults are : scale = 1, sil = 0, tol = 20000 and showsad = false ( which shows all vectors ).

MVChangeCompensate

MVChangeCompensate (clip vectors, clip)

Allows to change the compensation stored into the mvs stream.

MVIncrease

MVIncrease (clip, clip "vectors", int "horizontal", int "vertical", int "idx")

It allows to use vectors computed on a reduced version of the clip in order to make a compensation on a clip with the original size.

horizontal is the horizontal ratio between the width of the clip and the width of the reduced clip.

vertical is the vertical ratio between the height of the clip and the height of the reduced clip.

idx works the same as in MVAnalyse

MVDepan

MVDepan (clip, clip "vectors", bool "zoom", bool "rot", float "pixaspect", float "error", bool "info", string "log", float "wrong", float "zerow", int "range")

Get the motion vectors, estimate global motion and put data to output frame in special format for DePan plugin (by Fizick).

Inter-frame global motion (pan, zoom, rotation) is estimated by iterative procedure, with good blocks only.

Rejected blocks: 1) near frame borders; 2) with big SAD (by thSCD1 parameter); 3) with motion different from neighbors or global.

zoom and rot parameters switch zoom and rotation estimation, pixaspect is pixel aspect (1.094 for standard PAL, 0.911 for standard NTSC), error is maximum mean motion difference.

The frame estimated global motion is switched to null for big motion error or at scene change (by thSCD1, thSCD2 parameters).

info parameter allows to type global motion info for debug.

log parameter allows to set log file name in DeShaker, Depan format.

wrong parameter defines limit to disable blocks very different from neighbors.

zerow parameter defines weight of zero motion vectors (to decrease its influence).

range - number of previous (and also next) frames (fields) near requested frame to estimate their motion.

Defaults are : zoom = true, rot = true, pixaspect = 1.0, error = 15.0, info = false, wrong=10, zerow=0.05, range=0.

For global motion estimation of interlaced source, you must separate fields (for both MVAnalyse and MVDepan).

MVFlow

MVFlow (clip, clip "vectors", float "time", int "mode", bool "fields", clip "pelclip", int "idx")

Do a motion compensation of the frame not by blocks (like MVCompensation), but by pixels. Motion vector for every pixel is calculated by bilinear interpolation of motion vectors of current and neighbor blocks (according to pixel position). It means that the pixels pointed by the vector in the reference frame will be moved (flow) along the vectors to reach their places in the current frame. This flow motion compensation method does not produce any blocking artefactes, and is good for denoising, but sometimes can create very strange deformed pictures :). True motion estimation is strongly recommended for this function. Motion compensation may be full or partial (at intermediate time).
Limitation: vectors with components above 127 will be reset to zero length.

time: percent of motion compensation (default=100.0, full compensation), define time moment between reference and current frame.

mode can be either 0 ( default ), or 1.
mode=0 - fetch pixels to every place of destination. It is main producing mode.
mode=1 - shift pixels from every place of source (reference). It is debug (learning) mode with some empty spaces (with null intensity). It can be used for occlusion mask creation.

fields: if set to true and pel=2, then we add appropriate vertical shifts (by halfpixel) of fields for fieldbased video. Try use it for deinterlacing (not for denoising).
Default is false.

pelclip: upsampled source clip for using instead of internal subpixel interpolation (the same in MVAnalyse).
Default is not defined.

idx (may be) works the same way as idx in MVAnalyse.

MVFlowInter

MVFlowInter (clip, clip "mvbw", clip "mvfw", float "time", float "mL", clip "pelclip", int "idx")

Motion interpolation function. It is not the same (but similar) as MVInterpolate function of older MVTools version. It uses backward "mvbw" and forward "mvfw" motion vectors to create picture at some intermediate time moment between current and next (by delta) frame. It uses pixel-based (by MVFlow method) motion compensation from both frames. Internal forward and backward occlusion masks (MVMask kind=2 method) and time weighted factors are used to produce the output image with minimal artefactes. True motion estimation is strongly recommended for this function.

time: interpolation time position between frames (in percent, default=50.0, half-way)

mL: mask scale parameter. Lower values are corresponded to more strong occlusion mask (as in MVMask function, use it to tune and debug). Default=100.

pelclip: upsampled source clip for using instead of internal subpixel interpolation (the same in MVAnalyse).
Default is not defined.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

MVFlowFps

MVFlowFps (clip, clip "mvbw", clip "mvfw", int "num", int "den", int "mask", float "ml", clip "pelclip", int "idx")

Will change the framerate (fps) of the clip (and number of frames). The function can be use for framerate conversion, slow-motion effect, etc. It uses backward "mvbw" and forward "mvfw" motion vectors to create interpolated pictures at some intermediate time moments between frames. The function uses pixel-based motion compensation (as MVFlow, MVFlowInter). Internal forward and backward occlusion masks (MVMask kind=2 method) and time weighted factors are used to produce the output image with minimal artefactes. True motion estimation is strongly recommended for this function.

num: output clip fps numerator. Default=25.

den: fps denominator (default=1). Resultant fps = num/den. In particular for doubled NTSC fps=2*29.97 use num=60000 and den=1001, and for doubled NTSC FILM fps=2*23.976 use num=48000 and den=1001. When num or den is equal to 0, then doubled fps of input clip is assumed for output (since v1.8.1).

mask: processing mask mode:
mask=0 is simple backward and forward occlusion masks (used in versions up to 1.4.x, fastest);
mask=1 is similar masks with additional switching to static zero vectors at occlusion areas (similar to v1.5.x);
mask=2 is for using extra vectors from adjacent frames for decreasing objects halo at occlusion areas (v1.8, slowest). Default=2.

ml: mask scale parameter. The greater values are corresponded to more weak occlusion mask (as in MVMask function, use it to tune and debug). Default=100.

pelclip: upsampled source clip for using instead of internal subpixel interpolation (the same in MVAnalyse).
Default is not defined.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

MVFlowFps2

MVFlowFps2 (clip, clip "mvbw", clip "mvfw", clip "mvbw2", clip "mvfw2", int "num", int "den", int "mask", float "ml", clip "pelclip", int "idx", int "idx2")

Will change the framerate (fps) of the clip (and number of frames) like MVFlowFps, but with a little better quality (and slower processing) with external block overlapping.
(Notes: now MVFlowFps has more effective internal block overlapping, and MVFlowFps2 is obsolete. MVFlowFps2 does not work properly with overlap in MVAnalyse).
In addition to backward "mvbw" and forward "mvfw" motion vectors of original source clip, the function MVFlowFps2 uses backward "mvbw2" and forward "mvfw2" motion vectors of second (modified) source clip. Second clip must be produced from original source clip by cropping (i.e. diagonal shift) by half block size. It must be done with command Crop(a,a,-b,-b), where a=b=4 must be used for blksize=8, a=b=8 for blksize=16, and a=2, b=6 for blksize=4 (see example). Blocks boundaries will be at different parts of objects. MVFlowFps2 reverses the shift internally and averages motion vectors from these two sources to decrease motion estimation errors. The function uses pixel-based motion compensation (as MVFlow, MVFlowInter). Internal forward and backward occlusion masks (MVMask kind=2 method) and time weighted factors are used to produce the output image with minimal artefactes. True motion estimation is strongly recommended for this function.

num: output clip fps numerator. Default=25.

den: fps denominator (default=1). Resultant fps = num/den. In particular for doubled NTSC fps=2*29.97 use num=60000 and den=1001, and for doubled NTSC FILM fps=2*23.976 use num=48000 and den=1001. When num or den is equal to 0, then doubled fps of input clip is assumed for output (since v1.8).

mask: processing mask mode:
mask=0 is simple backward and forward occlusion masks (used in versions up to 1.4.x, fastest);
mask=1 is similar masks with additional switching to static zero vectors at occlusion areas (similar to v1.5.x);
mask=2 is for using extra vectors from adjacent frames for decreasing objects halo at occlusion areas (v1.8, slowest). Default=2.

ml: mask scale parameter. The greater values are corresponded to more weak occlusion mask (as in MVMask function, use it to tune and debug). Default=100.

pelclip: upsampled source clip for using instead of internal subpixel interpolation (the same in MVAnalyse).
Default is not defined.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

idx2 is MVAnalyse index of second (shifted) clip (must not coincide with first idx).

MVBlockFps

MVBlockFps (clip, clip "mvbw", clip "mvfw", int "num", int "den", int "mode", float "thres", int "idx")

The function uses block-based partial motion compensation to change the framerate (fps) of the clip (and number of frames). It uses backward "mvbw" and forward "mvfw" motion vectors to create interpolated pictures at some intermediate time moments between frames. Some internal forward and backward masks and time weighted factors are used to produce the output image. (Algorithm is based on MVInter function of old MVTools v1.9.12.) It is faster than MVFlowFps but may produce blocking and other artefactes. True motion estimation is strongly recommended for this function.

num: output clip fps numerator. Default=25.

den: fps denominator (default=1). Resultant fps = num/den. In particular for doubled NTSC fps=2*29.97 use num=60000 and den=1001, and for doubled NTSC FILM fps=2*23.976 use num=48000 and den=1001. When num or den is equal to 0, then doubled fps of input clip is assumed for output.

mode: processing mode:
0 - average of fetched forward and backward partial motion compensation (fastest, default).
1 - static median.
2 - dynamic median.
3 - time weigthed combination of fetched forward blocks masked by shifted backward and fetched backward masked by shifted forward.
4 - mode 3 mixed with simple static time average by occlusion mask of shifted blocks.
5 - occlusion mask (for debug).

thres: threshold for count of occlusions per block for mask binarization. Default=0, that is internally = blksize*blksizeV/4.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

MVFlowBlur

MVFlowBlur (clip, clip "mvbw", clip "mvfw", float "blur", int "prec", clip "pelclip", int "idx")

Experimental simple motion blur function. It may be used for FILM-effect (to simulate finite shutter time). It uses backward "mvbw" and forward "mvfw" motion vectors to create and overlay many copies of partially compensated pixels at intermediate time moments in some blurring interval around current frame. It uses pixel-based motion compensation (as MVFlow). True motion estimation is strongly recommended for this function.

blur: blur time interval between frames, open shutter time (in percent, default=50.0)

prec: blur precision in pixel units. Maximal step between compensated blurred pixels. Default =1 (most precise).

pelclip: upsampled source clip for using instead of internal subpixel interpolation (the same in MVAnalyse).
Default is not defined.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

MVDeGrain1, MVDeGrain2 and MVDegrain3

MVDeGrain1 (clip, clip "mvbw", clip "mvfw", int "thSAD", int "thSADC", int "plane", int "limit", clip "pelclip", int "idx")

MVDeGrain2 (clip, clip "mvbw", clip "mvfw", clip "mvbw2", clip "mvfw2", int "thSAD", int "thSADC", int "plane", int "limit", clip "pelclip", int "idx")

MVDeGrain3 (clip, clip "mvbw", clip "mvfw", clip "mvbw2", clip "mvfw2", clip "mvbw3", clip "mvfw3",int "thSAD", int "thSADC", int "plane", int "limit", clip "pelclip", int "idx")

Makes a temporal denoising with motion compensation. Blocks of previous and next frames are motion compensated and then averaged with current frame with weigthing factors depended on block differences from current (SAD). Functions support overlapped blocks mode.

Overlaped blocks processing is implemented as window block summation (like FFT3DFilter, overlap value up to blksize/2) for blocking artefactes decreasing.

MVDeGrain1 has temporal radius 1 (uses vectors of previous mvfw and next mvbw frames).

"thSAD" parameter defines soft threshold of block sum absolute differences. Block with SAD above threshold thSAD have a zero weight for averaging (denoising). Block with low SAD has highest weight. You must enter thSAD value reduced to block size 8x8. Low values can result in staggered denoising, large values can rezult in ghosting and artefactes.
Default thSAD=400.

"thSADC" is the threshold for chroma planes.
Default (if not defined) : thSADC=thSAD. If defined then thSADCis used for chroma and thSAD is used for luma.

plane parameter set procesed color plane:
0 - luma, 1 - chroma U, 2 - chroma V, 3 - both chromas, 4 - all. Default is 4.

limit: maximal change of pixel (like DeGrainMedian plugin to prevent some artefactes). Default is 255 (no limit).

pelclip: upsampled source clip for using instead of internal subpixel interpolation (the same in MVAnalyse).
Default is not defined.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

MVRecalculate

MVRecalculate (clip, clip vectors, int "thSAD", int "blksize", int "blksizeV", int "search", int "searchparam", int "lambda", bool "chroma", bool "truemotion", int "pnew", int "overlap", int "overlapV", string "outfile", int "sharp", clip "pelclip", int "dct", int "divide", bool "mc", int "idx", int "sadx264")

Refines and recalculates motion data of previously estimated (by MVAnalyse) motion vectors with different clip or new parameters set (e.g. lesser block size), after divide, etc. The two-stage method may be also useful for more stable (robust) motion estimation. The refining is at finest hierarchical level only. Interpolated vectors of old blocks are used as predictors for new vectors, with recalculation of SAD. Only bad quality new vectors with SAD above threshold thSAD will be re-estimated by search. Enter thSAD value reduced (scaled) to block size 8x8. Default thSAD=200. Good vectors are not changed, but its SAD will be updated (re-calculated).

III) Examples

In all examples we assume that some source clip is firstly defined:

source = AVISource("c:\test.avi") # or MPEG2Source, DirectShowSource, some previous filter, etc

To show the motion vectors ( forward ) :

vectors = source.MVAnalyse(isb = false)
source.MVShow(vectors)

To show the backward one :

vectors = source.MVAnalyse(isb = true)
source.MVShow(vectors)

To use MVMask :

vectors = source.MVAnalyse(isb = false)
source.MVMask(vectors)

To denoise :

backward_vec2 = source.MVAnalyse(isb = true, lambda = 1000, delta = 2)
backward_vec1 = source.MVAnalyse(isb = true, lambda = 1000, delta = 1)
forward_vec1 = source.MVAnalyse(isb = false, lambda = 1000, delta = 1)
forward_vec2 = source.MVAnalyse(isb = false, lambda = 1000, delta = 2)
source.MVDenoise(backward_vec2,backward_vec1,forward_vec1,forward_vec2,tht=10,thSAD=300)

To deblock the compensation stored into a mvs stream

vectors = source.MVAnalyse(isb = false, lambda = 1000)
compensation = source.MVCompensate(vectors, mode = 0)
compensation = compensation.Deblock() # use DeBlock function
vectors = vectors.MVChangeCompensate(compensation)

To denoise with pel = 2, efficiently :

backward_vec2 = source.MVAnalyse(isb = true, lambda = 1000, delta = 2, pel = 2, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, lambda = 1000, delta = 1, pel = 2, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, lambda = 1000, delta = 1, pel = 2, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, lambda = 1000, delta = 2, pel = 2, idx = 1)
source.MVDenoise(backward_vec2,backward_vec1,forward_vec1,forward_vec2,tht=10,thSAD=300)

To use MVIncrease :

vectors = source.reduceby2().mvanalyse(isb = true)
return source.MVIncrease(vectors, horizontal = 2, vertical = 2)

To use MVDepan with Depan plugin for interlaced source (DepanStabilize function example):

source = source.AssumeTFF().SeparateFields() # set correct fields order
vectors = source.MVAnalyse(isb = false)
globalmotion = source.MVDepan(vectors, pixaspect=1.094, thSCD1=400)
DepanStabilize(source, data=globalmotion, cutoff=2.0, mirror=15, pixaspect=1.094)
Weave()

To blur problem (blocky) areas of compensated frame with occlusion mask:

vectors = source.MVAnalyse(isb = false, lambda = 1000)
compensation = source.MVCompensate(vectors) # or use MVFlow function here
# prepare blurred frame with some strong blur or deblock function:
blurred = compensation.DeBlock(quant=51) # Use DeBlock function here
badmask = source.MVMask(vectors, kind = 2, ml=50)
overlay(compensation,blurred,mask=badmask) # or use faster MaskedMerge function of MaskTools

To recreate bad frames by interpolation with MVFlowInter:

backward_vectors = source.MVAnalyse(isb = true, truemotion=true, pel=2, delta=2, idx=1)
forward_vectors = source.MVAnalyse(isb = false, truemotion=true, pel=2, delta=2, idx=1)
inter = source.MVFlowInter(backward_vectors, forward_vectors, time=50, ml=70, idx=1)
# Assume bad frames are 50 and 60
source.trim(0,49) ++ inter.trim(49,-1) \
 ++ source.trim(51,59) ++ inter.trim(59,-1) ++ source.trim(61,0)

To change fps with MVFlowFps:

# assume progressive PAL 25 fps source
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
source.MVFlowFps(backward_vec, forward_vec, num=50, den=1, ml=100, idx=1) # get 50 fps

To double fps with MVFlowFps for fastest (almost) real-time playing:

# assume progressive PAL 25 fps or NTSC Film 23.976 source
backward_vec = source.MVAnalyse(blksize=16, isb = true, chroma=false, pel=1, searchparam=1, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(blksize=16, isb = false, chroma=false, pel=1, searchparam=1, idx=1)
source.MVFlowFps(backward_vec, forward_vec, num=2*FramerateNumerator(source), \
   den=FramerateDenominator(source), mask=0, idx=1)

To double fps with MVFlowFps for 'best' results (but slower processing):

# assume progressive PAL 25 fps or NTSC Film 23.976 source
backward_vec = source.MVAnalyse(overlap=4, isb = true, pel=2, search=3, idx=1)
# Use block overlap, halfpixel accuracy and Exhaustive search
forward_vec = source.MVAnalyse(overlap=4, isb = false, pel=2, search=3, idx=1)
source.MVFlowFps(backward_vec, forward_vec, num=2*FramerateNumerator(source), \
   den=FramerateDenominator(source), idx=1)

To change fps with MVFlowFps2:

# Assume progressive PAL 25 fps source. Lets try convert it to 50.
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1, search=3)
# we use explicit idx for more fast processing and use full search
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1, search=3)
cropped = source.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2, search=3)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2, search=3)
source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=50,idx=1,idx2=2)

To generate nice motion blur with MVFlowBlur:

backward_vectors = source.MVAnalyse(isb = true, truemotion=true)
forward_vectors = source.MVAnalyse(isb = false, truemotion=true)
source.MVFlowBlur(backward_vectors, forward_vectors, blur=15)

To denoise with some external denoiser filter (which uses 3 frames: prev, cur, next):

backward_vectors = source.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1)
# we use explicit idx for more fast processing
forward_vectors = source.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1)
forward_compensation = source.MVFlow(forward_vectors, idx=1, thSCD1=500) # or use MVCompensate
backward_compensation = source.MVFlow(backward_vectors, idx=1, thSCD1=500) # or use MVCompensate
# create interleaved 3 frames sequences
interleave(forward_compensation, source, backward_compensation)

DeGrainMedian() # place your preferred temporal (spatial-temporal) denoiser here

selectevery(3,1) # return filtered central (not-compensated) frames only

To use prefiltered clip for more reliable motion estimation, but compensate motion of not-prefiltered clip (denoising example)

# Use some denoiser (blur) or deflicker for prefiltering
prefiltered = source.DeGrainMedian()
backward_vectors = prefiltered.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1)
# we use explicit idx for more fast processing
forward_vectors = prefiltered.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1)
# use not-prefiltered clip for motion compensation (with other idx)
forward_compensation = source.MVFlow(forward_vectors, idx=2) # or use MVCompensate(mode=1)
backward_compensation = source.MVFlow(backward_vectors, idx=2) # or use MVCompensate(mode=1)
# create interleaved 3 frames sequences
interleave(forward_compensation, source, backward_compensation)

DeGrainMedian() # place your preferred temporal (spatial-temporal) denoiser here

selectevery(3,1) # return filtered central (not-compensated) frames only

To denoise by MVDegrain2 with overlapped blocks (blksize=8) and subpixel precision:

backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=1)

To denoise interlaced source by MVDegrain1 with overlapped blocks (blksize=8) and subpixel precision:

fields=source.AssumeTFF().SeparateFields() # or AssumeBFF
backward_vec2 = fields.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1)
forward_vec2 = fields.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1)
fields.MVDegrain1(backward_vec2,forward_vec2,thSAD=400,idx=1)
Weave()

To denoise interlaced source with function MVDegrain2i:

function MVDegrain2i(clip "source", int "overlap", int "dct", int "idx")
{
overlap=default(overlap,0) # overlap value (0 to 4 for blksize=8)
dct=default(dct,0) # use dct=1 for clip with light flicker
idx=default(idx,1) # use various idx for different sources in same script
fields=source.SeparateFields() # separate by fields
backward_vec2 = fields.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=overlap, idx = idx,dct=dct)
forward_vec2 = fields.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=overlap, idx = idx,dct=dct)
backward_vec4 = fields.MVAnalyse(isb = true, delta = 4, pel = 2, overlap=overlap, idx = idx,dct=dct)
forward_vec4 = fields.MVAnalyse(isb = false, delta = 4, pel = 2, overlap=overlap, idx = idx,dct=dct)
fields.MVDegrain2(backward_vec2,forward_vec2,backward_vec4,forward_vec4,thSAD=400,idx=idx)
Weave()
}

source=AVISource("video.avi")
mvdegrain2i(source,4,0,1)

How to use external subpixel interpolation clip:

# Load Eedi2 plugin (by tritical) with edge-directed interpolation function
LoadPlugin("Eedi2.dll")
# create upsampled clip of source
ups = source.EEDI2(field=1).LanczosResize(2*width(source), 2*height(source), src_left=0.25)
bv = source.MVAnalyse(isb = true, pel = 2, overlap=4, pelclip=ups, idx = 1)
source.MVCompensate(bv, pelclip=ups, idx=1)

How to use with MT filter and special multithreaded AviSynth:

# Load MT plugin by TSP
LoadPlugin("MT.dll")
avisource("some.avi")
global idx1 = 10  # global hint by IanB
MT("""
idx1 = idx1 + 1
# different threads for top and bottom half of frame must have different idx (trick by Foxishadis)
backward_vec2 = MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = idx1)
backward_vec1 = MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = idx1)
forward_vec1 = MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = idx1)
forward_vec2 = MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = idx1)
last.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=idx1)
""",2) # two threads

# Note: SetMTMode(2) mode of multithreaded AviSynth is also supported since MVTools v.1.8.4.1 (beta testing)

Same example with SetMTMode:

SetMTMode(5)
FFmpegSource("some.avi") # avisource doesn't work with SetMTMode on my machines - TSchniede

SetMTMode(2)
idx = 1
backward_vec2 = MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = idx)
backward_vec1 = MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = idx)
forward_vec1 = MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = idx)
forward_vec2 = MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = idx)
last.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=idx)

How to use prefiltered clip and recalculate motion data with original source (MVFlowFPS example)

prefiltered = source.DeGrainMedian() # some smoothing
backward = prefiltered.MVAnalyse(isb = true, idx = 1, blksize=16)
forward = prefiltered.MVAnalyse(isb = false, idx = 1, blksize=16)
# recalculate for original source clip and other block size
forward_re = MVRecalculate(source, forward, idx=2, blksize=8, thSAD=100)
backward_re = MVRecalculate(source, backward, idx=2, blksize=8, thSAD=100)
source.MVFlowFps(backward_re, forward_re, num=50, den=1, idx=2)

IV) Disclaimer

This plugin is distributed under terms of the GNU GPL license, without any warranty. See 'gpl.txt'.
Documentation is distributed under CreativeCommons BY-SA 3.0 license.

Please consider to make some donation for support.

MVTools uses portions of code of following projects:
Resizer (SimpleResize.cpp) is from SimpleResize Avisynth plugin by Tom Barry. http://www.trbarry.com
Fast DCT (fdct_mmx.asm) is from XVID MPEG-4 VIDEO CODEC. http://www.xvid.org
DCT general transform (see fftwlite.h) is from FFTW library (as DLL call). http://www.fftw.org
SATD (pixel-32.asm, pixel.asm) and alternative SADx264 (sad-a.asm, x86inc.asm, x86inc-32.asm, cpu-32.asm) are from x264 project. http://www.videolan.org/developers/x264.html (Loren Merritt, Laurent Aimar, Alex Izvorski et al).

V) Revisions

VI) Download

Download MVTools v.1.11.4.5

There are also new MVTools v2.0 branch under development.

Older MVTools v.0.9.9.1 can be downloaded from Manao's site