MaskTools2/mt lutxy

Description

mt_lutxy applies an expression to all the pixels of two masks / videos.

Syntax and Parameters

mt_lutxy (clip, clip, string "expr", string "yExpr", string "uExpr", string "vExpr", float "Y ", float "U", float "V", string "chroma", int "offX", int "offY", int "w", int "h", bool "sse2", bool "sse3", bool "ssse3", bool "sse4", bool "avx", bool "avx2", float "A", string "alpha", string "paramscale", bool "realtime", string "aExpr", string "scale_inputs", bool "clamp_float", int "use_expr", bool "clamp_float_UV")

clip   =

Input clip one.

clip   =

Input clip two.

string  expr = "x"

string  yExpr = "x"

string  uExpr = "x"

string  vExpr = "x"

Expression written is reverse polish notation.

If yexpr, uexpr or vexpr is noy defined, expr is used instead.

float  Y = 3

float  U = 2

float  V = 2

These values describe the actual processing mode that is to be used on each plane / channel. Here is how the modes are coded :

• x = -255...0 : all the pixels of the plane will be set to -x. (-255..0 range can of course differ when non-8 bit video formats are used)
• x = 1 : the plane will not be processed. That means the content of the plane after the filter is pure garbage.
• x = 2 : the plane of the first input clip will be copied.
• x = 3 : the plane will be processed with the processing the filter is designed to do.
• x = 4 : the plane of the second input clip will be copied.

U, V and A are defaulted to 2 (that way, the resulting clip contains the chroma (alpha) of clip1, and looks right).

string  chroma = ""

When defined, the value contained in this string will overwrite the u & v processing modes.

This is a nice addition proposed by mg262 that makes the filter more user friendly. Allowed values for chroma are:

• "process" : set u = v = 3.
• "copy" or "copy first" : set u = v = 2.
• "copy second" : set u = v = 4.
• "xxx", where xxx is a number : set u = v = -xxx.

int  offX = 0

int  offY = 0

offX and offY are the top left coordinates of the box where the actual processing shall occur. Everything outside that box will be garbage.

int  w = -1

int  h = -1

w and h are the width and height of the processed box. -1 means that the box extends to the lower right corner of the video. That also means that default settings are meant to process the whole picture.

bool  sse2 = true

bool  ssse3 = true

bool  sse4 = true

bool  avx = true

bool  avx2 = true

CPU optimizations; for debugging only.

float  Y = 1

Same as the Y,U,V parameters but for the alpha channel, if applicable.

string  alpha = ""

Same as the chroma parameter but for the alpha channel, if applicable.

string  paramscale = "i8"

Same as the chroma parameter but for the alpha channel, if applicable.

Parameter "paramscale" for filters working with threshold-like parameters (v2.2.5-)

Filters: mt_binarize, mt_edge, mt_inpand, mt_expand, mt_inflate, mt_deflate, mt_motion, mt_logic, mt_clamp

paramscale can be "i8" (default), "i10", "i10", "i12", "i14", "i16", "f32" or "none" or ""

Using "paramscale" tells the filter that parameters are given at what bit depth range.

By default paramscale is "i8", so existing scripts with parameters in the 0..255 range are working at any bit depths

bool  realtime = false

Parameter "realtime" for lut-type filters to override default behaviour.

string  aExpr = "x"

Same as the expr,yExpr,uExpr,vExpr parameters but for the alpha channel, if applicable.

string  scale_inputs = "none"

Autoscale any input (x,y,z,a) bit depths to 8-16 bit for internal expression use, the conversion method is either full range or limited YUV range. (Replaces clamp_f_i8, clamp_f_i10, clamp_f_i12, clamp_f_i14 or clamp_f_i16, clamp_f_f32 or clamp_f keywords)

The primary reason of this feature is the "easy" usage of formerly written expressions optimized for 8 bits.

Use

• "int" : scales limited range videos, only integer formats (8-16bits) to 8 (or bit depth specified by 'i8'..'i16')
• "intf" : scales full range videos, only integer formats (8-16bits) to 8 (or bit depth specified by 'i8'..'i16')
• "float" or "floatf" : only scales 32 bit float format to 8 bit range (or bit depth specified by 'i8'..'i16')
• "floatUV" : affects the chroma plane expressions of 32 bit float formats. Shifts the input up by 0.5 before processing it in the expression, thus values from -0.5..0.5 (zero centered) range are converted to the 0..1 (0.5 centered) one. Since the expression result internally has 0..1.0 range, this then is shifted back to the original -0.5..0.5 range. (since 2.2.20) Note: predefined constants such as cmin, cmax, range_min, range_max and range_half will be shifted as well, e.g. the expression will see range_half = 0.5
• "all" : scales videos to 8 (or bit depth specified by 'i8'..'i16') - conversion uses limited_range logic (mul/div by two's power)
• "allf" : scales videos to 8 (or bit depth specified by 'i8'..'i16') - conversion uses full scale logic (stretch)
• "none" : no magic

Usually limited range is for normal YUV videos, full scale is for RGB or known-to-be-fullscale YUV.

By default the internal conversion target is 8 bits, so old expressions written for 8 bit videos will probably work. This internal working bit-depth can be overwritten by the i8, i10, i12, i14, i16 specifiers.

When using autoscale mode, scaleb, scalef, yscaleb and yscalef keywords are meaningless for 8-16 bits, because there is nothing to scale. 32 bit (float) values will be scaled however when "float", "floatUV", "all", "allf" is specified.

How it works:

• This option scales (x,y,z,a) 8-32 bit inputs to a common bit depth value, which bit depth is 8 by default and can be set to 10, 12, 14 and 16 bits by the 'i10'..'i16' keywords For example: scale_inputs="all" converts any inputs to 8 bit range. No truncation occurs however (no precision loss), because even a 16 bit data is converted to 8 bit in floating point precision, using division by 256.0 (2^16/2^8). So the conversion is not a simple shift-right-8 in the integer domain, which would lose precision.

• Calculates expression

• Scales the result back to the output video bit depth. Clamping (clipping to valid range) and converting to integer occurs here.

The predefined constants such as 'range_max', etc. will behave according to the internal working bit depth.

Warning#1 This feature was created for easy porting earlier 8-bit-video-only expressions. You have to understand how it works internally.

Let's see a 16bit input in "all" and "allf" mode (target is the default 8 bits)

Limited range 16->8 bits conversion has a factor of 1/256.0 (Instead of shift right 8 in integer domain, float-division is used or else it would lose presision) Full range 16->8 bits conversion has a factor of 255.0/65535

Using bit shifts (really it's division and multiplication by 2^8=256.0): result = calculate_lut_value(input / 256.0) * 256.0 Full scale 16-8-16 bit mode ('intf', 'allf') result = calculate_lut_value(input / 65535.0 * 255.0 ) / 255.0 * 65535.0

Use scale_inputs = "all" ("int", "float") for YUV videos with 'limited' range e.g. in 8 bits: Y=16..235, UV=16..240). Use scale_inputs = "allf" (intf, floatf) for RGB or YUV videos with 'full' range e.g. in 8 bits: channels 0..255.

When input is 32bit float, the 0..1.0 (luma) and -0.5..0.5 (chroma) channel is scaled to 0..255 (8 bits), 0..1023 (i10 mode), 0..4095 (i12 mode), 0..16383(i14 mode), 0..65535(i16 mode) then back.

Warning#2 One cannot specify different conversion methods for converting before and after the expression. Neither can you specify different methods for different input clips (e.g. x is full, y is limited is not supported).

bool  clamp_float =

32 bit float video is always a bit different, as ususally no clamping is applied to valid ranges

This parameter along with clamp_float_UV changes this behaviour.

false: no clamp

true: standard clamp which is 0..1 for Luma or for RGB color space and -0.5..0.5 for YUV chroma UV chroma clamping can be set to 0..1 by using clamp_float_UV.

int  use_expr =

Use it when realtime calculation (interpreted pixel-by-pixel expression calculation) is slow and an appropriate AviSynth+ version (>r2712) is available.

By sending the expression to AviSynth+, lut filters can utilize a realtime JIT-compiled fast expression calculation.

Possible values:

• 0: uses lut and internal realtime calculation
• 1: Expr, when bit depth>=10 or lutxyza
• 2: When masktools would use realtime calc, passes the expressions and parameters to the "Expr" filter in AviSynth+
• 3: Expr, always passed (from 2.2.17)

For modes 1, 2 and 3: Passes the expressions, "scale_inputs", "clamp_float" and "clamp_float_UV" parameter to the "Expr" filter in AviSynth+

Note: clamp_float_UV is valid parameter only from AviSynth+ 3.5, and for compatibility reasons is passed only when it's true, so when it differs from the default value.

Note #1: AviSynth+ internal precision is 32bit float, masktools2 is double (usually no difference can be seen)

Note #2: Some keywords (e.g. bit shift) are not available on AviSynth+

Note #3: Since "Expr" can work only on full sized clips, offX, offY, w and h parameters are ignored.

bool  clamp_float_UV = false

false: standard clamp which is 0..1 for Luma or for RGB color space and -0.5..0.5 for YUV chroma UV

true: chroma UV clamp same as luma 0..1, used in conjunction with expressions written for integer (positive only) U/V values in mind.

Examples

• Compare 2 clips, allow only differences higher than the threshold[1]

thresh = 12
mt_lutxy(original, processed, expr="x y - abs "+string(thresh)+" > y x ?", Y=3, U=3, V=3)

• Luma difference (very strict)[2]

clip1 = this
clip2 = that
diff = mt_lutxy(clip1, clip2, expr="x y - abs 255 *")

or

diff = mt_lutxy(clip1, clip2, expr="x y - abs 2 - 255 12 / *")

This would make the difference ignore differences up to +/-2, and scale the remaining difference-mask in 12 steps from black to white.[3]

---

Back to MaskTools2 ←

---