AvsInpaint
From Avisynth wiki
| Abstract | |
|---|---|
| Author | AMSS0815, pinterf |
| Version | v1.2 |
| Download | AvsInpaint-v1.2.7z |
| Category | Logo removal |
| License | |
| Discussion | Doom9 Forum |
Contents |
Description
Image Inpainting is the art of restoring destroied parts of an image by using information of valid parts of the image in a way, so that the human eye does not recognize the damaged areas (at least not at a first sight). In video processing image inpainting is often applied to videos in order to remove TV station logos. This plugin comes with the intention to provide a suit for the removal of logos, whether opaque or transparent. It provides algorithms for these tasks:
- Logo detection: An obstacle is analyzed closely after being masked roughly by the user. The exact color map of the logo is determined as well as its alpha mask.
- Deblending: Given the color image and alpha mask of an overlayed logo, the logo is removed where it is not opaque.
- Logo inpainting: Opaque parts of the logo are interpolated. A fast method is used which has been published recently by Folkmar Bornemann and Tom März and which is roughly described below.
Finally, a function for calculating the distance function of a mask is provided, which can be used to expand or inflate masks as well as to provide smooth blending masks.
Requirements
Syntax and Parameters
Inpaint Function
InpaintLogo restores masked parts of video frames by interpolation with non-masked parts of the frame. Each masked pixel will be inpainted by averaging surrounding pixels. The weights of this averaging are determined by calculating a special structure tensor that contains information about the directions of isophotes near that pixel.
- InpaintLogo (clip Clip, clip "Mask", float "Radius", float "Sharpness", float "PreBlur", float "PostBlur", float "ChromaWeight", float "PreBlurSize", float "PostBlurSize", bool "ChromaTensor", float "PixelAspect", int "Steps")
- clip =
- Input clip; the damaged source clip. All color spaces are allowed. If no Mask is provided, the source clip must be RGB32 and the alpha channel is used as mask (see Mask for details). Otherwise all color channels of Clip will be inpainted. The resulting clip will have the same properties as the source clip.
- clip =
- clip Mask =
- Mask defines the parts of the video which will be treated by the plugin. This clip may either have a YUV colorspace (the Y channel is used) or RGB32 (the alpha channel is used). All pixels of the source clip where this mask has values greater than 127 will be inpainted. The mask clip must have the same (spatial) dimensions as the source clip. Moving logos are allowed: If the mask changes in time, the inpainted region in the source clip will change accordingly. If the logo is fixed (hence all mask frames are equal), the mask clip should be trimmed to one single frame indicating a static logo to the function. Since each movement of the inpainting region causes a lot new calculations, inpainting static logos is much faster.
- clip Mask =
- float Radius = 5.0
- Radius describes the neighborhood of a damaged pixel from where values are taken when the pixel is inpainted. Bigger values prevent isophotes being inpainted in the wrong direction, but also create more blur. (default: 5.0 Px)
- float Radius = 5.0
- float sharpness = 30.0
- Sharpness describes how faithfull the algorithm follows directional information contained in the structure tensor. Higher values can prevent blurring caused by high Radius values. (default: 30.0)
- float sharpness = 30.0
- float PreBlur = 0.5
- PreBlur is the standard deviation of the blur which is applied to the image before the structure tensor is computed. Higher values help connecting isophotes which have been cut by the inpainting region, but also increase CPU usage. PreBlur=0.0 disables pre-blurring. (default: 0.5 Px)
- float PreBlur = 0.5
- float PostBlur = 4.0
- PostBlur is the standard deviation of the blur which is applied to the structure tensors before they are used to determine the inpainting direction. Higher values help gathering enough directional information when there are only few valid pixels available, but also need lots of CPU cycles. (default: 4.0 Px)
- float PostBlur = 4.0
- float ChromaWeight = 0.0
- ChromaWeight describes how chroma channels are taken into account when the structure tensor is build. After tensors for each color channel are set up, tensors are added, where luma is weighted with 1.0-ChromaWeight, and all chroma tensors together with ChromaWeight. Tensors not needed are not computed. This parameter is ignored for RGB color spaces: Each channel is weighted equally then. (default: 0.0)
- float ChromaWeight = 0.0
- float PreBlurSize = 2×PreBlur
- float PostBlurSize = 2×PostBlur
- PreBlurSize and PostBlurSize describe the sizes of the blurring kernels used for pre blurring and post blurring, respectively. (default: 2×PreBlur and 2×PostBlur, resp.)
- float PreBlurSize = 2×PreBlur
- bool ChromaTensor = false
- When ChromaTensor is set to true, all structure tensors will be computed again before a chroma channel pixel is inpainted (after the corresponding luma pixel has been inpainted). Chroma values are always inpainted after a luma value has been inpainted in the same position. This can be especially usefull when chroma channels have a lower resultion than luma channels (e.g. YUY2, YV12), i.e. the chroma pixel is not in exactly the same position in which the just inpainted luma pixel is. This argument is ignored for RGB color spaces. (default: false)
- bool ChromaTensor = false
- float PixelAspect = 1.0
- PixelAspect is the ratio between width and height of a pixel. E.g. for 16:9 DVD movies with a resolution of 720x576, one would use PixelAspect=(16.0/9.0)*(576.0/720.0)≈1.422. This value distorts the circle defined by Radius as well as the blurring kernels and kernel extents and modifies the inpainting order of pixels. (default: 1.0)
- float PixelAspect = 1.0
- int Steps = -1
- If Steps is non-negative, only the first Steps pixels (in the inpainting order which is determined by the distance of the pixels to the border of the inpainting region) are inpainted, all remaining pixels will be untouched. This is mainly usefull for visualizing how the function works and for debugging purposes. If Steps is less then -1, a copyright message will be displayed (along with an error message). (default: -1)
- int Steps = -1
- If the mask is provided as a separate clip (Mask argument), all color channels available in the source Clip will be inpainted, including the alpha channel, if the source is RGB32. In most cases this is not desired since it needs additional CPU time, and information from the alpha channel is also used for inpainting the color channels which might tamper the results. To avoid this, use ConvertToRGB24 first when working with RGB32 clips.
- The following table shows which combinations of color spaces are allowed and which color channels will be inpainted and returned.
- YUV color spaces (YUY2, YV12) are usually better considering speed and quality (the latter because RGB colors easily become distorted when different channels are treated differently). Yet, color spaces with subsampled chroma channels (which applies to both, YUY2 and YV12) may expose difficulties when the inpainting mask does not respect the subsampling: If some of the chroma pixels are partly masked (i.e. some of the corresponding luma pixels are masked for inpainting, some are not), the respective chroma pixel will be masked for inpainting, although it might be possible there are not enough non-masked or yet inpainted chroma pixels around when it's its turn to be inpainted. In such chase, the chroma pixel will simply left untouched which will generate a reasonable result in most cases. However, to avoid such problems make sure each partly masked chroma pixel has a neighboring non-masked chroma pixel (e.g. by forcing your mask on the chroma grid, or by inflating and then deflating your mask by at least three pixels).
- There is no explicit mechanism that deals with parts of the picture which are destroied but which should not or cannot be recovered (like black bars on the top and the bottom of the movie). If you cannot Crop away these parts, mask them for inpaint. Otherwise the function would consider them as valid data and use them for inpainting (black bars usually create a big spurious edge!).
- It is in general a hard (or even impossible) task to find parameters for good inpainting a fixed mask for a long movie. The best choice of parameters depends on several factors, like
- the shape of the mask
- the amount of “good” pixels available near the mask border
- the complexity of the underlying pictures
- ...
- your personal flavor
- In most cases it is a good idea to pick some frames of the movie, stack them together with Stack*al and watch the changes with ScriptClip as you change the parameters.
- Beware: This algorithm has originally been developed for inpainting pictures, not movies. No temporal information is used for inpainting. You might observe strong discontinuities in time (jumping of shapes created by the algorithm). See included documentation for some details on how the algorithm works.
Deblend Function
DeblendLogo is roughly the inverse function of Layer: If a clip (the logo) is blended onto another clip (by using a given alpha mask), DeblendLogo tries to undo this and to restore the original clip.
- DeblendLogo(clip Clip, clip Logo, clip "Alpha")
- clip =
- Input clip; Clip is the source clip with the logo which you want to deblend. It can have any color space. The resulting clip will have the same properties.
- clip =
- clip =
- Logo contains the logo you want to remove from the source clip. The logo may be moving, but a static logo (Logo and Mask having only one frame) results in faster processing since some values can be pre-computed. If Mask is not provided, Logo must be RGB32, Clip RGB24, and the alpha channel is used as mask. Otherwise, Logo must have the same color space as Clip. painting region causes a lot new calculations, inpainting static logos is much faster.
- clip =
- clip Alpha =
- Mask contains the transparency mask of the logo you want to remove. This clip may either have a YUV colorspace (the Y channel is used) or RGB32 (the alpha channel is used). Pixel values of 0 mean full transparency (the source clip is returned unchanged here), values of 255 means full opacity (the logo cannot be deblended in these regions, the resulting pixels will have undefined values). The interval [0,255] also applies to YUV masks, so do not forget to add Matrix="PC.601" to the converting function when converting RGB masks to YUV.
- clip Alpha =
- The following table shows which combinations of color spaces are allowed and which color channels will be deblended and returned.
Example
External Links
- GitHub - Source code repository.
Back to External Filters ←
