RgTools

Abstract
Author	tp7, pinterf
Version	v1.3
Download	RgTools-1.3.7z
Category	Denoisers
License	MIT but binaries are GPLv2
Discussion	Doom9 Thread

Description

RgTools is a modern rewrite of RemoveGrain, Repair, BackwardClense, Clense, ForwardClense, VerticalCleaner and TemporalRepair all in a single plugin. RgTools is mostly backward compatible with the original plugins, incompatibilities mainly come from deprecated parameters in the original filters, read filter documentation for full details. Some routines might be slightly less efficient than original, some are faster. Output of some RemoveGrain modes are not exactly identical to the original due to some minor rounding differences; these differences will not affect your output in any significant way (it might make it better)[1]. Other functions should be identical.

Requirements

• [x86]: AviSynth+ or AviSynth 2.6
• [x64]: AviSynth+
• Supported color formats:
  • All planar formats (8/10/12/14/16/32-bit, Y(A), YUV(A), and RGB(A) with or without alpha)

• Latest supported Visual C++ Redistributable Package v14 (for Visual Studio 2017–2026)

Filters

Filter	Description	Color format
BackwardClense	Modified version of Clense that works on current and previous frames.	{{{4}}}
Clense	Temporal median of three frames.	{{{4}}}
ForwardClense	Modified version of Clense that works on current and next frames.	{{{4}}}
Repair	Repairs unwanted artifacts from (but not limited to) RemoveGrain, includes 28 modes.	{{{4}}}
RemoveGrain	Purely spatial denoising function, includes 28 (-1) modes.	{{{4}}}
VerticalCleaner	A very fast vertical median, it only has 2 modes.	{{{4}}}
TemporalRepair	TemporalRepair is a temporal filter, primarily useful for restoring static (non moving) details of spatial filters like RemoveGrain. Since 0.98.	{{{4}}}

Changelog

Version      Date            Changes

v1.3         20261030        Bug fixes
                             - RemoveGrain, Repair, VerticalCleaner: crash (memory corruption, also hangs) on planes of 1 row, and VerticalCleaner
                               mode 2 on planes of 2-3 rows (e.g. the chroma of a 2 or 4 row high 4:2:0 clip). A plane without an inner row is
                               copied now. Existed since 2013.
                             - RemoveGrain modes 13-16 (field interpolation), all code paths: with an odd plane height one row was not written
                               (random content): the last row (modes 13, 15) or the last odd row (modes 14, 16), e.g. the 4:2:0 chroma of
                               720x486. Existed since 2013.
                             - RemoveGrain, Repair (SSE2/SSE4.1/AVX2): wrong left border pixel, depending on memory content, when a plane is
                               exactly one vector + 1 pixel wide (8 bit: 17 or 33, 10-16 bit: 9 or 17, float: 5 or 9 pixels; e.g. the 4:2:0
                               chroma of a 34 or 66 pixel wide clip). Existed since 2013.
                             - RemoveGrain, Repair, TemporalRepair modes 1-3 (SSE2/SSE4.1, RemoveGrain also AVX2): with a plane width of a multiple
                               of the vector size, the last vector of a row read one pixel past the end of the row, in the last row past the end of the frame
                               (possible access violation). The output was not affected. Existed since 2013.
                             - TemporalRepair modes 1-3 (SSE2/SSE4.1): the right border column was overwritten when the row was a multiple of
                               16 bytes (e.g. the 4:2:2 chroma of a 64 pixel wide clip). Existed since v0.98.
                             - Repair: wrong output with a reference clip of a different frame pitch (e.g. a cropped reference) on the C code
                               path (CPUs without SSE2, opt=1, planes narrower than 17 pixels). Existed since 2013.
                             - Repair: the two clips must have the same bit depth and color family also when chroma is not processed (Y clips,
                               modeU=modeV=-1); was accepted with wrong output and reads past the reference rows. Existed since v0.93.
                             - ForwardClense, BackwardClense, 32 bit float YUV: negative chroma (U, V) was clipped to 0. Existed since the 32 bit
                               float support.
                             - Clense, ForwardClense, BackwardClense: with grey=true the alpha plane is copied (was random content). Existed
                               since v0.93.
                             - VerticalCleaner: a mode below -1 is an error (was accepted: undefined behaviour, possible crash). Existed since
                               2013.
                             - RemoveGrain mode 6, 32 bit float, SSE4.1 (opt=3, or auto on CPUs without AVX2): the costs were not clamped at
                               1.0 as on the other code paths; the result could differ. Existed since the float support.
                             - RemoveGrain mode 25, 10-14 bit, C code path: the result could differ from SIMD for some pixel patterns.
                               Existed since the 10-16 bit support.
                             
                             Changes
                             - RemoveGrain mode 11 on SSE2/SSE4.1/AVX2 (the x86 default): the exact (1 2 1)/16 kernel as documented and as the C
                               code, (4*c + 2*(a2+a4+a5+a7) + a1+a3+a6+a8 + 8) / 16. Before, it ran the mode 12 approximation (difference
                               at most 1). Mode 12 stays the faster approximation.
                             - RemoveGrain C code path (opt=1, CPUs without SSE2, very narrow planes): modes 12, 15, 16, 19 and 20 give the same
                               result as SIMD (they differed by up to 1, in float in the last bits). All code paths give the same result now.
                             - YUY2 and packed RGB give an error also with planar=true ("use ConvertToYV16"). Before, planar=true skipped the
                               format check and a YUY2 clip was processed as one plane.
                             - The "Invalid memory alignment. Unaligned crop?" errors are removed: AviSynth+ frames are always aligned
                               (classic AviSynth 2.6: use an aligned crop).
                             
                             New
                             - aarch64 NEON for all filters and bit depths (Linux, Windows on ARM64), the same result as C. Used when the host
                               reports NEON (AviSynth+: CPUF_ARM_NEON; SetMaxCPU("none") gives C). opt on aarch64: RemoveGrain, Repair: 1 = C,
                               any other value NEON; TemporalRepair: 0 = C, any other value NEON.
                             - The AviSynth+ V13 formats: YUV 4:1:1 at all bit depths, 4:4:0, 4:1:0 (with or without alpha) and YA (Y with alpha;
                               the alpha plane is copied).
                             - Documentation: all filters, modes and parameters in README.md; Expr equivalents of some RemoveGrain modes.
                             
                             Performance
                             - The row copies of the plane borders use memcpy instead of the AviSynth BitBlt.
                             - AVX2 (RemoveGrain): no manual vzeroupper (the compilers insert it; with Clang it caused saving and restoring all
                               vector registers in every plane function).
                             
                             Build environment
                             - Visual Studio 2026 solution (RgTools.slnx: Release with the v145 toolset, Rel_Clang with ClangCL); RgTools.sln
                               (Visual Studio 2022) only for the v141_xp build (Windows XP and 7). The old separate LLVM configurations are removed.
                             - CMake: rewritten (AviSynth+ style); detects x86 and aarch64 (NEON); a plain C build is possible
                               (ENABLE_INTEL_SIMD=OFF). GCC, Clang, MSVC: no warnings.
                             - Windows on ARM64 DLL (llvm-mingw cross build, build_arm64_mingw.bat); release pack script (build_release.bat).
                             - AviSynth headers updated to the V14 interface.
                             - Core test program (tests/core_test.cpp, CMake option RGTOOLS_BUILD_TESTS): SIMD vs C at all bit depths, also
                               under AddressSanitizer / UBSan and on aarch64 (qemu).
                             
                             Source
                             - The filter cores are separated from the AviSynth interface (*_core.cpp; prepares other front ends, e.g.
                               VapourSynth). The x86, aarch64 and C code are in separate files.
                             - One 10-16 bit code path instead of separate 10, 12, 14 and 16 bit templates (also usable for 9-15 bit).

v1.2         20210526          Fix: garbage in 10+ bit Backward/ForwardClense for non SSE4.1 processors (C code part)

v1.1         20210104          Source syntax for GCC
                               CMake build environment, builds on Linux, at least on my (pinterf) Ubuntu 19.10 WSL
                               (INTEL_INTRINSICS handling not implemented in the source, so it compiles on Intel at the moment)
                               Fix: artifacts in RemoveGrain mode 19 for 10-16 bits

v1.0         20200427        - RemoveGrain mode 26-28, (C, up to SSE4.1)
                               by the original author:
                               Mode 26 SmartRGC: 26 = medianblur. Based off mode 17, but preserves corners, but not thin lines.
                               Mode 27 SmartRGCL: 27 = medianblur. Same as mode 26 but preserves thin lines.
                               Mode 28 SmartRGCL2
                               Though mode 28 has different code from mode 27 so far no clips found producing different results :)
                             - Repair mode 26-28 (25 is nonexisting, only exists in RemoveGrain)
                             - add "opt" debug parameter to Repair. 0:auto, 1:c 2:SSE2 3:SSE4.1 4:AVX2 (latter is not used here atm)

v0.99        20200426        - RemoveGrain mode 25 ("nondestructivesharpen") finally reverse engineered 
                               and ported from the original RemoveGrain package. (no-no, Repair has still 24 modes)
                               Plus: 10-32 bits, Plus: sse4.1, avx2
                             - RemoveGrain: Revised, more correct 32 bit float in mode 6,7,8,9,15,16,23,24 
                               (proper clamping, chroma part specially treated)
                             - General: make C reference more similar to actual SIMD code
                             - fix: width<32 (or 16) with horizontal subsampling would use AVX2 (SSE2) on chroma
                             - (noticed: RemoveGrain: mode 11 and 12 SSE is simply avaraging
                               instead of really do 1/4 1/2 1/4 kernel blur like C (todo).
                               This is written in docs but even very old RemoveGrainSSE2 acts like this)
                             - Avisynth+ V8 header, frame property copy support
                             - opt debug parameter to RemoveGrain. 0:auto, 1:c 2:SSE2 3:SSE4.1 4:AVX2. Replaces bool "optAvx2"

v0.98        08/14/2019      - Include "TemporalRepair" filter from old RemoveGrainT package (rewritten C and SIMD intrinsics from pure inline asm)
                               Add Y8, YV16, YV24 besides YV12, drop YUY2 support.
                               Add 10-32 bit support for Y, YUV and planar RGB formats
                               Add int "opt" parameter (mainly for debug: 0=C 1=SSE2 2=SSE4.1) for testing specific code paths
                             - Codes for different processor targets (SSSE3 and SSE4.1) are now separated and are compiled using function attributes (clang, gcc).
                             - Other source changes for errorless gcc and clang build
                             - LLVM support, see howto in RgTools.txt
                               Note: use at least LLVM 9.0 build 21 June 2019 due to a clang compiler bug (_mm_avg_epu8 related, fixed on April 14 2019) 
                               older versions are up-to 1/3 slower than the Microsoft build. See latest snapshot builds at https://llvm.org/builds/
                             - GCC 8.3 support, CMakeFiles.txt, see howto in RgTools.txt
                             - RemoveGrain/Repair different code paths for SSE2/SSE4.1/AVX2 instead of SSE2/SSE3/AVX2.
                             - Add documentation (from old docs, new part: gcc/clang howto)
                             - todo: Overview 32 bit float parts for some RemoveGrain modes (6, 7, 8, 9, 15, 16, 23, 24)

v0.97        07/02/2018      - Remove some inherited clipping to 0..1 range for 32bit float.
                               In general we do not clamp in 32 bit float colorspaces, especially that U/V range is -0.5..+0.5 from Avisynth+ r2728

v0.96        06/09/2017      - RemoveGrain: AVX2. Available when Avisynth+ reports AVX2 usability
                               Can be disabled with new parameter: optAvx2=false
                             - Clense, ForwardClense, BackwardClense: ignore planar colorspace checking when planar=true. Like in RemoveGrain and Repair.
                             - Fix: Mode 11 and 13 for 32 bit float colorspaces (which worked like mode 10 and 12)

v0.95        03/14/2017      - Fix: RemoveGrain Mode 20: overflow at 14 and 16 bit depths in SSE4 (stripes)
                             - Repair: error on unaligned frames (unaligned crop) instead of access violation error

v0.94        01/26/2017      - Clense: new parameter (from v0.9): bool reduceflicker (default false)
                             - Clense: dummy compatibility parameters: bool planar, int cache
                             - autoregister filter MT modes as NICE_FILTER for Avisynth+
                               except for Clense: when reduceflicker is true, MULTI_INSTANCE MT mode is reported
                             - alignment check in Repair and RemoveGrain (anti-unaligned crop measures)  

v0.93        11/27/2016      - Same functionality as 0.92.1
                             - New bit depths 10, 12, 14, 16 bits and 32 bit float
                             - New formats: Planar RGB, Planar RGBA and YUVA support
                             - Use aligned loads if possible, internally
                             - 10+ bit formats require SSE4 for fast processing
                             - XP Support
                             - Built with VS2015, requires Visual Studio Redistributable Update 3 or newer
                             - Of course it works with classic Avisynth for 8 bit videos.

v0.92.1      02/10/2014      - Planar parameter added to VerticalCleaner registration function. It worked before but apparently not.

v0.92        02/10/2014      - Planar parameter added to VerticalCleaner for compatibility with QTGMC.
                               Same behavior as in RemoveGrain.

v0.91        11/30/2013      - Planar parameter is added to RemoveGrain and Repair. It allows to skip any colorspace 
                               checks whatsoever, making the filter always assume that planar input is passed.

v0.9         11/26/2013      - Initial release
                             - 24 modes of RemoveGrain, 24 modes of Repair, both modes of VerticalCleaner, and three 
                               variations of Clense implemented.

---

Back to External Filters ←