View source for Avs-mlrt

{{FilterCat6|External_filters|Plugins|Plugins_x64|Other_filters|multipurpose_Filters|Deep_color_tools}}
{{Filter3
|1={{Author/Asd-g}}
|2=1.0.1
|3=[https://github.com/Asd-g/avs-mlrt/releases/ avs-mlrt.7z]
|4=Multipurpose
|5=[https://github.com/Asd-g/avs-mlrt/blob/main/LICENSE GPLv3]
|6=}}

== Description ==
This project provides AviSynth+ ML filter runtimes for variety of platforms.

This is [https://github.com/AmusementClub/vs-mlrt a partial port of the VapourSynth plugin vs-mlrt].


To simplify usage, a wrapper [https://github.com/Asd-g/avs-mlrt/blob/main/mlrt.avsi mlrt.avsi] is provided for all bundled models.

Custom models can be found [https://forum.doom9.org/showthread.php?t=184768 in this doom9] thread.

'''Filters'''

* [[avs-mlrt#mlrt_ncnn|mlrt_ncnn]] - [https://github.com/Tencent/ncnn ncnn] is a popular AI inference runtime. mlrt_ncnn provides a vulkan based runtime for some AI filters. It includes support for on-the-fly ONNX to ncnn native format conversion so as to provide a unified interface across all runtimes provided by this project.
* [[avs-mlrt#mlrt_ov|mlrt_ov]] - [https://docs.openvino.ai/latest/index.html OpenVINO] is an AI inference runtime developed by Intel, mainly targeting x86 CPUs and Intel GPUs. The mlrt_ov plugin provides optimized pure CPU & Intel GPU runtime for some popular AI filters. Intel GPU supports Gen 8+ on Broadwell+ and the Arc series GPUs.
<br>
<br>
== Requirements ==
* [https://en.wikipedia.org/wiki/Vulkan Vulkan] compatible device (mlrt_ncnn only)
* Intel GPU (mlrt_ov only, device="GPU" only)
* [x64]: '''AviSynth+ r3928''' or greater (AviSynth+ 3.7.3 (test 6, r3935 can be [https://forum.doom9.org/showthread.php?p=1983250#post1983250 downloaded from here])
* Microsoft VisualC++ Redistributable Package 2022 (can be downloaded from [https://github.com/abbodi1406/vcredist/releases here])
<br>

== [[Script variables|Syntax and Parameters]] ==

===mlrt_ncnn===

:{{Template:FuncDef|mlrt_ncnn (clip[] input, string "network_path", int "overlap_w", int "overlap_h", int "tilesize_w", int "tilesize_h", int "device_id", int "num_streams", bool "builtin", string "builtindir", bool "fp16", bool "path_is_serialization", bool "list_gpu")}}

<br>
::{{Par2| |clip| }}
:::Clips to process.
:::They must be in RGB/Gray 32-bit planar format, have same dimensions and same number of frames.
<br>
::{{Par2|network_path|string|}}
::: Path to the model.
<br>
::{{Par2|overlap_w|int|0}}
::{{Par2|overlap_w|int|0}}
:::Overlap width and overlap height of the tiles, respectively.
:::Must be less than or equal to <code>tilesize_w</code> / <code>tilesize_h</code> / 2.
:::Default: 0.
<br>
::{{Par2|tilesize_w|int|input_width}}
::{{Par2|tilesize_h|int|input_height}}
:::Tile width and height, respectively.
:::Use smaller value to reduce GPU memory usage.
:::Must be specified when <code>overlap_w</code> / <code>overlap_h</code> > 0.
:::Default: input_width, input_height.
<br>
::{{Par2|device_id|int|}}
:::GPU device to use.
:::By default the default device is selected.
<br>
::{{Par2|num_streams|int|1}}
:::GPU parallel execution.
:::Default: 1.
<br>
::{{Par2|builtin|bool|True}}
:::Whether the models are in the same location with the plugin.
:::Default: True.
<br>
::{{Par2|builtindir|string|"models"}}
:::Root folder when <code>builtin</code> is used.
:::Default: "models".
<br>
::{{Par2|fp16|bool|False}}
:::Enable FP16 mode.
:::Default: False.
<br>
::{{Par2|path_is_serialization|bool|False}}
:::Whether the model is serialized into one contiguous memory buffer.
:::Default: False.
<br>
::{{Par2|list_gpu|bool|False}}
:::Simply print a list of available GPU devices on the frame and does nothing else.
:::Default: False.
<br>

===mlrt_ov===
<br>
Download the required OpenVINO runtimes from [https://github.com/Asd-g/avs-mlrt/blob/main/2022.3.7z here].
After there are few options:
*Add the extracted files to PATH.
*Place the extracted files in the same location as mlrt_ov.dll.
*(Requires [[LoadDLL]]) Create AutoLoadDll.avsi with following:
 LoadDLL("path_to\tbb.dll")
 LoadDLL("path_to\openvino.dll")
 LoadPlugin("mlrt_ov.dll")

:{{Template:FuncDef|mlrt_ov (clip[] input, string "network_path", int "overlap_w", int "overlap_h", int "tilesize_w", int "tilesize_h", string "device", bool "builtin", string "builtindir", bool "fp16", string "config", bool "path_is_serialization", bool "list_devices", string[] "fp16_blacklist_ops", string "dot_path")}}

<br>
::{{Par2| |clip| }}
:::Clips to process.
:::They must be in RGB/Gray 32-bit planar format, have same dimensions and same number of frames.
<br>
::{{Par2|network_path|string|}}
::: Path to the model.
<br>
::{{Par2|overlap_w|int|0}}
::{{Par2|overlap_w|int|0}}
:::Overlap width and overlap height of the tiles, respectively.
:::Must be less than or equal to <code>tilesize_w</code> / <code>tilesize_h</code> / 2.
:::Default: 0.
<br>
::{{Par2|tilesize_w|int|input_width}}
::{{Par2|tilesize_h|int|input_height}}
:::Tile width and height, respectively.
:::Use smaller value to reduce GPU memory usage.
:::Must be specified when <code>overlap_w</code> / <code>overlap_h</code> > 0.
:::Default: input_width, input_height.
<br>
::{{Par2|device|string|"CPU"}}
:::Device to use - CPU or GPU.
:::For example, if there are more than one GPU device, to use the first device - "GPU.0", to use the second device - "GPU.1"
:::Default: "CPU".
<br>
::{{Par2|builtin|bool|True}}
:::Whether the models are in the same location with the plugin.
:::Default: True.
<br>
::{{Par2|builtindir|string|"models"}}
:::Root folder when <code>builtin</code> is used.
:::Default: "models".
<br>
::{{Par2|fp16|bool|False}}
:::Enable FP16 mode.
:::Default: False.
<br>
::{{Par2|config|string|}}
:::Configuration parameters.
:::CPU configuration parameters can be found [https://docs.openvino.ai/2021.4/openvino_docs_IE_DG_supported_plugins_CPU.html#supported-configuration-parameters here].
:::GPU configuration parameters can be found [https://docs.openvino.ai/2021.4/openvino_docs_IE_DG_supported_plugins_GPU.html#supported-configuration-parameters here].
:::<code>KEY_</code> prefix must be omitted.
:::Format is: <code>param=value</code>.
:::If more than one parameter is specified, the parameters must be separated by space.
:::For example, to disable all internal CPU threading: <code>config="CPU_THROUGHPUT_STREAMS=0 CPU_THREADS_NUM=1 CPU_BIND_THREAD=NO"</code>
<br>
::{{Par2|path_is_serialization|bool|False}}
:::Whether the model is serialized into one contiguous memory buffer.
:::Default: False.
<br>
::{{Par2|list_devices|bool|False}}
:::Simply print a list of available CPU/GPU devices on the frame and does nothing else.
:::Default: False.
<br>
::{{Par2|fp16_blacklist_ops|string[]|["ArrayFeatureExtractor", "Binarizer", "CastMap", "CategoryMapper", "DictVectorizer", "FeatureVectorizer", "Imputer", "LabelEncoder", "LinearClassifier", "LinearRegressor", "Normalizer", "OneHotEncoder", "SVMClassifier", "TreeEnsembleRegressor", "ZipMap", "NonMaxSuppression", "TopK", "RoiAlign", "Range", "CumSum", "Min", "Max"]}}
:::Configurable FP16 operations black list.
:::Default: ["ArrayFeatureExtractor", "Binarizer", "CastMap", "CategoryMapper", "DictVectorizer", "FeatureVectorizer", "Imputer", "LabelEncoder", "LinearClassifier", "LinearRegressor", "Normalizer", "OneHotEncoder", "SVMClassifier", "TreeEnsembleRegressor", "ZipMap", "NonMaxSuppression", "TopK", "RoiAlign", "Range", "CumSum", "Min", "Max"].
<br>
::{{Par2|dot_path|string|}}
:::Path for .dot file.
:::Allows to serialize to xDot format.
<br>
== Examples ==
[[TODO]]
<br>
<br>
== mlrt_ncnn Changelog ==
 Version      Date            Changes<br>
 v1.0.1       2023/03/20      - Changed AviSynth+ requirements.<br>
 v1.0.0       2023/01/27      - Initial release

== mlrt_ov Changelog ==
 Version      Date            Changes<br>
 v1.0.0       2023/03/20      - Initial release

<br>
== External Links ==
*[https://github.com/Asd-g/avs-mlrt GitHub] - Source code repository.
*[https://forum.doom9.org/showthread.php?t=184768 Doom9] - ONNX models compatible with avs-mlr.
<br>
<br>
-----------------------------------------------
'''Back to [[External_filters#Averaging.2FLayering.2FMasking|External Filters]] &larr;'''