AviSynth+

From Avisynth wiki
Revision as of 14:13, 11 October 2017 by Raffriff42 (Talk | contribs)

Jump to: navigation, search


New Features

AviSynth+'s Plugin Autoloader

  • 1st October 2013 | Source: here and subsequent post.

Okay, so how do multiple plugin directories interact with plugin autoloading?

As a recap, here is how it used to work in the official Avisynth:

  • Look for the string HKEY_CURRENT_USER/Software/Avisynth/PluginDir2_5 in the registry. If it exists, load plugins from the path specified there and stop.
  • If the above string didn't exist, look in HKEY_LOCAL_MACHINE/Software/AviSynth/PluginDir2_5. Try to load plugins from the path specified there.
  • Done.

First thing to note, is that classic AviSynth only ever searches for plugins in one single directory. It only knows two directories (both specified in the registry), and it only tries the second path if there is no entry for the first one.

AviSynth+'s autoloader has a list of autoload directories. It iterates over all those directories and tries to load all plugins from each. But (and a big but!) it will not load a plugin from a directory if another plugin with the same basename is already loaded. The basename of a plugin is simply its file name without the extension.

The expected use case is that you can now overlay a new plugin directory on top of another one. AviSynth+ then would load all plugins from the first folder, then load only those plugins from the second that weren't loaded from the first, then those from the third that weren't loaded from the first or second and so on. For example, let's say your usual plugin folder has a lot of plugins you normally use. But at one time you have a small number of updated plugins that you only want to use from a few scripts, but you do not yet want to replace your existing plugins globally. Then you'd just add a new plugin overlay folder, with only the new plugins in it, and that's it. All scripts that specify the new folder will autoload all plugins from your usual one, except for the new plugins, which would get loaded from the new folder. All your other scripts will still use your old plugins.

By default, AviSynth+'s autoload folder list has four paths in it, in this order:

  1. PluginDir+ in Software/Avisynth in HKEY_CURRENT_USER
  2. PluginDir+ in Software/Avisynth in HKEY_LOCAL_MACHINE
  3. PluginDir2_5 in Software/Avisynth in HKEY_CURRENT_USER
  4. PluginDir2_5 in Software/Avisynth in HKEY_LOCAL_MACHINE

This means, if there are ever plugins which will only work with AviSynth+ but not with classic AviSynth, you can put them into one of the "PluginDir+" folders. AviSynth+ will then use the classic plugins from the normal AviSynth, but if there are versions of some plugins written for AviSynth+, it will use them instead, and the classic avisynth.dll will still not be bothered with them. This is all without you having to lift a finger (except for adding the "PluginDir+" values to the registry once, until we have an installer). So to summarize all this, you have the ability to define a plugin autoload folder in the registry which will only be used by AviSynth+, but not by AviSynth, in addition to your classic plugins.

New Functions

However, another new functionality offered by AviSynth+, is that now you can also specify autoload paths in the scripts. There are two functions for this:

  • AddAutoloadDir(string path, bool toFront): this will add a new autoload folder. The string parameter is obligatory, it is the folder path where to load from. The second boolean parameter is optional, and if true (default), it will add the path to the front/beginning of the autoloader's list, which means it will be searched earlier than the rest. If it is false, the path will get added to the end of the list, so it will get searched last (unless you again add another one to the end).
  • ClearAutoloadDirs(): This will clear all the paths from the autoloader's list. Note that it is NOT a reset to the default state. ClearAutoloadDirs() will clear all folders, so if you don't add new ones after that, you have disabled the autoload functionality. This is, BTW, also a way to disable autoloading for a particular script in AviSynth+.

Here's an important note: You can only call these functions if no plugin has been autoloaded yet. Autoloading happens if the first unknown function is looked up. This means you can only call AddAutoloadDir or ClearAutoloadDirs if you have only made calls to built-in functions up to that point in the script. I suggest you start your scripts with these calls to avoid any problems.

There is only one thing left to discuss: Are there any special directories you can reference from your script? You bet there are:

  • SCRIPTDIR is the folder of the most current script. It is the path of the imported script if your script calls import()
  • MAINSCRIPTDIR is the folder of your main script, the one where execution started
  • PROGRAMDIR is the folder of the executable running the current script
  • USER_PLUS_PLUGINS is the string stored in PluginDir+ in Software/Avisynth in HKEY_CURRENT_USER
  • MACHINE_PLUS_PLUGINS is the string stored in PluginDir+ in Software/Avisynth in HKEY_LOCAL_MACHINE
  • USER_CLASSIC_PLUGINS is the string stored in PluginDir2_5 in Software/Avisynth in HKEY_CURRENT_USER
  • MACHINE_CLASSIC_PLUGINS is the string stored in PluginDir2_5 in Software/Avisynth in HKEY_LOCAL_MACHINE

... all these special constants are case-sensitive for now.

Examples

  • If you want plugins to be autoloaded from the script's "autoload" directory too, you'd write:

AddAutoloadDir("MAINSCRIPTDIR/autoload")

  • If you want plugins to be autoloaded from the script's "autoload" directory, only from there and nowhere else, you'd write:

ClearAutoloadDirs()
AddAutoloadDir("MAINSCRIPTDIR/autoload")

  • If you wanted to manually recreate the default state of the autoloading folder list, you'd write:

ClearAutoloadDirs()
AddAutoloadDir("USER_PLUS_PLUGINS", false)
AddAutoloadDir("MACHINE_PLUS_PLUGINS", false)
AddAutoloadDir("USER_CLASSIC_PLUGINS", false)
AddAutoloadDir("MACHINE_CLASSIC_PLUGINS", false)

Notes

  • Both AviSynth and AviSynth+ already query interface versions. They try to load the 2.6 interface from a plugin first, and if that is not supported, they try to load the 2.5 interface. AviSynth+ also tries to load the C interface if both of the previous ones fail. In the future, the C interface should probably be prioritized over 2.5.
  • In what contexts do MAINSCRIPTDIR and the other 'special' names get replaced with the corresponding folders? In all strings, or only when used in the argument to AddAutoloadDir?
    -- Only in AddAutoloadDir(), and even there, only if they are at the very beginning of the string. These get replaced to absolute folder paths, so if they are not at the beginning of the string, replacing them would only result in an invalid path (e.g. you'd end up with "c:" in the middle of your path).
  • Source
  • AviSynth+ autoloads plugins if any of the following happens:[1]
    • AutoloadPlugins() is called
    • LoadPlugin() is called
    • A yet unknown (non-internal) function is called
  • avs_function_exists does not find the external source filter in this case because none of the above happened. So MasterNobody's patch is the right thing to do.

GScript

GScript has been used as the starting point for the implementation in AviSynth+, and changes compared to the GScript external plugin are not visible to the user (e.g. only important for AviSynth+ core developers). Two notable differences are between GScript and AviSynth+:[2]

  • In AviSynth+ there is no need to use GScript(""" and """) to encompass your GScript-specific code parts. The language extensions became native to AviSynth+ and can be used transparently like classic AviSynth syntax.
  • The "return" statement has been slightly changed to not only exit the inner-most code block, but to terminate the whole function (or script), as anybody with even the slightest scripting experience would expect. This is one of the very few incompatible changes compared to classic AviSynth.

Links

Logging Facility

Starting with r2069, AviSynth+ received a logging facility. You can enable it using SetLogParams(string target, int "level") at the beginning of your script.

  • 'target' can be either "stderr", "stdout", or a path to a file.
  • 'level' is 1: LOG_ERROR / 2 : LOG_WARNING / 3 : LOG_INFO/ 4 : LOG_DEBUG, with increasing verbosity. By default, logging is disabled (0).

Log messages can be output by scripts using LogMsg(string msg, int "level").

If logging is enabled, AviSynth+ will output log messages by itself too. It will automatically log errors, and will issue warnings and notes to the user to inform him about potential problems, buggy plugins, suboptimal settings et cetera. There are a couple of these log messages and they come in various colors.


MT Notes

So, how to use MT in AviSynth+? Most of it has been posted earlier actually, but let me summarize it.

By default, your script will run in single-threaded mode, just like with SEt's build. Also, just like in SEt's build, you'll have to make sure that filters use the correct MT mode, or else they might wreak havoc. There are three basic MT modes (1,2,3) and an experimental workaround mode (4) since r2440, and modes 1-3 are the same modes as in (yeah you guessed correctly) SEt's build. Which means you can use the same modes that you have used with AviSynth-MT.

There are some things though that are different and/or new in AviSynth+. The first difference is *how* you set the MT mode. In AviSynth-MT, you had to use SetMTMode(X), which caused all filters following that line to use mode X (until the next call to SetMTMode()). This meant if you needed to use multiple MT modes, you had to insert all those calls in the middle of your script, littered over many places.

Setting MT modes

AviSynth+ does it differently. In AviSynth+, you specify the MT-mode for only specific filters, and those filters will then automatically use their own mode, even if there were other MT-modes inbetween. This means you can specify all the MT modes at the beginning without polluting your script. You can even make a SetMTMode.avsi if you wish and let it autoload for all of your scripts, or import() it from their top. This is much cleaner, and it allows you to maintain all your MT-modes centrally at a single place. To make this distinction clear from AviSynth+, SetMTMode() is called SetFilterMTMode() in AviSynth+.

Enabling MT

The other difference is how you actually enable multithreading. Calling SetFilterMTMode() is not enough, it sets the MT mode, but the MT mode only has an effect if MT is enabled at all. Note this means you can safely include/import/autoload your SetFilterMTMode() calls in even single-threaded scripts, and they will not be messed up. Uhm, onto the point: You enable MT by placing a single call to Prefetch(X) at the *end* of your script, where X is the number of threads to use.

Example 

# This line causes all filters that don't have an MT mode explicitly use mode 2 by default.
# Mode 2 is a relatively safe choice until you don't know most of your calls to be either mode 1 or 3.
# Compared with mode 1, mode 2 trades memory for MT-safety, but only a select few filters will work with mode 1.
SetFilterMTMode("DEFAULT_MT_MODE", 2)
or
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)

# FFVideoSource(), like most of all source filters, needs MT mode 3. 
# Note: starting  with AviSynth+ r2069, it will now automatically recognize source filters.
# If it sees a source filter which has no MT-mode specified at all, it will automatically use 
# mode 3 instead of the default MT mode.
SetFilterMTMode("FFVideoSource", 3)
or 
SetFilterMTMode("FFVideoSource", MT_SERIALIZED)

# Now comes your script as usual
FFVideoSource(...)
Trim(...)
QTGMC(...)
...

# Enable MT!
Prefetch(4)

Help filling MT modes

The following script contains MT modes for various plugins, save it as mtmodes.avsi and place in your auto-load folder. The script is a work-in-progess, there's still lots of plugins that need to be tested and validated. When the script is finalized, the only thing the user will have to write in his script is the Prefetch call, all SetFilterMtMode calls will be hidden in a single .avsi script.

Choosing the correct MT mode

Please do check if the actual output is correct. Fast but corrupted output is useless. Easy way of checking would be using something like ColorBars(1920, 1080, "YV12").AddGrainC(10000, 10000, seed=1) as a source filter. It doesn't always work right but will do for most stuff.[3]

Source: http://forum.doom9.org/showthread.php?p=1667439#post1667439

  • MT_NICE_FILTER: Some filters (like nnedi3) use some buffers to do their dirty work and with mode 1 you get multiple threads writing data from different frames to the same buffer. This causes corruption when later someone tries to read from this buffer and gets not what was expected. Most of the "more complicated" filters use some kind of temporary storage thus won't work well with this mode. Simple filters might.
  • MT_MULTI_INSTANCE: Mode 2 doesn't have this issue because multiple threads will get their own buffers and no data will be shared. Hence mode 2 is the "default" mode which should work with most filters, but it wastes memory like crazy (take SangNom2 for example - for 1080p YV12 frame, size of temporary buffers is about 10MB, so with 4 threads you get 40MBs on single filter invocation. Now add some usual supersampling to this and multiple invocations in most aa scripts and... you get the idea).
  • MT_SERIALIZED: If the filter requires sequential access or uses some global storage, then mode 3 is the only way to go. Source filter (filters without clip parameter) are autodetected, they do not need an explicit MT mode setting, they will automatically use MT_SERIALIZED.
  • MT_SPECIAL_MT: Experimental. Now use only for MP_Pipeline, the filter is like a source filter (no input clip parameter), internally multithreaded, and suffer heavy performance degradation from any of the three regular mt modes. Really, this is a workaround. Available from AviSynth+ version r2440.

Closing notes (don't skip!)

  • Remember that MT is only stable as long as you have specified a correct MT mode for all filters.
  • Instead of the numbers 1-2-3-4, you can also use symbolic names for MT modes: MT_NICE_FILTER (1), MT_MULTI_INSTANCE (2), MT_SERIALIZED (3), MT_SPECIAL_MT (4)
  • Mode 3 (MT_SERIALIZED) is evil. It is necessary for some filters, and it is usually no problem for source filters, but it can literally completely negate all advantages of MT, if such a filter is placed near the end of your script. Let us know if you meet a non-source mode 3 filter, we might be able to do something about it, but in general, avoid such calls if you want performance. (And of course, insert what you have found into here.)
  • The new caches will save you a lot of memory in single-threaded scripts, but due to the way they work, they will also use more memory than before with MT enabled. The memory usage will scale much closer with the number of threads you have. Just something to keep in mind.
  • MT-enabled AviSynth+ triggers a latent bug in AvsPmod. Until a new version of AvsPmod is officially released, use this build. A thousand thanks to vdcrim for the fix.
  • Using too many threads can easily hurt performance a lot, because there are other bottlenecks too in your PC than just the CPU. For example, if you have a quad-core machine with 8 logical cores, less than 8 threads will often work much better than 8 or more.

Informational links

Links contain bits and pieces of how MT works in AviSynth+, correct usage, and other things MT.


Documentation elsewhere in this Wiki

The new content can be found:

AVS+ filter documentation is well under way. So far, we have:

Along with AVS+ information added to:

Items to do:

  • LogMsg and related functions – noted on Avisynthplus/Developers, should these have user docs as well?
  • AddAutoloadDir and related functions – ditto.
  • ColorSpaceNameToPixelType – looks like a developers' function?
  • Internal_functions#Global_Options – what do these options do?
  • Convert – fact checkers are welcome.
  • a note on notation a proposed multi-bit-depth notation, eg: 16d-235d (comments?)
  • Histogram – bits argument needs further explanation.
  • Tweak – realcalc argument needs further explanation.
  • Need updated change logs on all filter pages.


Changelog

Retrieved from "http://avisynth.nl/index.php?title=AviSynth%2B&oldid=9998"
Personal tools