Neural


Author V. C. Mohan
Date 1 Oct 2020

If one has an input video and one frame of it processed to get a desired image ( noise free or some other feature) then Neural network can be trained on that particular input frame and output target frame to get internal processing to match. For example if a 5x5 grid is used on that particular frame (with unknown weights) to get the target output then Neural learns by training how to get the nearest output. Works only if the target frame is result of linear processing of input

This Neural function assumes that the first frame of the input video was processed to get the tfn frame of the training clip. A window on the frame (that has all the range of amplitudes) is used for training. Starting with random weights for each input nodes through number of iterations the weights are adjusted with Resilient algorithm. This training may take several seconds to a few minutes.



All named parameters should be specified by name as the order in which they are shown here are different to the internal order.

After training these weights are used to process the entire video including the first frame. In case of RGB input, which color component be used for training can be specified. For YUV formats training is done on Y component only. While all 3 components of RGB formats are processed , in case of YUV only Y is processed

In some situations the error during each iteration may reach a local minimum and diverge. Best weights are stored for usage. In rare cases starting weights may not be suitable, in which case one can choose trying with several different starting weights and use those that give least error. Also instead of the inbuilt random numbers, a different seed can be used. However it should be borne in mind that the tme for learning will increase. Time is proportional to xpts * ypts * window_size * iter * bestof

Training window needs to be as small as possible to minimise time, but needs to encompass all possible range of values. A minimum 100x100 is accepted. If target frame was processed with an x * y grid, then xpts must be a minimum x and ypts be a minimum y

Packed RGB ( 8 and 16 bit), YUY2 and 8, 10 to 16 or 32 bit Planar formats are supported. This is thread safe. MT_NICE_FILTER

To save time in future for same situation the derived weights can be saved to a file. Instead of spending time on training the saved weights can be read from file and used. However please be sure that the saved weights are valid in the present situation. This file also includes number of target points, error at each iteration for each weight set. If reading from previously saved file, parameters listed after fname need not be given.

Details of parameters
Description Name Type Limits Default
Input clip to be processed clip none
whether weights derived to be saved for future or read from past file or not bother txt string "save", "read", and "none" "none"
file name with full path to be used fname string valid file for save and read options
training clip as target train clip none
number of grid points along width xpts integer 1 to 225. xpts * ypts must not exceed 225 or be less than 9 3
number of grid points along height ypts integer 1 to 225. xpts * ypts must not exceed 225 or be less than 9 xpts
top left x coordinate of training window tlx integer xpts / 2 +1. to reasonable value xpts
top y coordinate of training window tty integer ypts / 2 +1. to reasonable value ypts
top Right x coordinate of training window trx integer width - xpts / 2 +1. to reasonable value. Window width be atleast 100 width - xpts
bottom y coordinate of training window tby integer height - ypts / 2 +1. to reasonable value. Atleast 100 height - ypts
number of iterations with each set of starting weights iter integer +ve number. 2000
whether new seed to be used to generate starting weights? wset boolean true for yes, false for no false
number of different starting weight sets bestof integer 1 to 10 1
color to be used for training rgb data rgb integer 0 for red 1 for green and 2 for blue 1
frame number of train clip to be used as target for the input clip frame zero tfn integer must be within train clip 0

Usage examples:-
Neural(a,b, iter = 3000 , wset = false, bestof = 1,tty = 100, tlx = 100, trx = 300, tby = 300, xpts = 7)
# another though trivial example, takes long time to train:-
a=colorbars()
b = GBlur(a,rad = 7, sd = 2.0)
Neural(a,b, iter = 1000, bestof = 6, wset = false,tty = 100, tlx = 200, trx = 400, tby = 300, xpts = 7, txt = "save", fname = "C:\temp\neuralfile.txt")			
To my index page down loadmanyPlus plugin To Avisynth