******************************************************************************
Image Processing and Neural Networks laboratory
University of Texas at Arlington
Training of MLP using Levenberg-Marquardt Algorithm
Program written by Hema Chandrasekaran
For more information contact :
Prof. Michael T. Manry
Dept. of Electrical Engineering
University of Texas at Arlington
Arlington, TX 76019
e-mail : manry@uta.edu
NOTES:
1. This program trains a single hidden layer multilayer perceptron(MLP)
via Levenberg-Marquardt algorithm. This program performs much better
than the commercially available softwares that train MLP using
Levenberg-Marquardt algorithm.
Yet, this method is much slower than our HWO-OWO(hidden weight
optimization-output weight optimization) method which solves sets of
linear equations to train the MLP. We have a demo version of this software
available for download at http://nexus.uta.edu/eeweb/ip/software.html.
2. This program sets the following upper limits on the number of inputs,
hidden units, outputs.
MAXIMUM NUMBER OF HIDDENUNITS = 50
MAXIMUM NUMBER OF INPUTUNITS = 100
MAXIMUM NUMBER OF OUTPUTUNITS = 40
3. The mean square error(MSE) is calculated as follows:
y(i) is the output of the i th output unit.
t(i) is the desired or target output of the i th output unit.
Nv is the number of training vectors.
Nout is the total number of outputs.
SE is the total squred error
SE = Sum over p [ Sum over i ( {y(i) - t(i)}*{y(i) - t(i)}) ]
where the index i ranges from 1 to Nout, the index p ranges from 1 to Nv.
MSE = SE/Nv
4. SYSTEM REQUIREMENTS
486 / Pentium I or II with 32 MB memory
Windows 95/98 or Windows NT operating system
******************************************************************************
HELP FILE FOR LM_MLP.EXE
******************************************************************************
1. Enter input data filename :
Specify the data file name including the full path name. The data format
is explained in http://nexus.uta.edu/eeweb/ip/datahelp.html.
All mapping data files are in standard form. Standard form means that the
file is formatted, and that each pattern or vector has inputs on the left
and desired outputs on the right. You can view these data files using any
text editor.
2. Enter number of Iterations :
Enter the number of iterations (between 1 and 500).
3. Enter (1) to Initialize with random weights :
(2) Read weights from file :
Choose 1 if you want to initialize the network with small random numbers.
Choose 2 if you want to initialize the network with weights from a file.
If you choose 2, then the network structure and weights are read and
simulation begins.
The weight file generated by lm_mlp.exe is an ascii file.
First line contains the "number of hidden layers" which will be 1, since this
program trains only a single hidden layer MLP.
The subsequent 3 lines contain number of inputs, hidden units, outputs.
Then weights are written out in this order:
Weights connecting input units to hidden units (one hidden unit per line).
Weights connecting input units to output units (one output unit per line).
Weights connecting hidden units to output units (one output unit per line).
Hidden units' thresholds.
Output units' thresholds.
4. If you choose " Initialize with random weights " in (3) then you are prompted
to
Enter 1 to enter a new topology ,
2 to use old topology :
If you choose 1, ( to enter a new topology ), then you are asked to
4 (a) Enter filename to save new topology :
Give the file name to save your topology, including full path name.
4 (b) Enter number of Inputs ( 1 to 100) :
4 (c) Enter number of Outputs( 1 to 40) :
(These must be at the end of the pattern )
4 (d) Enter number of Units in Hidden Layer 1 (1, 30) :
5. Save the Outputs to a File? (y/n) :
Choose y if you want to save the outputs of the MLP to a file.
----------------------------------------------------------------------------
After training the 1 hidden layer MLP for specified number of iterations,
menu appears with the following options:
Enter 1 to continue training
2 to test weights on data file
0 to quit
Choose 1 to continue the training.
Choose 2 if you want test the weights on a validation set or tesing set.
Choose 0 to quit.
----------------------------------------------------------------------------------
This program generates the folowing types of files: (all are text files)
1. log files : There are two log files generated. One is quite extensive and another
log file is brief.
2. weight file : This contains the network configuartion followed by the network
weights.
3. out file : This contains the network generated outputs.
4. top file : This contains network topology.
******************************************************************************