******************************************************************************
	  Image Processing and Neural Networks laboratory
	           University of Texas at Arlington 
	 Training of MLP using Levenberg-Marquardt Algorithm


	 Program written by Hema Chandrasekaran

	 For more information contact :
	 Prof. Michael T. Manry
	 Dept. of Electrical Engineering
	 University of Texas at Arlington
	 Arlington, TX 76019
	 e-mail : manry@uta.edu


NOTES:

	1.	This program trains a single hidden layer  multilayer perceptron(MLP) 
		via Levenberg-Marquardt algorithm. This program performs much better 
		than the commercially available softwares that train MLP using 
		Levenberg-Marquardt algorithm.

			Yet, this method is much slower than our HWO-OWO(hidden weight
		optimization-output weight optimization) method which solves sets of
		linear equations to train the MLP. We have a demo version of this software
		available for download at http://nexus.uta.edu/eeweb/ip/software.html.

	2.	This program sets the following upper limits on the number of inputs,
		hidden units, outputs.

		MAXIMUM NUMBER OF HIDDENUNITS = 50
		MAXIMUM NUMBER OF INPUTUNITS  = 100
		MAXIMUM NUMBER OF OUTPUTUNITS =  40

	3.  The mean square error(MSE) is calculated as follows:

		y(i) is the output of the i th output unit.
		t(i) is the desired or target output of the i th output unit.
		Nv is the number of training vectors.
		Nout is the total number of outputs.
		SE is the total squred error
		

		SE = Sum over p [ Sum over i ( {y(i) - t(i)}*{y(i) - t(i)}) ]

		where the index i ranges from 1 to Nout, the index p ranges from 1 to Nv.

		MSE = SE/Nv

	4.	SYSTEM REQUIREMENTS

		486 / Pentium I or II with 32 MB memory
		Windows 95/98 or Windows NT operating system


******************************************************************************

HELP FILE FOR LM_MLP.EXE

******************************************************************************

 1. Enter input  data filename : 
	
	Specify the data file name including the full path name. The data format 
	is explained in  http://nexus.uta.edu/eeweb/ip/datahelp.html.

	All mapping data files are in standard form. Standard form means that  the
	file is formatted, and that each pattern or vector has inputs on the left 
	and desired outputs on the right. You can view these data files using any
	text editor.

2.	Enter number of Iterations : 

	Enter the number of iterations (between 1 and 500).

3.	Enter	(1) to Initialize with random weights :
			(2) Read weights from file  :

	Choose 1 if you want to initialize the network with small random numbers.
	
	Choose 2 if you want to initialize the network with weights from a file.

	If you choose 2, then the network structure and weights are read and
	simulation begins.

	The weight file generated by lm_mlp.exe is an ascii file. 

	First line contains the "number of hidden layers" which will be 1, since this
	program trains only a single hidden layer MLP.

	The subsequent 3 lines contain number of inputs, hidden units, outputs.

	Then weights are written out in this order:

	Weights connecting input units to hidden units (one hidden unit per line).

	Weights connecting input units to output units (one output unit per line).

	
	Weights connecting hidden  units to output units (one output unit per line).

	Hidden units' thresholds.

	Output units' thresholds.




4.	If you choose " Initialize with random weights " in (3) then you are prompted
	to 

		Enter 1 to enter a new topology ,
		      2 to use old topology		:

	If you choose 1, ( to enter a new topology ), then you are asked to 

	4 (a)	Enter filename to save new topology :

	Give the file name to save your topology, including full path name.

	4 (b)	Enter number of Inputs ( 1 to 100) :

	4 (c)	Enter number of Outputs( 1 to 40) :
		    (These must be at the end of the pattern )

	4 (d) 	Enter number of Units in Hidden Layer 1 (1, 30) :



	5.  Save the Outputs to a File? (y/n) : 

		Choose y if you want to save the outputs of the MLP to a file.


----------------------------------------------------------------------------

	After training the 1 hidden layer MLP for specified number of iterations,

	menu appears with the following options:

		 Enter 1 to continue training
			   2 to test weights on data file
			   0 to quit

	Choose 1 to continue the training.

	Choose 2 if you want test the weights on a validation set or tesing set.

	Choose 0 to quit.


----------------------------------------------------------------------------------
 This program generates the folowing types of files: (all  are text files)

1. log files	: There are two log files generated. One is quite extensive and another
				  log file is brief. 

2. weight file	: This contains the network configuartion followed by the network
				  weights. 

3. out file		: This contains the network generated outputs.

4. top file		: This contains  network topology.

******************************************************************************