Copyright 1996-2000 by Donald R. Tveter, http://www.dontveter.com, commercial use is prohibited. Short quotations are permitted provided proper attribution is given.

For people who need a really basic treatment of backprop including a simple numerical example and deriving the formulas I've put together the following:

• a 12 page postscript file derived from the not yet finished user manual for my Professional Backprop Software. Note that there is free Unix, Mac and PC software that will let you read and print postscript files on these systems. If you're serious about neural networking you will need postscript on your system some day because all the online papers use it.
• An HTML Version. This is new as of January 20, 2000 and composed of mostly HTML with some gifs for the figures, a total of about 100k. Mathematical symbols will come out screwy under X unless you take some extra steps, see: http://hutchinson.belmont.ma.us/tth/Xfonts.html. The quick fix described at the beginning of the document gives small size math symbols but that's nice enough that I did not try any of the other fixes.

There is a short online article by G. Thimm, P. Moerland and E. Fiesler that relates the gain to the learning rate and the size of the initial weights.

Backprop training can be quite complicated and to do it properly see the online article by Lutz Prechelt.

Lutz Prechelt has established a page on benchmarking algorithms at the University of Karlsruhe, Germany.

More background can be found in two online articles by Warren Sarle.

For an overview of some of the more well known variations on backprop see the article by Jondarr Gibb.