Neural Networks : a beginner's notes

Its not so difficult. I'll stop short of saying its easy, but its not difficult. And thats good news. (calming smiley)

During the implementation of a certain module of Recog, using Neural Networks turned out to be part of the design plan. It was a big deal for me because it was my first foray into the field of AI, and I had no idea where to start. I had my design requirements at hand, but no clue whatsoever about how to implement it. In my quest for understanding how to implement Neural Networks, I did encounter some pitfalls and also some helping hands, and I thought it would be useful if I recorded some of my findings here:

- As I discovered the hard way, borrowing a large number of books relating to ANNs (Artificial Neural Networks) from the library and trying to digest the mathematical formulae does not help.

- Spending time to read authors' lame attempts at citing analogies from real life to help you understand does not help you much.

- Trying to solve your design issue by using ANNs without knowing why exactly are you using it does not help. (You'll curse yourself later for not understanding what values to set for certain parameters. This is experience speaking - however small it might be.)

- Try to build your understanding part-by-part, and not shoving down the whole sandwich down your throat. Try to find the simplest explanation of what ANNs are. (open the wikipedia page on ANNs and scroll down. ignore the initial junk) Found it? Now read it. Done? Now read it again. Because there is no point in proceeding without knowing WHAT are the advantages/disadvantages of using ANNs. Does it fit the picture of your requirements? If yes, good. If not, move on. Read about real-life applications and the different learning paradigms (supervised, unsupervised, reinforcement).
Remember, using ANNs is not going to be the cool part in your project/app. The accuracy of final results (and the speed in which it is generated) is what makes it cool.

- Next, run to the Search Gods. If you think that there's going to be no solution implemented yet for the design problem that you're dealing with, think again. Chances are that you're very wrong. People (tons of them) all over the world are solving tons^tons of problems using ANNs, and almost all of them write a research paper on their solution (whether it works or not).
So, spend a lot of time on Microsoft Live Search Academic and Google Scholar to see if any kind of solution to your problem already exists
(I like the interface of the former more because it saves you trips to unnecessary pages by providing abstracts side-by-side).
Go through these papers (atleast half a dozen of them) and find out where certain approaches failed and where some succeeded.

- Finally, and most importantly, what I've learnt is that you've got to implement it to understand it. There are some Neural Network libraries available online which allow you to code and bring your neural network into actual existence. I'd suggest AForge and Brainnet libraries, both are C# .NET based libraries. Try downloading both of them, and see if any of the sample programs accompanying them resemble the method of solution to your design problem. The code is commented nicely and you won't have much problem in grasping the technique after a while.

I'd like to repeat this: you've got to implement it to understand it.

Hmm.. just remembered something. As I write this post, somewhere in Berkeley, people are trying to seek the tech beyond Neural Networks.
Something they claim is closer to mimicking the human brain.
Is that even possible? I believe so. But not yet... Not yet.

Also, just as I am about to finish writing this post, Vasu and Avi are trying to feed inputs to the neural network that we created (tested with few inputs yesterday and it worked!!! Now training it for a huge sample train. Fingers-crossed. Praying.)

Have fun connecting them neurons. They good.