PDA

View Full Version : A Neural Network Simulator



Viro
13-11-2002, 10:37 AM
Seeing as this is the place to post projects... I'll post a bit of info on mine.

For the past 3 months, I've been working on a Neural Network Simulator. I know, the name sounds like whoa, but it isn't really that amazing. All it really is, is a program that allows you to define a neural network, and train it.

At the moment, I've just got the delta rule implemented, and it learns via standard backpropagation. The input patterns are provided by means of XML (nice formatting). The Java version should be done in a couple of weeks. I'm not sure if there is any reason for me to do it in Delphi over Java :?:

Alimonster
13-11-2002, 10:49 AM
Well, I'll guess that you could make the decision once you've done the Java version ;). FWIW, Java GUIs are pretty bad :( - AWT is limited and Swing is very sluggish (even on an Athlon 1.1). The underlying number-crunching may or may not be good enough with a recent JVM. Java as a language is okay, it's simply hampered by really crappy GUI toolkits plus lack of proper enum's (the replacement kludge isn't acceptable IMO if you want clean code) and one or two other "things." Nice built-in library though (comprehensive).

So what tasks have you trained your ANN for? Are you going for something cool like recognising letters or pictures? You can have it check whether a given picture is programmer art - a big training set is available here from us here :lol:

BlueCat
13-11-2002, 10:55 AM
You don't need a neural network to recognize programmer art :mrevil:

Cool project, let us know how it goes. And yes save yourself some time and go straight to Delphi. I would say that though, wouldn't I? :P

Viro
13-11-2002, 02:06 PM
Well, I have actually thought long and hard about the language to implement it in, and here are my thoughts

Java
<ul>
<li>Fast for number crunching (very close to C++), but it takes far too much memory</li>
<li>Its cross platform (YESSS...), but you need to make sure everyone has the JVM (a major pain)</li>
<li>Good standard libraries. But this is where the UI falls short. Swing is dog slow, and I still don't like layout managers that much.</li>
<li>Looks good on your resume :)</li>
[/list]

C++
<ul>
<li>Powerful, but complex.</li>
<li>I'm very familiar with this language, so I'm still more productive here</li>
<li>cross platform hard to achieve</li>
<li>Looks good on your resume too</li>
[/list]

Delphi
<ul>
<li>Love dragging components on to a form. That's just plain easy, and practically painless</li>
<li>Lack of proper generics like C++ (Java has this problem too). It doesn't have a standard set of container classes , so I've got to write my own!!!</li>
<li>Lack of XML support. I could use MSXML ActiveX components, but that'll mean I need the Pro version. Don't have enough $$$ to $pend. OpenXML seems pretty ok, but I'll have to look into it more</li>
<li>Lack of programmer communities, but now that there's this site, this isn't so much of an issue :)</li>

Viro
13-11-2002, 02:08 PM
Hmm, HTML formatting didn't quite work there.

Here's a little bit more about my project. Below is a sample pattern file, coded up in XML. I hope the formatting stays.


<?xml version="1.0" encoding="UTF-8"?>

<!--begin to describe the neural network-->
<!--this neural network has 2 layers, only the Input and Output layer-->
<!-- There are 2 input units, and 1 output unit. There are no hidden units -->
<NeuralNet layers="2" inputs="2" outputs="1" hidden="0">
<Pattern id="1" width="2" height="1">
<Input>
0,0
</Input>
<Output>
0
</Output>
</Pattern>
<Pattern id="2" width="2" height="1">
<Input>
1,0
</Input>
<Output>
0
</Output>
</Pattern>
<Pattern id="3" width="2" height="1">
<Input>
0,1
</Input>
<Output>
0
</Output>
</Pattern>
<Pattern id="4" width="2" height="1">
<Input>
1,1
</Input>
<Output>
1
</Output>
</Pattern>
</NeuralNet>

Alimonster
13-11-2002, 02:41 PM
I have to say this, but a neural network without hidden layers is a bit... well, pointless, really ;). Make sure that it handles the "xor" problem too (if your example is the xor problem, should 1, 1 not give 0? Assuming that's what you're trying to do, of course)...

About the Delphi container classes: yes, this is a pain (I really wish that Borland would do generics some time soon!). In the meantime, try this library over at sourceforge, which might be interesting:

http://sourceforge.net/projects/decal/

Not perfect but it is a good start...

I might have to deal with good ol' overhyped XML soon :(. Anyone here have experience with XML and Delphi? If so, please pipe up.

lithander
13-11-2002, 03:06 PM
this might be a stupid question, but what are these generics/container classes you're talking about? What are they good for?

*feelsstupid*

-lith

Alimonster
13-11-2002, 03:27 PM
Container classes are used to contain stuff ;).

A container class is simply something that holds values - like an array, but with extra functionality. They usually store 'nodes', with each node containing the wanted value plus some more info, such as how to get to the next node. The nodes are dynamically allocated as necessary, so a container class isn't a fixed size.

For example, you get a linked list - this stores a node for each element, plus a pointer to get to the next node (so that you can have an unlimited amount of nodes - but you have to hop from one to the next using pointers, since it won't be a continuous area of memory like an array). You might also have a doubly-linked list, which would store two pointers - one for the previous node and one for the next. This would let you move both ways but with extra overhead (one more pointer, 4 bytes more per node).

You could also have a binary tree. These have nodes with the value plus two pointers - a left and a right one. New nodes are allocated on demand, again. You add to the left of the node if the value is less than the examined node, otherwise add it to the right. Recursion is handy here. You might have to rearrange things when you delete nodes, though, and it can be a pain keeping them balanced.

There are queues too - these map to the real-life concept. If you add something, it starts at the end of the queue. You remove from the start of the queue (imagine a line in a shop, for example). This is first in, first out.

In short, containers are fancy ways to store data. Each has a different set of properties (such as fast/slow insertion, fast/slow deletion, fast/slow search, high/low memory use, and so on). You pick a suitable container class and away you go!

There's also the concept of iterators: you get an iterator to a container class, then keep moving it on until the last element. This means that you don't care about the underlying structure of the object, and it also lets you specify ranges.

The problem is that the classes in the VCL are a bit lame, really. They do the job but with pointers, which is type-unsafe (you could coerce the compiler into, say, treating a pointer from one class as a pointer to another). If you used templates, you could outline the basic storage format ("store strings, store ints", whatever) and the compiler would generate classes for you. This would result in more type safety, but code bloat too (since each templated class w/ a type is totally separate).

Example, using hypothetical generic stuff in Delphi, a linked list node before-and-after might look like this:

type
PNode = ^TNode;
TNode = record
Data: Integer;
Next: PNode;
end;

//versus...

PNode = ^TNode;

template(MyType)
TNode = record
Data: MyType;
Next: PNode;
end;
end;
The first would be hard-coded to store only integers - the second could be passed in a type ("MyList: TNode<integer>" or "MyList: TNode<string>") and the compiler would generate the appropriate thing. This means you have a general-purpose prototype that can store any given value, with the compiler ensuring that it does the right thing.

Viro
13-11-2002, 03:36 PM
Yes, an ANN without a hidden layer is stupid. That's why mine does support hidden layers. I've made it support any number of hidden layers, limited by memory constraints of course (why? Don't know. It may be useful...). Its just that the example pattern file I posted doesn't have any hidden layers :) Using a hidden layer to solve a linearly separable problem is.... overkill.

I'm using OpenXML with Delphi. That library, kinda sucks. You can't parse XML files from within the IDE. You must execute the program separately, which is a pain. Get it from http://www.philo.de/xml.

An alternative is to use MSXML components, but it is a COM object, so there are problems with trying to get it to work with the Personal Edition of Delphi.

Alimonster
13-11-2002, 03:59 PM
Yes, an ANN without a hidden layer is stupid. That's why mine does support hidden layers. I've made it support any number of hidden layers, limited by memory constraints of course (why? Don't know. It may be useful...). Its just that the example pattern file I posted doesn't have any hidden layers :) Using a hidden layer to solve a linearly separable problem is.... overkill.
Ah, I see. Sorry!


I'm using OpenXML with Delphi. That library, kinda sucks. You can't parse XML files from within the IDE. You must execute the program separately, which is a pain. Get it from http://www.philo.de/xml.

An alternative is to use MSXML components, but it is a COM object, so there are problems with trying to get it to work with the Personal Edition of Delphi.
Hmm. I'm not looking forward to using XML, though I can see its uses (and the things it's not useful for, but which some people claim it is...). I have D5 Enterpise here, so I should be able to fiddle around with the MSXML parser. I'll have a nosey around for Delphi XML things and if I find anything, I'll let you know.

Viro
13-11-2002, 04:28 PM
I'm taking a look at Decal. On a cursory glance, it does look very nice.

Oh, I've got MSXML to work (Cheers, claps, whoops). For some reason, you need to import version 2 (2.6 and 3.0 don't work). Hehehehehe, I just might ditch Java after all, seeing as I haven't really coded much yet. Being an MS component, there's tonnes of information on it at MSDN.

lithander
13-11-2002, 04:31 PM
Thanks for the detailed explenation, Allimonster! ;)

Obviously i were allready using container classes without knowing what they are! :shock: ...at least i think so. For example I've a class for a Particle-Engine where you can add and delete Particles without caring about storing them. The class avoids that Particles are spread over the whole array returns (beside other functionallity) the highest occupied index, so you don't need to run through the whole array when updating. I guess this is an example for a container class?

sorry for getting of topic! I'll shut up now! :wink:

-lith

Viro
13-11-2002, 04:38 PM
The beauty of container classes, is that you get to reuse the implementation without changing a single line of code of the container. A good example would be C++'s STL.

For example, if you needed a vector of integers, floats & strings you'd just declare 3 vectors


vector<string> stringVector;
vector<int> intVector;
vector<float> floatVector;


This saves you time, because you know that if the vector works for 1, it should work for the rest. I like using prewritten containers, because I hate reinventing the wheel.

Useless Hacker
13-11-2002, 09:41 PM
Hmm, HTML formatting didn't quite work there.
Check you have HTML formatting enabled in your profile.

Viro
16-11-2002, 04:57 PM
My supervisor approved my design on Friday. Coding has started in earnest (I hope). I'm currently implementing it in Delphi as a RAD prototype. But if I like it a lot, I just might stick with Delphi.

I hope to have a demo out in the next few weeks.