There was a lot of feedback on my neural network implementation, mostly regarding architectural suggestion so i sat down and rewrote my neural network implementation, its pretty elegant now. I seperated the training of the network from the actual network itself and so i have a basic feed forward network, a data set reader and a neural network trainer. I also renamed several data structures to make things more understandable, also i wasnt lazy and used proper header files and includes 😛
Below is an updated class diagram of the new version:
Here’s the updated implementation (with a VS2k8 solution):
The original tutorials can be found here:
Taking Initiative 
i cannot download the source code 😦
Hey Bobby,
I just wanted to say good work on your neural net code and articles/tutorials. I study Theoretical Neuroscience and Artificial Intelligence as a PhD in England and I found I have very many similar interests to you (I used to run a CS 1.5/1.6 league in the UK called CSGN).
I come to the field of neural networks from a biological and psychological background, so the coding is my main obstacle. I mainly work with competitive networks (and Spatial Orientated Maps) for my modelling, so familiarising myself with the code for a back-prop net here was very useful!
Keep up the good work 🙂
I’m really glad you found it useful! As long as at least one person found it useful, then I’ve succeeded at what I wanted to do.
haha, a bit off topic, man i miss playing competitive CS!
Same, although it is only as good as the people you are playing with. I used to have shed loads of fun with the other 4/5 members of my team when I played. Without them it was enjoyable, but no where near as much!
I have managed to get to grips with much of your code, mainly thanks to all your helpful comments. I have a couple of questions regarding the training set-up – is it possible to read in a separate .csv files for training/testing/validation rather than using the % set-up currently in place? Feel free to shoot me an email 🙂
Hey, your project looks fantastic. I need a neural net for my final year project, to interpret shapes read in through a wiimote to enable gesture control.
Is there any chance of getting a VS 2005 version?
Thank you,
Scott Higgins
Much clearer. One question do perform gradient descent on
Ep = 1/2 * S(Tp – Op)2
or
E = SEp
Where “S” means sum
Thank you
Jermaine
gradient descent is the name of the weight update technique, the errors calculated are used in the update of the weights. As such i dont understand your question, also what dooe Tp and Op signify.
hi,
i can’t figure out how to use the outputted weights with the actual nn. how can i instatiate the nn with my trained weights to get my nn output?
Hi Bobby, I am a computer science student in the Brazilian city Bh. I loved your blog, which has helped me to understand neural networks. Often people who know a lot some knowledge, do not have much good will to teach, mainly put on the internet. Fortunately exixtem wonderful people like you. Kisses
PS: You’re really cute !!!!!!!!
Vania
Plz. Delete my previous post. Is corrupted:
I reupload it to pastebin: http://pastebin.com/yZLSmpXh
Is patch for compile in Linux.
Great Post!!
Ricardo.
Wow! Thanks Ricardo, that’s pretty awesome!
selaamalijkoum beste broederlet op de inhoud van je column Je bent hiervoor te intelligent voor en zet je zelf niet op oneErstandig.evn column schrijven is goed.Moge Allah je beschermen.Allah leid wie Hij wil.Wa AlijkoumSelaam
i saw it in bits & pieces while reading, it was dreadfully boring & the females on the show did not seem to represent any kind of beauty, clad, semiclad or inner.
Just found a bug in neuralNetwork::loadWeights, delete cstr instead of delete[] cstr.
Cheers,
Daniel
thanks for the catch, I’ll update it first chance I get 🙂
which artificial neural network algorithm we need to use to classify water pollution.
take inputs PH,chloride,fluoride. This is my project. and give me the code also and send how to execute.plz
I am not going to give you code past what is on my blog. It is your project so do it yourself. I think the tutorials on this site and the web are more than sufficient for you to figure it out. I am not going to do your work for you.
Jeeze you sound so rude. Plus your solution is really easy. Go read something.
Not sure if you are replying to me or the initial comment but I do tend to get annoyed when people ask me for code or questions especially considering how much information is available within the post and on the internet. I have said this numerous times on this blog that I will gladly answer questions and help when I can BUT I will not do people’s work for them or spoon-feed them…
Definitely replying to his original comment. It indents when it’s a reply to a specific comment above 😉
hi,ive followed your guide to make my first basic neural network( XOR network)
but it seems to train incorrectly 😦
ive looked in many sites\tutorials to seek where is my problem in the code
ill write the formulas ive used in my code ,and ill be glad if you take a look and maybe explain me where im wrong in the code implementation:)
im using 2 layers:input,hidden,output(2,2,1).
b-bias bw-biasweight,
each neuron has input,output and error.
for inputlayer:
neuron[i].input=(b*bw+inputs[i]);
while inputs is the array of inputs ive use to train the net.
neuron.output=sigmod(neuron.input);
for all other layers(hidden&output)
sum=
forevery neuron in the previous layer – neuron.output*weightWithme;
weightwithme is the coneection between the current neuron and the neuron from the prevois layer.
then,as input layer:neuron.input=(sum+b*bw).
neuron.output=sigmoid(neuron.input).
calc errors:
for output layer
neuron.error=myoutput*(1-myoutput)*(targetoutput-myoutput).
for all other layers(hidden&input)
sum=
for every neuron in the next layer-neuron.error*weightwithme.
neuron.error=neuron.output*(1-neuron.output)*sum;
update weights:
weight between neuron i-> j :Wij+=learnrate*i.output*j.error;
thank you very very much for your help:)
All your questions are covered by my NN tutorial posts, please read them. There are two neural network code examples on this blog and I’m sure there are dozens more on the web.
Hi I’m student in Thailand,I ‘am a beginner of ANN. I understand this is implement for training dataset. But I need to classification new data to return the actual class of its . Can you suggest me about testing phase in ANN. thank you
hi sir,i am from india.can u please provide me a code to implement artificial neural network for rainfall forecasting using back propogation algo.i wud be really indebted to u .i hav to submit it on 21st september.thanx in advance.
mail id is arihantbaid@yahoo.com
Just want to say your article is as surprising. The clarity for your publish is simply excellent and that i can assume you are knowledgeable in this subject. Well along with your permission allow me to take hold of your RSS feed to keep up to date with approaching post. Thank you one million and please continue the rewarding work.
This piece of writing is genuinely a good one it helps new
the webb visitors, who are wishing for blogging.
I blog often and I truly appreciate your content. This article has really peaked my interest.
I am going to take a note of your blog and keep checking for new details about once per week.
I opted in for your RSS feed too.
Thanks for the code. I am taking the Coursera Machine learning course which uses Octave to teach NNs. Since I already know C++, that’s one less thing I have to learn in order to experiment.
thanks for share,i’m also trying to code in c++, but i don’t used to in c++,here is my code can anyone have some time to see my problem. here i’m using gradient descent rule and delta rule, I is given value,and O is target value,in this code there are many syntax error,i need to find error.
#include
#include
using namespace std;
int main()
{
double n = 0.01; // iteration value
{
double I = [-0.5 0 0.5 1 ];
double O = [0.75 2 3.25 4.5];
double w0 = 2.2;
double w1 = 4.0;
{
for (int p = 0; p < 50000; ++p)
dw0 = 0;
dw1 = 0;
}
{
for i = 1:4
{
y(i) = (w0)+((w1)*I(i, 1));
dw1 = (dw1)+(O(i, 1) – y(i))*I(i, 1);
dw0 = (dw0)+(O(i, 1) – y(i));
}
w0 = (w0)+(dw0)*n;
w1 = (w1)+(dw1)*n;
}
y1 = w1*I + w0
cout << "the output of y1" << y1 << endl;
system("pause");
return 0;
}
Hi Bobby, great article.
I am iOS developer and ported your code to Xcode (Mac).
It is on GitHub here: https://github.com/lukaszmargielewski/NeuralNetworkMac
I hope you don’t mind, just let me know if you have something against it (I kept your credits in the source files).
I made a few small changes thou:
1. Tidied up neural network class and moved all methods related to training to the trainer class.
2. Changed memory allocation methods a bit (used malloc, but preparing to memalign it for later GPU ports).
3. Removed zero-ing and initializeWeights code since former is actually not necessary (values are zero-ed out during feedForward anyway) and later is needed only when training (therefore moved to trainer class and when training starts).
Please let me know if I made sore obvious mistakes here, but code seems to execute example training fine.
Cheers!
Hi Bobby
thank you for sharing the code, it has been very helpful to me.
i only don’t understand clampOutput function, this function suppose to return wether the output is zero or one since the target value is either zero or one , so i think it should be:
if(x=0.5) return 1
else return -1
PLEASE, correct me if i’m wrong.
oops!
i meant
if(x=0.5) return 1
else return -1
Christa,Thank you for sharing your wisdom. My question is, as an aspiring professional, how do I go about getting subtjces/models to pose for me so I can build up my portfolio? I have a small studio in my basement with a few lights and softboxes. I would love to specialize in photographing curvy girls. I’m a curvy girl myself so I know how it feels to not think yourself beautiful. I would like to bring that beautiful feeling to other women.Thanks again!
something wrong happens when i try to post a code
Hi!
have a Q on the tutorial, you say:
“… separate thresholds for each neuron (remember each neuron’s activation function took a weighted sum minus a threshold as input) , we simply create 2 extra neurons with a constant value of -1. These neurons are then hooked up to the rest of the network and have their own weights (these are technically the threshold values).”
would the wij’s sum upp to 1 ? (normed), otherwise the T will risk of be kind of smallish compared to sum;
actually the question is: what are the constraints and ranges on the variables ? (input, hidden, output; 0-1 ?, summation/normalizations ?)
guess thats in the code ?
tnx for advice and nice reading code
Georg