- #1
GProgramer
- 10
- 0
I'm trying to implement this example http://www.cs.bham.ac.uk/~jxb/INC/l3.pdf (page 15)
I'm trying to do Iteration training, but it seems as if the results always converging to a steady error rate that is too large to be acceptable, the values centering around 0 while they should be close to -1 and +1.
I don't know if there's something wrong with the code, or I have the training concept misunderstood?
I'm trying to do Iteration training, but it seems as if the results always converging to a steady error rate that is too large to be acceptable, the values centering around 0 while they should be close to -1 and +1.
I don't know if there's something wrong with the code, or I have the training concept misunderstood?
Code:
close all;clc;
M=3;N=1;
X=[-1 1.0 0.1;-1 2.0 0.2; -1 0.1 0.3; -1 2.0 0.3; -1 0.2 0.4; -1 3.0 0.4; -1 0.1 0.5; -1 1.5 0.5; -1 0.5 0.6; -1 1.6 0.7];
X=X';
d=[-1;-1;1;-1;1;-1;1;-1;1;1];
Wp=rand([M,N]);
Wp=Wp'/sum(Wp(:)); % theta is 1 so sum of Wp and W needs to be <1
W=rand([M,N]);
W=W'/sum(W(:));
V1=zeros(1,10); %Pre allocating for speed
Y1=zeros(1,10);
e=zeros(1,10);
while(1)
i=randi(length(X),1);
%---------------Feed forward---------------%
V1(i)=W*X(:,i);
Y1(i)=tanh(V1(i)/2);
e(i)=d(i)-Y1(i);
%------------Backward propagation---------%
delta1=e(i)*0.5*(1+Y1(i))*(1-Y1(i));
Wn(1,1)=W(1,1) + 0.1*(W(1,1)-Wp(1,1)) + 0.1*delta1*Y1(i);
Wn(1,2)=W(1,2) + 0.1*(W(1,2)-Wp(1,2)) + 0.1*delta1*Y1(i);
Wn(1,3)=W(1,3) + 0.1*(W(1,3)-Wp(1,3)) + 0.1*delta1*Y1(i);
Wp=W;
W=Wn;
figure(1);
stem(Y1);
axis([1 10 -1 1]);
drawnow;
end