Is this the correct way to implement a hopfield network?
raimannma opened this issue · comments
carrot/src/architecture/architect.js
Lines 466 to 498 in bf9ae22
If this is correct than a hopfield network is just a simple perceptron but with STEP activation function.
I found this pics of hopfield networks:
They seem like there is more ongoing then just a dense layer with step activation, or am I wrong?
Also Wikipedia says: "A Hopfield network is a form of recurrent artificial neural network" (Wikipedia). But there are only feedforward components.
You're absolutely right about it being more complex than just a dense feed-forward network. To my understanding in a hopfield network all nodes connect to all other nodes so implementation-wise this depends on whether the methods.connection.ALL_TO_ALL
connection method is implemented correctly inside of Group.connect()
ALL_TO_ALL
means
input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
but there should be backward pointing connections right ?
Like this?
input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
output0 -->input0
output0 -->input1
output1 -->input0
output1 -->input1
but there should be backward pointing connections right ?
Like this?
input0 -> output0 input0 -> output1 input1 -> output0 input1 -> output1 output0 -->input0 output0 -->input1 output1 -->input0 output1 -->input1
Yes exactly, we could create a new connection method that does this. Also, ALL_TO_ALL
is a bit misleading in this context I wonder if we should rename it
Ok, consider renaming it to ALL_TO_ALL_FORWARD
?
And we need no additional connection type we can just do:
input.connect(output, methods.connection.ALL_TO_ALL);
output.connect(input, methods.connection.ALL_TO_ALL);
That's true and much more elegant 👍 renaming sounds good also
We should also have some basic structural tests for architecture to avoid things like this slipping through