liquidcarrot / carrot

Lines 466 to 498 in bf9ae22

    
             /** 
        
             * Creates a hopfield network of the given size 
        
             * 
        
             * @param {number} size Number of inputs and outputs (which is the same number) 
        
             * 
        
             * @example <caption>Output will always be binary due to `Activation.STEP` function.</caption> 
        
             * let { architect } = require("@liquid-carrot/carrot"); 
        
             * 
        
             * var network = architect.Hopfield(10); 
        
             * var training_set = [ 
        
             *   { input: [0, 1, 0, 1, 0, 1, 0, 1, 0, 1], output: [0, 1, 0, 1, 0, 1, 0, 1, 0, 1] }, 
        
             *   { input: [1, 1, 1, 1, 1, 0, 0, 0, 0, 0], output: [1, 1, 1, 1, 1, 0, 0, 0, 0, 0] } 
        
             * ]; 
        
             * 
        
             * network.train(training_set); 
        
             * 
        
             * network.activate([0,1,0,1,0,1,0,1,1,1]); // [0, 1, 0, 1, 0, 1, 0, 1, 0, 1] 
        
             * network.activate([1,1,1,1,1,0,0,1,0,0]); // [1, 1, 1, 1, 1, 0, 0, 0, 0, 0] 
        
             * 
        
             * @return {Network} 
        
             */ 
        
             Hopfield: function(size) { 
        
               const input = new Group(size, 'input'); 
        
               const output = new Group(size, 'output'); 
        
               input.connect(output, methods.connection.ALL_TO_ALL); 
        
               output.set({ 
        
                 squash: methods.activation.STEP, 
        
               }); 
        
               return new architect.Construct([input, output]); 
        
             },

If this is correct than a hopfield network is just a simple perceptron but with STEP activation function.

I found this pics of hopfield networks:

They seem like there is more ongoing then just a dense layer with step activation, or am I wrong?

Also Wikipedia says: "A Hopfield network is a form of recurrent artificial neural network" (Wikipedia). But there are only feedforward components.

You're absolutely right about it being more complex than just a dense feed-forward network. To my understanding in a hopfield network all nodes connect to all other nodes so implementation-wise this depends on whether the methods.connection.ALL_TO_ALL connection method is implemented correctly inside of Group.connect()

ALL_TO_ALL means

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1

but there should be backward pointing connections right ?

Like this?

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
output0 -->input0
output0 -->input1
output1 -->input0
output1 -->input1

but there should be backward pointing connections right ?

Like this?
input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
output0 -->input0
output0 -->input1
output1 -->input0
output1 -->input1

Yes exactly, we could create a new connection method that does this. Also, ALL_TO_ALL is a bit misleading in this context I wonder if we should rename it

Ok, consider renaming it to ALL_TO_ALL_FORWARD ?

And we need no additional connection type we can just do:

input.connect(output, methods.connection.ALL_TO_ALL);
output.connect(input, methods.connection.ALL_TO_ALL);

That's true and much more elegant 👍 renaming sounds good also

We should also have some basic structural tests for architecture to avoid things like this slipping through

	/**
	* Creates a hopfield network of the given size
	*
	* @param {number} size Number of inputs and outputs (which is the same number)
	*
	* @example <caption>Output will always be binary due to `Activation.STEP` function.</caption>
	* let { architect } = require("@liquid-carrot/carrot");
	*
	* var network = architect.Hopfield(10);
	* var training_set = [
	* { input: [0, 1, 0, 1, 0, 1, 0, 1, 0, 1], output: [0, 1, 0, 1, 0, 1, 0, 1, 0, 1] },
	* { input: [1, 1, 1, 1, 1, 0, 0, 0, 0, 0], output: [1, 1, 1, 1, 1, 0, 0, 0, 0, 0] }
	* ];
	*
	* network.train(training_set);
	*
	* network.activate([0,1,0,1,0,1,0,1,1,1]); // [0, 1, 0, 1, 0, 1, 0, 1, 0, 1]
	* network.activate([1,1,1,1,1,0,0,1,0,0]); // [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
	*
	* @return {Network}
	*/
	Hopfield: function(size) {
	const input = new Group(size, 'input');
	const output = new Group(size, 'output');

	input.connect(output, methods.connection.ALL_TO_ALL);

	output.set({
	squash: methods.activation.STEP,
	});

	return new architect.Construct([input, output]);
	},

Is this the correct way to implement a hopfield network?