davidrmiller / biosim4

Biological evolution simulator

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Input neuron values not negative

Wingman8 opened this issue · comments

I was wondering why the input neuron output is 0 to 1 whereas other neurons are -1 to 1 and the output neurons seem to expect -1 to 1 as well.

In particular, input like LAST_MOVE_DIR_X maps -1 to 0 to 1 as 0 to .5 to 1. This seem asymmetrical and seems like the math would artificially bias toward one direction or the other.

Thanks so much for making this video! Before I watched it I had thought neural networks were way beyond my comprehension.

Hi @Wingman8, welcome to the world of neural nets. Your question is a good one. In the version of the simulator captured in GitHub, sensor values are always in the range 0..1. It's a somewhat arbitrary linear range that represents the minimum to maximum quantity of something. By adjusting its weights, a neuron can map that range to a positive or negative nonlinear curve. E.g., a neuron could flip the sign of the sensor value with a weight that is equally likely to be positive or negative, scale it, add a positive or negative bias, then pass that to tanh. You could think of the sensor value 0..1 as just one more coefficient in the formula for a curve that is equally likely to be positive or negative. I don't think that it biases direction movements, but I haven't thought about the mathematics all that deeply.

Thanks for the explanation. That make sense. One of the most fascinating things about neural nets is the fact that once they get even a little complex they become near impossible to understand exactly why they work.

Since in your implementation middle neurons can receive input from themselves or cyclically from other middle neurons, would you call it a recurrent neural network? I don't remember you calling it that in the video and maybe I totally misunderstand what an RNN is.

Agreed about the NN complexity. And yes, if feedback loops form, then it's a kind of recurrent network.