Same questions.

[Irina]

Hi, your tutorials are awesome! I just start learning python & machine learning and not without your help.

Regarding "Evolutionary-Algorithm / tutorial-contents / Using Neural Nets / Evolution Strategy with Neural Nets.py":

• is it possible to run your 'maze_env.py' on it? And how to do it? I try to do it for a week, but without result((
• how to save trained net for different games and load it, without new training?

Thanks a lots.

[Irina]

Save and load trained net completed :)
https://github.com/Sat0ri/Evolution-Strategy

Help me, please, to run your 'maze_env.py' on it.
https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow/blob/master/contents/5_Deep_Q_Network/maze_env.py

[Morvan]

It is possible, but I don't know if this is compatible with tkinter which the maze_env is based on. The tkinter may cause some issues when doing multiprocessing.

[Irina]

Ok, i try to do it without tkinter.

How can I add more layers to your net?

If I just add it in Build_net

s0, p0 = linear(CONFIG['n_feature'], 30)
s1, p1 = linear(30, 30)
s2, p2 = linear(30, 20)
s3, p3 = linear(20, CONFIG['n_action'])
return [s0, s1, s2, s3], np.concatenate((p0, p1, p2, p3))

it returns :
AssertionError: 15 (<class 'numpy.int64'>) invalid

[Morvan]

I don't have any problem with it after adding an additional layer. Please update it to my latest code in github, maybe this is caused by the old code.

[Irina]

Yes, I use your latest code, but still error.
Can you show me how to append one more layer, please.

[Morvan]

def build_net():
    def linear(n_in, n_out):  # network linear layer
        w = np.random.randn(n_in * n_out).astype(np.float32) * .1
        b = np.random.randn(n_out).astype(np.float32) * .1
        return (n_in, n_out), np.concatenate((w, b))
    s0, p0 = linear(CONFIG['n_feature'], 30)
    s1, p1 = linear(30, 30)
    s2, p2 = linear(30, 20)
    s3, p3 = linear(20, CONFIG['n_action'])
    return [s0, s1, s2, s3], np.concatenate((p0, p1, p2, p3))

and please make sure the generating of random noise looks like this:

noise_seed = np.random.randint(0, 2 ** 32 - 1, size=N_KID, dtype=np.uint32).repeat(2)    # mirrored sampling

[Irina]

I realy have now idea how, looks like the same code.. but now it works :D
Thanks