Loading a saved model the right way
QuantumCrazy opened this issue · comments
I just want to confirm if I am loading a saved model the right way on a test set.
So firstly I ran my preferred model and saved it
My env variable before the model load is only on the test set
Then I do
model = A2C.load(load_path, env=env)
obs = env.reset()
while True:
obs = obs[np.newaxis, ...]
action, _states = model.predict(obs)
........
Sorry I don't know how to indent lines of code in Github comments (I thought tab would do it)
It seems ok. Did you feel anything wrong with that?
I think its OK too, just wanted to make sure. The loaded model saves all the hidden layer info like weights and biases right? And what would you advise on vectorizing(normalizing) the env. Should we vectorize only the training env, testing env or both? At the moment I am vectorizing both training and testing envs because I would think that would help with computation time although I have heard some people only do it to training and not the test env.
Normalization should be applied on both sets, however, its parameters should be taken only from the training set. For example, the mean and variance of data should be extracted from the training set, and then the test set should be normalized using these two parameters.
My understanding is that when I load the saved model, it applies the actions and states in the model based on the training set, on the new observations from the test set. How can you check what parameters are loaded and if they are vectorized or not to make sure? In your example, I would have thought there is more than just variance and mean involved.
Neither stable_baselines nor any-trading doesn't apply any kind of normalization methods. So, it is all on you. You should save/load the required normalization parameters when you are creating the train/test env.
I just use VecNormalize() on the training env before running the model and saving it. Then VecNormalize() on the test set before loading the model. I hope that's enough
I think it's ok.