Same model behaves differently when run on different machines
QuantumCrazy opened this issue · comments
Salaam, its been a while since I posted here. I ran into a strange issue I am unable to fix or understand yet. When I ran a gym-anytrading RL model on my local PC and created the same model on another machine (on a virtual machine), the model behaved very differently. All the parameters are the same, the input files are the same, the code is the same. Its just on a different machine. I understand there are many variables at play, like different library versions, file permissions, machine configurations etc....but I do not understand why it would behave so differently. Could you help brainstorm why this could be?
The model on the local PC machine trades regularly. The same model on the second machine does not trade at all. It always generates BUY signals. I am starting the model with a BUY, so I don't know if that makes a difference.
Any thoughts would be welcome. Thanks!
Hi @QuantumCrazy , can you try it with the exact same versions of libraries?
Once, I had this problem with PyTorch, where a small minor change in the framework would reduce the model's accuracy by about 5%.
Hi @AminHP , I will to match the library versions but my concern is that the model is behaving very differently (basically not trading) while the first one is trading frequently. Did I mention they each have saved their own model from training, so not using the same trained model.
In theory with these AI models, if you set up two identical instances that generate their own saved model after training - are they supposed to generate the same signals? I would think there would be some slight variation but mostly should be the same. I have not measured or studied this other than in my example. This reminds me of chaos theory - one small change in initial conditions / parameters could result in a large magnified effect.
I am going to point both AI scripts to the same saved model from training. That way in theory there is 100% match between signals generated for both systems. I still don't fully understand why 2 models with the same parameters, input data and codes will save different trained models and behave very differently in their trading behavior.
Something interesting I came across in the documentation of stablebaselines:
Completely reproducible results are not guaranteed across Tensorflow releases or different platforms. Furthermore, results need not be reproducible between CPU and GPU executions, even when using identical seeds.
Which could explain why I am getting very different results across different machines (GPU/CPU) given the same hyperparamaters on both systems. This made me realize that for RL algorithms, they could not yet be natively stable in the sense that their results are not yet very reproducible (unless we set a deterministic predict() - something that I am testing at the moment ). What might be the case is that for a new machine, you need to perform your same method of hyperparameter tuning / testing to get the (best) hyperparameters on that particular machine, then deploy that bot into trading. Like the RL algorithm I'm also learning as I go along, very fascinating results.
Reproducibility issues aside, one particular model calibrated for one of my machines,- the one I regard as a "top" performer, has been successfully buying and selling rallies within a macro downtrend, which I what i am aiming for. As a result it not only is up per nominal fiat currency value but also up in (real) cryptocurrency value. For this one, training steps are 20 and training is done once per week for an interval snap of hourly price data. There are so many variables to play with, there is always some luck involved and plenty and plenty of testing and training and patience. This is not an easy field and far too experimental at this stage, but I would say there is some promise in these algorithms.
Thanks @QuantumCrazy for sharing this information