Scientific Papers Suggestion Thread
Ghost---Shadow opened this issue · comments
Messing around and trying random stuff can only get so far. If we want a more reliable network we should try surveying existing literature.
I think this paper(https://arxiv.org/abs/1608.02192) is a good start. Here is a video about it. (https://www.youtube.com/watch?v=QkqNzrsaxYc)
I totally agree with @Ghost---Shadow , we need some studies about the literature and pick or invent a low computational cost method to achieve it. The paper I refer is http://deepdriving.cs.princeton.edu/paper.pdf
While I think it's pointless to not read up to date research on the best methods, I also think it's unwise to assume you cannot possibly make progress without someone else showing you how to progress. Most of what we're doing here is still totally in the trial and error stage, no one really has the answer. If you get the answer to this project, you've got yourself a billion dollar business.
I am not sure if it is possible, but if you can rename this to "scientific papers" or something that calls people to action moreso to share academic research related to this topic, it might help.
Done, you should link this to the readme.md
Added a link in the readme here. Thanks!
Segmentation: https://arxiv.org/abs/1611.08323
github: https://github.com/TobyPDE/FRRN
Since many people were running out of GPU memory. I would recommend the SqueezeNet. My finals are going on so if nobody implements it by the end of the month, then I will.
EDIT: Done
Learning how to play Atari games from pixels:
https://www.nature.com/nature/journal/v518/n7540/full/nature14236.html
Some interesting info about what separate buffers GTA V renders, depth and stencil being of particular interest I think http://www.adriancourreges.com/blog/2015/11/02/gta-v-graphics-study/. Direct "snooping" of the individual buffers could I think be achieved with something like https://www.opengl.org/sdk/tools/GLIntercept/, dunno if it could be fast enough though.
NVIDIA's self-driving car model:
https://arxiv.org/pdf/1604.07316v1.pdf
https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/#disqus_thread
Reinforcement Learning: An Introduction, Sutton and Barto, book available online:
http://incompleteideas.net/sutton/book/ebook/the-book.html
A good introduction to policy gradient methods:
http://www.scholarpedia.org/article/Policy_gradient_methods
A very good survey of actor-critic reinforcement learning
http://ieeexplore.ieee.org/document/6392457/
Deterministic policy gradient algorithms:
http://proceedings.mlr.press/v32/silver14.pdf
@skotone recomended policy gradient methods. The problem with them is that they are mostly "on-policy" which means that the training data should be sampled under fixed NN weights and then they may be updated (though you can use training size of several frames easily with, for examle, A3C algorithm). On-policy methods are very inefficient in terms of the amount of simulation which needs to happen in order to train network, and it should be your biggest concern because running gta5 is very costly by itself.
I would stick to off-policy methods, which may learn optimal policy on ANY game records (even on human ones). With them you could make a big boofer of frames which comes from your gta5 agent and then use only it to make everything happen. Your primary interest should be Q-learning and TD (kinda more advanced q-learning) algorithms. Also, q-learning may have "stability" difficulties sometimes - use "double q-learning" if you experience problems
It is very unfortunate that I have neither windows nor mac computers (nor any gta5). If I had, I would take part in your deeds
here's a simple double q-learning implementation: https://github.com/npetrenko/oligopolyrl/blob/master/oligopolyrl.ipynb
Here is a very nice survey on Deep Reinforcement Learning (from 2017 by Li). Section 5 has a Robotics example that may be useful.
Inverse Cooperative Reinforcement Learning. I think this form of RL would best suited for this particular case.
Using semantically segmented data from GTA V, here they trained classifier only using GTA V data to train classifier which outperforms classifier trained on KITTI dataset in task of classifying real-world cars in photos:
https://arxiv.org/abs/1610.01983
https://github.com/umautobots/driving-in-the-matrix
This paper form NVIDIA team could help, https://arxiv.org/pdf/1604.07316v1.pdf
You can try with DeepMind control suite which is also in python but works only with MuJoCo physics engine.