deeplearning4j / deeplearning4j-examples

Deeplearning4j Examples (DL4J, DL4J Spark, DataVec)

Home Page:http://deeplearning4j.konduit.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

DQN_ALE NullPointerException in Transition.setNextObservation

phong-phuong opened this issue · comments

Version Information: 1.0.0-beta-7

I'm getting a fatal NullPointerException in replay memory handler after 1000 steps in DQN_ALE example running pong.bin.

Exception in thread "main" java.lang.NullPointerException
	at org.deeplearning4j.rl4j.learning.sync.Transition.setNextObservation(Transition.java:58)
	at org.deeplearning4j.rl4j.experience.ReplayMemoryExperienceHandler.setNextObservationOnPending(ReplayMemoryExperienceHandler.java:82)
	at org.deeplearning4j.rl4j.experience.ReplayMemoryExperienceHandler.setFinalObservation(ReplayMemoryExperienceHandler.java:58)
	at org.deeplearning4j.rl4j.learning.sync.qlearning.discrete.QLearningDiscrete.finishEpoch(QLearningDiscrete.java:186)
	at org.deeplearning4j.rl4j.learning.sync.qlearning.QLearning.trainEpoch(QLearning.java:126)
	at org.deeplearning4j.rl4j.learning.sync.SyncLearning.train(SyncLearning.java:96)
public void setNextObservation(Observation nextObservation) {
        // To conserve memory, only the most recent frame of the next observation is kept (if history is used).
        // The full nextObservation will be re-build from observation when needed.
        long[] nextObservationShape = nextObservation.getData().shape().clone(); // <=== error on this line
        nextObservationShape[0] = 1;
        this.nextObservation = nextObservation.getData()
                .get(new INDArrayIndex[] {NDArrayIndex.point(0)})
                .reshape(nextObservationShape);
    }