iterator of a gridworld distribution can sometimes return an invalid state
zsunberg opened this issue · comments
function trans_mat(mdp::MDP, policy::Policy)
n = n_states(mdp)
T = spzeros(n, n)
for s in iterator(states(mdp))
i = state_index(mdp, s)
tdist = transition(mdp, s, action(policy, s))
for sp in iterator(tdist)
j = state_index(mdp, sp)
T[i,j] = pdf(tdist, sp)
end
end
return T
end
pi = rand(actions(mdp), n_states(mdp));
Tpi = trans_mat(mdp, VectorPolicy(mdp, pi))
will produce this error
this seems old