Recieve ground truth data from Simulation

[Leon De Andrade]

As far as I understood the code the SimulationScene overwrites the ground truth data. It will still be available in the cache though:
https://github.com/nvr-avg/trajdata/blob/37ad56fe3dafc0aab83cbf35c660e6712e15a14d/src/trajdata/simulation/sim_df_cache.py#L36
which is used for calculating metrics and stats.

But I think there is another common use case: Plotting. Currently your simulation scene example doesn't plot ground truth data. And there is no straight forward way to do it I can see, because AgentBatch doesn't contain all the needed information.
So here is what I currently do:

 simulation_scene = SimulationScene(...) 
 simulation_scene_start = copy.deepcopy(simulation_scene)
 
 # Do stepping here 

agent_batch_start = sim_scene_start.get_obs()
current: Tensor = agent_batch_start.agent_hist[:, -1].cpu()
ground_truth: Tensor = agent_batch_start.agent_fut[:, :prediction_horizon, :].cpu()

agent_batch_final = simulation_scene.get_obs(get_map=False)
prediction: Tensor = agent_batch_final.agent_hist[:, simulation_scene.init_scene_ts:, :].cpu()

I would love if this could be simplified to have a data structure, which contains history, current, prediction and ground truth.

[Leon De Andrade]

@BorisIvanovic Friendly ping :)