Action in partially observable mode
Tudyx opened this issue · comments
I've noticed that in partially observable mode, in the observation we don't have the information about network addresses, services, OS and the target to compromise, as we expect.
But still, through the list of action we have the list and the network address of every host in the information system.
Does the environment make the assumption that the pentester has received a document describing the network architecture before? Sometimes it can happen in the real world, but sometimes this kind of document doesn't even exist.
It's not a criticism it's just to clarify because I've seen nothing about this in the doc.
Hey, @Tudyx really good point and it is definitely a limitation of NASim.
The main reason I implemented it that way was so that the action space size could be fixed, making it simpler to use with Deep RL algorithms at the time (i.e. with neural nets with fixed size input and output layers).
As you mention, ideally the action space would change as the pentester discovers more network addresses, or perhaps more realistic would be that the pentester's action space is mult-dimensional and includes choosing an address and exploit/scan/etc separately.
There actually is some support for this built into NASim with the nasim.envs.action.ParameterisedActionSpace action space, but even using that action space some information about the size of the network is given to the pentester.
At this point I don't intend to update NASim to support the no-information action space, only so to keep it simple and stable and because there are a lot of even better and more realistic environments being developed now (e.g. CybORG as you mentioned.)
Do you think it would be helpful to add a clarification to the docs? That's definitely something that could be added. Perhaps like a "Limitations" or "Sim-to-Real Gap" section or something, highlighting what's missing from NASim that needs to be handled in pentesters moving into more realistic scenarios.
Thank you for your detailed answer, it makes a lot of sense why it was designed that way at the beginning.
Do you think it would be helpful to add a clarification to the docs? That's definitely something that could be added. Perhaps like a "Limitations" or "Sim-to-Real Gap" section or something, highlighting what's missing from NASim that needs to be handled in pentesters moving into more realistic scenarios.
Yeah, I think a "Sim-to-Real Gap" section or something can be a good idea because not everybody will check this repository issues.
Ok cool, I'll look at adding something to the docs with the next update.