[RFC] New tests for shared_fs=none

[Wainer Moschetta]

Which feature do you think can be improved?

The confidential runtime classes (qemu-coco-dev, qemu-snp, qemu-tdx, qemu-sev) are all configured with shared_fs=virtio-9p which is actually represents a security flaw due the lack of a secure/trusted channels to share the host filesystem into the guest. @fidencio has worked to set changing the defaults to none in #9315 (which is blocked by guest pulling other issues).

On last CoCo CI working group meeting we talked about the fact that, apparently, there isn't any test case that directly verify shared_fs=none really won't export the host filesystem.

How can it be improved?

Create a test case, either unit or e2e, to certify the shared_fs=none behavior is as expected:

• it shouldn't share any host folders
• if needed it should copy the files over the guest. Resources like ConfigMap and Secrets should be copied, as well as downward API and projected volumes

Indirectly the k8s-configmap.bats, k8s-credentials-secrets.bats and k8s-projected-volume.bats will verify that shared_fs=none doesn't break features, however, afaik those tests don't check it falls back to the virtiofs-9p (which would be a bug).

Q: All in all, perhaps we should just introduce more checkers to the existing tests?

Additional Information

• I couldn't find a test for downstream API in our k8s suite

• Peer-pods is inherently shared_fs=none, no way a file will be shared without being copied. And it has tests for ConfigMap and Secrets

[Tobin Feldman-Fitzthum]

Probably out of scope of this test, but eventually we should try to show that the host cannot enable shared_fs without changing the measurement.

[Wainer Moschetta]

Probably out of scope of this test, but eventually we should try to show that the host cannot enable shared_fs without changing the measurement.

Good point. This can be an exploitable security breach indeed.

I was thinking also in simply remove the 9p driver from the guest kernel (if possible).

Anyway, I think this shared_fs=none discussion deserves an issue if not yet.

[Fabiano Fidêncio]

I was thinking also in simply remove the 9p driver from the guest kernel (if possible).

So, consider that in a very short term the kernel we ship for confidential and non-confidential will be the same.
If we disable 9p, we'll be disabling 9p for both. The same goes for disabling 9p support on QEMU.

What I'd rather do is enforce in the runtime code that if confidential_guest is set, some of the options will not be taken in consideration, such as virtio-fs and virtio-9p.

[Fabiano Fidêncio]

I sincerely think that just adding a new test, as we did with the pull in guest, is the wrong approach.
What we should do, instead, is just switch the whole test suit to run with shared_fs=none.

[Tobin Feldman-Fitzthum]

I sincerely think that just adding a new test, as we did with the pull in guest, is the wrong approach.
What we should do, instead, is just switch the whole test suit to run with shared_fs=none.

I think the idea is to use existing tests (either the confidential ones or all of them) to show that pulling with shared_fs=none works, but to also add more checks or maybe a new test to make sure that shared_fs=none actually results in no sharing.

[Fabiano Fidêncio]

On last CoCo CI working group meeting we talked about the fact that, apparently, there isn't any test case that directly verify shared_fs=none really won't export the host filesystem.

Do we need this? If we set shared_fs=none and do not set the nydus-snapshotter we'll simply fail to mount the rootfs.
I'm not against a negative test on this, just want to clarify this is really the priority we have.

[Fabiano Fidêncio]

• if needed it should copy the files over the guest. Resources like ConfigMap and Secrets should be copied, as well as downward API and projected volumes

This must be improved, indeed.

[Fabiano Fidêncio]

Indirectly the k8s-configmap.bats, k8s-credentials-secrets.bats and k8s-projected-volume.bats will verify that shared_fs=none doesn't break features, however, afaik those tests don't check it falls back to the virtiofs-9p (which would be a bug).

Those tests will simply fail using shared_fs=none, and the full list can be seen here:
#9667

[Fabiano Fidêncio]

I've also noticed failures related to initContainers (see: #9668, #9666, and #9664).

[Fabiano Fidêncio]

And #9665, which didn't fail, but broke silently, so I decided to disable it from the TDX runs as well.

[Fabiano Fidêncio]

#9315 is where I'm doing my experimentations, and I'd like to have the TDX tests merged (as in, not blocked), and then folks can work on their respective TEEs / common non-TEE at their own will.