Successive calls to `client.Containers(ctx)` then `c.Labels` may fail with `ErrNotFound` (possibly other methods as well)

[Apostasie]

Description

Unfortunately, it seems to be widespread across the codebase: we do make the assumption that whatever containers we get from client.Containers is still available down the road for other operations (like reading the Labels of the container).

This is of course not true. By the time we read the Labels, the container may no longer exist.

For a concrete example, see: https://github.com/containerd/nerdctl/blob/main/pkg/cmd/volume/rm.go#L34-L76

The problem is that this will make unrelated operations fail (that should not fail), when we hard error on that.
In the specific case I am pointing out above, this will make volume rm foo fail, even (especially) if the container in question has no relation to that volume whatsoever.

I am going to fix this particular instance, because it trips test parallelization I am working on, but there is obviously much more in there.

I do not have a good generic solution - introducing a global locking mechanism for nerdctl is likely a bad idea - it will significantly slow down concurrency performance, and will not even address some cases.

Also unfortunately, this is very tricky to reproduce and there is no simple way to trip these conditions unless we massively parallelize all our tests (which I want to do) and tackle them one by one...

The only solution I can think of is a careful review of the codebase to spot all these places where we are making assumptions about the availability of containers.

Steps to reproduce the issue

na

Describe the results you received and expected

na

What version of nerdctl are you using?

main

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

No response