VFS2 returns stale data with Kubernetes Downward API
impl opened this issue · comments
Description
In the gofer implementation for VFS2, an optimization in traversing relative symlinks that are chained together skips revalidation of paths in the middle of the chain. This can cause stale data to be returned in the sandbox.
In particular, this affects the Downward API in Kubernetes, which uses chained symlinks to atomically update state from the container's perspective. See https://github.com/kubernetes/kubernetes/blob/v1.23.5/pkg/volume/util/atomic_writer.go. Kubernetes containers will not see these updates for certain syscalls (until another, correctly working syscall causes revalidation).
This bug is isolated to the stepLocked
method of filesystem
when mayFollowSymlinks
is true
.
Fixed by #7296.
Steps to reproduce
First, mount a filesystem using p9 from the host into gVisor (e.g., /hostdir
to /mount
). Then construct the following file structure on the host:
# cd /hostdir
# mkdir real_1
# echo initial >real_1/target
# ln -s real_1 intermediate
# ln -s intermediate/target target
Then, in your gVisor sandbox, run stat(2)
in a loop against /mount/target
(log the struct fields, whatever works for you).
Go back to the host:
# mkdir real_2
# echo final >real_2/target
# ln -s real_2 intermediate_tmp
# mv intermediate_tmp intermediate
In the sandbox, observe that your stat(2)
calls are now returning stale data, e.g. st_ino
and st_size
have not updated to reflect the new intermediate symlink with data from real_2
.
runsc version
runsc version release-20220309.0-15-g34d2443eca11
spec: 1.0.2-dev
docker version (if using docker)
No response
uname
Linux k3d-relay-core-test-server-0 5.16.0-1-amd64 #1 SMP PREEMPT Debian 5.16.7-2 (2022-02-09) x86_64 GNU/Linux
kubectl (if using Kubernetes)
# kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:12:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.9+k3s1", GitCommit:"101917b0c493dd1effac1074feb1d5462b9a189b", GitTreeState:"clean", BuildDate:"2022-01-25T00:58:43Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3d-relay-core-test-server-0 Ready control-plane,master 14h v1.21.9+k3s1
repo state (if built from source)
release-20220309.0-14-g334090575
runsc debug logs (if available)
No response