Docker Issues and Tips (aufs/overlay/btrfs..)

Picked up and categorized subjectively from https://github.com/docker/docker/issues. Comments and pull requests are welcome.

⬜ = Open (maybe not up-to-date, please check the link by yourself!)

🔳 = Mostly resolved (ditto, plus subjective)

✅ = Resolved

Storage Drivers

Issue	Abstract	Impact	Reproducibility	Cause	Solution	Notes
✅ #783	Cannot access to a directory due to a permission error	😐 Medium	😃 Easy	Expected AUFS behavior. `dirperm1` mount option fixes this issue.	Update the kernel (AUFS >= 2008xxxx?) and Docker daemon (>= 1.7)	Confirm: `docker info
✅ #18180	A process becomes a zombie and hangs up	😱 High	😱 Hard(multiprocessor) 😃 Easy(uniprocessor)	Compatibility between the kernel and AUFS	Update the kernel (AUFS >= 20160111)	Java apps and MongoDB are known to be affected
🔳 #20199	`fcntl(F_SETFL, O_APPEND)` is ignored and hence data can be corrupted	😱 High	😃 Easy	AUFS bug	Update the kernel (AUFS >= 20160301)	Dovecot is known to be affected
⬜ #20240	Weird permission even though `dirperm1` is enabled	😐 Medium	😱 Hard	Unanalyzed	None
⬜ AUFS ML 2016-03-08	Hang up related to `O_DIRECT`	😱 High	😃 Easy	Unanalyzed	None	Percona is known to be affected
⬜ #24309	Unable to remove files previously committed	😱 High	😃 Easy	Unanalyzed	This article seems related, but perhaps slightly different(Japanese)

Non-bug issues:

AUFS is not available in the mainline kernel．Only a few distros (Ubuntu, Boot2Docker, ..) support AUFS, but even for Ubuntu, Canonical says "AUFS will disappear".
No support for extended attributes ("xattrs"), and might not ever get support (#1070, #8460).
rename(2) is not fully supported #25409

Issue	Abstract	Impact	Reproducibility	Cause	Solution	Notes
✅ #10180	RPMDB corruption	😱 High	😐 Medium	Expected overlay behavior	Use yum-{utils,plugins-ovl}-1.1.31-33.el7 (included in RHEL 7.2) or later. Kernel patch is also available.	Linux 4.6 or later prints human-friendly dmesg
✅ #12080	Cannot use UNIX domain sockets	😐 Medium	😃 Easy	Overlay Bug	Use Linux 4.7-rc4 or later
🔳 #12327	pip fails	😱 High	😃 Easy	Overlay Bug	Use Linux 4.5 or later
✅ #19082	Weird behavior after removing the current directory	😃 Low	😃 Easy	Overlay Bug	Use Linux 4.5 or later
⬜ #19647, coreos/bugs#1095	Untar fails intermittently	😱 High	😱 Hard	Overlay Bug	None	Analysis is in progress in coreos/bugs#1095
⬜ #19758	Daemon hangs up after frequent `docker run`	😱 High	😱 Hard	Unanalyzed (Overlay bug related to the number of processors?)	None
⬜ #20640	Container cannot be started	😐 Medium	😱 Hard	Unanalyzed	None	Possibly identical to #16902
✅ #20950	/dev/console: operation not permitted	😱 High	😃 Easy	Kernel Bug	Use recent Linux kernels
🔳 #21555	`docker build` fails intermittently	😱 High	😱 Hard	DiffDriver bug	Resolved in Overlay2 by design. Patch available for overlay1
✅ #24913	permissions broken after chown	😐 Medium	😃 Easy	Overlay Bug	Use Linux 4.6 or later
🔳 #25244	opaque flag not reset after directory copy up	😐 Medium	😃 Easy	Overlay Bug	None but patch is available	npm is known to be affected
✅ machine#3327	chmod fails with EPERM	😃 Low	😃 Easy	Overlay Bug	Use Linux 4.5 or later

Non-bug issues:

😱 High inode usage (resolved in overlay2, which will be available in Docker 1.12)
Red Hat says OverlayFS is Tech Preview
rename(2) is not fully supported #25409

Issue	Abstract	Impact	Reproducibility	Cause	Solution	Notes
✅ #19073	`sendfile(2)` can be unkillable	😃 Low	😃 Easy	BtrFS bug	None	Not likely to happen in production, but needs consideration for public PaaS
⬜ #20080	cgroups kmem limit leads crash and data corruption	😱 High	😃 Easy?	Btrfs bug	Avoid kmem limit configuration?

Non-bug issues:

Slow #10161
No page sharing (e.g. same DLLs are loaded redundantly) http://comments.gmane.org/gmane.comp.sysutils.docker.devel/1384
Docker says BtrFS is Experimental. Red Hat says BtrFS is Tech Preview.

Issue	Abstract	Impact	Reproducibility	Cause	Solution	Notes
✅ #20153	Some operations fail due to `EBUSY`	😐 Medium	😐 Medium	Daemon bug	Update Docker daemon

Non-bug issues:

Issue	Abstract	Impact	Reproducibility	Cause	Solution	Notes
✅ #4036	Mount fails	😱 High	😃 Easy	udev sync disabled	Use a Docker daemon binary which supports udev sync	Confirm: `docker info
⬜ #20401	Infinite “mount/remount” loop, which makes the system unresponsive	😱 High	😱 High	Unanalyzed (perhaps related to XFS)	None

Non-bug issues:

It totally depends on your workload, but Docker, Inc. says AUFS and Devicemapper (direct-lvm) are "production-ready".

Although not listed in the above table, VFS driver is also attractive for its robustness.

Links:

You know, containers should be "immutable" and "disposable".

For persistent data and some special temporary data, you should better consider using an external volume (docker run -v).

Links:

Issue	Abstract	Impact	Reproducibility	Cause	Solution	Notes
✅ #18776	TCP checksums are ignored	😱 High	😱 Hard	Kernel bug	Use Linux 4.4 or later	blog

Issue	Abstract	Impact	Reproducibility	Cause	Solution	Notes
⬜ #19209	GELF driver saturates CPU	😱 High	😃 Easy
✅ #18057,#20600	`cat /dev/zero` leads to out of memory	😱 High	😃 Easy	logger's stdio handling issue	Use Docker 1.13 or later (or just disable the logging)	Related: #21181
⬜ #22497	container cannot be stopped if many logs are being printed	😱 High	😱 Hard	logger's stdio handling issue
✅ #22502	logging blocks the container	😱 High	😃 Easy	logger's stdio handling issue	Use Docker 1.11 or later	affected versions: 1.10.0

Issue	Abstract	Impact	Reproducibility	Cause	Solution
✅ #17720	Docker daemon 1.9 serious performance issue	😱 High	😱 Hard	?	Use Docker 1.10
✅ #20670	/dev/pts unmounted on the HOST when you are using `-v /dev:/dev` (After that you can no longer open SSH nor xterm)	😱 High	😃 Easy	daemon bug related to mount namespace	Use Docker 1.11.1. (Or Spawn the docker daemon from systemd. Or do not use `-v /dev:/dev`)
✅ #20836	Daemon hangs up after frequent `docker run`	😱 High	😱 Hard	Daemon bug	Use Docker 1.11.1

Non-bug issues:

:whale: Docker Issues and Tips (aufs/overlay/btrfs..)