Giters

moby / moby

The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems

Home Page:https://mobyproject.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

start docker service fails in container

daixiang0 opened this issue · comments

commented 
[root@15d00ddbaf53 ~]$ systemctl start docker 
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
[root@15d00ddbaf53 ~]$ systemctl status docker -l
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Fri 2017-10-27 10:17:01 UTC; 4s ago
     Docs: https://docs.docker.com
  Process: 422 ExecStart=/usr/bin/dockerd (code=exited, status=1/FAILURE)
 Main PID: 422 (code=exited, status=1/FAILURE)

Oct 27 10:17:01 15d00ddbaf53 systemd[1]: Failed to start Docker Application Container Engine.
Oct 27 10:17:01 15d00ddbaf53 systemd[1]: Unit docker.service entered failed state.
Oct 27 10:17:01 15d00ddbaf53 systemd[1]: docker.service failed.
Oct 27 10:17:01 15d00ddbaf53 systemd[1]: docker.service holdoff time over, scheduling restart.
Oct 27 10:17:01 15d00ddbaf53 systemd[1]: start request repeated too quickly for docker.service
Oct 27 10:17:01 15d00ddbaf53 systemd[1]: Failed to start Docker Application Container Engine.
Oct 27 10:17:01 15d00ddbaf53 systemd[1]: Unit docker.service entered failed state.
Oct 27 10:17:01 15d00ddbaf53 systemd[1]: docker.service failed.
[root@15d00ddbaf53 ~]$ uname -a
Linux 15d00ddbaf53 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@15d00ddbaf53 ~]$ cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)

My host env:

[root@skydata_003 makeimage]# docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 28
Server Version: 17.06.2-ce
Storage Driver: devicemapper
 Pool Name: docker-253:1-1009778822-pool
 Pool Blocksize: 65.54kB
 Base Device Size: 53.69GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 1.042GB
 Data Space Total: 1.074TB
 Data Space Available: 475.8GB
 Metadata Space Used: 5.444MB
 Metadata Space Total: 2.147GB
 Metadata Space Available: 2.142GB
 Thin Pool Minimum Free Space: 10.74GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.140-RHEL7 (2017-05-03)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 6e23458c129b551d5c9871e5174f6b1b7f6d1170
runc version: 810190ceaa507aa2727d7ae6f4790c76ec150bd2
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 3.10.0-514.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 31.26GiB
Name: skydata_003
ID: 3FMA:CIIC:S4WS:UKGO:PVTU:FENF:2TWV:HQZS:VMFE:6HLT:ICDP:52TW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 192.168.20.38
 127.0.0.0/8
Live Restore Enabled: false

[root@skydata_003 makeimage]# uname -a
Linux skydata_003 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@skydata_003 makeimage]# cat /etc/redhat-release 
CentOS Linux release 7.3.1611 (Core)

My container file:

FROM centos:centos7.2.1511
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]

RUN systemctl enable sshd

CMD ["/usr/sbin/init"]
commented

This is not a bug; If you want to run systemd inside a container, the container must be started with additional privileges; it also needs access to certain paths on your hosts, which you need to bind-mount into the container.

Generally I'd really discourage using systemd; not only make it your containers less secure (because you have to run with additional privileged), containers are not VM's, and running multiple processes per container is most likely not the "way to go" (YMMV). Instead of using an ssh server, you may want to consider using docker exec <container> bash to get an interactive shell in the container (for debugging purposes).

Some pointers;

First of all, don't define a volume for the cgroups; for systemd to work, it needs access to the cgroups on the host, which has to be specified at runtime, so you can remove this line from your Dockerfile:

VOLUME [ "/sys/fs/cgroup" ]

Using something like this should work to get your container started;

docker run -dt \
  --cap-add=SYS_ADMIN \
  -e "container=docker" \
  -v /sys/fs/cgroup:/sys/fs/cgroup \
  centos:centos7.2.1511 /usr/sbin/init

But there are some older discussions on this topic that provide information about running systemd in a container, for example #30723, #28614, #28377 and others.

Please keep in mind that the GitHub issue tracker is not intended as a general support forum,
but for reporting bugs and feature requests. For other type of questions, consider using one of;

I'm closing this issue because this is not a bug, but feel free to continue the conversation

commented

If do not consider secure problem, can i just use --privileged to make it ok?

commented

See my example above; please don't use --privileged if not needed; it's bad practice

commented

I use above cmd and still fail:

Installed:
  docker.x86_64 2:1.12.6-61.git85d7426.el7.centos                                                                                                                                             

Dependency Installed:
  audit-libs-python.x86_64 0:2.7.6-3.el7                          checkpolicy.x86_64 0:2.5-4.el7                                container-selinux.noarch 2:2.28-1.git85ce147.el7             
  container-storage-setup.noarch 0:0.7.0-1.git4ca59c5.el7         device-mapper-event.x86_64 7:1.02.140-8.el7                   device-mapper-event-libs.x86_64 7:1.02.140-8.el7             
  device-mapper-persistent-data.x86_64 0:0.7.0-0.1.rc6.el7        docker-client.x86_64 2:1.12.6-61.git85d7426.el7.centos        docker-common.x86_64 2:1.12.6-61.git85d7426.el7.centos       
  libaio.x86_64 0:0.3.109-13.el7                                  libcgroup.x86_64 0:0.41-13.el7                                libseccomp.x86_64 0:2.3.1-3.el7                              
  libselinux-python.x86_64 0:2.5-11.el7                           libselinux-utils.x86_64 0:2.5-11.el7                          libsemanage-python.x86_64 0:2.5-8.el7                        
  lvm2.x86_64 7:2.02.171-8.el7                                    lvm2-libs.x86_64 7:2.02.171-8.el7                             oci-register-machine.x86_64 1:0-3.13.gitcd1e331.el7          
  oci-systemd-hook.x86_64 1:0.1.14-1.git1ba44c6.el7               oci-umount.x86_64 2:2.0.0-1.git299e781.el7                    policycoreutils.x86_64 0:2.5-17.1.el7                        
  policycoreutils-python.x86_64 0:2.5-17.1.el7                    python-IPy.noarch 0:0.75-6.el7                                selinux-policy.noarch 0:3.13.1-166.el7_4.5                   
  selinux-policy-targeted.noarch 0:3.13.1-166.el7_4.5             setools-libs.x86_64 0:3.3.8-1.1.el7                           skopeo-containers.x86_64 1:0.1.24-1.dev.git28d4e08.el7       
  xfsprogs.x86_64 0:4.5.0-12.el7                                  yajl.x86_64 0:2.0.4-4.el7                                    

Updated:
  dracut.x86_64 0:033-502.el7                                                                 systemd.x86_64 0:219-42.el7_4.4                                                                

Dependency Updated:
  audit-libs.x86_64 0:2.7.6-3.el7   device-mapper.x86_64 7:1.02.140-8.el7   device-mapper-libs.x86_64 7:1.02.140-8.el7   libselinux.x86_64 0:2.5-11.el7   libsemanage.x86_64 0:2.5-8.el7  
  libsepol.x86_64 0:2.5-6.el7       systemd-libs.x86_64 0:219-42.el7_4.4   

Complete!
[root@f62c0ed2610f /]# systemctl start docker 
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
[root@f62c0ed2610f /]# systemctl status docker -l
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2017-10-30 07:14:54 UTC; 1min 0s ago
     Docs: http://docs.docker.com
  Process: 391 ExecStart=/usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-runc --exec-opt native.cgroupdriver=systemd --userland-proxy-path=/usr/libexec/docker/docker-proxy-current $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $ADD_REGISTRY $BLOCK_REGISTRY $INSECURE_REGISTRY $REGISTRIES (code=exited, status=1/FAILURE)
 Main PID: 391 (code=exited, status=1/FAILURE)

Oct 30 07:14:54 f62c0ed2610f systemd[1]: Starting Docker Application Container Engine...
Oct 30 07:14:54 f62c0ed2610f dockerd-current[391]: time="2017-10-30T07:14:54.931059893Z" level=info msg="libcontainerd: new containerd process, pid: 401"
Oct 30 07:14:54 f62c0ed2610f dockerd-current[391]: time="2017-10-30T07:14:54.931258504Z" level=fatal msg="Failed to connect to containerd. Please make sure containerd is installed in your PATH or you have specificed the correct address. Got error: write /proc/401/oom_score_adj: permission denied"
Oct 30 07:14:54 f62c0ed2610f systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Oct 30 07:14:54 f62c0ed2610f systemd[1]: Failed to start Docker Application Container Engine.
Oct 30 07:14:54 f62c0ed2610f systemd[1]: Unit docker.service entered failed state.
Oct 30 07:14:54 f62c0ed2610f systemd[1]: docker.service failed.
commented

It looks like you still have the Red Hat fork of Docker installed?

docker.x86_64 2:1.12.6-61.git85d7426.el7.centos

and

ExecStart=/usr/bin/dockerd-current ........

(dockerd-current is the name they use for the docker daemon binary)

Make sure to uninstall those packages, and install the official ones https://docs.docker.com/engine/installation/linux/docker-ce/centos/

commented

thanks very much!