Giters

ceph / ceph-container

Docker files and images to run Ceph in containers

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

docker-compose setup dose not run as expected mds and osd

Vildnex opened this issue · comments

commented

Bug Report

What happened:

I am trying to build a docker-compose file via this configuration which should start a proper CEPH instance.

version: '3'
services:
  ceph-mon:
    image: ceph/daemon:v6.0.4-stable-6.0-pacific-centos-8
    environment:
      - MON_IP=127.0.0.1
      - CEPH_PUBLIC_NETWORK=0.0.0.0/0
    command: mon
    volumes:
      - ./ceph:/etc/ceph
    networks:
      - cluster-net

  ceph-mgr:
    image: ceph/daemon:v6.0.4-stable-6.0-pacific-centos-8
    environment:
      - CEPH_PUBLIC_NETWORK=0.0.0.0/0
    command: mgr
    depends_on:
      - ceph-mon
    volumes:
      - ./ceph:/etc/ceph
    networks:
      - cluster-net

  ceph-osd:
    image: ceph/daemon:v6.0.4-stable-6.0-pacific-centos-8
    environment:
      - CEPH_PUBLIC_NETWORK=0.0.0.0/0
    command: osd
    volumes:
      - ceph-vol:/var/lib/ceph/osd
      - ./ceph:/etc/ceph
    depends_on:
      - ceph-mon
    networks:
      - cluster-net

  ceph-mds:
    image: ceph/daemon:v6.0.4-stable-6.0-pacific-centos-8
    environment:
      - CEPH_PUBLIC_NETWORK=0.0.0.0/0
      - DEBUG=verbose
    command: mds
    volumes:
      - ./ceph:/etc/ceph
    networks:
      - cluster-net

volumes:
  ceph-vol:
networks:
  cluster-net:

The fallowing configuration will run with success the fallowing containers:

  • ceph-mgr
  • ceph-mon

But the rest of 2 dose not work as it should and I have an error:

  • ceph-mds: 
    2024-02-07 14:12:18  /opt/ceph-container/bin/entrypoint.sh: static: does not generate config
2024-02-07T14:12:18.352428927Z +/opt/ceph-container/bin/common_functions.sh:14: log(): return 0
2024-02-07T14:12:18.352454667Z +/opt/ceph-container/bin/start_mds.sh:6: start_mds(): check_config
2024-02-07T14:12:18.352489807Z +/opt/ceph-container/bin/common_functions.sh:30: check_config(): [[ ! -e /etc/ceph/ceph.conf ]]
2024-02-07T14:12:18.352493227Z +/opt/ceph-container/bin/start_mds.sh:11: start_mds(): '[' '!' -e /var/lib/ceph/mds/ceph-mds-f52d56bb22ed/keyring ']'
2024-02-07T14:12:18.352520651Z +/opt/ceph-container/bin/start_mds.sh:12: start_mds(): '[' '!' -e /var/lib/ceph/mds/ceph-f52d56bb22ed/keyring ']'
2024-02-07T14:12:18.352538257Z +/opt/ceph-container/bin/start_mds.sh:13: start_mds(): '[' -e /etc/ceph/ceph.client.admin.keyring ']'
2024-02-07T14:12:18.352555873Z +/opt/ceph-container/bin/start_mds.sh:14: start_mds(): keyring_opt=(--name client.admin --keyring "$ADMIN_KEYRING")
2024-02-07T14:12:18.352594597Z +/opt/ceph-container/bin/start_mds.sh:22: start_mds(): timeout 10 ceph --cluster ceph --name client.admin --keyring /etc/ceph/ceph.client.admin.keyring health
2024-02-07T14:12:28.355980176Z +/opt/ceph-container/bin/start_mds.sh:22: start_mds(): exit 1
  • ceph-osd: 
    ceph-ab6ade99e3b0.1 | 2024-02-07T14:12:06.912+0000 7eff0898e080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-ab6ade99e3b0//keyring: (2) No such file or directory
2024-02-07T14:12:06.914003823Z ceph-ab6ade99e3b0.1 | 2024-02-07T14:12:06.912+0000 7eff0898e080 -1 AuthRegistry(0x55e5a866aa40) no keyring found at /var/lib/ceph/osd/ceph-ab6ade99e3b0//keyring, disabling cephx
2024-02-07T14:12:06.914013578Z ceph-f1628a8ee531.1 | 2024-02-07T14:12:06.912+0000 7fb0964ed080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-f1628a8ee531//keyring: (2) No such file or directory
2024-02-07T14:12:06.914016204Z ceph-f1628a8ee531.1 | 2024-02-07T14:12:06.912+0000 7fb0964ed080 -1 AuthRegistry(0x5638ad656140) no keyring found at /var/lib/ceph/osd/ceph-f1628a8ee531//keyring, disabling cephx
2024-02-07T14:12:06.914116063Z ceph-a68d3868d163.1 | 2024-02-07T14:12:06.912+0000 7fec3f285080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-a68d3868d163//keyring: (2) No such file or directory
2024-02-07T14:12:06.914135928Z ceph-a68d3868d163.1 | 2024-02-07T14:12:06.912+0000 7fec3f285080 -1 AuthRegistry(0x55e0a3010a40) no keyring found at /var/lib/ceph/osd/ceph-a68d3868d163//keyring, disabling cephx
2024-02-07T14:12:06.914189324Z ceph-ad84ed8e7949.1 | 2024-02-07T14:12:06.912+0000 7f96f7b96080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-ad84ed8e7949//keyring: (2) No such file or directory
2024-02-07T14:12:06.914204708Z ceph-ad84ed8e7949.1 | 2024-02-07T14:12:06.912+0000 7f96f7b96080 -1 AuthRegistry(0x556374136a40) no keyring found at /var/lib/ceph/osd/ceph-ad84ed8e7949//keyring, disabling cephx
2024-02-07T14:12:06.914225571Z ceph-ab6ade99e3b0.1 | 2024-02-07T14:12:06.912+0000 7eff0898e080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-ab6ade99e3b0//keyring: (2) No such file or directory
2024-02-07T14:12:06.914230900Z ceph-ab6ade99e3b0.1 | 2024-02-07T14:12:06.912+0000 7eff0898e080 -1 AuthRegistry(0x7ffc5b9c8120) no keyring found at /var/lib/ceph/osd/ceph-ab6ade99e3b0//keyring, disabling cephx
2024-02-07T14:12:06.914398793Z ceph-ab6ade99e3b0.1 | failed to fetch mon config (--no-mon-config to skip)
2024-02-07T14:12:06.914420140Z ceph-f1628a8ee531.1 | 2024-02-07T14:12:06.912+0000 7fb0964ed080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-f1628a8ee531//keyring: (2) No such file or directory
2024-02-07T14:12:06.914428124Z ceph-f1628a8ee531.1 | 2024-02-07T14:12:06.912+0000 7fb0964ed080 -1 AuthRegistry(0x7ffd70fb20e0) no keyring found at /var/lib/ceph/osd/ceph-f1628a8ee531//keyring, disabling cephx
2024-02-07T14:12:06.914528035Z ceph-f1628a8ee531.1 | failed to fetch mon config (--no-mon-config to skip)
2024-02-07T14:12:06.914647252Z ceph-ad84ed8e7949.1 | 2024-02-07T14:12:06.912+0000 7f96f7b96080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-ad84ed8e7949//keyring: (2) No such file or directory
2024-02-07T14:12:06.914659038Z ceph-ad84ed8e7949.1 | 2024-02-07T14:12:06.912+0000 7f96f7b96080 -1 AuthRegistry(0x7ffdf35be940) no keyring found at /var/lib/ceph/osd/ceph-ad84ed8e7949//keyring, disabling cephx
2024-02-07T14:12:06.914661290Z ceph-a68d3868d163.1 | 2024-02-07T14:12:06.912+0000 7fec3f285080 -1 auth: unable to find a keyring on /var/lib/ceph/osd/ceph-a68d3868d163//keyring: (2) No such file or directory
2024-02-07T14:12:06.914664358Z ceph-a68d3868d163.1 | 2024-02-07T14:12:06.912+0000 7fec3f285080 -1 AuthRegistry(0x7ffcb27552a0) no keyring found at /var/lib/ceph/osd/ceph-a68d3868d163//keyring, disabling cephx
2024-02-07T14:12:06.914749538Z ceph-ad84ed8e7949.1 | failed to fetch mon config (--no-mon-config to skip)
2024-02-07T14:12:06.915774356Z forego              | sending SIGTERM to ceph-f1628a8ee531.1
2024-02-07T14:12:06.915807845Z forego              | sending SIGTERM to ceph-ad84ed8e7949.1
2024-02-07T14:12:06.915820098Z forego              | sending SIGTERM to ceph-a68d3868d163.1
2024-02-07T14:12:06.919030693Z teardown: managing teardown after SIGCHLD
2024-02-07T14:12:06.919075181Z teardown: Waiting PID 68 to terminate 
2024-02-07T14:12:06.919082489Z teardown: Process 68 is terminated
2024-02-07T14:12:06.919119464Z teardown: Bye Bye, container will die with return code 0

comand to run: docker compose up -d

Environment:

  • OS (e.g. from /etc/os-release): Ubuntu 22.04.3 LTS
  • Kernel (e.g. uname -a): 5.14.0-1052-oem
  • Docker version (e.g. docker version): Docker version 25.0.1, build 29cf629
  • Ceph version (e.g. ceph -v): daemon:v6.0.4-stable-6.0-pacific-centos-8
commented

Anyone?

commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

commented

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.