Unable to setup CLP in Debian OS

[mantrimahitha]

Hi Team,
I am trying to follow the steps mentioned in package-template, when I am trying to start clp, it is failing with this error. The system has enough memory. Can you please edit the readme.md and post all the required softwares.

Traceback (most recent call last):
  File "/root/clp/tools/packager/out/clp-package-ubuntu-focal-x86_64-v0.0.1/etc/../sbin/start-clp", line 462, in main
    start_queue(instance_id, clp_config)
  File "/root/clp/tools/packager/out/clp-package-ubuntu-focal-x86_64-v0.0.1/etc/../sbin/start-clp", line 271, in start_queue
    subprocess.run(cmd, stdout=subprocess.DEVNULL, check=True)
  File "/opt/conda/default/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['docker', 'exec', '-it', 'clp-queue-209e', 'rabbitmqctl', 'wait', '/tmp/rabbitmq.pid']' returned non-zero exit status 137.

[kirkrodrigues]

Hi mantrimahitha,

Sorry, I haven't seen that issue before. From my own tests, the container in your error message uses a maximum of about 200MB during startup.

Could you give us the following details about your system to help us debug?

• Amount of RAM
• Docker version
• OS version
• clp commit hash that you're building from

Thanks!

[bossjie]

@kirkrodrigues I met similar issue when running cmd "python3 start-clp"
here are my server info.

RAM: 64GB
Docker version 24.0.7
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-1160.49.1.el7.x86_64
Architecture: x86-64
clp last commit 2b884bd ( I believe so as the package tool pulls newest code from github)

error log:
Traceback (most recent call last):
File "start-clp", line 462, in main
start_queue(instance_id, clp_config)
File "start-clp", line 271, in start_queue
subprocess.run(cmd, stdout=subprocess.DEVNULL, check=True)
File "/usr/lib64/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'exec', '-it', 'clp-queue-938f', 'rabbitmqctl', 'wait', '/tmp/rabbitmq.pid']' returned non-zero exit status 137.

And I consider the compress is seperate with clp process(Hope I'm wrong). so I run "python3 compress xxl-service.log"
and got below error. I'm heading to install using single node. the experience is like a blackbox touching by me. If we have better doc to explain the components( I see it trys to run docker images to support it) and install flow, it would be great. As the log info is limited.

docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /vdb/clp-package-ubuntu-focal-x86_64-v0.0.2/var/data/archives.
See 'docker run --help'.
Traceback (most recent call last):
File "compress", line 168, in
sys.exit(main(sys.argv))
File "compress", line 159, in main
subprocess.run(cmd, check=True)
File "/usr/lib64/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'run', '-i', '--rm', '--network', 'host', '-w', '/opt/clp', '-u', '0:0', '--name', 'clp-compressor-502b', '--mount', 'type=bind,src=/vdb/clp-package-ubuntu-focal-x86_64-v0.0.2,dst=/opt/clp', '--mount', 'type=bind,src=/,dst=/mnt/logs,readonly', '--mount', 'type=bind,src=/vdb/clp-package-ubuntu-focal-x86_64-v0.0.2/var/data/somehostname.company.com,dst=/opt/clp/var/data', '--mount', 'type=bind,src=/vdb/clp-package-ubuntu-focal-x86_64-v0.0.2/var/log/somehostname.company.com,dst=/opt/clp/var/log', '--mount', 'type=bind,src=/vdb/clp-package-ubuntu-focal-x86_64-v0.0.2/var/data/archives,dst=/mnt/archive-output', 'ghcr.io/y-scope/clp/clp-execution-x86-ubuntu-focal:main', '/opt/clp/sbin/native/compress', '--config', '/opt/clp/var/log/.clp-compressor-502b-config.yml', '--remove-path-prefix', '/mnt/logs', '/mnt/logs/vdb/clp-package-ubuntu-focal-x86_64-v0.0.2/sbin/xxl-service.log']' returned non-zero exit status 125.

[bossjie]

by downloading from https://github.com/y-scope/clp/releases/download/v0.0.2/clp-package-ubuntu-focal-x86_64-v0.0.2.tar.gz and install in same type of centos server, run pip3 install -r requirements-pre-3.7.txt successfully but
it's not working

[root@hostname clp-package-ubuntu-focal-x86_64-v0.0.2]# sbin/start-clp

Status: Downloaded newer image for rabbitmq:3.9.8
2023-11-16 09:17:04,807 [INFO] [clp] Stopping clp-db-bffd...
2023-11-16 09:17:05,254 [INFO] [clp] Stopped clp-db-bffd.
2023-11-16 09:17:05,283 [ERROR] [clp] Failed to start CLP.
Traceback (most recent call last):
File "sbin/start-clp", line 462, in main
start_queue(instance_id, clp_config)
File "sbin/start-clp", line 271, in start_queue
subprocess.run(cmd, stdout=subprocess.DEVNULL, check=True)
File "/usr/lib64/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'exec', '-it', 'clp-queue-bffd', 'rabbitmqctl', 'wait', '/tmp/rabbitmq.pid']'

[bossjie]

I even upgrade python3 to 3.7 and retry, it fails also.
2023-11-16 09:44:10,263 [INFO] [clp] Starting database...
2023-11-16 09:44:11,904 [INFO] [clp] Started database.
2023-11-16 09:44:11,904 [INFO] [clp] Creating database tables...
2023-11-16 09:44:12,617 [INFO] [clp] Created database tables.
2023-11-16 09:44:12,617 [INFO] [clp] Starting queue...
2023-11-16 09:44:16,752 [INFO] [clp] Stopping clp-db-0862...
2023-11-16 09:44:17,159 [INFO] [clp] Stopped clp-db-0862.
2023-11-16 09:44:17,179 [ERROR] [clp] Failed to start CLP.
Traceback (most recent call last):
File "sbin/start-clp", line 462, in main
start_queue(instance_id, clp_config)
File "sbin/start-clp", line 271, in start_queue
subprocess.run(cmd, stdout=subprocess.DEVNULL, check=True)
File "/usr/local/lib/python3.7/subprocess.py", line 468, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'exec', '-it', 'clp-queue-0862', 'rabbitmqctl', 'wait', '/tmp/rabbitmq.pid']' returned non-zero exit status 137.

[Jack Luo]

After chatting offline with @bossjie, several issues were identified which prevented the successful startup of CLP package.

1. Docker bug in CentOS 7 which prevents the MariaDB SQL database and rabbitmq container image from being pulled and started from docker-hub

• solution: upgrade to a recent docker release such as docker-ce-24.0.7-1

2. Permission issues with rabbitmq container's log file when started with the root user

• temporary solution: use another user to start CLP package
• permanent solution: a PR will be pushed out to resolve the underlying permission issue

After the two issues above are addressed, we were able to successfully start CLP package on CentOS 7.