Giters

semaphoreui / semaphore

Modern UI for Ansible, Terraform, OpenTofu, Bash, Pulumi.

Home Page:https://semaphoreui.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Problem: Installation of Python Modules via requirements.txt doesnt seem to work as intended

Fabl0s opened this issue · comments

commented

Issue

Hi,

we encouter an Issue regarding installing a Python Module into Semaphore.
We want to use ARA as callback plugin for all our runs, and while this used to work, in recent changes it seem to broke.

Before it was enough to install the Module into the Ansible Repo's venv, but that no longer seems to work.
We added ara into the requirements.txt to be installed on Container start, and according to logs it is getting installed.

However on Runs triggered by the WebUI, we always get this line from Debug:

[WARNING]: Skipping plugin (/home/semaphore/.local/lib/python3.11/site-packages/ara/plugins/callback/ara_default.py), cannot load: No module named 'ara'

Which is odd - Going into the container as semaphore user, I can import the ara module just fine.
The only workarround that works so far was also installing the ara module as root, after that its working via Web perfectly fine.

Doing everything by hand within the Container or on the Host within the venv it works from CLI just fine.
This only seems to affect the web triggered Runs.

Impact

Ansible (task execution)

Installation method

Docker

Database

Postgres

Browser

Microsoft Edge

Semaphore Version

v2.9.112-4f95ac8-1717065276

Ansible Version

external venv:

ansible [core 2.16.7]
  config file = /projects/ansible/ansible.cfg
  configured module search path = ['/projects/ansible/library']
  ansible python module location = /projects/ansible/env/lib64/python3.11/site-packages/ansible
  ansible collection location = /projects/ansible/collections
  executable location = /projects/ansible/env/bin/ansible
  python version = 3.11.7 (main, Jan 22 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/projects/ansible/env/bin/python3.11)
  jinja version = 3.1.4
  libyaml = True


semaphore web:

ansible-playbook [core 2.16.7]
  config file = /tmp/semaphore/ansible.cfg
  configured module search path = ['/tmp/semaphore/library']
  ansible python module location = /opt/semaphore/venv/lib/python3.11/site-packages/ansible
  ansible collection location = /tmp/semaphore/collections
  executable location = /opt/semaphore/venv/bin/ansible-playbook
  python version = 3.11.9 (main, Apr 14 2024, 13:40:00) [GCC 13.2.1 20231014] (/opt/semaphore/venv/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Logs & errors

Podman Logs:

No additional system dependencies to install
Installing additional python dependencies
Requirement already satisfied: requests in /opt/semaphore/venv/lib/python3.11/site-packages (from -r /etc/semaphore/requirements.txt (line 1)) (2.32.3)
Collecting molecule (from -r /etc/semaphore/requirements.txt (line 2))
  Downloading molecule-24.2.1-py3-none-any.whl.metadata (5.8 kB)
Collecting redis (from -r /etc/semaphore/requirements.txt (line 4))
  Downloading redis-5.0.4-py3-none-any.whl.metadata (9.3 kB)
Collecting hvac==1.2.0 (from -r /etc/semaphore/requirements.txt (line 5))
  Downloading hvac-1.2.0-py3-none-any.whl.metadata (3.4 kB)
Collecting ara (from -r /etc/semaphore/requirements.txt (line 6))
  Downloading ara-1.7.1-py2.py3-none-any.whl.metadata (9.0 kB)
Collecting pyhcl<0.5.0,>=0.4.4 (from hvac==1.2.0->-r /etc/semaphore/requirements.txt (line 5))
  Downloading pyhcl-0.4.5-py3-none-any.whl.metadata (3.9 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (2024.2.2)
Collecting ansible-compat>=4.1.8 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading ansible_compat-24.6.0-py3-none-any.whl.metadata (2.9 kB)
Requirement already satisfied: ansible-core>=2.12.10 in /opt/semaphore/venv/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (2.16.7)
Collecting click<9,>=8.0 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting click-help-colors>=0.9 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading click_help_colors-0.9.4-py3-none-any.whl.metadata (4.1 kB)
Collecting enrich>=1.2.7 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading enrich-1.2.7-py3-none-any.whl.metadata (3.9 kB)
Collecting jsonschema>=4.9.1 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading jsonschema-4.22.0-py3-none-any.whl.metadata (8.2 kB)
Requirement already satisfied: Jinja2>=2.11.3 in /opt/semaphore/venv/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (3.1.4)
Requirement already satisfied: packaging in /usr/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (23.2)
Collecting pluggy<2.0,>=0.7.1 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading pluggy-1.5.0-py3-none-any.whl.metadata (4.8 kB)
Requirement already satisfied: PyYAML>=5.1 in /opt/semaphore/venv/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (6.0.1)
Collecting rich>=9.5.1 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading rich-13.7.1-py3-none-any.whl.metadata (18 kB)
Collecting wcmatch>=8.1.2 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading wcmatch-8.5.2-py3-none-any.whl.metadata (4.8 kB)
Collecting pbr!=2.1.0,>=2.0.0 (from ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading pbr-6.0.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting cliff (from ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading cliff-4.7.0-py3-none-any.whl.metadata (2.1 kB)
Collecting subprocess-tee>=0.4.1 (from ansible-compat>=4.1.8->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading subprocess_tee-0.4.1-py3-none-any.whl.metadata (2.8 kB)
Requirement already satisfied: cryptography in /opt/semaphore/venv/lib/python3.11/site-packages (from ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (42.0.7)
Requirement already satisfied: resolvelib<1.1.0,>=0.5.3 in /opt/semaphore/venv/lib/python3.11/site-packages (from ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/semaphore/venv/lib/python3.11/site-packages (from Jinja2>=2.11.3->molecule->-r /etc/semaphore/requirements.txt (line 2)) (2.1.5)
Collecting attrs>=22.2.0 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl.metadata (3.0 kB)
Collecting referencing>=0.28.4 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading referencing-0.35.1-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading rpds_py-0.18.1-cp311-cp311-musllinux_1_2_x86_64.whl.metadata (4.1 kB)
Collecting markdown-it-py>=2.2.0 (from rich>=9.5.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich>=9.5.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB)
Collecting bracex>=2.1.1 (from wcmatch>=8.1.2->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading bracex-2.4-py3-none-any.whl.metadata (3.6 kB)
Collecting PrettyTable>=0.7.2 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading prettytable-3.10.0-py3-none-any.whl.metadata (30 kB)
Collecting autopage>=0.4.0 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading autopage-0.5.2-py3-none-any.whl.metadata (7.9 kB)
Collecting cmd2>=1.0.0 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading cmd2-2.4.3-py3-none-any.whl.metadata (12 kB)
Collecting stevedore>=2.0.1 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading stevedore-5.2.0-py3-none-any.whl.metadata (2.3 kB)
Collecting pyperclip>=1.6 (from cmd2>=1.0.0->cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading pyperclip-1.8.2.tar.gz (20 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting wcwidth>=0.1.7 (from cmd2>=1.0.0->cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading wcwidth-0.2.13-py2.py3-none-any.whl.metadata (14 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=9.5.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Requirement already satisfied: cffi>=1.12 in /opt/semaphore/venv/lib/python3.11/site-packages (from cryptography->ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (1.16.0)
Requirement already satisfied: pycparser in /opt/semaphore/venv/lib/python3.11/site-packages (from cffi>=1.12->cryptography->ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (2.22)
Downloading hvac-1.2.0-py3-none-any.whl (149 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.9/149.9 kB 22.5 MB/s eta 0:00:00
Downloading molecule-24.2.1-py3-none-any.whl (113 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.0/114.0 kB 17.6 MB/s eta 0:00:00
Downloading redis-5.0.4-py3-none-any.whl (251 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 252.0/252.0 kB 38.7 MB/s eta 0:00:00
Downloading ara-1.7.1-py2.py3-none-any.whl (233 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 233.7/233.7 kB 37.3 MB/s eta 0:00:00
Downloading ansible_compat-24.6.0-py3-none-any.whl (23 kB)
Downloading click-8.1.7-py3-none-any.whl (97 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 18.2 MB/s eta 0:00:00
Downloading click_help_colors-0.9.4-py3-none-any.whl (6.4 kB)
Downloading enrich-1.2.7-py3-none-any.whl (8.7 kB)
Downloading jsonschema-4.22.0-py3-none-any.whl (88 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.3/88.3 kB 16.6 MB/s eta 0:00:00
Downloading pbr-6.0.0-py2.py3-none-any.whl (107 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 107.5/107.5 kB 18.3 MB/s eta 0:00:00
Downloading pluggy-1.5.0-py3-none-any.whl (20 kB)
Downloading pyhcl-0.4.5-py3-none-any.whl (50 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.2/50.2 kB 9.7 MB/s eta 0:00:00
Downloading rich-13.7.1-py3-none-any.whl (240 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.7/240.7 kB 33.7 MB/s eta 0:00:00
Downloading wcmatch-8.5.2-py3-none-any.whl (39 kB)
Downloading cliff-4.7.0-py3-none-any.whl (81 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.2/81.2 kB 14.4 MB/s eta 0:00:00
Downloading attrs-23.2.0-py3-none-any.whl (60 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 9.6 MB/s eta 0:00:00
Downloading autopage-0.5.2-py3-none-any.whl (30 kB)
Downloading bracex-2.4-py3-none-any.whl (11 kB)
Downloading cmd2-2.4.3-py3-none-any.whl (147 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 147.2/147.2 kB 26.4 MB/s eta 0:00:00
Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl (18 kB)
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 13.8 MB/s eta 0:00:00
Downloading prettytable-3.10.0-py3-none-any.whl (28 kB)
Downloading pygments-2.18.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 65.5 MB/s eta 0:00:00
Downloading referencing-0.35.1-py3-none-any.whl (26 kB)
Downloading rpds_py-0.18.1-cp311-cp311-musllinux_1_2_x86_64.whl (1.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 46.0 MB/s eta 0:00:00
Downloading stevedore-5.2.0-py3-none-any.whl (49 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.7/49.7 kB 7.6 MB/s eta 0:00:00
Downloading subprocess_tee-0.4.1-py3-none-any.whl (5.1 kB)
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Downloading wcwidth-0.2.13-py2.py3-none-any.whl (34 kB)
Building wheels for collected packages: pyperclip
  Building wheel for pyperclip (pyproject.toml): started
  Building wheel for pyperclip (pyproject.toml): finished with status 'done'
  Created wheel for pyperclip: filename=pyperclip-1.8.2-py3-none-any.whl size=11124 sha256=3675f8a744fdbacd1ddcde1949ba1d1c2c6c0d9728a09fc6ded7098954b35c0c
  Stored in directory: /home/semaphore/.cache/pip/wheels/70/bd/ba/8ae5c080c895c9360fe6e153acda2dee82527374467eae061b
Successfully built pyperclip
Installing collected packages: wcwidth, pyperclip, pyhcl, subprocess-tee, rpds-py, redis, pygments, PrettyTable, pluggy, pbr, mdurl, click, bracex, autopage, attrs, wcmatch, stevedore, referencing, markdown-it-py, hvac, cmd2, click-help-colors, rich, jsonschema-specifications, cliff, jsonschema, enrich, ara, ansible-compat, molecule
  WARNING: The script pygmentize is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script pbr is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script markdown-it is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script jsonschema is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts ara and ara-manage are installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script molecule is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed PrettyTable-3.10.0 ansible-compat-24.6.0 ara-1.7.1 attrs-23.2.0 autopage-0.5.2 bracex-2.4 click-8.1.7 click-help-colors-0.9.4 cliff-4.7.0 cmd2-2.4.3 enrich-1.2.7 hvac-1.2.0 jsonschema-4.22.0 jsonschema-specifications-2023.12.1 markdown-it-py-3.0.0 mdurl-0.1.2 molecule-24.2.1 pbr-6.0.0 pluggy-1.5.0 pygments-2.18.0 pyhcl-0.4.5 pyperclip-1.8.2 redis-5.0.4 referencing-0.35.1 rich-13.7.1 rpds-py-0.18.1 stevedore-5.2.0 subprocess-tee-0.4.1 wcmatch-8.5.2 wcwidth-0.2.13
Starting semaphore server
Loading config
Validating config
Postgres ##########@####################:5432 ##########
Tmp Path (projects home) /tmp/semaphore
Semaphore v2.9.112-4f95ac8-1717065276
Interface
Port :3000
Server is running

time="2024-06-03T09:06:25Z" level=error msg="websocket: close sent" fields.level=Error
time="2024-06-03T09:06:25Z" level=error msg="close tcp 172.29.0.20:3000->172.29.0.19:55060: use of closed network connection" fields.level=Error
time="2024-06-03T09:06:32Z" level=info msg="Task 98 added to queue"
time="2024-06-03T09:06:37Z" level=info msg="Set resource locker with TaskRunner 98"
time="2024-06-03T09:06:37Z" level=info msg="Task 98 removed from queue"
time="2024-06-03T09:06:40Z" level=info msg="Stopped running TaskRunner 98"
time="2024-06-03T09:06:40Z" level=info msg="Release resource locker with TaskRunner 98"
time="2024-06-03T09:13:20Z" level=info msg="Task 99 added to queue"
time="2024-06-03T09:13:22Z" level=info msg="Set resource locker with TaskRunner 99"
time="2024-06-03T09:13:22Z" level=info msg="Task 99 removed from queue"
time="2024-06-03T09:13:25Z" level=info msg="Stopped running TaskRunner 99"
time="2024-06-03T09:13:25Z" level=info msg="Release resource locker with TaskRunner 99"

time="2024-06-03T09:22:54Z" level=error msg="websocket: close sent" fields.level=Error
time="2024-06-03T09:22:54Z" level=error msg="close tcp 172.29.0.20:3000->172.29.0.19:57488: use of closed network connection" fields.level=Error
time="2024-06-03T09:23:04Z" level=info msg="Task 100 added to queue"
time="2024-06-03T09:23:07Z" level=info msg="Set resource locker with TaskRunner 100"
time="2024-06-03T09:23:07Z" level=info msg="Task 100 removed from queue"
time="2024-06-03T09:23:10Z" level=info msg="Stopped running TaskRunner 100"
time="2024-06-03T09:23:10Z" level=info msg="Release resource locker with TaskRunner 100"
exit code: 0```


Web Run Logs:

```Task 98 added to queue
Started: 98
Run TaskRunner with template: Vault Test
Preparing: 98
installing static inventory
No collection/requirements.yml file found. Skip galaxy install process.
No collection/requirements.yml file found. Skip galaxy install process.
No role/requirements.yml file found. Skip galaxy install process.
No role/requirements.yml file found. Skip galaxy install process.
ansible-playbook [core 2.16.7]
  config file = /tmp/semaphore/ansible.cfg
  configured module search path = ['/tmp/semaphore/library']
  ansible python module location = /opt/semaphore/venv/lib/python3.11/site-packages/ansible
  ansible collection location = /tmp/semaphore/collections
  executable location = /opt/semaphore/venv/bin/ansible-playbook
  python version = 3.11.9 (main, Apr 14 2024, 13:40:00) [GCC 13.2.1 20231014] (/opt/semaphore/venv/bin/python3)
  jinja version = 3.1.4
  libyaml = True
Using /tmp/semaphore/ansible.cfg as config file
setting up inventory plugins
Loading collection ansible.builtin from 
Parsed /tmp/semaphore/inventory_98.yml inventory source with yaml plugin
redirecting (type: callback) ansible.builtin.yaml to community.general.yaml
Loading collection community.general from /opt/semaphore/venv/lib/python3.11/site-packages/ansible_collections/community/general
redirecting (type: callback) ansible.builtin.yaml to community.general.yaml
Loading callback plugin community.general.yaml of type stdout, v2.0 from /opt/semaphore/venv/lib/python3.11/site-packages/ansible_collections/community/general/plugins/callback/yaml.py
[WARNING]: Skipping plugin (/home/semaphore/.local/lib/python3.11/site-
packages/ara/plugins/callback/ara_default.py), cannot load: No module named 'ara'
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.```

### Manual installation - system information

_No response_

### Configuration

_No response_

### Additional information

_No response_
commented

@fiftin
Same issue when running from the Web UI. Maybe it has something to do with all requirements.txt modules being installed in /home/semaphore and not in the venv

8a04df013a1c:~$ cat /etc/semaphore/requirements.txt
pywinrm
pymssql
netaddr
passlib
8a04df013a1c:~$ /opt/semaphore/venv/bin/pip3 list -v
Package            Version     Location                                            Installer
------------------ ----------- --------------------------------------------------- ---------
ansible            9.4.0       /opt/semaphore/venv/lib/python3.11/site-packages    pip
ansible-core       2.16.7      /opt/semaphore/venv/lib/python3.11/site-packages    pip
boto3              1.34.115    /opt/semaphore/venv/lib/python3.11/site-packages    pip
botocore           1.34.115    /opt/semaphore/venv/lib/python3.11/site-packages    pip
certifi            2024.2.2    /opt/semaphore/venv/lib/python3.11/site-packages    pip
cffi               1.16.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
charset-normalizer 3.3.2       /opt/semaphore/venv/lib/python3.11/site-packages    pip
cryptography       42.0.7      /opt/semaphore/venv/lib/python3.11/site-packages    pip
idna               3.7         /opt/semaphore/venv/lib/python3.11/site-packages    pip
Jinja2             3.1.4       /opt/semaphore/venv/lib/python3.11/site-packages    pip
jmespath           1.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
MarkupSafe         2.1.5       /opt/semaphore/venv/lib/python3.11/site-packages    pip
netaddr            1.3.0       /home/semaphore/.local/lib/python3.11/site-packages pip
packaging          23.2        /usr/lib/python3.11/site-packages
passlib            1.7.4       /home/semaphore/.local/lib/python3.11/site-packages pip
pip                24.0        /opt/semaphore/venv/lib/python3.11/site-packages    pip
pycparser          2.22        /opt/semaphore/venv/lib/python3.11/site-packages    pip
pymssql            2.3.0       /home/semaphore/.local/lib/python3.11/site-packages pip
pyparsing          3.1.1       /usr/lib/python3.11/site-packages
pyspnego           0.10.2      /home/semaphore/.local/lib/python3.11/site-packages pip
python-dateutil    2.9.0.post0 /opt/semaphore/venv/lib/python3.11/site-packages    pip
pywinrm            0.4.3       /home/semaphore/.local/lib/python3.11/site-packages pip
PyYAML             6.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
requests           2.32.3      /opt/semaphore/venv/lib/python3.11/site-packages    pip
requests-ntlm      1.2.0       /home/semaphore/.local/lib/python3.11/site-packages pip
resolvelib         1.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
s3transfer         0.10.1      /opt/semaphore/venv/lib/python3.11/site-packages    pip
setuptools         65.5.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
six                1.16.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
urllib3            2.2.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
xmltodict          0.13.0      /home/semaphore/.local/lib/python3.11/site-packages pip

The UI reports the error: msg: 'winrm or requests is not installed: No module named ''winrm'''
However, if I run the ansible-playbook command logged into the container

  • docker exec -it docker-semaphore-1 bash
  • /opt/semaphore/venv/bin/python3 /opt/semaphore/venv/bin/ansible-playbook -i inventory.yml --user svc-ansible --ask-pass --extra-vars '{"ANSIBLE_HOST_KEY_CHECKING":false,"ansible_connection":"winrm","ansible_port":"5986","ansible_winrm_server_cert_validation":"ignore","semaphore_vars":{"task_details":{"id":2,"url":null,"username":"admin"}}}' --fork 30 ping.yml

The above works OK and does not report the same module missing error

8a04df013a1c:~$ semaphore version
v2.9.112-4f95ac8-1717065276

maybe related to the fix implemented on: #2060 and the install command

semaphore/deployment/docker/server/server-wrapper

Line 193 in 80e2c8e

pip3 install --upgrade --user \

looks like the --user flag for pip is not recommended/supported within venv

commented

A PR which removes the --user flag from the server/runner wrapper is incoming, I'm on it. Hopefully this will resolve the remaining issue related to requirements.txt

commented

Just tried the current develop docker build - works now, thank you