Molecule seems to fail when Ansible is installed from rpm
alexsaezm opened this issue · comments
Issue Type
- Bug report
Molecule and Ansible details
ansible 2.8.2
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/asaezmor/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.6/site-packages/ansible
executable location = /usr/bin/ansible
python version = 3.6.8 (default, Apr 3 2019, 17:26:03) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)]
molecule, version 2.20.1Molecule installation method (one of):
- pip
Ansible installation method (one of):
- pip
- rpm
Desired Behavior
The execution should work no matter how Ansible was installed.
Actual Behaviour
When installed from a rpm file (ansible-2.8.2-1.el8ae.noarch.rpm) the execution failed.
Using pip, everything works.
$ molecule --debug converge
--> Validating schema /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix
└── default
├── dependency
├── create
├── prepare
└── converge
--> Scenario: 'default'
--> Action: 'dependency'
Skipping, missing the requirements file.
--> Scenario: 'default'
--> Action: 'create'
DEBUG: ANSIBLE ENVIRONMENT
---
ANSIBLE_CONFIG: /tmp/molecule/fedora-lab/default/ansible.cfg
ANSIBLE_FILTER_PLUGINS: /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/filters:/tmp/molecule/fedora-lab/default/plugins/filters:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/plugins/filters
ANSIBLE_LIBRARY: /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/libraries:/tmp/molecule/fedora-lab/default/library:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/library
ANSIBLE_ROLES_PATH: /tmp/molecule/fedora-lab/default/roles:/home/asaezmor/Developer/src/github.com/alexsaezm
DEBUG: MOLECULE ENVIRONMENT
---
MOLECULE_DEBUG: 'True'
MOLECULE_DEPENDENCY_NAME: galaxy
MOLECULE_DRIVER_NAME: docker
MOLECULE_ENV_FILE: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/.env.yml
MOLECULE_EPHEMERAL_DIRECTORY: /tmp/molecule/fedora-lab/default
MOLECULE_FILE: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/molecule.yml
MOLECULE_INSTANCE_CONFIG: /tmp/molecule/fedora-lab/default/instance_config.yml
MOLECULE_INVENTORY_FILE: /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml
MOLECULE_LINT_NAME: yamllint
MOLECULE_PROJECT_DIRECTORY: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab
MOLECULE_PROVISIONER_LINT_NAME: ansible-lint
MOLECULE_PROVISIONER_NAME: ansible
MOLECULE_SCENARIO_DIRECTORY: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default
MOLECULE_SCENARIO_NAME: default
MOLECULE_VERIFIER_LINT_NAME: flake8
MOLECULE_VERIFIER_NAME: testinfra
MOLECULE_VERIFIER_TEST_DIRECTORY: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/tests
DEBUG: SHELL REPLAY
ANSIBLE_CONFIG=/tmp/molecule/fedora-lab/default/ansible.cfg ANSIBLE_FILTER_PLUGINS=/home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/filters:/tmp/molecule/fedora-lab/default/plugins/filters:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/plugins/filters ANSIBLE_LIBRARY=/home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/libraries:/tmp/molecule/fedora-lab/default/library:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/library ANSIBLE_ROLES_PATH=/tmp/molecule/fedora-lab/default/roles:/home/asaezmor/Developer/src/github.com/alexsaezm MOLECULE_DEBUG=True MOLECULE_DEPENDENCY_NAME=galaxy MOLECULE_DRIVER_NAME=docker MOLECULE_ENV_FILE=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/.env.yml MOLECULE_EPHEMERAL_DIRECTORY=/tmp/molecule/fedora-lab/default MOLECULE_FILE=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/molecule.yml MOLECULE_INSTANCE_CONFIG=/tmp/molecule/fedora-lab/default/instance_config.yml MOLECULE_INVENTORY_FILE=/tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml MOLECULE_LINT_NAME=yamllint MOLECULE_PROJECT_DIRECTORY=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab MOLECULE_PROVISIONER_LINT_NAME=ansible-lint MOLECULE_PROVISIONER_NAME=ansible MOLECULE_SCENARIO_DIRECTORY=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default MOLECULE_SCENARIO_NAME=default MOLECULE_VERIFIER_LINT_NAME=flake8 MOLECULE_VERIFIER_NAME=testinfra MOLECULE_VERIFIER_TEST_DIRECTORY=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/tests
DEBUG: COMMAND
/usr/bin/ansible-playbook --skip-tags=molecule-notest,notest --diff --inventory=/tmp/molecule/fedora-lab/default/inventory /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml -vvv
ansible-playbook 2.8.2
config file = /tmp/molecule/fedora-lab/default/ansible.cfg
configured module search path = ['/home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/libraries', '/tmp/molecule/fedora-lab/default/library', '/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/library']
ansible python module location = /usr/lib/python3.6/site-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 3.6.8 (default, Apr 3 2019, 17:26:03) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)]
Using /tmp/molecule/fedora-lab/default/ansible.cfg as config file
host_list declined parsing /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml as it did not pass it's verify_file() method
script declined parsing /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml as it did not pass it's verify_file() method
Parsed /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml inventory source with yaml plugin
PLAYBOOK: create.yml ***********************************************************
1 plays in /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml
[WARNING]: Skipping plugin (/home/asaezmor/.local/lib/python3.6/site-
packages/molecule/provisioner/ansible/plugins/filters/molecule_core.py) as it
seems to be invalid: No module named 'molecule'
PLAY [Create] ******************************************************************
META: ran handlers
TASK [Log into a Docker registry] **********************************************
task path: /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml:8
fatal: [localhost]: FAILED! => {
"msg": "An unhandled exception occurred while templating '{{ lookup('file', molecule_file) | molecule_from_yaml }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: no filter named 'molecule_from_yaml'. String: {{ lookup('file', molecule_file) | molecule_from_yaml }}"
}
PLAY RECAP *********************************************************************
localhost : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
ERROR: 
Have you tried using code from master branch? If you cannot replicate it with master we can close it.
Good catch. With master works. Thanks!
I replicated it using the latest version. My bad, I thought it was fixed but I didn't try exactly the same thing.
So:
- Ansible from rpm
- molecule-2.22rc3
$ molecule init role -r test-molecule
--> Initializing new role test-molecule...
Initialized role in /tmp/test-molecule successfully.
$ cd test-molecule/
$ cat molecule/default/molecule.yml
---
dependency:
name: galaxy
driver:
name: podman
lint:
name: yamllint
platforms:
- name: instance
image: centos:7
provisioner:
name: ansible
lint:
name: ansible-lint
verifier:
name: testinfra
lint:
name: flake8
$ molecule test
$ molecule test
--> Validating schema /tmp/test-molecule/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix
└── default
├── lint
├── dependency
├── cleanup
├── destroy
├── syntax
├── create
├── prepare
├── converge
├── idempotence
├── side_effect
├── verify
├── cleanup
└── destroy
--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /tmp/test-molecule/...
Lint completed successfully.
--> Executing Flake8 on files found in /tmp/test-molecule/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /tmp/test-molecule/molecule/default/playbook.yml...
[701] Role info should contain platforms
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'gala$y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: author
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'gala$y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: description
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'gala$y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: company
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galax
y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: license
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galax
y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
An error occurred during the test sequence action: 'lint'. Cleaning up.
--> Scenario: 'default'
--> Action: 'cleanup'
Skipping, cleanup playbook not configured.
--> Scenario: 'default'
--> Action: 'destroy'
[WARNING]: Skipping plugin (/home/asaezmor/Developer/src/github.com/ansible/mo
lecule-2.22rc3/molecule/provisioner/ansible/plugins/filters/molecule_core.py)
as it seems to be invalid: No module named 'molecule'
PLAY [Destroy] *****************************************************************
TASK [Destroy molecule instance(s)] ********************************************
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ lookup('file', molecule_file) | molecule_from_yaml }}'. Error was a <class 'ansible.errors.AnsibleError'>, origina
l message: template error while templating string: no filter named 'molecule_from_yaml'. String: {{ lookup('file', molecule_file) | molecule_from_yaml }}"}
PLAY RECAP *********************************************************************
localhost : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
ERROR: - When I use Ansible from pip:
$ molecule test
--> Validating schema /tmp/test-molecule/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix
└── default
├── lint
├── dependency
├── cleanup
├── destroy
├── syntax
├── create
├── prepare
├── converge
├── idempotence
├── side_effect
├── verify
├── cleanup
└── destroy
--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /tmp/test-molecule/...
Lint completed successfully.
--> Executing Flake8 on files found in /tmp/test-molecule/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /tmp/test-molecule/molecule/default/playbook.yml...
[701] Role info should contain platforms
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: author
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: description
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: company
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
[703] Should change default metadata: license
/tmp/test-molecule/meta/main.yml:2
{'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}
An error occurred during the test sequence action: 'lint'. Cleaning up.
--> Scenario: 'default'
--> Action: 'cleanup'
Skipping, cleanup playbook not configured.
--> Scenario: 'default'
--> Action: 'destroy'
PLAY [Destroy] *****************************************************************
TASK [Destroy molecule instance(s)] ********************************************
changed: [localhost] => (item=None)
changed: [localhost]
TASK [Wait for instance(s) deletion to complete] *******************************
changed: [localhost] => (item=None)
changed: [localhost]
PLAY RECAP *********************************************************************
localhost : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
--> Pruning extra files from scenario ephemeral directorySo I think something is different between the rpm and the pip installations.
Wait but now it is just failing with linting errors?
(Excuse me from the side.)
I think that the RPM does not work because all of filters and plugins (.py files) under molecule/provisioner/ansible/ are missing (and might not be installed) from it.
...
ASK [Destroy molecule instance(s)] ********************************************
fatal: [localhost]: FAILED! => {"msg": ... no filter named 'molecule_from_yaml'. ...
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
We faced the same issue yesterday at https://bugs.launchpad.net/tripleo/+bug/1837143 and apparently this is caused by the decision of making the scripts installed use python -s which disables access to user installed packages. I am not sure why this was done but is clearly affecting a huge number of users.
One thing is clear: this is not a molecule bug and is likely what I would call a packaging bug (hopefully not an intended feature). I would usually try to raise it as an Ansible bug but if you look at https://releases.ansible.com/ansible/rpm/release/ you will not find an ansible 2.8 based on python3.6 so.... you mainly used a non-official ansible distribution. Time to file it un bugzilla against RHEL.
@decentral1se no, sorry, was another test in a new role.
@ssbarnea I used the rpm file from Red Hat.
I have the same problem on fedora 30 with ansible 2.8.2 installed from the official repository.
@alexsaezm do you use Fedora?
I've removed molecule from pip3 and pip2, remove ansible by dnf and then install ansible and molecule from pip3 and it's working fine for me now.
Should we close this or?
Can it be reopened until the issue is fixed on Ansible rpm packaging side?
It is there RH Bugzilla or issue already posted anywhere?
@tutunak your workaround works for me thanks!
Snippet for anybody else:
sudo yum -y remove ansible
pip3 install ansible --user
pip3 install "molecule[lint]" --user
pip3 install "molecule[docker]" --user