Unable to find files (but not really)

[Jeffrey van Pelt]

Issue Type

• Bug report

Ansible and Ansible Lint details

$ ansible --version
ansible 2.8.0
  config file = None
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/ansible/2.8.0/local/lib/python2.7/site-packages/ansible
  executable location = /opt/ansible/2.8.0/bin/ansible
  python version = 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516]
$ ansible-lint --version
ansible-lint 4.1.0

• ansible installation method: pip
• ansible-lint installation method: pip

Actual Behaviour (Bug report only)

When running ansible-lint (via Molecule or without) I get the following warnings about it being unable to find some files:

$ ansible-lint .
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': - No such file or directory
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_http01_create.yml'} - No such file or directory
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_dns01_create.yml'} - No such file or directory
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_http01_cleanup.yml'} - No such file or directory
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_dns01_cleanup.yml'} - No such file or directory

When running with -vvv, you can see it IS able to find and lint the files, but wants to do something with what seems a Python list interpreted quite litterally.

$ ansible-lint . -vvv
Examining /root/roles/acme_ssl/tasks/acme_dns01_create.yml of type tasks
Examining /root/roles/acme_ssl/tasks/main.yml of type tasks
Examining /root/roles/acme_ssl/tasks/acme_create_account.yml of type tasks
Examining /root/roles/acme_ssl/tasks/acme_http01_create.yml of type tasks
Examining /root/roles/acme_ssl/tasks/acme_dns01_cleanup.yml of type tasks
Examining /root/roles/acme_ssl/tasks/acme_http01_cleanup.yml of type tasks
Examining /root/roles/acme_ssl/tasks/{u'file': of type tasks
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': - No such file or directory
Examining /root/roles/acme_ssl/tasks/{u'file': u'acme_http01_create.yml'} of type tasks
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_http01_create.yml'} - No such file or directory
Examining /root/roles/acme_ssl/tasks/{u'file': u'acme_dns01_create.yml'} of type tasks
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_dns01_create.yml'} - No such file or directory
Examining /root/roles/acme_ssl/tasks/{u'file': u'acme_http01_cleanup.yml'} of type tasks
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_http01_cleanup.yml'} - No such file or directory
Examining /root/roles/acme_ssl/tasks/{u'file': u'acme_dns01_cleanup.yml'} of type tasks
WARNING: Couldn't open /root/roles/acme_ssl/tasks/{u'file': u'acme_dns01_cleanup.yml'} - No such file or directory

The repo in question is: https://github.com/Thulium-Drake/ansible-role-acme_ssl/

Any clue what is going on here?

[Nathon Fowlie]

I'm seeing very similar behaviour, however in my case it appears that ansible-lint is somehow getting an incorrect path for a sub-set of the task files. I'm using the most recent version of ansible-lint.

WARNING: Couldn't open /repositories/ansible-docker/tasks/tasks/upgrade/docker-uninstall-rhel7.yml - No such file or directory
WARNING: Couldn't open /repositories/ansible-docker/tasks/tasks/upgrade/docker-pre-upgrade.yml - No such file or directory
WARNING: Couldn't open /repositories/ansible-docker/tasks/tasks/attestation/docker-attestation.yml - No such file or directory
WARNING: Couldn't open /repositories/ansible-docker/tasks/tasks/deployments/docker-rpm-functional-tests.yml - No such file or directory
WARNING: Couldn't open /repositories/ansible-docker/tasks/tasks/attestation/docker-security-check.yml - No such file or directory
WARNING: Couldn't open /repositories/ansible-docker/tasks/upgrade/local-facts.yml - No such file or directory

The every error, the file exists, but in a different location to where ansible-lint thinks it exists.

path as determined by ansible-lint	correct path
/repositories/ansible-docker/tasks/tasks/attestation/docker-security-check.yml	/repositories/ansible-docker/tasks/attestation/docker-security-check.yml (ditto for all other paths containing /tasks/tasks)
/repositories/ansible-docker/tasks/upgrade/local-facts.yml	/repositories/ansible-docker/tasks/local-facts.yml

There's another task files under /repositories/ansible-docker/tasks and /respositories/ansible-docker/tasks/upgrade that ansible-lint correctly processes, but for some reason it doesn't like these 6 particular files. I've double checked and they use utf-8 encoding and unix line feeds so it doesn't seem to be an encoding issue. Running ansible-lint in verbose mode unfortunately doesn't help to shed much light either.

[Justin Georgeson (Work)]

I just ran into this issue, same behavior with ansible-lint v4.1.0 and 4.2.0. I was able to resolve it by using inline form of include_tasks

Produces warning

- name: include extra file
  include_tasks:
    file: my_tasks.yml
    apply:
      become: yes

Does not produce warning

- name: include extra file
  include_tasks: my_tasks.yml
  args:
    apply:
      become: yes

[Nick Lopez]

@jghal's solution worked for me - same ansible-lint versions.

[morin_river]

Present a simple example.

- name: simple path include test
  include_tasks: included_file.yml

- name: relative path include test
  include_tasks: tasks/included_file.yml

This playbook works fine on Ansible.
However, ansible-lint gives an error that file cannot be found.

$ WARNING: Couldn't open /ansible_dir/roles/test_role/tasks/tasks/included_file.yml - No such file or directory

It seems that the reference destination of the path of ansible and include_task is different.
It is good to ask rules for include_task , but the error that file cannot be found is not good.

[Jeffrey van Pelt]

Confirmed fixed as of ansible-lint 4.3.4, same repo, don't mind the errors, need to fix those ;-)

$ ansible-lint --version
ansible-lint 4.3.4
$ ansible-lint .
[208] File permissions not mentioned
tasks/acme_create_account.yml:4
Task/Handler: Create basedir - ACME account

[208] File permissions not mentioned
tasks/acme_http-01_create.yml:4
Task/Handler: Checking token directory on webserver - {{ certificate.domain }}

[208] File permissions not mentioned
tasks/acme_http-01_create.yml:9
Task/Handler: Place challenge token on webserver - {{ certificate.domain }}

[208] File permissions not mentioned
tasks/acme_request.yml:4
Task/Handler: Check certificate directory

[208] File permissions not mentioned
tasks/acme_request.yml:154
Task/Handler: Create destdir on target - {{ certificate.domain }}

[208] File permissions not mentioned
tasks/acme_request.yml:160
Task/Handler: Deploy certificates to target - {{ certificate.domain }}

[204] Lines should be no longer than 160 chars
tasks/acme_request.yml:224
          sudo -u zimbra /opt/zimbra/bin/zmcertmgr deploycrt comm /etc/ssl/acme/{{ certificate.domain }}/{{ certificate.domain }}.cer /etc/ssl/acme/{{ certificate.domain }}/fullchain_ca.crt

You can skip specific rules by adding them to the skip_list section of your configuration file:                      

┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│# .ansible-lint                                                                                                    │
│warn_list:  # or 'skip_list' to silence them completely                                                            │
│  - '204'  # Lines should be no longer than 160 chars'                                                             │
│  - '208'  # File permissions not mentioned'                                                                       │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

[Stephen Ryan]

This isn't solved in all cases. Example on a multi level includes option.

Example setup

folder/test.yml

---
- name: Ansible-lint subfolder include test
  hosts: localhost
  connection: local
  gather_facts: false
  tasks:
    - include_tasks: subfolder/subsubfolder/tasks.yml

folder/relativefolder/tasks.yml

---
- debug:
    msg: "Relative import"

folder/subfolder/subsubfolder/tasks.yml

---
- include_tasks: relativefolder/tasks.yml

Ansible run

# ansible-playbook folder/test.yml

PLAY [Ansible-lint subfolder include test] ******************

TASK [include_tasks] ******************
included: /usr/share/dwansible/folder/subfolder/subsubfolder/tasks.yml for localhost

TASK [include_tasks] ******************
included: /usr/share/dwansible/folder/relativefolder/tasks.yml for localhost

TASK [debug] ******************
ok: [localhost] =>
  msg: Relative import

PLAY RECAP 

Ansible-lint

# ansible-lint folder/test.yml
WARNING  Couldn't open /folder/subfolder/relativefolder/tasks.yml - No such file or directory [try:0]

The issue appears to be in how the basedir in /lib/ansiblelint/utils.py is getting passed around / setup. In ansible-playbook, basedir would be set to folder/ for all include setups. In ansible-lint it appears to be getting set to folder/subfolder/subsubfolder/ when it is checking the second file.

Ansible uses path_dwim_relative internally for these best I can tell instead of path_dwim. That includes all the other search paths.

Switching isn't a simple drop in drop replacement and looks like it would require a fair bit of testing. @ssbarnea is their any reason why I'm missing that would stop ansible-lint from switching over?

[Jeffrey van Pelt]

It seems this still needs more attention :-)

[Jo Rhett]

Exact same problem with include_vars.... The file in question is in vars/ubuntu.yml underneath the role.

This Fails

include_vars: ubuntu.yml

gets:

WARNING  Couldn't open /Users/username/src/repo/roles/rolename/ubuntu.yml - No such file or directory [try:0]

Succeeds

include_vars:
  file: ubuntu.yml

[Adrián Tóth]

I confirm that I also had this problem using ansible-lint 4.3.5 and import_tasks.

[Sorin Sbarnea]

A huge number of changes happened since this bug was originally opened and the mentioned errors cannot be reproduced with current v5 version, so I am closing it.

[Stephen Ryan]

The test case I listed above still shows the issue.

# ansible-lint --version
ansible-lint 5.0.0
# ansible-lint folder/test.yml
WARNING  Overriding detected file kind 'yaml' with 'playbook' for given positional argument: folder/test.yml
WARNING  Failed to discover yaml files to lint using git: fatal: not a git repository (or any of the parent directories): .git
WARNING  Listing 1 violation(s) that are fatal
load-failure: [Errno 2] No such file or directory: 'folder/subfolder/relativefolder/tasks.yml' (filenotfounderror)
folder/subfolder/relativefolder/tasks.yml:0

You can skip specific rules or tags by adding them to your configuration file:
# .ansible-lint
warn_list:  # or 'skip_list' to silence them completely
  - load-failure  # Failed to load or parse file
Finished with 1 failure(s), 0 warning(s) on 3 files.

Is there another version with this fixed after 5.0.0 that hasn't been released yet that I can try?

[Adrián Tóth]

@ssbarnea I can still reproduce this bug, this issue is not resolved yet.

load-failure: [Errno 2] No such file or directory: 'tasks/tasks/subtasks/subtask_2.yml' (filenotfounderror)
tasks/tasks/subtasks/subtask_2.yml:0

As you can see, the path is incorrect, while there is no tasks (child) folder in tasks (parent) folder.

I have added the folder - ansible-lint-bug.tar.gz, which includes the bug scenario. In this folder run the following commands as I did (see the screenshot).

[Lorenzo Pistone]

I am seeing the same problem with included tasks like include_tasks: ../task-file.yaml. This is not resolved.

[Sorin Sbarnea]

I will be happy to review a pull-request that addresses this bug.

[Marc Jay]

@ssbarnea I think acknowledging that this is still a possible issue by reopening it would be the first necessary step? I am encountering this issue still as well

[Sorin Sbarnea]

I do not want to reopen the same bug because it already collected too much information and at least some of the reported issues were clearly fixed. It would worth creating a new bug for this, so we have a clean history. In fact the ideal was would to add a test that replicates the bug, even if it fails (you can add xfail marker to it).

To add a new test, just add the files needed into examples, and add one more method like https://github.com/ansible-community/ansible-lint/blob/master/test/TestExamples.py#L51-L56 -- it is that simple. Add @pytest.mark.xfail("Bug <url>") before the function to indicate it is still failing.