[docker] docker-py naming conflict since 2.0.2 release
dminca opened this issue · comments
Since the release of docker v2.0.2 both docker-py and docker libs cannot reside together as it'll throw a naming conflict error.
This change has already been applied in docker/docker-py#1399 in order to fix docker/docker-py#1395 docker/docker-py#1370 and last but not least docker/compose#4344 , thus docker-compose v1.10.1+ will require the docker lib instead of the docker-py one.
ISSUE TYPE
- Bug Report
- Documentation Report
COMPONENT NAME
docker_containerdocker_imagedocker_image_factsdocker_logindocker_network
ANSIBLE VERSION
ansible 2.2.1.0
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides
CONFIGURATION
OS / ENVIRONMENT
"N/A"
SUMMARY
Possible naming conflict on the docker-py package since version 2.0.2 will prevent user from keeping both docker-py and the new docker libs.
STEPS TO REPRODUCE
- install the docker suite:
docker-enginev1.13.1docker-composev1.11.2
- install
dockerviapipmodule - via the
shellorcommandmodule, run adocker-compose upon adocker-compose.yml - run a task via
docker_containermodule
- name: deploy master node via docker-compose
environment:
RANCHER_SRV_MASTER: "{{ rancher_server.master }}"
shell: "docker-compose -f docker-compose.yaml -f master.yaml up -d"
args:
chdir: /opt/rancher/rancher-stack
when: "inventory_hostname == rancher_server.master"
- name: return registration tokens of Rancher servers
uri:
method: POST
status_code: 201
url: "http://{{ rancher_server.master }}:{{ rancher_server.port }}/v1/registrationtokens?projectId={{ project_ids.stdout_lines[0] }}"
return_content: true
register: rancher_token_url
- name: get registration URLs from Rancher servers
uri:
method: GET
url: "{{ rancher_token_url.json['links']['self'] }}"
return_content: true
register: rancher_token
- name: register Rancher Agents with Rancher Servers
become: true
become_method: sudo
docker_container:
name: "{{ rancher_agent.name }}"
image: "rancher/agent:{{ rancher_agent.version }}"
state: started
privileged: true
detach: true
cleanup: true
env:
CATTLE_AGENT_IP: "{{ rancher_server.master }}"
volumes:·
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/rancher:/var/lib/rancher
command: "{{ rancher_token.json['registrationUrl'] }}"EXPECTED RESULTS
- run the
docker-compose.ymlvia thecommandorshellmodule to lift up infrastructure - successfully start the
rancher-agentcontainer via thedocker_containermodule
ACTUAL RESULTS
- ansible will throw:
Failed to import docker-py - No module named docker. Try "pip install docker-py"dockeranddocker-pycannot reside together sincedocker v2.0.2docker-composewill not function withdocker-pysince compose v1.10.1
failed: [web-node001.org] (item=4a4f54604e9d) => {
"failed": true,
"invocation": {
"module_args": {
"api_version": null,
"blkio_weight": null,
"cacert_path": null,
"capabilities": null,
"cert_path": null,
"cleanup": false,
"command": null,
"cpu_period": null,
"cpu_quota": null,
"cpu_shares": null,
"cpuset_cpus": null,
"cpuset_mems": null,
"debug": false,
"detach": true,
"devices": null,
"dns_opts": null,
"dns_search_domains": null,
"dns_servers": null,
"docker_host": null,
"entrypoint": null,
"env": null,
"env_file": null,
"etc_hosts": null,
"exposed_ports": null,
"filter_logger": false,
"force_kill": true,
"groups": null,
"hostname": null,
"ignore_image": false,
"image": null,
"interactive": false,
"ipc_mode": null,
"keep_volumes": false,
"kernel_memory": null,
"key_path": null,
"kill_signal": null,
"labels": null,
"links": null,
"log_driver": null,
"log_options": null,
"mac_address": null,
"memory": "0",
"memory_reservation": null,
"memory_swap": null,
"memory_swappiness": null,
"name": "4a4f54604e9d",
"network_mode": null,
"networks": null,
"oom_killer": null,
"oom_score_adj": null,
"paused": false,
"pid_mode": null,
"privileged": false,
"published_ports": null,
"pull": false,
"purge_networks": false,
"read_only": false,
"recreate": false,
"restart": false,
"restart_policy": null,
"restart_retries": null,
"security_opts": null,
"shm_size": null,
"ssl_version": null,
"state": "absent",
"stop_signal": null,
"stop_timeout": null,
"timeout": null,
"tls": null,
"tls_hostname": null,
"tls_verify": null,
"trust_image_content": false,
"tty": false,
"ulimits": null,
"user": null,
"uts": null,
"volume_driver": null,
"volumes": null,
"volumes_from": null
},
"module_name": "docker_container"
},
"item": "4a4f54604e9d",
"msg": "Failed to import docker-py - No module named docker. Try `pip install docker-py`"
Hi,
I have issue in run() as well as pull():
client = docker.DockerClient(base_url='tcp://172.17.0.1:2375', version='auto')
container = client.containers.run(image='alpine:latest', command='echo hello world!')
#client.images.pull('alpine:latest')
container = client.containers.run(image='alpine:latest', command='echo hello world!')
File "/usr/lib/python2.7/site-packages/docker/models/containers.py", line 655, in run
self.client.images.pull(image)
File "/usr/lib/python2.7/site-packages/docker/models/images.py", line 256, in pull
self.client.api.pull(name, **kwargs)
File "/usr/lib/python2.7/site-packages/docker/api/image.py", line 333, in pull
header = auth.get_config_header(self, registry)
AttributeError: 'module' object has no attribute 'get_config_header'
I have "pip install docker". There is no docker-py
How to go about fixing this?
thx
-a
@alpeshspatel if you have previously installed docker-py, just :
pip uninstall docker-pythenpip install docker
The code needs to be updated to reference the new Python Docker library name.
Aborting. No time.
Can someone explain why this issue has been closed?
I'm running Ansible 2.3.1.0 on Ubuntu 16.04. I have installed the package python-docker (version 1.9), but when running a playbook with Docker, I get the error: Failed to import docker-py - No module named requests.exceptions. Try 'pip install docker-py'.
I tried to remove the python-docker package and do pip install docker-py. it installed version 1.10.6. But I still get the same error. Then I did uninstall that package (using uninstall instead of install) and installed docker with pip install docker. I got version 2.3.0. Bit I still have the same problem.
So at the moment, I have no ways to use the Docker modules in Ansible!
@jcberthon Are you still encountering this problem? ansible 2.3.0 added detection with docker-py >= 2, make sure you have the correct python installation with correct version of docker-py and python-docker.
@kassiansun I just ran into this using docker 2.0.1 (and docker-py 1.10.6) on python 3.5 and ansible 2.3.1.0
@sigmavirus24 Well, from docker-compose's requirements file, now it depends on docker-py 2.4.2, and this should work well with docker_container(on my pc it runs smoothly).
So I didn't get why this is happening, what are the versions of you guys' docker-compse, docker-py, python-docker? I'm working with all latest version, with no error.
So I don't even have docker-compose installed. Literally did pip install ansible docker in a virtualenv on macOS (and homebrew python3) and I'm getting the
AttributeError: 'module' object has no attribute 'get_config_header'
exception with
- name: "Repro for 22993"
hosts: localhost
tasks:
- name: "Pull hello-world"
docker_image:
name: "hello-world"@sigmavirus24 What's the version of docker-py?
Actually I don't think ansible can run with pip(3) install, ansible 2.3.0 is the offical version but it can't run with python3(at least not officially supported)
Yeah, I run ansible on Python 3 enough that I don't think it's that.
I started off with docker-py 1.10.6 (as above) and when I encountered the same AttributeError I tried docker instead but got the same error. I even recreated the virtualenv from scratch and still get this error.
If you want to run ansible with python3, the git version is better(v2.4 is working on python3 support), and I'm running it on my python3 environment.
docker-py runs on your target machine, not local, I have not much experience with virtualenv, this should be setup before you run ansible on remote.
On you local machine, pip install ansible is all you need to do. Then install docker-py or any other dependencies on target machine. ansible only need itself to run on local machine, it copies the modules to remote, and the modules runs on remote might need some other dependencies(like docker-py).
Hi @kassiansun
I forgot to update my comment. I found the problem. When I read the document it was written to install docker-py for dependency on the host that executes the module. That sentence was not clear to me and I installed docker-py on the host that executes the playbook (where I execute ansible-playbook ...). Of course that did not work.
Now I understand that "executes the module" refer to the machine where the ansible module is actually executed, so those are the hosts defined in the playbook. After I added a pre-step which install on the remote hosts the docker-py dependency, then it worked.
I guess the documentation could be less ambiguous. When you read it twice, you think it is on the host where you execute your playbook. After banging your head against a wall (it does help putting back some neurone in place ;-) ) and reading it a 3rd time, then you understand that it could be meant differently. So you try and you realise your mistake.
I would suggest updating the documentation as follow:
Requirements
Those are applicable for all hosts (remote or local) where this module will be executed.
<list of req>
@jcberthon Actually I think this is the source of this kind of problems, many people don't know how ansible works - copy the module to remote and execute it. And the docker-py/python-docker problem distracts people, make it even harder to locate what's happening.
Hope when people search for missing module problem, they will be directed to here.
I found this problem today, in a bit weird way, with ansible 2.3.1.0
With given tasks:
# Install Docker
- name: Install Docker
apt: name={{ docker_package }} state=latest update-cache=yes
# Install docker-py
- name: Install docker-py
pip: name={{ docker_py_package }} state=latest
the results are:
kbiszta@ubuntu-xenial:~$ docker --version
Docker version 17.05.0-ce, build 89658be
kbiszta@ubuntu-xenial:~$ pip show docker-py
Name: docker-py
Version: 1.10.6
Summary: Python client for Docker.
Home-page: https://github.com/docker/docker-py/
Author: Joffrey F
Author-email: joffrey@docker.com
License: UNKNOWN
Location: /usr/local/lib/python2.7/dist-packages
Requires: backports.ssl-match-hostname, six, websocket-client, ipaddress, docker-pycreds, requests
So, everything seems to be fine, but when next I run the task:
# Get gitlab-ci-runner images
- name: Get the gitlab-ci-runner docker images
docker_image:
name: 'gitlab/gitlab-runner'
tag: '{{ item }}'
state: present
with_items:
- 'latest'
- 'alpine'
I get the "msg": "Error pulling image gitlab/gitlab-runner:latest - 'module' object has no attribute 'get_config_header'". I read on web that uninstalling docker-pyhelps to solve this issue, but after removing it I got "msg": "Failed to import docker-py - No module named docker. Try 'pip install docker-py'". So, after installing it back again, everything went fine, what made me confused.
Any ideas why reinstalling the docker-py with exactly the same version of the module made it work properly?
Best,
Kris.
@szarlatan I had the same issue, comparing another environment I have, doing:
pip3 uninstall docker
pip3 install docker==2.1.0
fixes the issue. I can now use again the module. Maybe a newer version works, I don't know.
Has this been fixed for Ansible 2.5.0? I'm not too familiar with the docker APIs but I think that this PR, released in 2.5.0 may have fixed it: #36973 at least for many of the docker modules.
needs_info
@abadger no, it hasn't. Unfortunately docker and docker-py occupy the same namespace (docker), and depending on install order they will overwrite each others files.
I was just thinking about this earlier if there was a way we could detect it, and not sure that is possible, as it can manifest as an import error in some situations.
Ultimately, you cannot have both docker and docker-py installed together if you want a functional library.
Also, at this point docker-py is deprecated, and no one should be using it going forward.
@abadger yes, that is correct. You can have either docker or docker-py installed to use our modules. docker is preferred at this point.
However, you cannot have both python libraries installed at the same time.
@dminca This issue is waiting for your response. Please respond or the issue will be closed.
Since docker-py is now legacy, no issues are reported so far, although there are still traces of docker-py in the Ansible codebase. Should we replace them with the latest docker lib?
@dminca there is really no need. On another note, and I failed to come back to this issue, 2.6 (devel) and 2.5.3 will include extra protection around preventing docker and docker-py from being installed side by side.
If you have further questions please stop by IRC or the mailing list:
- IRC: #ansible on irc.freenode.net
- mailing list: https://groups.google.com/forum/#!forum/ansible-project
Hi！@jcberthon i got the same problem as yours. But i just have merely one machine (ubuntu 16.04), so i don't know how to solve it. Do you have any idea?
Thank you!
Huimin
Hi @huiminzeng
Sorry for the long delay, I was away most of May and did not catch with all the emails notifications.
If you have only one machine, then do you something like delegate_to: 127.0.0.1 or equivalent? Or do you connect to a VM on the same machine?
Basically, you need to install docker-py on the "destination" machine where you want your want your Docker commands to be executed.
I ran into this same issue as well and tried to remove the docker python modules (with pip uninstall), but it still failed. Eventually I found out that ansible was using the pip2 environment, where both docker and docker-py was installed and I had to do pip2 uninstall and then re-install again.
So, make sure to check both pip and pip2 if you run into this issue.