docker / for-win

Bug reports for Docker Desktop for Windows

Home Page:https://www.docker.com/products/docker#/windows

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

docker-compose bind mount docker.sock not a valid Windows path

ameier38 opened this issue · comments

Expected behavior

Should be able to bind mount the docker socket as explained
in http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ using docker-compose

Actual behavior

I can successfully run the command

docker run -v /var/run/docker.sock:/var/run/docker.sock -it docker

But when I try to use within a docker-compose.yml file I get the following error

Creating docker_docker_1 ... error

ERROR: for docker_docker_1  Cannot create container for service docker: b'Mount denied:\nThe source path "\\\\var\\\\run\\\\docker.sock:/var/run/docker.sock"\nis not a valid Windows path'

ERROR: for docker  Cannot create container for service docker: b'Mount denied:\nThe source path "\\\\var\\\\run\\\\docker.sock:/var/run/docker.sock"\nis not a valid Windows path'
ERROR: Encountered errors while bringing up the project.

Information

Docker Community Edition

Version 18.03.0-ce-rc3-win56 (16433)
Channel: edge
b6990be

Note: Just updated Docker and I did not receive this error in previous version.

docker-compose.yml

version: '3'
services:
    docker:
        image: docker
        stdin_open: true
        tty: true
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock

Running docker-compose up gives the following error

λ  docker-compose up
Creating docker_docker_1 ... error

ERROR: for docker_docker_1  Cannot create container for service docker: b'Mount denied:\nThe source path "\\\\var\\\\run\\\\docker.sock:/var/run/docker.sock"\nis not a valid Windows path'

ERROR: for docker  Cannot create container for service docker: b'Mount denied:\nThe source path "\\\\var\\\\run\\\\docker.sock:/var/run/docker.sock"\nis not a valid Windows path'
ERROR: Encountered errors while bringing up the project.

Steps to reproduce the behavior

  1. Run docker-compose up using above docker-compose.yml

Reverting to

Version 17.12.0-ce-win47 (15139)
Channel: stable
9c692cd

works without an error.

THis is critical for us, too!

@ameier38 Where can I find this older verison of docker?

@ChristianSauer Go to install page and then download the stable channel.

It would be really helpful for us Edge users if you could keep the previous Edge versions available for download so when something fundamental like this breaks we can just roll back one Edge release, instead of having to revert all the way back to Stable.

You can download previous versions from this page https://docs.docker.com/docker-for-windows/release-notes/

@creativearmenia So you can, I missed that. Thank you!

It's still an issue in 18.03.0-ce-rc4

The windows version of docker doesn't use unix socket (/var/run/docker.sock) but npipe (npipe:////./pipe/docker_engine).
So you have either

  • to switch to linux container which runs docker in a full virtualized linux with unix socket
  • to pass the npipe instead of the unix socket to the container (windows container only)
  • to use a network socket (should work with linux and windows container)

The doc is here : https://docs.docker.com/docker-for-windows/faqs/#how-do-i-connect-to-the-remote-docker-engine-api

@stalb This is not about connecting to the API, it's about mounting the /var/run/docker.sock unix socket (from the Hyper-V host Linux VM) into a running container.

Looks like this issue is in stable now, and just wiped out an environment that uses an nginx proxy, listening to docker.sock to auto-proxy sites.

https://github.com/jwilder/nginx-proxy

Docker Version:
18.03.0-ce-win59

We are getting into same issues on stable driver, when can we get fix for this.? Do we need to switch back to previous stable release.? Is there a workaround for this until we get fix in next stable driver.?

@jagraj I reverted to the last version, 17.12.0-ce-win47, for now. I wasn't able to identify any workarounds or locate a docker.sock that was accessible from the host file system.

I am wondering how this blocking issue ended up in stable release where customers already reported in edge drivers.

Docker-for-Windows team: Please let us know that you've registered this critical regression! It's on track to break our product for Windows users.

We've also had this reported for some of our Windows lando users.

commented

Docker For Windows updated to 18.03.0-ce-win59 (16762) on the stable channel today and I ran into this issue as well. Docker-compose is unable to bind to /var/run/docker.sock

I've managed to work around this issue by setting environment variable COMPOSE_CONVERT_WINDOWS_PATHS to something truthy (1, 'true', 'on') as per docker/compose#4240. Release notes at the time suggest Docker for Windows did not need to set this but clearly this has now been broken, see https://github.com/docker/compose/blob/master/CHANGELOG.md#190-2016-11-16.

Issue seems to have been introduced by this commit docker/compose@2fbec60, most likely docker/compose@2fbec60#diff-e2ff86ffa7e639ddeff3723cabf428eeR150.
Original PR - docker/compose#5560

So this doesn't seem to be a Docker for Windows issue directly and probably should be fixed in compose?

@jimgolfgti 's marvelous workaround in #1829 (comment) is the ticket for us. Thanks so much for sharing that.

The workaround from @jimgolfgti didn't work for me in Git for Windows 2.16.2.1 bash shell. Same error with and without the environment variable. Uninstalled d4w 18.x and installed 17.12.0-ce-win47 (15139), and now everything works as before.

commented

Came here to say the same, update broke things with docker-compose and the ability to bind "/var/run/docker.sock".

commented

run $Env:COMPOSE_CONVERT_WINDOWS_PATHS=1 in PowerShell and see if it does the deference

@sv1slim
Worked like a charm

I can confirm that it is working on my Git Bash. @shpros the quickest way to test it works is by doing something like:

export COMPOSE_CONVERT_WINDOWS_PATHS=1
docker-compose down && docker-compose up -d

If you do not export, the environment variable may not make it all the way through the processes spawned -- especially if you are using other services that run docker-compose. Once confirmed working through this route, you can add it to your environment through more permanent means, through editing .bashrc in your home directory or otherwise. You can check that the environment variable is successfully set via echo $COMPOSE_CONVERT_WINDOWS_PATHS.

Hey folks,

Compose maintainer here - can someone try the following versions [1] of Compose: 1.18.0, 1.19.0, 1.20.1 against Docker For Windows 18.03.0-ce and report if they all exhibit the problem? That'd help a lot with pinpointing the issue.

Thanks!

[1] you can download the .exe files from https://github.com/docker/compose/releases

@shin- I can confirm that I the issue occurs with the following

  • Docker version 18.03.0-ce, build 0520e24
  • docker-compose version 1.20.1, build 5d8c71b2

Same for me (same versions as @deisterhold)

For me on a windows box with Docker Version 18.03.0-ce-win59 (16762) Channel: stable d70d80d

set COMPOSE_CONVERT_WINDOWS_PATHS=1
docker-compose down && Docker-compose up -d

did the job. Thanks for the steer @johnrom and others

Also having this issue atm.
I'll use the ENV var for now.

Setting COMPOSE_CONVERT_WINDOWS_PATHS to 1 on Windows 10 helped to fix this issue running Docker and Compose via Cygwin.

Subscribing.

Came back to this thread because I mistakenly allowed Docker to upgrade itself and found myself in the same situation. However, this time, export COMPOSE_CONVERT_WINDOWS_PATHS=1 worked for me. Windows 10 Enterprise, using Git bash that comes with Git for Windows.

commented

Thanks for the solution @jimgolfgti !

Are we supposed to explicitly set export COMPOSE_CONVERT_WINDOWS_PATHS=1 in all of our scripts from now on? Or is this going to be the default in some newer version of either compose or docker-for-win?

Same issue for me on latest Windows 10 and Docker For Windows Version 18.03.1-ce-win65 (17513)
I can confirm that setting $Env:COMPOSE_CONVERT_WINDOWS_PATHS=1 fixes the issue, but this is not a very sustainable workaround

How is this even allowed in stable? Do people not test things nowadays?

I have the same issue, but unfortunately, the COMPOSE_CONVERT_WINDOWS_PATHS solution doesn't work for me.

This PyCharm bug isn't on GitHub, so no automatic cross-references:

https://youtrack.jetbrains.com/issue/PY-29994

But this has basically forced me to return to command line development and give up all the groovy remote debugging I could do with PyCharm.

If you're using the Windows command prompt, setting COMPOSE_CONVERT_WINDOWS_PATHS to 1 in the system environment variables works, but make sure you restart your command prompt after you change that setting. It's easy to forget.

commented

As a quick workaround I simply put COMPOSE_CONVERT_WINDOWS_PATHS=1 inside the .env of the project.

@rlundy, modifying the system environment is not the best idea unless you know the implications of what you are doing, and is not kind of a suggestion to throw into the pot without a word of caution. Setting this in your user environment works just as well. Also, for a one-off, you can always do set COMPOSE_CONVERT_WINDOWS_PATHS=1 in the current shell (set COMPOSE_CONVERT_WINDOWS_PATHS= without the value to remove).

When working with multiple hosts, I create a cmd file for each with the required sets, and a shortcut to launch this command prompt with the command cmd /k docenv_foo.cmd. The /k tells cmd to run the file and then stay interactive. If you throw the title command into this initialization script, the window title will identify the environment.

Oh, @rlundy, let's 😄 and be ☺️ here, and be 👨‍🏫 👩‍🏫 to each other and learn from each other, so please explain why do you think preferring the user over the system environment is a 👎 . At the very least, it should not look like trading a 👎 for a 👎 on this professional site! As for the batch files setting environments, Microsoft themselves think it's a 👍 , and this is exactly how they set up different build environments with multiple Visual Studios, compilers and simulators: shortcuts with all the various correct environment initialization .cmd files invoked with the /k switch. You open the console with a click, and all are all set. On Windows we are playing on their course, and I see nothing wrong adapting their best practices that I personally do not see as incorrect either. To me it's simple, I am setting not only this variable, but also DOCKER_HOST, DOCKER_TLS and others when needed, and also a command like title Docker on FOOHOST in such a .cmd file helps identify the console window when there are multiple hosts and multiple environments open at the same time. But I am very far from claiming to be an expert or an authority, and am genuinely interested why do you think this is a 👎 workflow. Cheers 🥂 !

@kkm000 I feel like somebody who doesn't understand the implications of modifying a system environment variable is probably not the kind of person who's going to (a) be on Github at all, and (b) know anything about Docker, and (c) be interested in an issue called "docker-compose bind mount docker.sock not a valid Windows path".

As everyone, $Env:COMPOSE_CONVERT_WINDOWS_PATHS=1 is ok as workaround.

But this is clearly a bug. docker-compose is meant to remove the "it works on my machine syndrome" and having to an set environment variable to make it works contradict this goal.

But this is clearly a bug. docker-compose is meant to remove the "it works on my machine syndrome" and having to an set environment variable to make it works contradict this goal.

Completely agree!

Before finding this workaround, I had to ditch Docker for Windows, and started to use a staging Linux guest with VirtualBox, with a shared folder to my development Windows computer.

It worked like a charm. Thanks a lot!

commented

This worked for me.
export COMPOSE_CONVERT_WINDOWS_PATHS=1
docker-compose down && docker-compose up -d

using export or $Env doesnt work for me. I am on the latest version of Docker for Windows.

I had to use export COMPOSE_CONVERT_WINDOWS_PATHS=1, added extra "/" to look like this "//var/run/docker.sock:/tmp/docker.sock:ro" and then run docker-compose up -d in git bash

$Env:COMPOSE_CONVERT_WINDOWS_PATHS=1 not working for me.
Windows 10 and Docker 18.06.0-ce-win72 (19098)

@thiagosoeiro try it in git bash instead. Windows Powershell option didnt work for me either

@subrato30 none of the options is working for me. :(
dunno what to do

@thiagosoeiro as mentioned in my previous comment, try uninstalling d4w 18.x and backing off to the latest 17.x version. You'll have to tell it not to update itself, or you'll be back where you started.

Is there any update on this? Having serious issues with many workloads on Windows.

Docker 18.06.1-ce-win73 (19507) running on Windows 10 giving error:

Cannot create container for service portainer: b'Mount denied:\nThe source path "\\\\var\\\\run\\\\docker.sock:/var/run/docker.sock"\nis not a valid Windows path'

Trying to get Traefik to work, I was able to set "COMPOSE_CONVERT_WINDOWS_PATHS=1" in a ".env" file (same directory location as docker-compose.yml) and "/var/run/docker.sock:/var/run/docker.sock" as a volume in the docker-compose.yml file.

Loading the volume was failing until I added the .env file with the property as mentioned above.

Win10

Docker for Windows:
Version 18.06.0-ce-win72 (19098)
Channel: stable
5513f91

Docker Compose:
docker-compose version 1.22.0, build f46880fe

So is this bug just marked as, "we don't care".....I'm on Stable Docker [18.06.1-ce-win73 (19507)] and still having this issue, why do I have to set this variable on Windows? If its just a stupid workaround only for windows, why is it not default?

I have been trying to get Traefik to work for days now and finally discovered this stupid ENV variable.... Please change and add automated regression testing to ensure it does not come back.

The fix is already available on the edge channel, and will be on the stable channel once the next version of Docker (18.09.0) reaches GA. If you want to stay on the stable channel but still want the fix, you can also download the latest Compose RC.

Using this metioned solutions in last docker version in windows 10:
$Env:COMPOSE_CONVERT_WINDOWS_PATHS=1
docker-compose down && docker-compose up -d
the deployment works fine.

commented

Hey folks,

Compose maintainer here - can someone try the following versions [1] of Compose: 1.18.0, 1.19.0, 1.20.1 against Docker For Windows 18.03.0-ce and report if they all exhibit the problem? That'd help a lot with pinpointing the issue.

Thanks!

[1] you can download the .exe files from https://github.com/docker/compose/releases

@shin-

Hey Shin,

I've tried it and I'm still getting the same error.
Docker version 18.06.1-ce, build e68fc7a
docker-compose version 1.23.1, build b02f1306

** Windows 10 Pro

I tried using --privileged flag as well as the pipe for socket. But does not seem to be working for me , my test dsl is just

pipeline {
    agent {
        docker { image 'node:7-alpine' }
    }
    stages {
        stage('Test') {
            steps {
                sh 'node --version'
            }
        }
    }
}

But I get this

Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/test-pipeline
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . node:7-alpine
/var/jenkins_home/workspace/test-pipeline@tmp/durable-8d0344e5/script.sh: 2: /var/jenkins_home/workspace/test-pipeline@tmp/durable-8d0344e5/script.sh: docker: not found
[Pipeline] sh
+ docker pull node:7-alpine
/var/jenkins_home/workspace/test-pipeline@tmp/durable-dbf75ea1/script.sh: 2: /var/jenkins_home/workspace/test-pipeline@tmp/durable-dbf75ea1/script.sh: docker: not found
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 127
Finished: FAILURE

This is how I am running jenkins
docker run --privileged -it -e -d -p 8080:8080 -v //./pipe/docker_engine://./pipe/docker_engine -v C:\Users\mithuns\jenkins_config:/var/jenkins jenkins/jenkins:lts

PS C:\WINDOWS\system32> docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:21:34 2018
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:29:02 2018
  OS/Arch:          linux/amd64
  Experimental:     true

And docker-compose

PS C:\WINDOWS\system32> docker-compose version
docker-compose version 1.22.0, build f46880fe
docker-py version: 3.4.1
CPython version: 3.6.6
OpenSSL version: OpenSSL 1.0.2o  27 Mar 2018

Hi,
I am on windows machine and running the Git Bash that executes linux script which spawns docker (and in there there is another docker).

In order to make this work I need to setup export COMPOSE_CONVERT_WINDOWS_PATHS=1
and then the path must look like this: (the two brackets in beginning cost me a lot of hours)
-v //var/run/docker.sock:/var/run/docker.sock

To work on both - windows-git-bash and linux I used this workaround

docker_socket=/var/run/docker.sock:/var/run/docker.sock
cat /etc/os-release || docker_socket=//var/run/docker.sock:/var/run/docker.sock
cat /etc/os-release || export COMPOSE_CONVERT_WINDOWS_PATHS=1
docker run -v $docker_socket myservice
commented

@LiborVilimekMassive, //var/run/docker.sock://var/run/docker.sock will work on most GNU/Linux OSs too. Hence, you don't need to define it twice (with and without the leading slashes).

Still having the error here, on Windows Server 2019 (10.0.17763 N/A Build 17763)

PS C:\Users\Administrator\docker\traefil> docker-compose version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.0.2q  20 Nov 2018
PS C:\Users\Administrator\docker\traefil> docker version
Client: Docker Engine - Enterprise
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        f660560464
 Built:             07/25/2019 20:59:52
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Enterprise
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.12.5
  Git commit:       f660560464
  Built:            07/25/2019 20:57:41
  OS/Arch:          windows/amd64
  Experimental:     true

None of the COMPOSE_CONVERT_WINDOWS_PATHS (with cmd, powershell, in an .env) tricks works.

Still getting

PS C:\Users\Administrator\docker\traefil> docker-compose up -d reverse-proxy
Creating traefil_reverse-proxy_1 ... error

ERROR: for traefil_reverse-proxy_1  Cannot create container for service reverse-proxy: invalid volume specification: '/var/run/docker.sock:/var/run/docker.sock:rw'

With this docker-compose.yml:

version: '3.4'

services:
  reverse-proxy:
    image: traefik # The official Traefik docker image
    command: --api --docker # Enables the web UI and tells Traefik to listen to docker
    ports:
      - "80:80"     # The HTTP port
      - "8080:8080" # The Web UI (enabled by --api)
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:rw

And this .env :

COMPOSE_CONVERT_WINDOWS_PATHS=1

Replacing --privileged with --user root works with /var/run/docker.sock:

docker run --user root -it -e -d -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock -v C:\Users\mithuns\jenkins_config:/var/jenkins jenkins/jenkins:lts

$Env:COMPOSE_CONVERT_WINDOWS_PATHS=1 not working for me.
Windows 10 and Docker 18.06.0-ce-win72 (19098)

Did you solve it?

@tehffs Were you able to find a workaround? I am trying to do the same thing on Windows Server 2019 with Traefik

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked