Bug: Running inside container when mounting /var/run/docker. sock in container does not work

[Arnaud Clauss]

Describe the bug

My test does not work:

tests_integration/utils/custom_elasticsearch_container.py:84: in start
    super().start()
../env/lib/python3.10/site-packages/testcontainers/core/container.py:87: in start
    Reaper.get_instance()
../env/lib/python3.10/site-packages/testcontainers/core/container.py:188: in get_instance
    Reaper._instance = Reaper._create_instance()
../env/lib/python3.10/site-packages/testcontainers/core/container.py:242: in _create_instance
    raise last_connection_exception
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
cls = <class 'testcontainers.core.container.Reaper'>
    @classmethod
    def _create_instance(cls) -> "Reaper":
        logger.debug(f"Creating new Reaper for session: {SESSION_ID}")
    
        Reaper._container = (
            DockerContainer(c.ryuk_image)
            .with_name(f"testcontainers-ryuk-{SESSION_ID}")
            .with_exposed_ports(8080)
            .with_volume_mapping(c.ryuk_docker_socket, "/var/run/docker.sock", "rw")
            .with_kwargs(privileged=c.ryuk_privileged, auto_remove=True)
            .with_env("RYUK_RECONNECTION_TIMEOUT", c.ryuk_reconnection_timeout)
            .start()
        )
        wait_for_logs(Reaper._container, r".* Started!")
    
        container_host = Reaper._container.get_container_host_ip()
        container_port = int(Reaper._container.get_exposed_port(8080))
    
        last_connection_exception: Optional[Exception] = None
        for _ in range(50):
            try:
                Reaper._socket = socket()
>               Reaper._socket.connect((container_host, container_port))
E               ConnectionRefusedError: [Errno 111] Connection refused
../env/lib/python3.10/site-packages/testcontainers/core/container.py:228: ConnectionRefusedError

To Reproduce

Run your test on linux in a container while mounting /var/run/docker. sock without specifying DOCKER_HOST.

Runtime environment

Docker image: python:3.10
Testcontainers version: 4.4.0

Source of the bug

Reaper._container.get_container_host_ip() does not return the correct value.

In docker_client.py:

    def host(self) -> str:
        """
        Get the hostname or ip address of the docker host.
        """
        # https://github.com/testcontainers/testcontainers-go/blob/dd76d1e39c654433a3d80429690d07abcec04424/docker.go#L644
        # if os env TC_HOST is set, use it
        host = os.environ.get("TC_HOST")
        if not host:
            host = os.environ.get("TESTCONTAINERS_HOST_OVERRIDE")
        if host:
            return host
        try:
            url = urllib.parse.urlparse(self.client.api.base_url)

        except ValueError:
            return None
        if "http" in url.scheme or "tcp" in url.scheme:
            return url.hostname
        if inside_container() and ("unix" in url.scheme or "npipe" in url.scheme):
            ip_address = default_gateway_ip()
            if ip_address:
                return ip_address
        return "localhost"

This method return localhost because the client api base_url is http+docker://localhost/v1.41/containers/create
So the first condition is true, and the method return localhost.

It should test if we are inside a container and if true, get the default_gateway_ip().

Furthermore this method so throw an error if the subprocess return an error. For now, it just return None if the command ip is not installed.

def default_gateway_ip() -> str:
    """
    Returns gateway IP address of the host that testcontainer process is
    running on

    https://github.com/testcontainers/testcontainers-java/blob/3ad8d80e2484864e554744a4800a81f6b7982168/core/src/main/java/org/testcontainers/dockerclient/DockerClientConfigUtils.java#L27
    """
    cmd = ["sh", "-c", "ip route|awk '/default/ { print $3 }'"]
    try:
        process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        ip_address = process.communicate()[0]
        if ip_address and process.returncode == 0:
            return ip_address.decode("utf-8").strip().strip("\n")
    except subprocess.SubprocessError:
        return None

Last weird comportement in get_exposed_port:

    def get_exposed_port(self, port: int) -> str:
        mapped_port = self.get_docker_client().port(self._container.id, port)
        if inside_container():
            gateway_ip = self.get_docker_client().gateway_ip(self._container.id)
            host = self.get_docker_client().host()

            if gateway_ip == host:
                return port
        return mapped_port

I don't understand the condition if we are inside a container, if the host is the same as the gateway ip we should return the mapped_port.
In this case, we are in a container, so using the gateway_ip as host we should use the mapped_port to connect to the mapped port on the host.

[David Ankin]

this has not worked since #388 - i do not know how to test it so it has not been reverted in any meaningful way but you can discover more of these threads about the dind issue here #517 including, ultimately the workaround - #475 (comment)

I do think this project should fix itself but im going to close this issue because i dont think it differs from the other issues. feel free to correct if this is not the case.