Compatibilty with aiohttp.ClientSession?

[Michał Lowas-Rzechonek]

Hi,

First of all, thanks for this. I was able to speed up our tests by an order of magnitude using this package.

I use it in somewhat unorthodox manner, by creating several TestClient instances for a few services I want to mock in tests, then patch aiohttp.ClientSession so that I can intercept outgoing calls made by my (non-web) application and route them to one of the TestClients.

Unfortunately, the signature of TestClient's http methods is a bit different than corresponding methods in aiohttp.ClientSession. To forward these calls, I need to frob incoming arguments before passing them to TestClient, and then wrap resulting Response object into async context managers and other shenanigans.

I guess TestClient's API was designed to match requests, not aiohttp, is that right?

If so, what do you think about adding a compatibility layer that would match aiohttp.ClientSession API?

[Jordi Masip]

Hi,

First of all, thanks for this. I was able to speed up our tests by an order of magnitude using this package.

I'm glad to hear this :)

I use it in somewhat unorthodox manner, by creating several TestClient instances for a few services I want to mock in tests, then patch aiohttp.ClientSession so that I can intercept outgoing calls made by my (non-web) application and route them to one of the TestClients.

Unfortunately, the signature of TestClient's http methods is a bit different than corresponding methods in aiohttp.ClientSession. To forward these calls, I need to frob incoming arguments before passing them to TestClient, and then wrap resulting Response object into async context managers and other shenanigans.

I guess TestClient's API was designed to match requests, not aiohttp, is that right?

Well, async-asgi-testclient is based on a previous version of Quart's Test Client and maybe this one is based on Flask. Also, it has borrowed some idea from the synchronous Starlette TestClient.

If so, what do you think about adding a compatibility layer that would match aiohttp.ClientSession API?

It's a difficult question. I'd like to keep this library as simple and small as possible and I'm not sure if this feature will be interesting for the people that wants a small library for testing ASGI web-apps.

That said, I wonder if adding this aiohttp compatiblity layer could be done just wrapping around the TestClient or it will need changes on the actual TestClient and Reponse object. I'm afraid to have to maintain two TestClients / Reponse object or increase the complexity of the current client. Could you provide more details on your implementation?

[Michał Lowas-Rzechonek]

Sure. I think it's rather incomplete, but does the job for us:

class AsyncResponse:
    def __init__(self, response: requests.Response):
        self.response = response

    async def json(self):
        return self.response.json()

    async def text(self):
        return self.response.text

    async def read(self):
        return self.response.text

    @property
    def status(self):
        return self.response.status_code

    @property
    def headers(self):
        return self.response.headers

    @property
    def reason(self):
        return self.response.reason


class AsyncRequest:
    def __init__(self, request: Awaitable[requests.Response]):
        self.request = request

    async def __aenter__(self):
        return AsyncResponse(await self.request)

    async def __aexit__(self, *args, **kwargs):
        pass

and then, for example:

@pytest.fixture
async def service_mocks(monkeypatch, event_loop):
    async with AsyncTestClient(influx_mock()) as influx_client:
        def select_client(url):
            if url.host == "influx.local":
                return influx_client

            raise KeyError(url)

        def _request(method, url, *args, **kwargs):
            url = URL(url)
            client = select_client(url)
            # massage requests arguments to match AsyncTestClient's
            kwargs["query_string"] = kwargs.pop("params", None)
            return event_loop.run_until_complete(method(client, str(url.relative()), *args, **kwargs))

        def post(*args, **kwargs):
            return _request(AsyncTestClient.post, *args, **kwargs)

        def get(*args, **kwargs):
            return _request(AsyncTestClient.get, *args, **kwargs)

        def put(*args, **kwargs):
            return _request(AsyncTestClient.put, *args, **kwargs)

        def patch(*args, **kwargs):
            return _request(AsyncTestClient.patch, *args, **kwargs)

        monkeypatch.setattr("requests.post", post)
        monkeypatch.setattr("requests.get", get)
        monkeypatch.setattr("requests.put", put)
        monkeypatch.setattr("requests.patch", patch)

        class ClientSession:
            def __init__(self, *args, **kwargs):
                pass

            def _request(self, method, url, *args, **kwargs):
                url = URL(url)
                client = select_client(url)

                # massage aiohttp.ClientSession arguments to match
                # AsyncTestClient's
                kwargs["query_string"] = kwargs.pop("params", None)

                data = kwargs.pop("data", None)
                kwargs["form" if isinstance(data, dict) else "data"] = data

                return AsyncRequest(method(client, str(url.relative()), *args, **kwargs))

            def request(self, method, url, *args, **kwargs):
                return getattr(self, method.lower())(url, *args, **kwargs)

            def get(self, *args, **kwargs):
                return self._request(AsyncTestClient.get, *args, **kwargs)

           def post(self, *args, **kwargs):
                return self._request(AsyncTestClient.post, *args, **kwargs)

            def put(self, *args, **kwargs):
                return self._request(AsyncTestClient.put, *args, **kwargs)

            def patch(self, *args, **kwargs):
                return self._request(AsyncTestClient.patch, *args, **kwargs)

            async def close(self):
                pass

        monkeypatch.setattr("aiohttp.ClientSession", ClientSession)

        yield

[Michał Lowas-Rzechonek]

What would be ideal, was some kind of AsyncTestClient.client_session() call that would return something compatible with aiohttp.ClientSession.

Compat with requests is much lower on the priority list, as I'd rather refactor our tests to be async everywhere (via pytest-asyncio) and remove direct requests usage altogether.

[Jordi Masip]

Sorry for delayed answer.

I have a few questions/comment:

1- AsyncTestClient is the same as async_asgi_testclient.TestClient?

2-

What would be ideal, was some kind of AsyncTestClient.client_session() call that would return something compatible with aiohttp.ClientSession.

I wouldn't add the client_session() in the existing TestClient but I'm ok with creating a new AiohttpTestClient that has de client_session() method. But maybe, it's easy as ClientSession class in module aiohttp_compat.py, so you can do something like:

from async_asgi_testclient import aiohttp_compat as aiohttp

async with aiohttp.ClientSession() as session:
   ....

3. This compatibility layer need the package aiohttp to be installed?

In any case, I'm still not sure if this aiohttp/requests compatibility layer should be in this repo/library or should be in a new one that uses async-asgi-testclient as a dependency.