Exception httpx.ResponseNotRead thrown randomly when using cassettes
justanotherboy opened this issue · comments
I recently upgraded vcrpy version to 6.0.1and I started getting
httpx.ResponseNotReadexceptions randobly in my tests using cassettes. These cassettes are in the format of the version
4.2.1`.
I got these errors more in my CI pipeline than in my local environment, but still I get these exceptions at least 10% of the time, while Jenkins is about 80% of the time. Both environments bring up a docker container based on the same image, and then execute pytest
to run the unit tests.
Most of my code uses a ThreadPoolExecutor
when making HTTP Requests with httpx, so we use the sync httpx Client. We also use httpx response hooks to raise_for_status
.
Checking the code in vcrpy I ended in the part where the Response was generated in the httpx_stubs and found the read
and the close
methods are patched. During my debugging, adding a simple print("hello")
within the _from_serialized_response
function made the exception to be thrown 100% of the time.
Now, if you go to the httpx._models
where the Response constructor is you can see the Response uses read to build the content
attribute of the instance. You can also see that content
that is passed as parameter to the constructor is never set as an attribute directly, but is encoded into a stream first and then read will set the content
attribute of the Response instance.
I was checking when the httpx support was added that the function _from_serialized_response was not patching the read
method. The next commit on the httpx stubs patched read but set _content
attribute manually. Since then, the patch has been kept for this function.
Checking the httpx code and the vcrpy code I removed the patch
statements from the _from_serialized_response
function and the tests were successful 100% of the time, in my local environment and in the CI pipeline. I'm submitting the patch, and it works for me that I don't use the async httpx Client and have limited knowledge of the vcrpy project.
Please let me know if I can provide more details, and I hope I was able to make myself clear in the previous wall of text.
I couldn't open a PR, but here is the fix that worked for me.