Error in requests version 2.31.0 with urllib3 2.2.1 and 2.10 raise IncompleteRead(self._fp_bytes_read, self.length_remaining)

[pcor]

We have updated from urllib3 1.26 to urllib3 2.xx and now we have same requests with this error.

lib/python3.10/site-packages/urllib3/response.py", line 833, in _raw_read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(119510 bytes read, -59755 more expected)

We have moved to version 1.26.18 and it works correctly with the same requests. We have done several tests and there are some requests that always fail with urllib3 version 2.1 or 2.2.1 and work correctly with version 1.26.

[pcor]

I have installed an empty enviroment with conda and requests:

python 3.12.2
requests 2.31.0
urllib3 1.26.18

And we have no errors, when I install urllib3 2.1.0, the program crashes, Below is a simple test program that crashes.

python 3.12.2
requests 2.31.0
urllib3 2.1.0

import requests
from io import BytesIO


class Test():
    def __init__(self):
        self.proxy = None
        self.headers = {}
        self.headers['Connection'] = 'close'
        self.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36'
        self.headers['Accept'] = '*/*'
        self.headers['Accept-Encoding'] = 'gzip, deflate'
        self.session = requests.Session()

    def get(self, url):
        self.session.headers = self.headers
        r = self.session.get(url, proxies=self.proxy, allow_redirects=True, stream=True)
        raw_content = r.raw.read()
        content_length = len(raw_content)
        r.raw._fp = BytesIO(raw_content)
        content=r.text
        with open('test.html', 'w', encoding='utf-8') as fo:
            fo.write(content)

lurl = 'https://www.nolon.es/es/es/comprar?MainUseId=&Invest=False&ClassId=&HighlightTypeId=&PropertyTypeId=&DistrictId=30&CountyId=&AreaRange.MinValue=&AreaRange.MaxValue=&PriceRange.MinValue=0&PriceRange.MaxValue=&CampaignId=&PropertyId=&ResultsPerPage=27&SortOrderBy=DescendingDate&SortOrderBy=DescendingDate'
Test().get(lurl)`

This is the error I get:

traceback (most recent call last):
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/urllib3/response.py", line 712, in _error_catcher
yield
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/urllib3/response.py", line 833, in _raw_read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(42084 bytes read, -21042 more expected)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/requests/models.py", line 816, in generate
yield from self.raw.stream(chunk_size, decode_content=True)
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/urllib3/response.py", line 934, in stream
data = self.read(amt=amt, decode_content=decode_content)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/urllib3/response.py", line 877, in read
data = self._raw_read(amt)
^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/urllib3/response.py", line 811, in _raw_read
with self._error_catcher():
File "/opt/miniconda3/envs/test/lib/python3.12/contextlib.py", line 158, in exit
self.gen.throw(value)
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/urllib3/response.py", line 729, in _error_catcher
raise ProtocolError(f"Connection broken: {e!r}", e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(42084 bytes read, -21042 more expected)', IncompleteRead(42084 bytes read, -21042 more expected))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/datos1/test/test.py", line 28, in
Test().get(lurl)
File "/home/datos1/test/test.py", line 23, in get
content=r.text
^^^^^^
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/requests/models.py", line 923, in text
if not self.content:
^^^^^^^^^^^^
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/requests/models.py", line 899, in content
self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/test/lib/python3.12/site-packages/requests/models.py", line 818, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(42084 bytes read, -21042 more expected)', IncompleteRead(42084 bytes read, -21042 more expected))

[Ian Stapleton Cordasco]

You must never call r.text or r.content after consuming the underlying response data. That's you're actual bug. I suspect requests and urllib3 were silently returning nothing but you're code has always been the issue