URL.with_query can not escape slash
whg517 opened this issue · comments
Describe the bug
The query content in the URL.with_query
generated by yarl is inconsistent with that generated by urllib.parse.urlencode
. The former does not encode the characters in the argument. Like ://
To Reproduce
yarl
from yarl import URL
url = URL('https://example.com')
url.with_query({'redirect_url': 'https://example.com'})
URL('https://example.com/?redirect_url=https://example.com')
urllib.parse
from urllib import parse
schema, netloc, url, params, query, fragment = parse.urlparse('https://example.com')
query = parse.urlencode({'redirect_url': 'https://example.com'})
parse.urlunparse((schema, netloc, url, params, query, fragment))
'https://example.com?redirect_url=https%3A%2F%2Fexample.com'
the url query is different.
Expected behavior
I'm not sure if Yarl did this in accordance with the URL specification, but the parameters of Query are not fully encoded, which will cause the target server to return invalid parameters of my request. Because when I use OpenID Connect, redirect_url
has to be encode.
So, I want yarl.with_query
to fully encode the parameter content. Or control with a parameter, such as strict
. Or add safe
parameter, pass safe content like urllib.parse.urlencode
. Or if someone tells me that there is no specification in the URL specification, I will give up using yarl.with_query
in this part of the logic.
Thanks.
Logs/tracebacks
no
Python Version
❯ python --version
Python 3.10.0
multidict Version
❯ python -m pip show multidict
Name: multidict
Version: 5.2.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/kevin/.virtualenvs/fastapi-keykloak-ztrS_yJ1/lib/python3.10/site-packages
Requires:
Required-by: yarl
yarl Version
❯ python -m pip show yarl
Name: yarl
Version: 1.7.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/kevin/.virtualenvs/fastapi-keykloak-ztrS_yJ1/lib/python3.10/site-packages
Requires: idna, multidict
Required-by:
OS
Debian
Additional context
No response
Code of Conduct
- I agree to follow the aio-libs Code of Conduct
I'm having this exact issue, not sure why this was closed.
And to add a bit of extra information, the URL constructor also has this problem.
yarl
>>> from yarl import URL
>>> URL('https://example.com/?query=%2F')
URL('https://example.com/?query=/')
urllib.parse
>>> from urllib import parse
>>> parse.urlunparse(parse.urlparse('https://example.com/?query=%2F'))
'https://example.com/?query=%2F'
This is breaking redirects when using aiohttp (if the server expects to receive the %2F
as it was in the Location header).