aio-libs / yarl

Yet another URL library

Home Page:https://yarl.aio-libs.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

URL.with_query can not escape slash

whg517 opened this issue · comments

Describe the bug

The query content in the URL.with_query generated by yarl is inconsistent with that generated by urllib.parse.urlencode. The former does not encode the characters in the argument. Like ://

To Reproduce

yarl

from yarl import URL
url = URL('https://example.com')
url.with_query({'redirect_url': 'https://example.com'})
URL('https://example.com/?redirect_url=https://example.com')

urllib.parse

from urllib import parse
schema, netloc, url, params, query, fragment = parse.urlparse('https://example.com')
query = parse.urlencode({'redirect_url': 'https://example.com'})
parse.urlunparse((schema, netloc, url, params, query, fragment))
'https://example.com?redirect_url=https%3A%2F%2Fexample.com'

the url query is different.

Expected behavior

I'm not sure if Yarl did this in accordance with the URL specification, but the parameters of Query are not fully encoded, which will cause the target server to return invalid parameters of my request. Because when I use OpenID Connect, redirect_url has to be encode.

So, I want yarl.with_query to fully encode the parameter content. Or control with a parameter, such as strict. Or add safe parameter, pass safe content like urllib.parse.urlencode. Or if someone tells me that there is no specification in the URL specification, I will give up using yarl.with_query in this part of the logic.

Thanks.

Logs/tracebacks

no

Python Version

python --version
Python 3.10.0

multidict Version

python -m pip show multidict
Name: multidict
Version: 5.2.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/kevin/.virtualenvs/fastapi-keykloak-ztrS_yJ1/lib/python3.10/site-packages
Requires: 
Required-by: yarl

yarl Version

python -m pip show yarl

Name: yarl
Version: 1.7.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/kevin/.virtualenvs/fastapi-keykloak-ztrS_yJ1/lib/python3.10/site-packages
Requires: idna, multidict
Required-by:

OS

Debian

Additional context

No response

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct

I'm having this exact issue, not sure why this was closed.

And to add a bit of extra information, the URL constructor also has this problem.

yarl

>>> from yarl import URL
>>> URL('https://example.com/?query=%2F')
URL('https://example.com/?query=/')

urllib.parse

>>> from urllib import parse
>>> parse.urlunparse(parse.urlparse('https://example.com/?query=%2F'))
'https://example.com/?query=%2F'

This is breaking redirects when using aiohttp (if the server expects to receive the %2F as it was in the Location header).