Request is fired to packages.httpie.io on every invocation of `http`
thenbe opened this issue · comments
A request is fired to packages.httpie.io on every invocation of http. Not sure if this related to update notifications (#1136). If so, then it seems overkill to check for updates on every single invocation.
You can see and/or block this request using something like opensnitch (linux) or littlesnitch (mac).
Checklist
- I've searched for similar issues.
- I'm using the latest version of HTTPie.
Minimal reproduction code and steps
- Run
http example.com - Use
opensnitch/littlesnitch(links above) to view request topackages.httpie.io - Again, run
http example.com - Again, use
opensnitch/littlesnitch(links above) to view request topackages.httpie.io - repeat...
Current result
Unnecessary/excessive requests to packages.httpie.io.
…
Expected result
No unnecessary/excessive requests to packages.httpie.io.
…
Debug output
Please re-run the command with --debug, then copy the entire command & output and paste both below:
$ http --debug example.com
HTTPie 3.2.2
Requests 2.31.0
Pygments 2.15.1
Python 3.10.12 (main, Jun 6 2023, 22:43:10) [GCC 12.3.0]
/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/bin/python3.10
Linux 6.1.49
<Environment {'apply_warnings_filter': <function Environment.apply_warnings_filter at 0x7f1fdd949510>,
'args': Namespace(),
'as_silent': <function Environment.as_silent at 0x7f1fdd9493f0>,
'colors': 256,
'config': {'default_options': []},
'config_dir': PosixPath('/home/nbe/.config/httpie'),
'devnull': <property object at 0x7f1fdd92b880>,
'is_windows': False,
'log_error': <function Environment.log_error at 0x7f1fdd949480>,
'program_name': 'http',
'quiet': 0,
'rich_console': <functools.cached_property object at 0x7f1fdd935780>,
'rich_error_console': <functools.cached_property object at 0x7f1fdd9373d0>,
'show_displays': True,
'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
'stderr_isatty': True,
'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
'stdin_encoding': 'utf-8',
'stdin_isatty': True,
'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
'stdout_encoding': 'utf-8',
'stdout_isatty': True}>
<PluginManager {'adapters': [],
'auth': [<class 'httpie.plugins.builtin.BasicAuthPlugin'>,
<class 'httpie.plugins.builtin.DigestAuthPlugin'>,
<class 'httpie.plugins.builtin.BearerAuthPlugin'>],
'converters': [],
'formatters': [<class 'httpie.output.formatters.headers.HeadersFormatter'>,
<class 'httpie.output.formatters.json.JSONFormatter'>,
<class 'httpie.output.formatters.xml.XMLFormatter'>,
<class 'httpie.output.formatters.colors.ColorFormatter'>]}>
>>> requests.request(**{'auth': None,
'data': RequestJSONDataDict(),
'headers': <HTTPHeadersDict('User-Agent': b'HTTPie/3.2.2')>,
'method': 'get',
'params': <generator object MultiValueOrderedDict.items at 0x7f1fdd4033e0>,
'url': 'http://example.com'})
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 597962
Cache-Control: max-age=604800
Content-Encoding: gzip
Content-Length: 648
Content-Type: text/html; charset=UTF-8
Date: Mon, 04 Sep 2023 15:08:27 GMT
Etag: "3147526947"
Expires: Mon, 11 Sep 2023 15:08:27 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dcb/7F83)
Vary: Accept-Encoding
X-Cache: HIT
<!doctype html>
<html>
<head>
<title>Example Domain</title>
<meta charset="utf-8" />
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style type="text/css">
body {
background-color: #f0f0f2;
margin: 0;
padding: 0;
font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Aria
l, sans-serif;
}
div {
width: 600px;
margin: 5em auto;
padding: 2em;
background-color: #fdfdff;
border-radius: 0.5em;
box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
}
a:link, a:visited {
color: #38488f;
text-decoration: none;
}
@media (max-width: 700px) {
div {
margin: 0 auto;
width: auto;
}
}
</style>
</head>
<body>
<div>
<h1>Example Domain</h1>
<p>This domain is for use in illustrative examples in documents. You may use this
domain in literature without prior coordination or asking for permission.</p>
<p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>Additional information, screenshots, or code examples
…
Thanks for the report. This is supposed to happen infrequently. Will fix.
Still happening. Very annoying testing a proxy server getting an extra request on every request.
Still happening on the latest version (3.2.2).
While having a periodic update check may be useful, perhaps there could be an option in the config file to disable it entirely?
Either way, the additional request on every invocation makes it significantly harder to use for testing a proxy setup.
Any solution would be appreciated.
I tried to reproduce but without success. Can, everyone in this thread, post the debug output so that we can establish the common factor?
I can reproduce this still on 3.2.2. @Ousret To help you reproduce this, can you share what method you're using to detect the request? (I mentioned a couple of example methods in my original post)
@Ousret running mitmproxy listening on port 8080.
Running HTTPS_PROXY=http://localhost:8080 https --verify=no get example.com once results in
Running the same command twice results in
No matter what kind of request I make with HTTPie, it always fires off a check to packages.httpie.io every time.
Getting the debug output:
Hopefully this is useful.
Found the origin of this situation. A fix is available in #1531
To preview it, run the following pip install "git+https://github.com/Ousret/HTTPie.git@feature-tryout-niquests" -U
Can you test the patch and report back? If it solved your issue or not.
@Ousret well, it's certainly different.
It seems that HTTPie is still issuing the requests, but now doesn't trust the proxy's cert (even with --verify=no).
HTTPie initially set verif=False (v3.x) for fetching latest versions information from the server, the given PR let it (cert verif) by default. If you don't use mitmproxy and the request succeed, it will stop doing it.
Nonetheless, I applied an additional measure that (i) consider a failure sufficient to stop trying (ii) forward the verify argument to it.
You can try to install the patch again, it should be OK now.