Currency fetching doesn't respect proxies

Question

Currency fetching doesn't respect proxies

rossburton opened this issue 4 years ago · comments

My work laptop has a tight VPN so to access the real internet a proxy needs to be used. However the currency script doesn't appear to use the proxies, so never actually fetches the data.

Dean Jackson · Answer 1 · Thu Apr 16 2020 04:03:01 GMT+0800 (China Standard Time)

It does use a proxy if one is set. UNIX-y tools and languages automatically use proxies specified by http_proxy and https_proxy, which Alfred sets if it's configured up correctly.

Have you configured Alfred to "Use macOS http proxy settings for scripts" in Alfred Preferences > Advanced?

Note: that only works if you've configured static proxies in System Preferences > Network (i.e. you're using "Web Proxy (HTTP)" and "Secure Web Proxy (HTTPS)".

If you're using auto discovery/configuration, it won't work.

Ross Burton · Answer 2 · Thu Apr 16 2020 04:07:19 GMT+0800 (China Standard Time)

Proxies are set in System Prefs. Export proxies is set in Alfred.

If I type "conv 10 usd" I get two results: "fetching currencies" and "set APP_KEY". APP_KEY is set in the workflow variables to my key on openexchangerates.org.

Dean Jackson · Answer 3 · Thu Apr 16 2020 04:09:29 GMT+0800 (China Standard Time)

and "set APP_KEY".

Yeah, I think that's an unrelated bug. It should still fetch the rates. Check the log file (enter conv workflow:openlog into Alfred) to see what the currency updater says. Its output isn't visible in Alfred's debugger because it runs in the background.

Ross Burton · Answer 4 · Thu Apr 16 2020 05:00:38 GMT+0800 (China Standard Time)

Well, two things.

21:08:37 convert.py:74 ERROR    [parser] unit usd is a fiat currency, but OpenExchangeRates.org API key isn't set

It really is set in the workflow variables.

Looks like the proxy isn't reaching the update checker too.

21:55:29 update.py:564 ERROR    <urlopen error timed out>
Traceback (most recent call last):
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/update.py", line 557, in <module>
    check_update(repo, version, prereleases)
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/update.py", line 466, in check_update
    dls = get_downloads(repo)
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/update.py", line 410, in get_downloads
    js = wf().cached_data(key, _fetch, max_age=60)
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/workflow.py", line 1701, in cached_data
    data = data_func()
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/update.py", line 405, in _fetch
    r = web.get(url)
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/web.py", line 591, in get
    stream=stream)
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/web.py", line 579, in request
    return Response(req, stream)
  File "/Users/ross/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.091084E9-1F3C-4F4C-95B8-A8CA7A79843A/workflow/web.py", line 223, in __init__
    self.raw = urllib2.urlopen(request)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1241, in https_open
    context=self._context)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1198, in do_open
    raise URLError(err)
URLError: <urlopen error timed out>
21:55:29 background.py:285 ERROR    [__workflow_update_check] command failed with status 1

Dean Jackson · Answer 5 · Thu Apr 16 2020 05:33:36 GMT+0800 (China Standard Time)

It really is set in the workflow variables.

Yes. I told you: it's an unrelated bug. That message is triggered by there being no exchange rates available, not by APP_KEY not being set.

As you can see from my log, it's showing the same "error" but fetching the exchange rates all the same:

23:05:58 convert.py:74 ERROR    [parser] unit eur is a fiat currency, but OpenExchangeRates.org API key isn't set
23:05:58 workflow.py:2103 DEBUG    ---------- finished in 0.084s ----------
23:05:58 currency.py:137 DEBUG    [200] https://openexchangerates.org/api/latest.json?app_id=XXX
23:05:58 currency.py:144 DEBUG    [OpenExchangeRates.org] 1 USD = 127.368704 DZD
23:05:58 currency.py:144 DEBUG    [OpenExchangeRates.org] 1 USD = 18.14 NAD
23:05:58 currency.py:144 DEBUG    [OpenExchangeRates.org] 1 USD = 5.824632 GHS
...
...

Looks like the proxy isn't reaching the update checker too.

And like I also told you, Python automatically uses the proxy if it's properly configured, so you need to double-check your setup because that's where the problem is.

Open the conv Script Filter and add this at the top of the script:

echo "http_proxy=$http_proxy" >&2
echo "https_proxy=$https_proxy" >&2

Then check what's in Alfred's debugger when you run the workflow.

I'll fix the incorrect "APP_KEY is not set" error, but that isn't actually stopping the workflow downloading exchange rates.

Ross Burton · Answer 6 · Thu Apr 16 2020 18:21:48 GMT+0800 (China Standard Time)

[11:18:05.799] STDERR: Convert[Script Filter] http_proxy=
https_proxy=

'Use macOS proxy settings' is checked. Proxies are set correctly, redacted screenshot attached. I'm guessing you'll call this an Alfred bug now?

Dean Jackson · Answer 7 · Thu Apr 16 2020 18:36:11 GMT+0800 (China Standard Time)

I'm guessing you'll call this an Alfred bug now?

I'm still pretty sure it's a problem with your setup. Uncheck "Automatic Proxy Configuration" and try again.

Ross Burton · Answer 8 · Thu Apr 16 2020 18:38:40 GMT+0800 (China Standard Time)

No change (and that would be a bug in Alfred, as I'm only setting http/https proxies for applications too dumb to use the PAC).

Dean Jackson · Answer 9 · Thu Apr 16 2020 18:40:32 GMT+0800 (China Standard Time)

No change

Try restarting Alfred. It might not have picked up the changes.

and that would be a bug in Alfred, as I'm only setting http/https proxies for applications to dumb to use the PAC

Alfred can't use PAC.

Ross Burton · Answer 10 · Thu Apr 16 2020 18:46:49 GMT+0800 (China Standard Time)

No change

Try restarting Alfred. It might not have picked up the changes.

Still no change.

and that would be a bug in Alfred, as I'm only setting http/https proxies for applications to dumb to use the PAC

Alfred can't use PAC.

Or more accurately, typical shell scripts can't use PAC. I understand how PAC and friends works.

Dean Jackson · Answer 11 · Thu Apr 16 2020 19:01:54 GMT+0800 (China Standard Time)

Still no change

Then I don't know what the problem is. It works correctly on my machine. If "Web Proxy" and/or "Secure Web Proxy" are set, Alfred also sets them in the workflow environment, and the workflow also uses them.

It looks like Alfred isn't finding the settings, so I suggest you go over your setup again. Make sure you're setting them on the right interface, pressing "Apply" etc.

If that doesn't work, you'll have to set the variables yourself on any workflows that need them.

At any rate, it's not an issue with the workflow, and there's nothing I can do about it.

Or more accurately, typical shell scripts can't use PAC

Exactly.

Dean Jackson · Answer 12 · Sat Aug 01 2020 04:30:33 GMT+0800 (China Standard Time)

@rossburton Could you update to v3.7.1?

I think your issue might be fixed.

Ross Burton · Answer 13 · Fri Aug 14 2020 03:23:33 GMT+0800 (China Standard Time)

Sorry, I've left the job with the evil proxy of doom.