xmunoz / sodapy

Python client for the Socrata Open Data API

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can't access from behind proxy

asarchami opened this issue · comments

I'm trying to use python behind proxy and I get this error:

HTTPSConnectionPool(host='hostname', port=443): Max retries exceeded with url: /api/views/hdsc-ubkz.json (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fc509240110>: Failed to establish a new connection: [Errno 113] No route to host',))

Is there a way to specify proxy settings with sodapy?

sodapy uses the python library requests to make HTTP requests. According to the requests documentation, you can pass proxy settings as an argument (which would require modifying sodapy), or you can set environment variables, which should work without modifying sodapy. Can you try setting the environment variables? The example they give is for mac/linux, but you can do it in windows with a slightly different syntax that I don't recall off-hand.

Any luck with @timwis's suggestion, @asarchami? I feel like the environment variable is a good compromise, but if that doesn't work, it shouldn't be too hard to set a proxy argument in the client that is transparently passed to the requests library.

Environment variables didn't work for me. I haven't try sodapy since but according to my past experience I have to pass proxy settings as an argument. My working environment and proxy is kind of weird.

I busted out tcpdump and did manage to get this working with a proxy through environment variables. Something important to note is that sodapy connects via https by default, so the environment variable needs to be something like os.environ["HTTPS_PROXY"] = "https://13.74.23.178:3128", otherwise you will experience the errors above. If your proxy doesn't support https, you can set the client to connect with http by configuring your own adapter.

from requests.adapters import HTTPAdapter
myadapter = {}
myadapter["prefix"] = "http://"
adapter = HTTPAdapter()
myadapter["adapter"] = adapter
client = Socrata(DOMAIN, APPTOKEN, session_adapter=myadapter)

Can you try something like what I described above and let me know how it goes?

Edit: I should probably add something to the documentation about this...