Proxy support
JustMikey opened this issue · comments
Hi, I was wondering if you would consider adding an option for using the proxy for requests?
And thank you for the package, it's great :)
Thanks a lot for the kind words!
I'm not 100% sure of what you mean by using the proxy for request? If you mean that Rackla should be able to proxy incoming requests to another server then I think that would be a great idea which I'll definitely look in to.
After reading #17, I think it would be a good start to be able to convert an incoming "Plug request" to a "Rackla request". If we then just modify (at least) the url, I think you could use that functionality to proxy requests easily.
Like, take for example a situation when you have an API but it's behind the firewall so you have to make the outgoing requests go through that firewall so the proxy option would be a good thing to have. Hackney seems to support proxies so it should be easy enough to do.
So what I am offering is to add an option for the gateway to be able to send requests to the APIs via another proxy.
Aha, now I understand what you mean and yes, I can absolutely add this to Rackla. Since it's supported by Hackney it's a fairly trivial change.
I'll try to add it over the weekend!
That would be great, thank you very much :)
I've added support for proxies in develop now. I need to do some more testing and I have one more thing I want to add before I make a proper release - but if you wish to try it out before the release, just clone the latest develop!
Example usages:
%Rackla.Request{url: "http://api.ipify.org", options: %{proxy: %Rackla.Proxy{type: :socks5, host: "localhost", port: 8080}}} |> Rackla.request |> Rackla.collect"http://api.ipify.org" |> Rackla.request(proxy: %Rackla.Proxy{type: :socks5, host: "localhost", port: 8080}) |> Rackla.collectThank you very much for your work. I will try it after this weekend. Also I wanted to point out that it may make sense to add a way to use authentication for the proxy, if it's is required. I believe hackney has the proxy_auth option for that.
Username and password is already present in the Rackla.Proxy struct but I see now that it is only used for socks5 and not http tunnelling (which uses proxy_auth). I'll add support for that also.
Yeah, that would be great :)
I just pushed a new commit to develop, username and password should work on both socks5 and HTTP tunnels (connect).
It's not tested though so it would be great if you could report back if everything works as expected when you try it out !
Sure, will do :) Do you have any idea of how it could be covered with tests?
No, that's a tough one! I'm guessing it's hard to mock and actually
launching two different proxies just for unit testing feels like a heavy
dependency to include. Maybe Hackney have some tests that can be
repurposed?
Yeah I agree, I am not sure how to test it as well. I will look into it tomorrow.
That's because the Elixir string didn't work (and that's because Erlang and Elixir represents strings differently but I didn't think about that when I wrote the). Using to_charlist is probably a better approach or do you have another suggestion?
Well, converting it to an atom may be dangerous because GC doesn't collect those so it may lead to the memory leaks if somebody uses it dynamically. I tested the hackney's request function in the iex and it seems to work with the elixir strings.
It's true that they aren'g GCed but I reasoned that using a lot of dynamic proxies seemed like a very unlikely scenario :) Nevertheless, it's better to avoid them if possible.
I will double check the issue I had with Elixir strings to see what it was about.
I did the testing using a SOCKS5 proxy, are you that as well?
Well, I tried it with HTTP proxy and it did not work, but then I believe I removed the transformation from the string to an atom for the host and it did work.
Ok, I'll try both proxy types in a couple of hours and sort it all out. Thanks for the update!
Thank you!
Release as part of version 1.1.0!
Thank you so much for the package and your time :)