apify / proxy-chain

Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.

Home Page:https://www.npmjs.com/package/proxy-chain

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Limiting amount of incoming requests and put them to pending state

korotovsky opened this issue · comments

Hi,

I found this library very useful even though puppeter nowadays supports page.authenticate() but for some of my proxies I don't get authentication popup so only this library is the last hope or solutions like Squid. So thanks a lot.

Meanwhile I need a bit extend it and I'm curious now how to make it in a better way, because I have proxies that are limits me with concurrent connections, let's say 5 but browser for sure generate a waaaay more. I need something like a global throttler but not a network throttler, throttler for amount of active connections. If we reached limit then all further connections are coming into queue and awaiting of competition one of the processes that was queued already.

I checked the code a bit and basically my needs would solve a simple Promise inside onConnect callback. But to add it is not that trivial. Maybe you would have a better idea?

Thanks for any feedback.

You can simply return a Promise from your prepareRequestFunction() and then the connection to the proxy will wait. Just beware that the client connection might time out if you wait too long.

Closing this since this is not a bug.