jaeles-project / gospider

Gospider - Fast web spider written in Go

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Proxy settings not fully honored

ocervell opened this issue · comments

It seems that proxies are not honored, by looking at Wireshark traffic I see some requests not going through any proxy.

I think this is related to gocolly/colly#392

We probably need to disable keep alives like:

c.WithTransport(&http.Transport{
  DisableKeepAlives: true,
})

in the code here