aivarsk / scrapy-proxies

Random proxy middleware for Scrapy

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

I am confused about the non-username-password proxy logic

tonylin75 opened this issue · comments

if proxy_user_pass:
            request.meta['proxy'] = proxy_address
            basic_auth = 'Basic ' + base64.b64encode(proxy_user_pass.encode()).decode()
            request.headers['Proxy-Authorization'] = basic_auth
else:
        log.debug('Proxy user pass not found')
        log.debug('Using proxy <%s>, %d proxies left' % (
        proxy_address, len(self.proxies)))

I am very confused here as a noob python developer. From this part of the logic in randomProxy file. It seems like if the proxy provided in the list.txt is in the http://username:password@host2:port format, then it will work by assigning proxy_address to the request, otherwise, do nothing but logging debug...

What am I missing here?

I think you can edit to get proxy working without username and password like this.
image