oscardelben / rawler

Rawler is a tool that crawls the links of your website

Home Page:http://oscardelben.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Authorization header is sent even without --username and --password

agross opened this issue · comments

rawler https://nativescript.org/

fails because the request looks like this:

GET https://www.nativescript.org/ HTTP/1.1
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
User-Agent: Ruby
Authorization: Basic Og==
Connection: close
Host: www.nativescript.org

The server returns HTTP/500 only because of Authorization: Basic Og==.