Authorization header is sent even without --username and --password
agross opened this issue · comments
Alexander Groß commented
rawler https://nativescript.org/
fails because the request looks like this:
GET https://www.nativescript.org/ HTTP/1.1
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
User-Agent: Ruby
Authorization: Basic Og==
Connection: close
Host: www.nativescript.org
The server returns HTTP/500 only because of Authorization: Basic Og==
.