humble

HTTP Headers Analyzer

"A journey of a thousand miles begins with a single step. - Lao Tzu"

"And if you don't keep your feet, there's no knowing where you might be swept off to. - Bilbo Baggins"

Features
Screenshots
Installation & Update
Usage
Advanced Usage
Checks: Missing Headers
Checks: Fingerprint Headers
Checks: Deprecated Headers and Insecure Values
Checks: Empty Values
Guidelines included
To-Do
Further Reading
Contribute
Acknowledgements
License

Features

✔️ 14 checks of missing HTTP response headers.
✔️ 1005 checks of fingerprinting through HTTP response headers.
✔️ 87 checks of deprecated HTTP response headers/protocols or with insecure/wrong values.
✔️ TLS/SSL checks: requires https://testssl.sh/ and Unix machine!.
✔️ Browser compatibility check for enabled security headers.
✔️ Two types of analysis: brief and detailed, along with HTTP response headers.
✔️ Export of analysis to HTML5, JSON, PDF 1.4 and TXT.
✔️ The analysis includes dozens of references, official documentation and technical articles.
✔️ i10n: show the analysis, messages and nearly all errors in English or Spanish.
✔️ Saves each analysis, showing (at the end) the improvements or deficiencies in relation to the last one.
✔️ Shows analysis statistics: either against a specific URL or all of them.
✔️ Shows fingerprint statistics: either against a specific term or the Top 20.
✔️ Code reviewed via Flake8, SonarLint and Sourcery.
✔️ Tested, one by one, on thousands of URLs.
✔️ Fully tested and working on Windows (10 20H2 - 19042.985) and Linux (Kali 2021.1).
✔️ Almost all the code under one of the most permissive licenses: MIT.
✔️ Regularly updated.
✔️ Technical resource in OWASP Secure Headers Project and Kali Linux (this should be the latest Release).
✔️ Developed entirely in my spare time, no strings attached: feel free to try it out and integrate it into your projects!.
✔️ And with the approval of an AI :)!.

Screenshots

.: (Windows) - Brief analysis.

.: (Linux) - Brief analysis and retrieved HTTP headers.

.: (Linux) - Detailed analysis in Spanish.

.: (Linux) - TLS/SSL checks (requires https://testssl.sh/ and Unix machine, for now!).

Options used: -f -g -p -U -s --hints

.: (Linux) - List of HTTP fingerprint headers based on a specific term.

.: (Windows) - Detailed analysis saved as PDF. Example.

.: (Linux) - Detailed analysis saved as HTML. Example.

.: (Linux) - Brief analysis saved as JSON. Example.

.: (Linux) - Analysis history file: Date, URL, Missing, Fingerprint, Deprecated/Insecure, Empty headers & Total warnings (the four previous totals).

.: (Linux) - Statistics of the analysis performed against a specific URL.

.: (Linux) - Statistics of the analysis performed against all URLs.

Installation & Update

NOTE: Python 3.9 or higher is required.

# install python3 and python3-pip if not exist
(Windows) https://www.python.org/downloads/windows/
(Linux) if not installed by default, install them via, e.g. Synaptic, apt, dnf, yum ...

# install git
(Windows) https://git-scm.com/download/win
(Linux) https://git-scm.com/download/linux

# clone the repository
$ git clone https://github.com/rfc-st/humble.git

# change the working directory to humble
$ cd humble

# install the requirements
$ pip3 install -r requirements.txt

# update humble (every week, inside humble's working directory)
$ git pull

# or download the latest release (every four to five weeks)
https://github.com/rfc-st/humble/releases

Usage

(Windows) $ py humble.py
(Linux)   $ python3 humble.py

usage: humble.py [-h] [-a] [-b] [-df] [-e [PATH]] [-f [TERM]] [-g] [-l {es}] [-o {html,json,pdf,txt}] [-r] [-u URL] [-v]

humble (HTTP Headers Analyzer) - https://github.com/rfc-st/humble

options:
  -h, --help              show this help message and exit
  -a                      show statistics of the performed analysis (will be global if '-u URL' is omitted)
  -b                      show a brief analysis (if omitted, a detailed one will be shown)
  -df                     do not follow redirects (if omitted, the last redirection will be the one analyzed)
  -e [PATH]               show TLS/SSL checks; requires the PATH of testssl.sh (https://testssl.sh/) and Unix (for now!)
  -f [TERM]               show fingerprint statistics (will be the Top 20 if "TERM", e.g. "Google", is omitted)
  -g                      show guidelines for securing popular web servers/services
  -l {es}                 the language for displaying analyses, messages and errors (if omitted it will be in English)
  -o {html,json,pdf,txt}  save analysis to 'URL_headers_yyyymmdd.ext' file (.json files will contain a brief analysis)
  -r                      show full HTTP response headers and a detailed analysis
  -u URL                  schema and URL to analyze. E.g. https://google.com
  -v, --version           show the version of this tool and check for updates

Advanced Usage

.: (Linux) - Show only the analysis summary.

$ python3 humble.py -u https://www.spacex.com | grep -A 8 "\!." | sed $'1i \n'

.: (Windows, in Spanish) - show only the analysis summary (PowerShell >= 7 required).

$ py humble.py -u https://www.spacex.com -l es | Select-String -Pattern '!.' -Context 1,8 -NoEmphasis

.: (Linux) - Show only the URL, date and analysis summary.

$ python3 humble.py -u https://www.spacex.com | grep -A7 -E "0. Info|\!." | grep -v "^\[1\." | sed 's/[--]//g' | sed -e '/./b' -e :n -e 'N;s/\n$//;tn' | sed $'1i \n'

.: (Linux) - Show only the deprecated headers/protocols and insecure values.

$ python3 humble.py -u https://www.spacex.com | sed '/3. /,/4. /!d' | sed '$d' | sed $'1i \n'

.: (Linux) - Check for HTTP client errors (4XX).

$ python3 humble.py -u https://block.fiverr.com | grep -A1 -B5 'Note : \|Nota : ' --color=never

.: (Linux) - Analyze multiple URLs and save the results as PDFs.

$ datasets=('https://facebook.com' 'https://www.microsoft.com' 'https://www.spacex.com'); for dataset in "${datasets[@]}"; do python3 humble.py -u "$dataset" -o pdf; done

Checks: Missing Headers

Show / Hide


`Cache-Control`	`Clear-Site-Data`	`Content-Type`
`Content-Security-Policy`	`Cross-Origin-Embedder-Policy`	`Cross-Origin-Opener-Policy`
`Cross-Origin-Resource-Policy`	`NEL`	`Permissions-Policy`
`Referrer-Policy`	`Strict-Transport-Security`	`X-Content-Type-Options`
`X-Frame-Options`	`X-Permitted-Cross-Domain-Policies`

Checks: Fingerprint headers

Check this file.

Checks: Deprecated headers/protocols and insecure values

Check this file.

Checks: Empty values

Any HTTP response header.

Guidelines included to enable security HTTP headers

Amazon AWS
Apache HTTP Server
Cloudflare
MaxCDN
Microsoft Internet Information Services
Nginx

To-do

Add more header/value checks (only security-oriented)
Google Style Python Docstrings and maybe documentation via Sphinx.

Contribute

Report a Bug.
Create a Feature request.
Report a Security Vulnerability.
Send me an email with your suggestions!: rafael.fcucalon@gmail.com

Thanks for your time!! :).

Acknowledgements

Azathothas for reporting this bug.
bulaktm for this suggestion.
David, for believing in the usefulness of this tool.
Eduardo, for making possible the first Demo and for the example "(Linux) - Analyze multiple URLs and save the results as PDFs".
gl4nce for this suggestion.
İDRİS BUDAK for reporting the need to this check.
manuel-sommer for this, this and this!.
stanley101music for this suggestion.

ehlewis / humble