ExtractHosts
Extracts hosts (IP/Hostnames) from files. Hosts can be extracted from text files, PE, etc. Any file that keeps the host in plaintext without obscuring it, this should extract it.
The name came about when polling random people on the street about the idea, and they responded, "eh?". With this installed, all you need to type is 'eh' to start pulling hosts from input.
Installation
git clone https://github.com/bwall/ExtractHosts.git
cd ExtractHosts
sudo python setup.py install
Examples
The following are just some example usages help
bwall@research:~$ eh -h
usage: ./eh [-h] [-v] [-r] [-f] [-d] [-s] [-T] [-4] [-6] [-D]
[path [path ...]]
Identifies and extracts domains and IPs from files
positional arguments:
path Paths to files or directories to scan (if not
supplied, stdin is the file being read)
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-r, --recursive Scan paths recursively
-f, --show-files Show file names along with results
-d, --hide-duplicates
Hide duplicate results (hides per file when show-files
is enabled)
-s, --strict Stricter processing of domains
-T, --test Run some quick self tests
-4, --ipv4 Return IPv4 results
-6, --ipv6 Return IPv6 results
-D, --domain Return domain results
/usr/local/bin/eh v1.3.1 by Brian Wallace (@botnet_hunter)
wget
bwall@research:~$ wget http://bwall.github.io/ -qO- | eh -d
bwall.github.io
twitter.com
gmail.com
github.com
README.md
ajax.googleapis.com
crypto-js.googlecode.com
google-analytics.com
malware
The 0686429b86844d9d1a14a159a0263b9bfcea4fd247c77537aa0278c9c5cb4ac3 file is a sample of the POS malware, Dexter, created for demo purposes.
bwall@research:~$ eh 0686429b86844d9d1a14a159a0263b9bfcea4fd247c77537aa0278c9c5cb4ac3
houseofcarders.com
File system recursion
bwall@research:~$ eh -drf Downloads/PEStudio/
/home/bwall/Downloads/PEStudio/PeStudioFunctionsDeprecated.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioFunctionsDeprecated.xml msdn.microsoft.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListFunctions.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioEvasions.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioIndicators.xml www.winitor.com
/home/bwall/Downloads/PEStudio/ChangeLog.txt winitor.com
/home/bwall/Downloads/PEStudio/ChangeLog.txt www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioThresholds.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioWhiteListSections.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListLibraries.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioOrdinals.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioVirusTotal.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioCodePages.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioCodePages.xml msdn.microsoft.com
/home/bwall/Downloads/PEStudio/PeStudioWhiteListLibraries.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioFeatures.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudio.exe 6.0.0.0
/home/bwall/Downloads/PEStudio/PeStudioSettings.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml internetmailru.cdnmail.ru
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml sputnik.mail.ru
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml mail.ru
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml Command.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml 127.0.0.1
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml 2.0.0.1
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml www.memtest86.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml boxedapp.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml PAYPAL.COM
/home/bwall/Downloads/PEStudio/PeStudioBlackListStrings.xml start.spoon.net
/home/bwall/Downloads/PEStudio/PeStudioFunctionsUndocumented.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioWellKnownResources.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioFunctionsMapping.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeParser.dll 776::06
/home/bwall/Downloads/PEStudio/PeParser.dll ::
/home/bwall/Downloads/PEStudio/PeParser.dll 3::
/home/bwall/Downloads/PEStudio/PeStudioTranslations.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListLanguages.xml www.winitor.com
/home/bwall/Downloads/PEStudio/PeStudioBlackListLanguages.xml msdn.microsoft.com
Change log
Version 1.3.0
- Added UTF-16LE support in IPs and Domains
Version 1.2.0
- Added arguments to filter results for only desired host types (IPv4, IPv6, Domain)
Version 1.1.0
- Files are no longer loaded completely into RAM so larger files can be processed (performance increase noticed)
- Strict mode added, where domains are only considered if all lower case
TODO
- IPv6 regex needs to be shortened and heavily tested
- Heavier testing
- Improve performance
- Multiple core processing support