lothos612 / regex-for-OSINT

A list of application tools and information resources to help you effectively use regular expressions in OSINT (Open Source Intelligence)

Regex for OSINT

A list of application tools and information resources to help you effectively use regular expressions in OSINT (Open Source Intelligence)

@cyb_detective

If you are serious about OSINT but do not yet use regular expressions in your practice, I recommend that you read this article to realize all the amazing possibilities this technology offers:

How regular expressions can be useful in OSINT. Theory and some practice using Google Sheets

---

Table of contents

• Best Regex Cheatsheets

• Regex online courses

• Regex Lists and Libraries

• Regex generators and constructors

• Regex testers

• Understanding regex

• Command line tools for searching and extracting data with regex

• Text editors with regex support

• Search engines with regex support

• Examples of OSINT tools with regex support

• Google Chrome Extensions for searching and extracting data with regex

Theory

Best Regex Cheatsheets

Link	Description
Devhints	Simple universal regex cheatsheet
Remram44 Cheatsheet	for Perl/PCR, Python's re, POSIX (BRE), POSIX extended (ERE)and Vim regex
DaveChild Regex Cheatsheet	Posted on Cheatography cheatsheet with downloadable PDF version.

Regex online courses

Link	Description
Regex Learn	Complete 56 short online exercises and learn how to use regular expressions.
Regex One	25 lessons with interactive exercises

Regex Lists and Libraries

Link	Description
UI Bakery Regex Library	A short list of a couple dozen of the most useful and widely used regular expressions.
Regex101	16000+ regular expressions + editor/debugger with quick reference
XRegexp	JavaScript regular expressions library
"Secret regex list"	List of regular expressions to search for API keys and other juicy info. Useful when searching in public code repositories as well as when researching files on the computer/server that is your target.
Stackoverflow regex tag	250 000 + questions with examples of regex

Important:

If you happen to find some regular expression on the Internet and it doesn't work, check its compatibility with the application or programming language you are using. All the differences between the different types of regular expressions (PCRE, JavaScript, Python, POSIX ERE, etc.) are described in detail here

Regex generators and constructors

Link	Description
Autoregex	A tool that generates regular expressions against a verbal description using AI
GREX	Command line tool written by Rust that generates a regular expression corresponding to entered string. There are dozens of different options, each of which can produce completely different results.
RegexpConstructor	Online tool for creating regular expressions based on typed string. Very little automation, very many options and settings. The result depends almost entirely on the user's effort.
Regular expression builder	Online tool for writing regular expressions by selecting values and parameters from a drop-down list.
Strings to regex	Generates a regular expression matching a long list of string values.
Regex Help	Generating regular expressions based on example strings.
Regex Nodes	Regex Visual Editor

Regex testers

Link	Description
Regexly JS	A tool for testing various JavaScript methods with a combination of "regular expression" + "string".
Rubular	Ruby regular expression editor
PyRegex	Python regular expression tester
RegexHero	.NET regex tester
PHPLive Regex	PHP Regex tester
Regex101	Golang, Java, .NET, Python, JavaScript, PCRE regex tester

Understanding regex

Link	Description
Regex Translator	An online tool that generates a text description for any regular expression
Stanford regex analyzer	Another online tool that generates a text description for any regular expression
Regexper	A tool that converts a regular expression into a block diagram
Extend Class Regex Visualizer	Test, debug and visaulize regex online

Practice

Important: Regular expressions are supported by a huge number of applications. Even Google Docs and Adobe Photoshop (in JavaScript automations) support them. Before you install any new application to use regular expressions, think about whether you can use something you already have.

Command line tools for searching and extracting data with regex

Link	Description
FD	Written in Rust analog of the standard Linux find utility. Faster and more User friendly
SD	Written in Rust analog of the standard Linux sed utility. Faster and more User friendly
Awesome GREP	A list of various tools for working with different file types that have similar functionality to the standard Linux grep utility
VIM	The most powerful command line text editor with many functions for working with regular expressions.
AWK	AWK is a domain-specific language designed for text processing and typically used as a data extraction and reporting tool

Text editors with regex support

Link	Description
Notepad++
Sublime Text
Edit Pad
Atom
Visual Studio

Search engines with regex support

Link	Description
Public WWW	Source Code Search Engine. Find any alphanumeric snippet, signature or keyword in the web pages HTML, JS and CSS code.
Grep.app	Search across half of million Github repos
NetlasIO	Search service for all devices connected to the Internet. You can search by IP addresses, port, protocols, certificates, vulnerability names and other parameters.
Sourcegraph	Source code search engine
Live Grep	Search across Github repos

Examples of OSINT tools with regex support

Link	Description
WayMore	Search archived links to domain in Wayback Machine and Common Crawl (+ Urlscan and Alien Vault OTX). Very extensive options for filtering search results by keywords (using Regexp), date and time, and other parameters.
Secret Finder	Tool for find sensitive data (apikeys, accesstoken,jwt,..) or search anything with #regexp on #javascript files.
DumpsterDiver	Tool can analyze big volumes of data and find some "secrets" in the files (passwords and hardcoded password, SSH, Azure and AWS keys etc). Uncompress archived files, support regular expressions and advanced search rules.
Rextract	This extreme simple tool extracts the strings corresponding to a certain #regex from the html code of the list of URLs.
GREP for OSINT	Set of very simple shell scripts that will help you quickly analyze a text or a folder with files for data useful for investigation (phone numbers, bank card numbers, URLs, emails and nicknames).

Google Chrome Extensions for searching and extracting data with regex

Link	Description
Regex Checker	Search and highlight (in webpage): Emails, Phone numbers, Dates, Prices, Adresses
find+	Regex Find-in-Page Tool
Chrome regex search	Dynamically highlight regex matches on the web page

---

Thank you for following me! https://cybdetective.com