Best way to add multiple ci.NAME_TYPE =

[bedford1234]

Hello,

I would like to be able to pull multiple features from the certificate such as dNSName and emailAddress. What is the best way to add that to the crtsh_enum_logs.py script?

Your assistance is greatly appreciated!

[Bharath]

Hey,

The script here https://github.com/appsecco/the-art-of-subdomain-enumeration/blob/master/crtsh_enum_psql.py already extracts domain names(dNSName). If you want to extract emails then there is a script that does exactly that https://gist.github.com/yamakira/d449e502f432278f772bd672ec785d7c . I have also written a script that extracts sub-domain names and emails from SSL/TLS certs using censys.io https://github.com/yamakira/censys-enumeration .

If you think there is something that these scripts are missing, please feel free to mention. I'll be happy to guide you on writing another script.

[bedford1234]

Hey yamakira,

Thank you for responding to my post. If I was to adjust the crtsh_enum_psql.py script and I wanted to gather serial number instead of dNSName, is there a listing of the proper naming conventions used by crt.sh? For example is serial number: serialNumber?

Thanks

[Bharath]

There are couple of ways to go about it. crt.sh provides access to PSQL database(certwatch) so you can take a look into the tables to find out what is the naming convention for the field that you are interested in.

https://groups.google.com/forum/#!msg/crtsh/sUmV0mBz8bQ/K-6Vymd_AAAJ
The table you might be interested in is certificate_identity

The other way is to use showSQL=Y parameter with crt.sh web interface which will show a SQL statement that will work on the certwatch database to extract the data you are looking for.

https://crt.sh/?q=www.comodo.com&showSQL=Y

[bedford1234]

Hi,

Thanks for getting back to me. After connecting to the crt.sh db as a guest using postgres and using the \dt command to view the tables I only see 36 tables. It seems like I'm missing a lot of the important ones. Any clues to why this could be happening?

[Bharath]

If I'm not wrong those are the only tables that are related to "certwatch" database. certificate_identity table is where the domain names are stored. You could explore the tables to understand how to pull specific data.

[bedford1234]

I'm trying to edit the emails_from_ct_logs.py script to open a text file and read a domain from the .txt file one by one. Below is my code. When I run it, it tells me that one unique email is found. The .txt file has thousands of domains in it. Any assistance would be greatly appreciated!

from future import print_function

author = 'Bharath'
version = "0.1.0"

try:
import psycopg2
except ImportError:
raise ImportError('\n\033[33mpsycopg2 library missing. pip install psycopg2\033[1;m\n')
sys.exit(1)
import re
import sys
import json

DB_HOST = 'crt.sh'
DB_NAME = 'certwatch'
DB_USER = 'guest'

def connect_to_db(domain_name):
try:
conn = psycopg2.connect("dbname={0} user={1} host={2}".format(DB_NAME, DB_USER, DB_HOST))
cursor = conn.cursor()
cursor.execute("SELECT ci.NAME_VALUE NAME_VALUE FROM certificate_identity ci WHERE ci.NAME_TYPE = 'emailAddress' AND reverse(lower(ci.NAME_VALUE)) LIKE reverse(lower('%{}'));".format(domain_name))
except:
print("\n\033[1;31m[!] Unable to connect to the database\n\033[1;m")
return cursor

def get_unique_emails(cursor, domain_name):
unique_emails = []
for result in cursor.fetchall():
matches=re.findall(r"'(.+?)'",str(result))
for email in matches:
#print(email)
if email not in unique_emails:
if "{}".format(domain_name) in email:
unique_emails.append(email)
return unique_emails

def print_unique_emails(unique_emails):
print("\033[1;32m[+] Total unique emails found: {}\033[1;m".format(len(unique_emails)))
for unique_email in sorted(unique_emails):
print(unique_email)

def write_unique_emails(unique_emails):
with open('unique_emails.json', 'w') as outfile:
json.dump(unique_emails, outfile, sort_keys=True, indent=4)

def get_domain_name():
with open('file.txt') as f:
get_domain_name = [line.split() for line in f.readlines()]

if name == 'main':
domain_name = get_domain_name()
cursor = connect_to_db(domain_name)
unique_emails = get_unique_emails(cursor, domain_name)
print_unique_emails(unique_emails)
write_unique_emails(unique_emails)