The password extractor framework uses image processing with Python
and OpenCV
to recover passwords given thermal images.
This README outlines how the framework operates and provides information needed to run the framework.
There are four main steps that have to be taken to find the passwords on a thermal image:
- Key region detection on the image,
- Labeling of keys,
- Residue detection,
- Password recovery and password guessing.
In keyboards, keys are usually separated from each other to prevent unwanted key presses. This property creates an observable effect on the thermal image since there are certain temperature differences between a key and its surroundings. We use this property to detect the keys on a given thermal image (i.e., a reference image) in the first step of the password discovery process. Users also can draw regions that could not be found by the framework.
Detected Key Regions on an Image of a KeyboardAfter the key regions are marked on the image, the user is asked to enter the labels (e.g., characters on a keyboard) for keys. The advantage of this approach is that it generalizes the framework to work with any types of password/PIN entry devices (e.g., keyboards, PIN pads). After labeling is completed, the label data can be saved to be used later.
New Key Regions are Added by the User and LabeledAfter the keys are labeled, a thermal image is chosen by the user and the framework tries to detect thermal residues on the image using blob detection. The preprocessing step consists of detecting important regions (e.g., areas above certain temperature) and filtering the other regions out. If a blob is in a labeled key region, the corresponding label is used to construct the password.
Blobs Found on a Thermal ImageIn the last step, the framework uses a dictionary of passwords to find the "best" match using a distance metric. Passwords are sorted according to their distance to the pressed keys found on the thermal image and displayed for further inspection.
Progress Information of Password RecoveryThe framework can be run with Python 2.x
and uses cv2
, numpy
, pickle
, and tkinter
.
blob.py
is the main script. It can be run with the command
python blob.py SEQ3709/SEQ_3709_58.bmp
In addition, utils
folder contains additional helper scripts.