Queried Compessed BreachCompilation: #1
pakerte opened this issue · comments
I've download "BreachCompilation: #1"
It has 44GB of text files.
So I gzipped these files
$ cd BreachCompilation
$ find data/ -type f -exec gzip -v {} \;
$
$ du -sh data/
13G data/
Then I copied query.sh to zquery.sh
And edited to be like this:
(see the bottom of this message)
Use it instead query.sh
./zquery.sh example@example.com
It still quiet fast as the original condition.
But now I have 30G more free space
Hope that help you too.
Thank you
@pakerte
#!/bin/bash
dir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
if [ "$1" != "" ]; then
letter1=$(echo ${1,,}|cut -b1)
if [[ $letter1 == [a-zA-Z0-9] ]]; then
if [ -f "$dir/data/$letter1.gz" ]; then
zgrep -ai "^$1" "$dir/data/$letter1.gz"
else
letter2=$(echo ${1,,}|cut -b2)
if [[ $letter2 == [a-zA-Z0-9] ]]; then
if [ -f "$dir/data/$letter1/$letter2.gz" ]; then
echo "zgrep -ai ^$1" "$dir/data/$letter1/$letter2.gz"
zgrep -ai "^$1" "$dir/data/$letter1/$letter2.gz"
else
letter3=$(echo ${1,,}|cut -b3)
if [[ $letter3 == [a-zA-Z0-9] ]]; then
if [ -f "$dir/data/$letter1/$letter2/$letter3.gz" ]; then
zgrep -ai "^$1" "$dir/data/$letter1/$letter2/$letter3.gz"
fi
else
if [ -f "$dir/data/$letter1/$letter2/symbols" ]; then
zgrep -ai "^$1" "$dir/data/$letter1/$letter2/symbols"
fi
fi
fi
else
if [ -f "$dir/data/$letter1/symbols" ]; then
zgrep -ai "^$1" "$dir/data/$letter1/symbols"
fi
fi
fi
else
if [ -f "$dir/data/symbols" ]; then
zgrep -ai "^$1" "$dir/data/symbols"
fi
fi
else
echo "[*] Example: ./zquery name@domain.com"
fi