Enpass 6.x compatibility
acrewdson opened this issue · comments
Hi,
Thanks for your work on this. I noticed in the README you mentioned that you were looking for a new maintainer. I'd be willing to pitch in there, and would also like to help with the 6.x compatibility. Let me know how to proceed.
Thanks!
Hi, i’m testing Enpass 6.c with Go code on: https://github.com/HazCod/enpass-cli-test
Some context: you can run it with go run main.go.
My main issue right now is that i'm able to decrypt the SQLite database, but it appears to be empty.
@hazcod Thanks, that's a helpful pointer. I'm planning to start looking at this over the weekend and will get back to you.
For reference, this works in python:
> #!/usr/bin/env python3
>
> import hashlib
> from pysqlcipher3 import dbapi2 as sqlite
> from Crypto.Cipher import AES
>
> # Sources:
> # - https://www.enpass.io/docs/security-whitepaper-enpass/vault.html
> # - https://discussion.enpass.io/index.php?/topic/4446-enpass-6-encryption-details/
> # - https://www.zetetic.net/sqlcipher/sqlcipher-api/
>
> enpass_db_file = 'vault.enpassdb'
> enpass_master_password = b'secret-master-password'
>
> # The first 16 bytes of the database file are used as salt
> enpass_db_salt = open(enpass_db_file, 'rb').read(16)
>
> # The database key is derived from the master password
> # and the database salt with 100k iterations of PBKDF2-HMAC-SHA512
> enpass_db_key = hashlib.pbkdf2_hmac('sha512', enpass_master_password, enpass_db_salt, 100000)
>
> # The raw key for the sqlcipher database is given
> # by the first 64 characters of the hex-encoded key
> enpass_db_hex_key = enpass_db_key.hex()[:64]
>
> # Decrypt sqlcipher DB
> conn = sqlite.connect(enpass_db_file)
> c = conn.cursor()
> c.row_factory = sqlite.Row
> c.execute('PRAGMA key="x\'' + enpass_db_hex_key + '\'";')
> c.execute('PRAGMA cipher_compatibility = 3;')
>
> c.execute("SELECT * FROM Identity;")
> identity = c.fetchone()
>
> print(identity['Info'].hex())
@hazcod I spent some time with the go code today. Still no idea why it doesn't work. One question: if the Python version above works (I confirm it does), why the need to switch to go?
I think I got it working: acrewdson/enpass-cli-test@bfc5a8a
It was mainly this acrewdson/enpass-cli-test@bfc5a8a#diff-3b69a238bd40d5d8d8a4f4711cf70fa3R36
and this acrewdson/enpass-cli-test@bfc5a8a#diff-80e212c6e46fd430ea9b00d46e983af4R14
Good one @acrewdson ! I'll look into further
Edit: Seems like table Cards is gone;
2019/01/28 08:37:22 data: Identity
2019/01/28 08:37:22 data: sqlite_sequence
2019/01/28 08:37:22 data: vault_info
2019/01/28 08:37:22 data: item
2019/01/28 08:37:22 data: itemfield
2019/01/28 08:37:22 data: folder
2019/01/28 08:37:22 data: folder_items
2019/01/28 08:37:22 data: attachment
2019/01/28 08:37:22 data: custom_icon
2019/01/28 08:37:22 data: preferences
2019/01/28 08:37:22 data: share_info
2019/01/28 08:37:22 data: template
2019/01/28 08:37:22 data: category
2019/01/28 08:37:22 data: password_history
item:
&ID
&INTEGER
&uuid
&TEXT
&created_at
&INTEGER
&meta_updated_at
&INTEGER
&field_updated_at
&INTEGER
&title
&TEXT
&subtitle
&TEXT
¬e
&TEXT
&icon
&TEXT
&favorite
&INTEGER
&trashed
&INTEGER
&archived
&INTEGER
&deleted
&INTEGER
&auto_submit
&INTEGER
&form_data
&TEXT
&category
&TEXT
&template
&TEXT
&wearable
&INTEGER
&usage_count
&INTEGER
&last_used
&INTEGER
&key
&BLOB
&extra
&TEXT
&updated_at
&INTEGER
It seems that key in item is 44 bytes long, instead of the usual 47 bytes.
@hazcod That's odd. Also, an you help me understand why we can't use Python for this?
We can, but i'm currently practicing writing Go code. Since it has static typing and compiles to a binary, it's easier.
TL;DR: because I can
Edit: also it should be considerably faster compared to an interpreted language like Python.
@hazcod OK, cool, just wanted to make sure I understood 👍 It would definitely be nice to get the go version working.
@hazcod I can start taking another look at this, but first wanted to check if you have noticed anything new since your last update above? Thanks!
Also, I just spotted this. Wanted to see if you have looked at it yet, whether it has anything useful?
@acrewdson He's using 24.000 iterations for the key derivation, which makes me think that code is for the old Enpass. In Enpass 6.x, this is upped to 100.000
In my old code: https://github.com/HazCod/enpass-cli/blob/master/pass.py#L158
Any updates here?
@okgolove I haven't made any progress. Are you interested in getting an Enpass CLI working? My recollection is that getting a Python version working might be more viable.
@acrewdson I am, but I'm not sure I will be able to implement something.
Last time I tried I was still stuck at #16 (comment) ...
@hazcod I am able to go further with #16 (comment) but stuck at decrypting the key in item table
For example, when I try to decrypt the encrypted key in item table with key and iv I get -
# Get params from stream
i = 16 # First 16 bytes are for "mHashData", which is unused
ret["iv"] = bytearray()
salt = bytearray()
while i <= 31:
ret["iv"].append(info[i])
i += 1
while i <= 43:
salt.append(info[i])
i += 1
ret["iv"] = bytes(ret["iv"])
ret["key"] = generateKey(identity["Hash"].encode('utf-8'), salt)
c.execute("SELECT * FROM item;")
identity = c.fetchone()
print(decrypt(identity['key'], ret['key'], ret['iv']))Console -
Traceback (most recent call last):
File "enpass_cli_hazcod.py", line 81, in <module>
print(decrypt(identity['key'], ret['key'], ret['iv']))
File "enpass_cli_hazcod.py", line 19, in decrypt
return unpad(str(cipher.decrypt(enc), 'utf-8'))
File "/Volumes/data/venv3/lib/python3.7/site-packages/Crypto/Cipher/blockalgo.py", line 295, in decrypt
return self._cipher.decrypt(ciphertext)
ValueError: Input strings must be a multiple of 16 in length
I am suspecting that my key and iv is wrong. Could you please help me here ?
Thanks.
@Akasurde It would be great it Enpass could hop in and help us out.. i've tried reaching out, but no response.
Here my attempts at decrypting the contents.
https://github.com/upekkha/enpass6-encryption-details
Basically trying to guess where the key and iv may be stored and looping over these candidates. Unfortunately, so far, no luck in identifying the plaintext in the decrypted strings.
Hi!
Here is how to assemble the master password when somebody uses a key file:
import binascii
import hashlib
from pathlib import Path
from pysqlcipher3 import dbapi2 as sqlite
from Crypto.Cipher import AES
def make_master_password(password: bytes, key_path: Path):
key_hex_xml = Path(key_path).read_bytes()
# no need to use XML lib for such a simple string operation
cut_key_value = slice(5, -6)
key_hex = key_hex_xml[cut_key_value]
key_bytes = binascii.unhexlify(key_hex)
return password + key_bytesThis can be used for the pkdf2_hmac:
master_password = make_master_password(PASSWORD, KEY_FILE)
# The first 16 bytes of the database file are used as salt
enpass_db_salt = open(ENPASS_DB, "rb").read(16)
# The database key is derived from the master password
# and the database salt with 100k iterations of PBKDF2-HMAC-SHA512
enpass_db_key = hashlib.pbkdf2_hmac(
"sha512", master_password, enpass_db_salt, PBKDF2_ROUNDS
)Thanks @kissgyorgy , were you able to read items with this?
Yes, after this you can do anything, you can even use the database with the sqlcipher binary!
@kissgyorgy Can you please share the details with us? Thanks.
@kissgyorgy Can you please share the details with us? Thanks.
There is no more details, I took the above program and confirmed that by using a key, this works and I can issue any query, see tables and schema, etc...
#!/usr/bin/env python3
import os
import binascii
import hashlib
from pathlib import Path
from pysqlcipher3 import dbapi2 as sqlite
from Crypto.Cipher import AES
# Sources:
# - https://www.enpass.io/docs/security-whitepaper-enpass/vault.html
# - https://discussion.enpass.io/index.php?/topic/4446-enpass-6-encryption-details/
# - https://www.zetetic.net/sqlcipher/sqlcipher-api/
PBKDF2_ROUNDS = 100_000
ENPASS_DB = os.environ["ENPASS_DB"]
KEY_FILE = os.environ["ENPASS_KEY_FILE"]
PASSWORD = os.environb[b"ENPASS_PASSWORD"]
def make_master_password(password: bytes, key_path: Path):
key_hex_xml = Path(key_path).read_bytes()
# no need to use XML lib for such a simple string operation
cut_key_value = slice(5, -6)
key_hex = key_hex_xml[cut_key_value]
key_bytes = binascii.unhexlify(key_hex)
return password + key_bytes
def main():
master_password = make_master_password(PASSWORD, KEY_FILE)
# The first 16 bytes of the database file are used as salt
enpass_db_salt = open(ENPASS_DB, "rb").read(16)
# The database key is derived from the master password
# and the database salt with 100k iterations of PBKDF2-HMAC-SHA512
enpass_db_key = hashlib.pbkdf2_hmac(
"sha512", master_password, enpass_db_salt, PBKDF2_ROUNDS
)
# The raw key for the sqlcipher database is given
# by the first 64 characters of the hex-encoded key
enpass_db_hex_key = enpass_db_key.hex()[:64]
conn = sqlite.connect(ENPASS_DB)
c = conn.cursor()
c.row_factory = sqlite.Row
c.execute(f"PRAGMA key=\"x'{enpass_db_hex_key}'\";")
c.execute("PRAGMA cipher_compatibility = 3;")
c.execute("SELECT * FROM Identity;")
identity = c.fetchone()
print(identity["Info"].hex())
And using the enpass_db_hex_key you got here, you can connect to the sqlcipher database:
$ sqlcipher my.enpassdb
sqlite> PRAGMA cipher_compatibility = 3;
sqlite> PRAGMA KEY="x'<here comes enpass_db_hex_key>'";
sqlite> SELECT * FROM item; # works
I think the passwords are not yet visible, you need to implement the password decrypt part too.
@kissgyorgy I have that part working as wel, but I am failing to decrypt the separate items.
Thanks @kissgyorgy and @hazcod. I was able to successfully decrypt the items in the database.
It turns out they use:
AES-256-GCMforauthenticated encryption (AE)- Key length is
32 bytes(obviously) - Nonce length is
12 bytes - As
additional authenticated data (AAD)resp. header they use the item's UUID (without the dashes) - (The hash used in
itemfieldis unsaltedSHA-1)
Here is an example with two items in my vault:
sqlite> SELECT uuid, hex(key) FROM item;
a2ec30c0-aeed-41f7-aed7-cc50e69ff506|E13FB959F5CCAFBC94F02BBDA2EF2F086A6942D818E1B1BD2ECE182896712E788E10AAAF51E21C58B1758BFC
d9a5c594-214c-4f24-8e82-981bb5643a30|3537174F6ED41149421D45390334021ACFC0C89286ED15FAC397C2A38CB95AAFB149848FCA0538F032F802BF
and:
sqlite> SELECT item_uuid, value, hash FROM itemfield WHERE type = "password";
a2ec30c0-aeed-41f7-aed7-cc50e69ff506|a9279ed095d6c5f6624bb4863e8283bb501e6f15884209e86bfed86fda7bc28cc32d7476cc|c322e152482986d1caaa73fd0fe8be48c5bb4045
d9a5c594-214c-4f24-8e82-981bb5643a30|d19dde4390f74ab0598a779c09b3b093f4de26ce376f4ccce02cd6299efa6cb50dbf34|38fad679fb5bd255791576234bcc371fd2bbca2a
Let's join the tables together for easier processing later:
sqlite> SELECT i.uuid, hex(i.key), if.value, if.hash FROM item i, itemfield if WHERE if.type = "password" AND i.uuid = if.item_uuid;
a2ec30c0-aeed-41f7-aed7-cc50e69ff506|E13FB959F5CCAFBC94F02BBDA2EF2F086A6942D818E1B1BD2ECE182896712E788E10AAAF51E21C58B1758BFC|a9279ed095d6c5f6624bb4863e8283bb501e6f15884209e86bfed86fda7bc28cc32d7476cc|c322e152482986d1caaa73fd0fe8be48c5bb4045
d9a5c594-214c-4f24-8e82-981bb5643a30|3537174F6ED41149421D45390334021ACFC0C89286ED15FAC397C2A38CB95AAFB149848FCA0538F032F802BF|d19dde4390f74ab0598a779c09b3b093f4de26ce376f4ccce02cd6299efa6cb50dbf34|38fad679fb5bd255791576234bcc371fd2bbca2a
Here is the code snippet for decrypting every row in the joined table (instead of selecting the Identity like @kissgyorgy did it):
c.execute("SELECT i.uuid, i.key, if.value, if.hash FROM item i, itemfield if WHERE if.type = \"password\" AND i.uuid = if.item_uuid;")
for row in c:
# The key object is saved in binary from and actually consists of the
# AES key (32 bytes) and a nonce (12 bytes) for GCM
key = row["key"][:32]
nonce = row["key"][32:]
# The value object holds the ciphertext (same length as plaintext) +
# (authentication) tag (16 bytes) and is stored in hex
length = len(row["value"])
ciphertext = bytearray.fromhex(row["value"][:length - 32])
tag = bytearray.fromhex(row["value"][length - 32:])
# As additional authenticated data (AAD) they use the UUID but without
# the dashes: e.g. a2ec30c0aeed41f7aed7cc50e69ff506
uuid = row["uuid"]
header = uuid.replace("-", "")
# Now we can initialize, decrypt the ciphertext and verify the AAD.
# You can compare the SHA-1 output with the value stored in the db
cipher = AES.new(key, AES.MODE_GCM, nonce=nonce)
cipher.update(bytearray.fromhex(header))
password = cipher.decrypt(ciphertext)
try:
cipher.verify(tag)
h = hashlib.new('sha1')
h.update(password)
print("SHA-1(\"" + password.decode("utf-8") + "\")\t= " + h.hexdigest())
except ValueError:
print("Key incorrect or message corrupted")
and the output generated is:
ubuntu :: ~ % ./enpass.py
SHA-1("Thank you, kissgyorgy") = c322e152482986d1caaa73fd0fe8be48c5bb4045
SHA-1("And you too, hazcod") = 38fad679fb5bd255791576234bcc371fd2bbca2a
You can compare the SHA-1 values with the hashes in itemfield manually.
@jmastr : are you sure about the algo? This is a fresh vault.json which mentions aes-256-cbc:
{
"creating_device": "laptop",
"encryption_algo": "aes-256-cbc",
"have_keyfile": 0,
"kdf_algo": "pbkdf2",
"kdf_iter": 100000,
"last_modified_device": "django",
"last_modified_time": 1607085617,
"last_password_changed_time": 1607085524,
"last_password_changing_device": "laptop",
"vault_att_count": 0,
"vault_icon": "vault/v2",
"vault_items_count": 1,
"vault_name": "Primary",
"vault_uuid": "primary",
"version": 6
}Can you share your enpass.py?
@hazcod yes I am sure. aes-256-cbc is used to encrypt the whole vault (via SQLCipher). This is actually stated on the Security page from Enpass: https://www.enpass.io/docs/security-whitepaper-enpass/vault.html#sqlcipher-and-enpass
CBC does not offer authenticated encryption and therefore no integrity protection. I guess that’s why they use GCM on a per item basis.
I will share my enpass.py when I am back at my laptop.
ubuntu :: ~ % cat enpass.py
#!/usr/bin/env python3
import os
import binascii
import hashlib
import hmac
import base64
from pathlib import Path
from pysqlcipher3 import dbapi2 as sqlite
from Crypto.Cipher import AES
# Sources:
# - https://www.enpass.io/docs/security-whitepaper-enpass/vault.html
# - https://discussion.enpass.io/index.php?/topic/4446-enpass-6-encryption-details/
# - https://www.zetetic.net/sqlcipher/sqlcipher-api/
PBKDF2_ROUNDS = 100_000
ENPASS_DB = os.environ["ENPASS_DB"]
KEY_FILE = os.environ["ENPASS_KEY_FILE"] # can be 'export ENPASS_KEY_FILE=""'
PASSWORD = os.environb[b"ENPASS_PASSWORD"]
def make_digest(message, key):
message = bytes(message, 'UTF-8')
digester = hmac.new(key, message, hashlib.sha1)
signature = digester.hexdigest()
return signature
def make_master_password(password: bytes, key_path: Path):
if not key_path:
return password
key_hex_xml = Path(key_path).read_bytes()
# no need to use XML lib for such a simple string operation
cut_key_value = slice(5, -6)
key_hex = key_hex_xml[cut_key_value]
key_bytes = binascii.unhexlify(key_hex)
return password + key_bytes
def main():
master_password = make_master_password(PASSWORD, KEY_FILE)
# The first 16 bytes of the database file are used as salt
enpass_db_salt = open(ENPASS_DB, "rb").read(16)
# The database key is derived from the master password
# and the database salt with 100k iterations of PBKDF2-HMAC-SHA512
enpass_db_key = hashlib.pbkdf2_hmac(
"sha512", master_password, enpass_db_salt, PBKDF2_ROUNDS
)
# The raw key for the sqlcipher database is given
# by the first 64 characters of the hex-encoded key
enpass_db_hex_key = enpass_db_key.hex()[:64]
enpass_mac_hex_key = enpass_db_key.hex()[64:]
conn = sqlite.connect(ENPASS_DB)
c = conn.cursor()
c.row_factory = sqlite.Row
c.execute(f"PRAGMA key=\"x'{enpass_db_hex_key}'\";")
c.execute("PRAGMA cipher_compatibility = 3;")
c.execute("SELECT i.title, i.uuid, i.key, if.value, if.hash FROM item i, itemfield if WHERE if.type = \"password\" AND i.uuid = if.item_uuid;")
for row in c:
# The key object is saved in binary from and actually consists of the
# AES key (32 bytes) and a nonce (12 bytes) for GCM
key = row["key"][:32]
nonce = row["key"][32:]
# If you deleted an item from Enpass, it stays in the database, but the
# entries are cleared
if not nonce:
continue
# The value object holds the ciphertext (same length as plaintext) +
# (authentication) tag (16 bytes) and is stored in hex
length = len(row["value"])
ciphertext = bytearray.fromhex(row["value"][:length - 32])
tag = bytearray.fromhex(row["value"][length - 32:])
# As additional authenticated data (AAD) they use the UUID but without
# the dashes: e.g. a2ec30c0aeed41f7aed7cc50e69ff506
uuid = row["uuid"]
header = uuid.replace("-", "")
# Now we can initialize, decrypt the ciphertext and verify the AAD.
# You can compare the SHA-1 output with the value stored in the db
cipher = AES.new(key, AES.MODE_GCM, nonce=nonce)
cipher.update(bytearray.fromhex(header))
password = cipher.decrypt(ciphertext)
try:
cipher.verify(tag)
h = hashlib.new('sha1')
h.update(password)
print(row["title"] + ":\t" + "SHA-1(\"" + password.decode("utf-8") + "\")\t= " + h.hexdigest())
except ValueError:
print("Key incorrect or message corrupted")
if __name__ == "__main__":
main()
@hazcod Does it work now?
@jmastr thank you for your sample, I've updated my test code at https://github.com/hazcod/enpass-cli-test/blob/master/enpasscli/vault.go
However, I always receive a query error file is not a database while the database open works fine.
I've opened up an issue in the meanwhile: mutecomm/go-sqlcipher#21
Can you try attached vault with password mymasterpassword?
enpassvault.zip
The relevant pragmas in my library: https://github.com/hazcod/enpass-cli-test/blob/master/enpasscli/vault.go#L44
@hazcod you mean with my enpass.py:
ubuntu :: ~ % ./enpass.py
mylogin: SHA-1("mypassword") = 91dfd9ddb4198affc5c194cd8ce6d338fde470e2
or do you mean with your go code?
@jmastr I would really like to rewrite this in Go for easier support & performance..
If I get to figure this out. :-/
@hazcod There you go: https://github.com/hazcod/enpass-cli-test/pull/2 :) (pun intended)
@jmastr you're the man ❤️
I'll pump out a full cli client tomorrow
FYI @jmastr almost done on https://github.com/hazcod/enpass-cli-test/ , just need to fix a bug.
Amazing! Good job!
Wow! I can't believe the issue was closed!