Sending / Receiving encrypted messages fails (but only for one account)
angelo-fuchs opened this issue · comments
Android Version: 13
Device: Fairphone 3
Delta Chat 1.50.2 GPlay
Expected behavior: Send and Receive works normally
Actual behavior:
I have several DC Accounts, which work fine, except one. This one can't send or receive encrypted mails since 8th December.
The same account on Desktop (1.48.0 (git: v1.33.0-1163-g0349c14e) Delta Chat Core v1.148.7, Debian) works fine.
On receiving I see the message "Diese Nachricht kann nicht entschlüsselt werden [...]"
On sending I get a red exclamation mark and the info:
error while parsing composed key: Message("unexpected packed data: ParsingError(TagBits)")
The log does not contain a stacktrace, but just the same message.
I did not change the keys of the account or something like that in the near past.
@angelo-fuchs Was this key generated by Delta Chat or imported from somewhere?
Core 1.148.7 used rPGP 0.13.2. Delta Chat for Android 1.50.2 uses core 1.151.5 and rPGP 0.14.2.
There is a suspicious commit rpgp/rpgp@a9de958 that was merged before 0.14.0.
@link2xt I don't know for sure, but I think that I imported the key, as I was using the same one to sign PGP stuff prior to using Delta Chat (I think).
I just installed Android 1.48.3 from fDroid and imported a backup from the failing DC into the new one. The 1.48.3 fDroid works as expected (except, that already failed messages are not repaired)
I attempted to recreate the issue like this:
Create a new PGP Key with my regular Desktop
Create a new DeltaChat Account on Desktop
Import the Secrect Key to the new Account
Send messages around
Backup the new Account and transfer Backup to Android
Import Backup on Android
Send messages around
That works as expected. Is there something special the key in the failing Account could have, that I should pay attention to?
To try and narrow down what's going wrong, could you attempt to use the problematic OpenPGP private key on a desktop machine, with the https://crates.io/crates/rsop tool?
(rsop is also based on rPGP, just like Delta Chat).
You could attempt to perform a signing operation like this (with enabled debug output):
echo "hello world" | RUST_LOG=debug rsop sign private-key.pgp
I'd expect that this will also fail, but will give us solid insights into the problematic property of the key.
I have the same problem.
I tried uninstalling the android application and reinstalling it (1.50.2). Then added my smarpthone as a second device from the Desktop application (1.48.0).
After the transfer, the same problem persists, impossible to receive or send messages on the android device.
Logs:
12-11 18:52:43.616 12618 12654 🔴 DeltaChat: [accId=1] Failed to send message: Failed to create send jobs: "error while parsing composed key: Message(\"unexpected packet data: ParsingError(TagBits)\")"
12-11 18:52:43.617 12618 12654 🔴 DeltaChat: Failed to send message: Failed to create send jobs: "error while parsing composed key: Message(\"unexpected packet data: ParsingError(TagBits)\")"
I tried to export the keys from the android app to test with rsop but I couldn't :
IMEX failed to complete: errors while exporting keys
I don't know if this is related.
@pepea28 Did you import the key at some point or was it generated by Delta Chat?
If exporting the key does not work, you can go into .config/DeltaChat/accounts/<your account>, open dc.db with sqlitebrowser, open "Browse Data" tab, switch to "keypairs" table, select private_key column and click "Export to file" button. The file is directly usable with rsop sign, rsop sign is fine with binary keys as well as ascii-armored keys.
This key is quite old, but I believe they were generated by DeltaChat.
In the keypairs table, there were three entries with the same addr (and is_default : 1).
I tested all three keys (id 1, 3 and 4). For id 3 and 4, it worked.
However, for id 4, I got the same error:
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] primary key: KeyId(ead46491f50598c9)
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] signatures
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] user
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] peek UserId
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] user data: UserId
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: SignatureCreationTime 5ca35605
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyExpirationTime 00000000
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: PrimaryUserId 01
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyFlags 03
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: PreferredSymmetricAlgorithms 0907030201
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: PreferredHashAlgorithms 08090a0b02
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: PreferredCompressionAlgorithms 02
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: Features 01
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: Issuer ead46491f50598c9
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] peek SecretSubkey
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] subkeys
[2024-12-11T20:23:55Z DEBUG pgp::composed::signed_key::key_parser] peek SecretSubkey
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: SignatureCreationTime 5ca35605
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyExpirationTime 00000000
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyFlags 0c
[2024-12-11T20:23:55Z DEBUG pgp::packet::signature::de] parsing subpacket: Issuer ead46491f50598c9
Bad data Err(Message("error while parsing composed key: Message(\"unexpected packet data: ParsingError(TagBits)\")"))
thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rpgpie-sop-0.4.3/src/lib.rs:168:43:
FIXME: Message("No TSKs found")
stack backtrace:
0: rust_begin_unwind
at /build/rustc-60UC9b/rustc-1.75.0+dfsg0ubuntu1~bpo0/library/std/src/panicking.rs:645:5
1: core::panicking::panic_fmt
at /build/rustc-60UC9b/rustc-1.75.0+dfsg0ubuntu1~bpo0/library/core/src/panicking.rs:72:14
2: core::result::unwrap_failed
at /build/rustc-60UC9b/rustc-1.75.0+dfsg0ubuntu1~bpo0/library/core/src/result.rs:1653:5
3: <rpgpie_sop::Keys as sop::Load<rpgpie_sop::RPGSOP>>::from_reader
4: sop::cli::real_main
5: sop::cli::main
6: rsop::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
I tested all three keys (id 1, 3 and 4). For id 3 and 4, it worked.
However, for id 4, I got the same error:
Probably a typo, for 3 and 4 it worked but for 1 there is an error or something like this?
Seems it failed without printing actual subpacket that failed, but maybe @hko-s can guess what it is from surrounding packets.
If you have some way to install sequoia, could you run sq toolbox packet dump <keyfile> to see the packet structure of the key and check what subpacket comes after "Issuer"?
Or use gpg --list-packets, it does almost the same thing as sq toolbox packet dump, maybe even better given that the key was likely generated with GnuPG (if it's really about key 4 and not 1).
Yes sorry it was a typo, id 1 and 3 worked, but not 4.
Here's the result of gpg --list-packets for the problematic key :
gpg: [don't know]: invalid packet (ctb=02)
# off=0 ctb=c5 tag=5 hlen=3 plen=920 new-ctb
:secret key packet:
version 4, algo 1, created 1554208260, expires 0
pkey[0]: [2048 bits]
pkey[1]: [17 bits]
skey[2]: [2046 bits]
skey[3]: [1024 bits]
skey[4]: [1024 bits]
skey[5]: [1023 bits]
checksum: 3d5f
keyid: EAD46491F50598C9
# off=923 ctb=cd tag=13 hlen=2 plen=17 new-ctb
:user ID packet: "<xxxxxxxxxx@xxxxxxx>"
# off=942 ctb=c2 tag=2 hlen=3 plen=316 new-ctb
:signature packet: algo 1, keyid EAD46491F50598C9
version 4, created 1554208261, md5len 0, sigclass 0x13
digest algo 8, begin of digest 95 4e
hashed subpkt 2 len 4 (sig created 2019-04-02)
hashed subpkt 9 len 4 (key does not expire)
hashed subpkt 25 len 1 (primary user ID)
hashed subpkt 27 len 1 (key flags: 03)
hashed subpkt 11 len 5 (pref-sym-algos: 9 7 3 2 1)
hashed subpkt 21 len 5 (pref-hash-algos: 8 9 10 11 2)
hashed subpkt 22 len 1 (pref-zip-algos: 2)
hashed subpkt 30 len 1 (features: 01)
subpkt 16 len 8 (issuer key ID EAD46491F50598C9)
data: [2048 bits]
# off=1261 ctb=c7 tag=7 hlen=3 plen=920 new-ctb
:secret sub key packet:
version 4, algo 1, created 1554208261, expires 0
pkey[0]: [2048 bits]
pkey[1]: [17 bits]
skey[2]: [2047 bits]
skey[3]: [1024 bits]
skey[4]: [1024 bits]
skey[5]: [1024 bits]
checksum: 34f1
keyid: 683A0916AD9D9A7F
# off=2184 ctb=c2 tag=2 hlen=3 plen=293 new-ctb
:signature packet: algo 1, keyid EAD46491F50598C9
version 4, created 1554208261, md5len 0, sigclass 0x18
digest algo 8, begin of digest 81 4c
hashed subpkt 2 len 4 (sig created 2019-04-02)
hashed subpkt 9 len 4 (key does not expire)
hashed subpkt 27 len 1 (key flags: 0C)
subpkt 16 len 8 (issuer key ID EAD46491F50598C9)
data: [2048 bits]
# off=2481 ctb=fc tag=60 hlen=2 plen=170 new-ctb
:unknown packet: type 60, length 170
dump: 38 4b 5c EOF
# off=2481 ctb=fc tag=60 hlen=2 plen=170 new-ctb looks like some packet with type 60, the first one from "Private or Experimental Use" according to https://www.iana.org/assignments/openpgp/openpgp.xhtml
I think it's unlikely to be random garbage at the end of the key, hlen=2 and plen=170 look like perfectly valid header/body length numbers.
@pepea28 What is the length of the keyfile according to wc -c? If it is 2481 + 170 = 2651 or something like this, then it's a valid packet and rPGP should learn to ignore it even if we don't know what it is.
Ok, then it is just 5 bytes of garbage at the end of the key, 2 bytes of header and then "38 4b 5c". And the header is not of a valid packet because the packet length is declared to be 170-bytes long.
So it seems there are 5 bytes fc aa 38 4b 5c trailing the actual key material. I'm at a loss why that could be.
(And I can't immediately decide if it's reasonable to fail so hard on this input.)
I now assume the key could be "fixed" by chopping these 5 bytes off, manually, though.
E.g. by doing dd if=keyfile.old bs=1 count=2481 of=keyfile.new
For rPGP it seems reasonable to fail hard when the key is truncated. Even if the packet is "experimental", it has length 170 and there are not enough bytes so this is definitely not ok. I am surprised this was not an error before, actually, but I guess nom naturally treats EOF as just some sort of input that does not match the parser and parses everything up to EOF as just a sort of erroneous packet.
Regardless of whether we make rPGP accept such keys, I think Delta Chat core should ignore keys it fails to parse when loading the keyring if they are not selected as default and not fail sending messages because the key it was not going to use anyway failed to parse. For reception also makes sense to try to decrypt with usable keys and ignore the keys we cannot load, even if they are default. If default key cannot be loaded and we want to send then I don't know what to do, generating a new key seems risky.
I now assume the key could be "fixed" by chopping these 5 bytes off, manually, though.
There is also a public key in the database that should match. But it should be possible to delete the key from the database, then import properly truncated secret key and let Delta Chat generate the public key. If the key is not selected as the default and is not really used, maybe not worth fixing it and better just delete it from the database.
and
is_default:1
is_default column is not used anymore, selected key is determined by key_id in the config table.
I removed the last 5 bytes of the key in dc.db.
I reinstalled DeltaChat android 1.50.2 and added it as a second device from the desktop version (whose private key has been corrected).
I got the same error.
I also did: dd if=keyfile.old bs=1 count=2481 of=keyfile.new and redid an rsop sign and the error is still there:
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] primary key: KeyId(ead46491f50598c9)
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] signatures
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] user
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] peek UserId
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] user data: UserId
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: SignatureCreationTime 5ca35605
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyExpirationTime 00000000
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: PrimaryUserId 01
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyFlags 03
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: PreferredSymmetricAlgorithms 0907030201
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: PreferredHashAlgorithms 08090a0b02
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: PreferredCompressionAlgorithms 02
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: Features 01
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: Issuer ead46491f50598c9
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] peek SecretSubkey
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] subkeys
[2024-12-11T21:53:23Z DEBUG pgp::composed::signed_key::key_parser] peek SecretSubkey
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: SignatureCreationTime 5ca35605
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyExpirationTime 00000000
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: KeyFlags 0c
[2024-12-11T21:53:23Z DEBUG pgp::packet::signature::de] parsing subpacket: Issuer ead46491f50598c9
Bad data Err(Message("error while parsing composed key: Message(\"unexpected packet data: ParsingError(TagBits)\")"))
thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rpgpie-sop-0.4.3/src/lib.rs:168:43:
FIXME: Message("No TSKs found")
stack backtrace:
0: rust_begin_unwind
at /build/rustc-60UC9b/rustc-1.75.0+dfsg0ubuntu1~bpo0/library/std/src/panicking.rs:645:5
1: core::panicking::panic_fmt
at /build/rustc-60UC9b/rustc-1.75.0+dfsg0ubuntu1~bpo0/library/core/src/panicking.rs:72:14
2: core::result::unwrap_failed
at /build/rustc-60UC9b/rustc-1.75.0+dfsg0ubuntu1~bpo0/library/core/src/result.rs:1653:5
3: <rpgpie_sop::Keys as sop::Load<rpgpie_sop::RPGSOP>>::from_reader
4: sop::cli::real_main
5: sop::cli::main
6: rsop::main
@pepea28 What does gpg --list-packets show for the truncated keyfile? It should at least not produce any dump line.
By the way this looks a lot like my old key generated with Delta Chat from 2019 (algorithm preferences etc.), probably using C core and NetPGP back then:
hashed subpkt 2 len 4 (sig created 2019-04-02)
hashed subpkt 9 len 4 (key does not expire)
hashed subpkt 25 len 1 (primary user ID)
hashed subpkt 27 len 1 (key flags: 03)
hashed subpkt 11 len 5 (pref-sym-algos: 9 7 3 2 1)
hashed subpkt 21 len 5 (pref-hash-algos: 8 9 10 11 2)
hashed subpkt 22 len 1 (pref-zip-algos: 2)
hashed subpkt 30 len 1 (features: 01)
So the key is likely from old C core of Delta Chat. My key does not have garbage at the end though, but I did not import it, it's the first key. Maybe importing was buggy at that time and copied some extra bytes from memory, it was in C after all.
I removed the last 5 bytes of the key in
dc.db.I reinstalled DeltaChat android 1.50.2 and added it as a second device from the desktop version (whose private key has been corrected).
I got the same error.
I also did:
dd if=keyfile.old bs=1 count=2481 of=keyfile.newand redid anrsop signand the error is still there
[...]
I think the offset in the GnuPG dump does not work the way I thought it did. So the shortened key (based on my dd command suggestion above) is probably cut in the wrong place, and I can't figure out what the right place is.
You could look at the original exported private key file in a hex dump/hex editor, and look for the bytes fc aa very close to the end, to figure out at which exact point the key file actually needs to be cut. The bytes fc aa should not remain, the key file should be cut right before them.
dd if=keyfile.old bs=1 count=2481 of=keyfile.new
This should be dd if=keyfile.old bs=1 count=2480 of=keyfile.new. At least 2184 + 3 + 293 = 2180 calculated from # off=2184 ctb=c2 tag=2 hlen=3 plen=293 new-ctb. Not sure why gpg --list-packets printed 2181 as the next offsets, all previous offsets add up.
I think there are 6 bytes of garbage, 02 fc aa 38 4b 5c. Warning about 02 got printed to stderr as gpg: [don't know]: invalid packet (ctb=02).
I think there are 6 bytes of garbage,
02 fc aa 38 4b 5c. Warning about 02 got printed to stderr asgpg: [don't know]: invalid packet (ctb=02).
It took me a long time to follow, but I now find this theory compelling.
I have checked out old C core and at least can say that it correctly strips -----END PGP PRIVATE KEY BLOCK----- before passing it into mailmime_base64_body_parse. But I have a suspicion that mailmime_base64_body_parse (which comes from libetpan) does not know anything about ASCII armor checksum in the end which Delta Chat also passes inside.
I suspect that there was a bug in older C core with importing ASCII-armored keys.
Here is a mashup of the code extracted from the old core:
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#include <stdlib.h>
/**
* Remove all `\r` characters from the given buffer.
* This function will not convert anything else in the future, so it can be used
* safely for marking thinks to remove by `\r` and call this function afterwards.
*
* @param buf The buffer to convert.
* @return None.
*/
void dc_remove_cr_chars(char* buf)
{
const char* p1 = buf; /* search for first `\r` */
while (*p1) {
if (*p1=='\r') {
break;
}
p1++;
}
char* p2 = (char*)p1; /* p1 is `\r` or null-byte; start removing `\r` */
while (*p1) {
if (*p1!='\r') {
*p2 = *p1;
p2++;
}
p1++;
}
/* add trailing null-byte */
*p2 = 0;
}
static inline signed char get_base64_value(char ch)
{
if ((ch >= 'A') && (ch <= 'Z'))
return ch - 'A';
if ((ch >= 'a') && (ch <= 'z'))
return ch - 'a' + 26;
if ((ch >= '0') && (ch <= '9'))
return ch - '0' + 52;
switch (ch) {
case '+':
return 62;
case '/':
return 63;
case '=': /* base64 padding */
return -1;
default:
return -1;
}
}
char* dc_strdup(const char* s) /* strdup(NULL) is undefined, save_strdup(NULL) returns an empty string in this case */
{
char* ret = NULL;
if (s) {
if ((ret=strdup(s))==NULL) {
exit(16); /* cannot allocate (little) memory, unrecoverable error */
}
}
else {
if ((ret=(char*)calloc(1, 1))==NULL) {
exit(17); /* cannot allocate little memory, unrecoverable error */
}
}
return ret;
}
void dc_ltrim(char* buf)
{
size_t len = 0;
const unsigned char* cur = NULL;
if (buf && *buf) {
len = strlen(buf);
cur = (const unsigned char*)buf;
while (*cur && isspace(*cur)) {
cur++; len--;
}
if ((const unsigned char*)buf!=cur) {
memmove(buf, cur, len + 1);
}
}
}
void dc_rtrim(char* buf)
{
size_t len = 0;
unsigned char* cur = NULL;
if (buf && *buf) {
len = strlen(buf);
cur = (unsigned char*)buf + len - 1;
while (cur!=(unsigned char*)buf && isspace(*cur)) {
--cur, --len;
}
cur[isspace(*cur) ? 0 : 1] = '\0';
}
}
void dc_trim(char* buf)
{
dc_ltrim(buf);
dc_rtrim(buf);
}
/* Split data from PGP Armored Data as defined in https://tools.ietf.org/html/rfc4880#section-6.2.
The given buffer is modified and the returned pointers just point inside the modified buffer,
no additional data to free therefore.
(NB: netpgp allows only parsing of Version, Comment, MessageID, Hash and Charset) */
int dc_split_armored_data(char* buf, const char** ret_headerline, const char** ret_setupcodebegin, const char** ret_preferencrypt, const char** ret_base64)
{
int success = 0;
size_t line_chars = 0;
char* line = buf;
char* p1 = buf;
char* p2 = NULL;
char* headerline = NULL;
char* base64 = NULL;
#define PGP_WS "\t\r\n "
if (ret_headerline) { *ret_headerline = NULL; }
if (ret_setupcodebegin) { *ret_setupcodebegin = NULL; }
if (ret_preferencrypt) { *ret_preferencrypt = NULL; }
if (ret_base64) { *ret_base64 = NULL; }
if (buf==NULL || ret_headerline==NULL) {
goto cleanup;
}
dc_remove_cr_chars(buf);
while (*p1) {
if (*p1=='\n') {
/* line found ... */
line[line_chars] = 0;
if (headerline==NULL) {
/* ... headerline */
dc_trim(line);
if (strncmp(line, "-----BEGIN ", 11)==0 && strncmp(&line[strlen(line)-5], "-----", 5)==0) {
headerline = line;
if (ret_headerline) {
*ret_headerline = headerline;
}
}
}
else if (strspn(line, PGP_WS)==strlen(line)) {
/* ... empty line: base64 starts on next line */
base64 = p1+1;
break;
}
else if ((p2=strchr(line, ':'))==NULL) {
/* ... non-standard-header without empty line: base64 starts with this line */
line[line_chars] = '\n';
base64 = line;
break;
}
else {
/* header line */
*p2 = 0;
dc_trim(line);
if (strcasecmp(line, "Passphrase-Begin")==0) {
p2++;
dc_trim(p2);
if (ret_setupcodebegin) {
*ret_setupcodebegin = p2;
}
}
else if (strcasecmp(line, "Autocrypt-Prefer-Encrypt")==0) {
p2++;
dc_trim(p2);
if (ret_preferencrypt) {
*ret_preferencrypt = p2;
}
}
}
/* prepare for next line */
p1++;
line = p1;
line_chars = 0;
}
else {
p1++;
line_chars++;
}
}
if (headerline==NULL || base64==NULL) {
goto cleanup;
}
/* now, line points to beginning of base64 data, search end */
if ((p1=strstr(base64, "-----END "/*the trailing space makes sure, this is not a normal base64 sequence*/))==NULL
|| strncmp(p1+9, headerline+11, strlen(headerline+11))!=0) {
goto cleanup;
}
*p1 = 0;
dc_trim(base64);
if (ret_base64) {
*ret_base64 = base64;
}
success = 1;
cleanup:
return success;
}
static int mailmime_base64_body_parse_impl(
const char * message, size_t length,
size_t * indx, char ** result,
size_t * result_len, int partial)
{
size_t cur_token, last_full_token_end;
char chunk[4];
int chunk_index;
char out[3];
char * mmapstr;
int res;
int r;
size_t written;
chunk[0] = 0;
chunk[1] = 0;
chunk[2] = 0;
chunk[3] = 0;
cur_token = * indx;
last_full_token_end = * indx;
chunk_index = 0;
written = 0;
mmapstr = malloc((length - cur_token) * 3 / 4);
if (mmapstr == NULL) {
exit(1);
}
while (1) {
signed char value;
value = -1;
while (value == -1) {
if (cur_token >= length)
break;
value = get_base64_value(message[cur_token]);
cur_token ++;
}
if (value == -1)
break;
chunk[chunk_index] = value;
chunk_index ++;
if (chunk_index == 4) {
out[0] = (chunk[0] << 2) | (chunk[1] >> 4);
out[1] = (chunk[1] << 4) | (chunk[2] >> 2);
out[2] = (chunk[2] << 6) | (chunk[3]);
chunk[0] = 0;
chunk[1] = 0;
chunk[2] = 0;
chunk[3] = 0;
chunk_index = 0;
last_full_token_end = cur_token;
/*
if (mmap_string_append_len(mmapstr, out, 3) == NULL) {
res = MAILIMF_ERROR_MEMORY;
goto free;
}
*/
written += 3;
}
}
if (chunk_index != 0 && !partial) {
size_t len;
len = 0;
out[0] = (chunk[0] << 2) | (chunk[1] >> 4);
len ++;
if (chunk_index >= 3) {
out[1] = (chunk[1] << 4) | (chunk[2] >> 2);
len ++;
}
/*
if (mmap_string_append_len(mmapstr, out, len) == NULL) {
res = MAILIMF_ERROR_MEMORY;
goto free;
}
*/
written += len;
}
if (partial) {
cur_token = last_full_token_end;
}
/*
r = mmap_string_ref(mmapstr);
if (r < 0) {
res = MAILIMF_ERROR_MEMORY;
goto free;
}
*/
* indx = cur_token;
//* result = mmapstr->str;
* result_len = written;
return 0;
free:
//mmap_string_free(mmapstr);
err:
return res;
}
int mailmime_base64_body_parse(const char * message, size_t length,
size_t * indx, char ** result,
size_t * result_len)
{
return mailmime_base64_body_parse_impl(message, length, indx, result, result_len, 0);
}
static int set_self_key(const char* armored)
{
int success = 0;
char* buf = NULL;
const char* buf_headerline = NULL; // pointer inside buf, MUST NOT be free()'d
const char* buf_preferencrypt = NULL; // - " -
const char* buf_base64 = NULL; // - " -
char* self_addr = NULL;
buf = dc_strdup(armored);
if (!dc_split_armored_data(buf, &buf_headerline, NULL, &buf_preferencrypt, &buf_base64)
|| strcmp(buf_headerline, "-----BEGIN PGP PRIVATE KEY BLOCK-----")!=0 || buf_base64==NULL) {
return 1;
}
size_t result_len = 0;
char* result = NULL;
size_t indx = 0;
printf("base64: %s\nXXX\n", buf_base64);
mailmime_base64_body_parse(buf_base64, strlen(buf_base64), &indx, &result/*must be freed using mmap_string_unref()*/, &result_len);
printf("Dearmored key length: %zd\n", result_len);
/*
if (!dc_key_set_from_base64(private_key, buf_base64, DC_KEY_PRIVATE)
|| !dc_pgp_is_valid_key(context, private_key)
|| !dc_pgp_split_key(context, private_key, public_key)) {
return 1;
}
*/
}
int main(int argc, char *argv[]) {
FILE *fp = fopen(argv[1], "r");
if (fp == NULL) return 1;
char armored[4096];
fread(armored, sizeof armored[0], sizeof armored / sizeof armored[0], fp);
printf("%s\n", armored);
set_self_key(armored);
return 0;
}Here key length is 471, but this C code reports that it wants to produce 474 (actual writing of base64 decoded bytes is commented out), so 3 bytes of garbage:
$ rsop generate-key alice@example.org > key.asc
$ gcc dckeybug.c
$ ./a.out key.asc
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1o2iRYJKwYBBAHaRw8BAQdAwoN6aodg9kIIkn4/vTUYGWYr7XPIwPDRDIP2
ANlWAZ0AAQDMdgPZtapv3yD3onvGQiRYphDDmR4ds4LPG/Ov1tv6NhFzzRFhbGlj
ZUBleGFtcGxlLm9yZ8KPBBAWCAA3AhkBBQJnWjaJAhsDCAsJCAcKDQwLBRUKCQgL
AhYCAScWIQTEmKEqZFmwkpq8iZDaKIHpEluVEQAKCRDaKIHpEluVEZ/jAQDMkNh4
VQI4D5+fZrCWLrqXAuTeICzoaU1EoP2gAYqfvwD9F0odPw/FP4wJTm0MU61BNt0X
ZJsHeTieX5SXXREA8QHHXQRnWjaJEgorBgEEAZdVAQUBAQdAjGBlMXUeV77npD+a
3160M2+O1MvyTNnNGGdVf6uqsT8DAQgHAAD/fsg0v51P6i751DuPhNoSCoWI7mrK
2R0rGL80R6Lc23gRwMJ4BBgWCAAgBQJnWjaJAhsMFiEExJihKmRZsJKavImQ2iiB
6RJblREACgkQ2iiB6RJblREyzwEA5R/gOjOEfFLoCGM14V+pBBNukYHlpt/xiADt
Ini8Pe4A/0NF8FWaFPpGNWfvhgWDq5zaQSvNUnOINvv176MpAmwH
=IcSU
-----END PGP PRIVATE KEY BLOCK-----
base64: xVgEZ1o2iRYJKwYBBAHaRw8BAQdAwoN6aodg9kIIkn4/vTUYGWYr7XPIwPDRDIP2
ANlWAZ0AAQDMdgPZtapv3yD3onvGQiRYphDDmR4ds4LPG/Ov1tv6NhFzzRFhbGlj
ZUBleGFtcGxlLm9yZ8KPBBAWCAA3AhkBBQJnWjaJAhsDCAsJCAcKDQwLBRUKCQgL
AhYCAScWIQTEmKEqZFmwkpq8iZDaKIHpEluVEQAKCRDaKIHpEluVEZ/jAQDMkNh4
VQI4D5+fZrCWLrqXAuTeICzoaU1EoP2gAYqfvwD9F0odPw/FP4wJTm0MU61BNt0X
ZJsHeTieX5SXXREA8QHHXQRnWjaJEgorBgEEAZdVAQUBAQdAjGBlMXUeV77npD+a
3160M2+O1MvyTNnNGGdVf6uqsT8DAQgHAAD/fsg0v51P6i751DuPhNoSCoWI7mrK
2R0rGL80R6Lc23gRwMJ4BBgWCAAgBQJnWjaJAhsMFiEExJihKmRZsJKavImQ2iiB
6RJblREACgkQ2iiB6RJblREyzwEA5R/gOjOEfFLoCGM14V+pBBNukYHlpt/xiADt
Ini8Pe4A/0NF8FWaFPpGNWfvhgWDq5zaQSvNUnOINvv176MpAmwH
=IcSU
XXX
Dearmored key length: 474
$ rsop dearmor < key.asc | wc -c
471
Removing checksum (=IcSU) manually makes "Dearmored key length" match the value produced with rsop.
So I think this bug affects basically all Delta Chat users who imported keys into Delta Chat with C core before it switched to Rust at the end of 2019. This includes keys transferred using Autocrypt Setup Message. Such imported keys have a bit of random garbage at the end.
I get 3 bytes of garbage on each import. 6 bytes of garbage is explained by importing the key twice as suggested by @hko-s in the chat.
So the problem is that CRC-24 like =IcSU which should not be parsed was parsed by old core when importing ASCII-armored keys, producing 3 more bytes.
For example:
$ rsop generate-key 'alice@example.org' > key.asc
$ cat key.asc
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1pAbhYJKwYBBAHaRw8BAQdA3zUZ8h7cLavP1k/X2fgXNfvSaGlrWIVd6bFQ
4/SKaUQAAP9ISqSuVTG6HGPjefXDCOIlyPnKW7yZEUkllfsoIS+dsxDXzRFhbGlj
ZUBleGFtcGxlLm9yZ8KPBBAWCAA3AhkBBQJnWkBuAhsDCAsJCAcKDQwLBRUKCQgL
AhYCAScWIQS9GYMcePPc4XbP8MJhV7KhKehjyAAKCRBhV7KhKehjyEQiAQDGg9Gs
cCYXYUrDoVjSk2PLESvnfzhb/2V8Nj0L037hWQEAx+rkRWspix1nGHHRJgth09ej
+IcRg2qBjR6E1UE16wXHXQRnWkBuEgorBgEEAZdVAQUBAQdABMF+yZcYgnA5hwxD
3I0asNf423d+cYB1XqygfEQRSR4DAQgHAAD/aQ4WkQzR2RCYrRW7yTeqJCziQCrt
ZtUOyWyXQa5szggPccJ4BBgWCAAgBQJnWkBuAhsMFiEEvRmDHHjz3OF2z/DCYVey
oSnoY8gACgkQYVeyoSnoY8hhZAD/eOlvL1aivrP3M6ePh5PEqaRUwlpJd5nC6L2L
vGN4uI8A/iMm5biYSHTQf9G4wHS4fVgD7gHYCOqbKyLyv75/B7IA
=XZj3
-----END PGP PRIVATE KEY BLOCK-----
$ sed '/^-----/d;s/=//' < key.asc | base64 -d | rsop armor
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1pAbhYJKwYBBAHaRw8BAQdA3zUZ8h7cLavP1k/X2fgXNfvSaGlrWIVd6bFQ
4/SKaUQAAP9ISqSuVTG6HGPjefXDCOIlyPnKW7yZEUkllfsoIS+dsxDXzRFhbGlj
ZUBleGFtcGxlLm9yZ8KPBBAWCAA3AhkBBQJnWkBuAhsDCAsJCAcKDQwLBRUKCQgL
AhYCAScWIQS9GYMcePPc4XbP8MJhV7KhKehjyAAKCRBhV7KhKehjyEQiAQDGg9Gs
cCYXYUrDoVjSk2PLESvnfzhb/2V8Nj0L037hWQEAx+rkRWspix1nGHHRJgth09ej
+IcRg2qBjR6E1UE16wXHXQRnWkBuEgorBgEEAZdVAQUBAQdABMF+yZcYgnA5hwxD
3I0asNf423d+cYB1XqygfEQRSR4DAQgHAAD/aQ4WkQzR2RCYrRW7yTeqJCziQCrt
ZtUOyWyXQa5szggPccJ4BBgWCAAgBQJnWkBuAhsMFiEEvRmDHHjz3OF2z/DCYVey
oSnoY8gACgkQYVeyoSnoY8hhZAD/eOlvL1aivrP3M6ePh5PEqaRUwlpJd5nC6L2L
vGN4uI8A/iMm5biYSHTQf9G4wHS4fVgD7gHYCOqbKyLyv75/B7IAXZj3
=AAAA
-----END PGP PRIVATE KEY BLOCK-----
We got 3 more bytes on the key, but if we repeat the process new checksum is 0 (=AAAA), so we don't get totally random garbage by importing twice.
But maybe C code works differently, need to uncomment the rest of string-producing code and see how it actually works.
I have finished dckeydebug.c:
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#include <stdlib.h>
/**
* Remove all `\r` characters from the given buffer.
* This function will not convert anything else in the future, so it can be used
* safely for marking thinks to remove by `\r` and call this function afterwards.
*
* @param buf The buffer to convert.
* @return None.
*/
void dc_remove_cr_chars(char* buf)
{
const char* p1 = buf; /* search for first `\r` */
while (*p1) {
if (*p1=='\r') {
break;
}
p1++;
}
char* p2 = (char*)p1; /* p1 is `\r` or null-byte; start removing `\r` */
while (*p1) {
if (*p1!='\r') {
*p2 = *p1;
p2++;
}
p1++;
}
/* add trailing null-byte */
*p2 = 0;
}
static inline signed char get_base64_value(char ch)
{
if ((ch >= 'A') && (ch <= 'Z'))
return ch - 'A';
if ((ch >= 'a') && (ch <= 'z'))
return ch - 'a' + 26;
if ((ch >= '0') && (ch <= '9'))
return ch - '0' + 52;
switch (ch) {
case '+':
return 62;
case '/':
return 63;
case '=': /* base64 padding */
return -1;
default:
return -1;
}
}
char* dc_strdup(const char* s) /* strdup(NULL) is undefined, save_strdup(NULL) returns an empty string in this case */
{
char* ret = NULL;
if (s) {
if ((ret=strdup(s))==NULL) {
exit(16); /* cannot allocate (little) memory, unrecoverable error */
}
}
else {
if ((ret=(char*)calloc(1, 1))==NULL) {
exit(17); /* cannot allocate little memory, unrecoverable error */
}
}
return ret;
}
void dc_ltrim(char* buf)
{
size_t len = 0;
const unsigned char* cur = NULL;
if (buf && *buf) {
len = strlen(buf);
cur = (const unsigned char*)buf;
while (*cur && isspace(*cur)) {
cur++; len--;
}
if ((const unsigned char*)buf!=cur) {
memmove(buf, cur, len + 1);
}
}
}
void dc_rtrim(char* buf)
{
size_t len = 0;
unsigned char* cur = NULL;
if (buf && *buf) {
len = strlen(buf);
cur = (unsigned char*)buf + len - 1;
while (cur!=(unsigned char*)buf && isspace(*cur)) {
--cur, --len;
}
cur[isspace(*cur) ? 0 : 1] = '\0';
}
}
void dc_trim(char* buf)
{
dc_ltrim(buf);
dc_rtrim(buf);
}
/* Split data from PGP Armored Data as defined in https://tools.ietf.org/html/rfc4880#section-6.2.
The given buffer is modified and the returned pointers just point inside the modified buffer,
no additional data to free therefore.
(NB: netpgp allows only parsing of Version, Comment, MessageID, Hash and Charset) */
int dc_split_armored_data(char* buf, const char** ret_headerline, const char** ret_setupcodebegin, const char** ret_preferencrypt, const char** ret_base64)
{
int success = 0;
size_t line_chars = 0;
char* line = buf;
char* p1 = buf;
char* p2 = NULL;
char* headerline = NULL;
char* base64 = NULL;
#define PGP_WS "\t\r\n "
if (ret_headerline) { *ret_headerline = NULL; }
if (ret_setupcodebegin) { *ret_setupcodebegin = NULL; }
if (ret_preferencrypt) { *ret_preferencrypt = NULL; }
if (ret_base64) { *ret_base64 = NULL; }
if (buf==NULL || ret_headerline==NULL) {
goto cleanup;
}
dc_remove_cr_chars(buf);
while (*p1) {
if (*p1=='\n') {
/* line found ... */
line[line_chars] = 0;
if (headerline==NULL) {
/* ... headerline */
dc_trim(line);
if (strncmp(line, "-----BEGIN ", 11)==0 && strncmp(&line[strlen(line)-5], "-----", 5)==0) {
headerline = line;
if (ret_headerline) {
*ret_headerline = headerline;
}
}
}
else if (strspn(line, PGP_WS)==strlen(line)) {
/* ... empty line: base64 starts on next line */
base64 = p1+1;
break;
}
else if ((p2=strchr(line, ':'))==NULL) {
/* ... non-standard-header without empty line: base64 starts with this line */
line[line_chars] = '\n';
base64 = line;
break;
}
else {
/* header line */
*p2 = 0;
dc_trim(line);
if (strcasecmp(line, "Passphrase-Begin")==0) {
p2++;
dc_trim(p2);
if (ret_setupcodebegin) {
*ret_setupcodebegin = p2;
}
}
else if (strcasecmp(line, "Autocrypt-Prefer-Encrypt")==0) {
p2++;
dc_trim(p2);
if (ret_preferencrypt) {
*ret_preferencrypt = p2;
}
}
}
/* prepare for next line */
p1++;
line = p1;
line_chars = 0;
}
else {
p1++;
line_chars++;
}
}
if (headerline==NULL || base64==NULL) {
goto cleanup;
}
/* now, line points to beginning of base64 data, search end */
if ((p1=strstr(base64, "-----END "/*the trailing space makes sure, this is not a normal base64 sequence*/))==NULL
|| strncmp(p1+9, headerline+11, strlen(headerline+11))!=0) {
goto cleanup;
}
*p1 = 0;
dc_trim(base64);
if (ret_base64) {
*ret_base64 = base64;
}
success = 1;
cleanup:
return success;
}
static int mailmime_base64_body_parse_impl(
const char * message, size_t length,
size_t * indx, char ** result,
size_t * result_len, int partial)
{
size_t cur_token, last_full_token_end;
char chunk[4];
int chunk_index;
char out[3];
char * mmapstr;
int res;
int r;
size_t written;
chunk[0] = 0;
chunk[1] = 0;
chunk[2] = 0;
chunk[3] = 0;
cur_token = * indx;
last_full_token_end = * indx;
chunk_index = 0;
written = 0;
mmapstr = malloc((length - cur_token) * 3 / 4);
if (mmapstr == NULL) {
exit(1);
}
while (1) {
signed char value;
value = -1;
while (value == -1) {
if (cur_token >= length)
break;
value = get_base64_value(message[cur_token]);
cur_token ++;
}
if (value == -1)
break;
chunk[chunk_index] = value;
chunk_index ++;
if (chunk_index == 4) {
out[0] = (chunk[0] << 2) | (chunk[1] >> 4);
out[1] = (chunk[1] << 4) | (chunk[2] >> 2);
out[2] = (chunk[2] << 6) | (chunk[3]);
chunk[0] = 0;
chunk[1] = 0;
chunk[2] = 0;
chunk[3] = 0;
chunk_index = 0;
last_full_token_end = cur_token;
mmapstr[written++] = out[0];
mmapstr[written++] = out[1];
mmapstr[written++] = out[2];
}
}
if (chunk_index != 0 && !partial) {
size_t len;
len = 0;
out[0] = (chunk[0] << 2) | (chunk[1] >> 4);
len ++;
if (chunk_index >= 3) {
out[1] = (chunk[1] << 4) | (chunk[2] >> 2);
len ++;
}
for (int i = 0; i < len; i++) {
mmapstr[written++] = out[i];
}
written += len;
}
if (partial) {
cur_token = last_full_token_end;
}
* indx = cur_token;
* result = mmapstr;
* result_len = written;
return 0;
free:
free(mmapstr);
err:
return res;
}
int mailmime_base64_body_parse(const char * message, size_t length,
size_t * indx, char ** result,
size_t * result_len)
{
return mailmime_base64_body_parse_impl(message, length, indx, result, result_len, 0);
}
static int set_self_key(const char* armored)
{
int success = 0;
char* buf = NULL;
const char* buf_headerline = NULL; // pointer inside buf, MUST NOT be free()'d
const char* buf_preferencrypt = NULL; // - " -
const char* buf_base64 = NULL; // - " -
char* self_addr = NULL;
buf = dc_strdup(armored);
if (!dc_split_armored_data(buf, &buf_headerline, NULL, &buf_preferencrypt, &buf_base64)
|| strcmp(buf_headerline, "-----BEGIN PGP PRIVATE KEY BLOCK-----")!=0 || buf_base64==NULL) {
return 1;
}
size_t result_len = 0;
char* result = NULL;
size_t indx = 0;
mailmime_base64_body_parse(buf_base64, strlen(buf_base64), &indx, &result/*must be freed using mmap_string_unref()*/, &result_len);
fwrite(result, 1, result_len, stdout);
fflush(stdout);
}
int main(int argc, char *argv[]) {
FILE *fp = fopen(argv[1], "r");
if (fp == NULL) return 1;
char armored[4096];
fread(armored, sizeof armored[0], sizeof armored / sizeof armored[0], fp);
set_self_key(armored);
return 0;
}The result is identical:
$ ./a.out alice.asc | rsop armor
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1pDvxYJKwYBBAHaRw8BAQdAWFWG7B4xlcBMIVkrwZq+5kyOt8TfvUhurJUW
w2BDhxIAAQCE+7D59g49hIhlSkX/fzFP9Y2BPid3h5JYCuY2cAGPxQ+jzRFhbGlj
ZUBleGFtcGxlLm9yZ8KPBBAWCAA3AhkBBQJnWkO/AhsDCAsJCAcKDQwLBRUKCQgL
AhYCAScWIQTFMF1Wsz4Sp8gsSDk588zEG6j4FQAKCRA588zEG6j4FTJbAQC73wiu
+A/YXbvC2U+G6hRroqGRqZa9FX8g4ITTPiXgGAEA1IZqRmOp9+FhLc4QTA6LzCCi
HSXQGQ+feXeT6xmrcwrHXQRnWkO/EgorBgEEAZdVAQUBAQdA6EnzS9tv0vssfqim
HhSIpPIsZVHyG7OeUMAfA3f3+gcDAQgHAAD/Y/Grg/BA1mrKh0JrkK3synGMoOny
WPDxxLX2dhAeusAVIMJ4BBgWCAAgBQJnWkO/AhsMFiEExTBdVrM+EqfILEg5OfPM
xBuo+BUACgkQOfPMxBuo+BUVmgD/XZV/slIrQKgWVphmtaBGIwqxWjwMn9jXD6TV
hyYuUMABAKxLxgsMdnD82v5tqcomL4khIqMvEcblRksqKmc1TA0NIeoP
=AAAA
-----END PGP PRIVATE KEY BLOCK-----
$ sed '/^-----/d;s/=//' < alice.asc | base64 -d | rsop armor
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1pDvxYJKwYBBAHaRw8BAQdAWFWG7B4xlcBMIVkrwZq+5kyOt8TfvUhurJUW
w2BDhxIAAQCE+7D59g49hIhlSkX/fzFP9Y2BPid3h5JYCuY2cAGPxQ+jzRFhbGlj
ZUBleGFtcGxlLm9yZ8KPBBAWCAA3AhkBBQJnWkO/AhsDCAsJCAcKDQwLBRUKCQgL
AhYCAScWIQTFMF1Wsz4Sp8gsSDk588zEG6j4FQAKCRA588zEG6j4FTJbAQC73wiu
+A/YXbvC2U+G6hRroqGRqZa9FX8g4ITTPiXgGAEA1IZqRmOp9+FhLc4QTA6LzCCi
HSXQGQ+feXeT6xmrcwrHXQRnWkO/EgorBgEEAZdVAQUBAQdA6EnzS9tv0vssfqim
HhSIpPIsZVHyG7OeUMAfA3f3+gcDAQgHAAD/Y/Grg/BA1mrKh0JrkK3synGMoOny
WPDxxLX2dhAeusAVIMJ4BBgWCAAgBQJnWkO/AhsMFiEExTBdVrM+EqfILEg5OfPM
xBuo+BUACgkQOfPMxBuo+BUVmgD/XZV/slIrQKgWVphmtaBGIwqxWjwMn9jXD6TV
hyYuUMABAKxLxgsMdnD82v5tqcomL4khIqMvEcblRksqKmc1TA0NIeoP
=AAAA
-----END PGP PRIVATE KEY BLOCK-----
So I think there is no memory corruption or anything like that on C side, added "garbage" is indeed CRC-24 dearmored as key contents.
This does not explain why 6 bytes all look random-ish. Normally feeding the key concatenated with CRC-24 into CRC-24 results in 0 checksum: https://en.wikipedia.org/wiki/Computation_of_cyclic_redundancy_checks#One-pass_checking
I cannot find the code in C core that could have modified the key between importing and storing in the database, maybe something happens in NetPGP while writing armored version of invalid key.
In any case we cannot assume anything about these garbage bytes except that their number is divisable by 3. So on the core side I guess we have to chop 3 bytes from the end until the key parses or up to some limit.
Regardless of whether we make rPGP accept such keys, I think Delta Chat core should ignore keys it fails to parse when loading the keyring if they are not selected as default and not fail sending messages because the key it was not going to use anyway failed to parse. For reception also makes sense to try to decrypt with usable keys and ignore the keys we cannot load, even if they are default. If default key cannot be loaded and we want to send then I don't know what to do, generating a new key seems risky.
I have looked into the code now. Delta Chat already ignores the keys it cannot load in the keyring by filtering them out:
https://github.com/deltachat/deltachat-core-rust/blob/cc672b81faaf1cde62f665b1ef5d2af525f02878/src/key.rs#L182
So I think @pepea28 has keys 1, 3 and 4 with key_id equal to 4 in the config table, and this key 4 is corrupted. Probably the same with @angelo-fuchs, primary key that was imported later is corrupted.
As ignoring the keys which fail to load is already implemented, and for primary key errors cannot be ignored, I am going to make it more robust by removing 3 last bytes if it fails to load.
I managed to produce some garbage by changing the user ID so the key itself has some = padding at the end.
Running this script (with alic@example.org instead of alice@example.org) and C program compiled from the source above:
#!/bin/sh
rsop generate-key alic@example.org | rsop armor > alice.asc
cat alice.asc
./a.out alice.asc | rsop armor > alice2.asc
cat alice2.asc
./a.out alice2.asc | rsop armor > alice3.asc
cat alice3.asc
The result is:
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1qQLBYJKwYBBAHaRw8BAQdAiVL3tP0kWyXCGaqyOJnCOxtJYsW4mbELRiLA
0fLgTFUAAQCAp9Vbt2Yhc+OgPkMgnhbBtVF9U44ZcdRIPK9nx7VchA9azRBhbGlj
QGV4YW1wbGUub3Jnwo8EEBYIADcCGQEFAmdakCwCGwMICwkIBwoNDAsFFQoJCAsC
FgIBJxYhBKrF98GAQ61HZ9rxoV0o2J6NbXHdAAoJEF0o2J6NbXHdgNUBAM8eFw8b
udCzNcfsRlXqHWSepaI8g0iSXZwzxyLvvOJFAP4sShJr2tsv0Akx6A8OmAoQhurm
eTN84cFw72C2mERWBsddBGdakCwSCisGAQQBl1UBBQEBB0A4Et5PeW2+GWTcSC+H
2qb3Dx3qfgPQh2HEuXHs4MEgNwMBCAcAAP9Lyr4Ylz4T3vkOCgkT6D2YVocJrLCH
3BEv7+M1jOpQ+BBEwngEGBYIACAFAmdakCwCGwwWIQSqxffBgEOtR2fa8aFdKNie
jW1x3QAKCRBdKNiejW1x3TrNAQCRxza+l6XsASCtMQekJwpjrDfK4VumZ2SCKRKw
e/4vFAD9HMgDMX2rCzjXYcRFNiyzxc50ZcPdVfwzA1tEahwAYA4=
=Aqyb
-----END PGP PRIVATE KEY BLOCK-----
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1qQLBYJKwYBBAHaRw8BAQdAiVL3tP0kWyXCGaqyOJnCOxtJYsW4mbELRiLA
0fLgTFUAAQCAp9Vbt2Yhc+OgPkMgnhbBtVF9U44ZcdRIPK9nx7VchA9azRBhbGlj
QGV4YW1wbGUub3Jnwo8EEBYIADcCGQEFAmdakCwCGwMICwkIBwoNDAsFFQoJCAsC
FgIBJxYhBKrF98GAQ61HZ9rxoV0o2J6NbXHdAAoJEF0o2J6NbXHdgNUBAM8eFw8b
udCzNcfsRlXqHWSepaI8g0iSXZwzxyLvvOJFAP4sShJr2tsv0Akx6A8OmAoQhurm
eTN84cFw72C2mERWBsddBGdakCwSCisGAQQBl1UBBQEBB0A4Et5PeW2+GWTcSC+H
2qb3Dx3qfgPQh2HEuXHs4MEgNwMBCAcAAP9Lyr4Ylz4T3vkOCgkT6D2YVocJrLCH
3BEv7+M1jOpQ+BBEwngEGBYIACAFAmdakCwCGwwWIQSqxffBgEOtR2fa8aFdKNie
jW1x3QAKCRBdKNiejW1x3TrNAQCRxza+l6XsASCtMQekJwpjrDfK4VumZ2SCKRKw
e/4vFAD9HMgDMX2rCzjXYcRFNiyzxc50ZcPdVfwzA1tEahwAYA4AqyYAAA==
=F7AL
-----END PGP PRIVATE KEY BLOCK-----
-----BEGIN PGP PRIVATE KEY BLOCK-----
xVgEZ1qQLBYJKwYBBAHaRw8BAQdAiVL3tP0kWyXCGaqyOJnCOxtJYsW4mbELRiLA
0fLgTFUAAQCAp9Vbt2Yhc+OgPkMgnhbBtVF9U44ZcdRIPK9nx7VchA9azRBhbGlj
QGV4YW1wbGUub3Jnwo8EEBYIADcCGQEFAmdakCwCGwMICwkIBwoNDAsFFQoJCAsC
FgIBJxYhBKrF98GAQ61HZ9rxoV0o2J6NbXHdAAoJEF0o2J6NbXHdgNUBAM8eFw8b
udCzNcfsRlXqHWSepaI8g0iSXZwzxyLvvOJFAP4sShJr2tsv0Akx6A8OmAoQhurm
eTN84cFw72C2mERWBsddBGdakCwSCisGAQQBl1UBBQEBB0A4Et5PeW2+GWTcSC+H
2qb3Dx3qfgPQh2HEuXHs4MEgNwMBCAcAAP9Lyr4Ylz4T3vkOCgkT6D2YVocJrLCH
3BEv7+M1jOpQ+BBEwngEGBYIACAFAmdakCwCGwwWIQSqxffBgEOtR2fa8aFdKNie
jW1x3QAKCRBdKNiejW1x3TrNAQCRxza+l6XsASCtMQekJwpjrDfK4VumZ2SCKRKw
e/4vFAD9HMgDMX2rCzjXYcRFNiyzxc50ZcPdVfwzA1tEahwAYA4AqyYAAAF7AAA=
=FhcP
-----END PGP PRIVATE KEY BLOCK-----
Length increases not in step of 3, but first by 5 then by 4:
$ rsop dearmor < alice.asc | wc -c
470
$ rsop dearmor < alice2.asc | wc -c
475
$ rsop dearmor < alice3.asc | wc -c
479
Have not tried to figure out how to get 6 non-zero bytes, but there are more possibilities than just getting 3 bytes of garbage once.
So I will have to try removing 1 byte from the end up to 10 bytes or so until it works as a workaround, because it is hard to make any assumptions
@link2xt I confirm that in the config table, my key_id = 4.
I also confirm that by removing the 6 bytes, rsop sign works.
gpg --list-packets on the corrected key (having removed the 6 bytes) gives :
# off=0 ctb=c5 tag=5 hlen=3 plen=920 new-ctb
:secret key packet:
version 4, algo 1, created 1554208260, expires 0
pkey[0]: [2048 bits]
pkey[1]: [17 bits]
skey[2]: [2046 bits]
skey[3]: [1024 bits]
skey[4]: [1024 bits]
skey[5]: [1023 bits]
checksum: 3d5f
keyid: EAD46491F50598C9
# off=923 ctb=cd tag=13 hlen=2 plen=17 new-ctb
:user ID packet: "<xxxxxxxxxxxxx@xxxxxxxxxx>"
# off=942 ctb=c2 tag=2 hlen=3 plen=316 new-ctb
:signature packet: algo 1, keyid EAD46491F50598C9
version 4, created 1554208261, md5len 0, sigclass 0x13
digest algo 8, begin of digest 95 4e
hashed subpkt 2 len 4 (sig created 2019-04-02)
hashed subpkt 9 len 4 (key does not expire)
hashed subpkt 25 len 1 (primary user ID)
hashed subpkt 27 len 1 (key flags: 03)
hashed subpkt 11 len 5 (pref-sym-algos: 9 7 3 2 1)
hashed subpkt 21 len 5 (pref-hash-algos: 8 9 10 11 2)
hashed subpkt 22 len 1 (pref-zip-algos: 2)
hashed subpkt 30 len 1 (features: 01)
subpkt 16 len 8 (issuer key ID EAD46491F50598C9)
data: [2048 bits]
# off=1261 ctb=c7 tag=7 hlen=3 plen=920 new-ctb
:secret sub key packet:
version 4, algo 1, created 1554208261, expires 0
pkey[0]: [2048 bits]
pkey[1]: [17 bits]
skey[2]: [2047 bits]
skey[3]: [1024 bits]
skey[4]: [1024 bits]
skey[5]: [1024 bits]
checksum: 34f1
keyid: 683A0916AD9D9A7F
# off=2184 ctb=c2 tag=2 hlen=3 plen=293 new-ctb
:signature packet: algo 1, keyid EAD46491F50598C9
version 4, created 1554208261, md5len 0, sigclass 0x18
digest algo 8, begin of digest 81 4c
hashed subpkt 2 len 4 (sig created 2019-04-02)
hashed subpkt 9 len 4 (key does not expire)
hashed subpkt 27 len 1 (key flags: 0C)
subpkt 16 len 8 (issuer key ID EAD46491F50598C9)
data: [2048 bits]
I don't know if it helps