Signing keys clarification needed
kapitainsky opened this issue · comments
Trying to sign AppImage with my pgp keys:
Embedding MD5 digest
gpg2 and sha256sum are installed and user requested to sign, hence signing
Error: cannot embed key in AppImage: size exceeds reserved ELF section size
Could not find any information about what keys are permitted, obviously mine is too long (4096 bits). Any details on this somewhere?
The issue is in reference to linuxdeploy/linuxdeploy-plugin-appimage#9
Hello @kapitainsky and welcome to AppImage.
In the current implementation, ELF section .sig_key is 8192 bytes long. It is not part of the spec yet, but we are working on it.
I have been using 4096 bit GPG keys successfully. The ASCII ("armored") version of the public key needs to fit into 8192 bytes, and it does easily (for me):
https://travis-ci.org/probonopd/cool-retro-term/builds/616227444#L791
Could it be that you have a key with subkeys, or something like that?
I realize that setting up signing is currently a cumbersome, manual process and I am writing a tool to automate it.
cc @TheAssassin
Indeed I have a subkey. Have to look at it closer.
But as I see that all signing is still work in progress. It is fine. I will stick in the meantime to detached signature file.
Will come back to it when it is more mature. Thanks a lot!
One last thought on this. I might have one or more subkeys but it should not mean that AppImage should include all of them. Only one used for signing will be sufficient. Also if definition of .sig_key section is not part of spec maybe it would be wise not to limit its lengths? Who knows what future holds for keys. You used gpg to do all job anyway. And if some key is 8KB or 20KB does not really matter given size of typical AppImage
Currently the maximum size must be known beforehand, as it needs to be pre-allocated. This will be fixed in future type specifications.
Remember, your key doesn't require the very generous 8kB very likely. We only save the public part into the AppImage.
Correct. But as I have subkeys overall size can be bigger. It is only necessary to save key used for signing not all of them which is probably what is done today. And IMHO format specification should be future proof as much as possible. Why to limit yourself to 8k? Make it variable and not limited by design decision.
Why to limit yourself to 8k? Make it variable and not limited by design decision.
The keys are saved in ELF sections that are allocated at compile time of the AppImage runtime, when we don't know the size of the key yet. The key (and the signature) are not saved inside the AppImage filesystem but in ELF sections to make it possible to exchange/update them without having to recreate the whole filesystem.
The solution is to use a dedicated signing key that is only used for this purpose. I am working on a tool to make setting this up almost fully automatic.
ok... people have keys. and keys have history and built trust. You cant just say Appimage needs new special key. So far only option is to just say --sign. I cant choose what key to use. Think from user perspective. I have had my key for years. It works. gpg does good work. I sign email and files for years. and suddenly it is oups you need another key.
maybe whole concept to including public part of the key is just wrong? why not only signature? AppImage is not going to solve keys trust issues. and should not even try.
Pretty shortsighted point of view. Of course you need the public key, it's essential to actually check if a signature is valid. It doesn't mean you have to trust the signature. That trust must be created differently.
AppImageUpdate for instance is making heavy use of the embedded pubkey. The trust is created using a transitive model (à la "I have the old AppImage, I trust it, so if the new AppImage's signature is valid and the old signature is valid as well, I can trust the new AppImage if it's signed with the same key as the old one"). You have to have both keys in order to make such a comparison.
The problem here is not that the key is embedded. The problem is the "signing algorithm" that makes way too many assumptions, which is due to the annoying point of view seen in many AppImage related tools which I often disagree with, that configuration is bad and unnecessary and the tool always knows better than the user.
So, don't hate the signature embedding, hate the tool which does the embedding in a way that a) breaks and b) is not what you want. The assumption that the size of the section is sufficient was made on facts; you can calculate the space one single key requires nowadays, and 8k is a good bit larger than that. The space could always be increased.
I have written the algorithm to embed the keys back in the good ol' days, and basically followed the scheme that the original signature embedding was following, i.e., just calling gpg with some export parameter and embedding the result. We can make the embedding more configurable to allow you to select the right key to embed. Or, even better, just export the required subkey.
When it comes to signing, signatures, and trust, it is important to know the "threat model" and what we want to achieve. Since this is a complex topic, I would suggest that we write a "theory of operation" that explains it. @TheAssassin has already pointed some points out.
We can make the embedding more configurable
Actually, I'd propose to the exact opposite - make it zero-configuration, "just works". My experience is that everything that can be configured will increase complexity and breakage. I have an example implementation in the works.
ideally would be both worlds together @probonopd "just work" should be default but @TheAssassin is right that giving "advanced" configuration options for "power users" make all tool much more powerful
@probonopd your experience doesn't apply to build tools then, I'd say. The solution can't be "ah, you just have to set up a custom GPG home, export your key on your old one, import it in the new one, then make appimagetool use the new, and clean up everything afterwards; not to mention you need to make sure your keys remain confidential in the process". Compared to "just export GPG_KEY_ID=<my key ID>", not making this configurable is nonsense. What's more complex: maintaining a tiny switch in the codebase or requiring huge complicated and complex workarounds to solve real-world problems?
Actually, I've been working in my experimental go implementation to make all those steps (and more) fully automatic. For me, the ideal tool is a one-liner that does everything in a standardized way without needing any configuration. (And no, I'm not there yet...)
@probonopd your experience doesn't apply to build tools then, I'd say.
My experience with "traditional" build tools is that nothing works without having to "fiddle around". Notable exception: Xcode on the Mac. You click the "compile" button, and that's basically it.
Hi!
About the ELF section sizes, I understand we need to know the public key size before compiling in order to determine the size of .sig_key. However, you could already do that if you run a strlen of the gpg --export command before compiling, and allocating the required amount of space for that purpose. This should alleviate the requirement for different sig_key sizes.
Consider also that different keys may produce different signature sizes, although the same key will produce the same signature size for different messages of the same size. Thus, you can also use this approach to calculate the .sha256_sig header when building.
Also, couple of questions on this that would appreciate clarification, both in the current state, but also to "where the spec is going".
I am implementing a verifier for signed AppImages with a controlled trust source. I don't want to rely on either validate.c nor AppImageUpdate code for several reasons, one of them is that my codebase is Python-based and the ELF format is also well-supported to read there with portable code. Also, because AppImage being a spec, it should be feasible and make for stronger spec compliance.
From my experience and use case considerations, I would strongly advocate for keeping separate sections for the public key and the signature. One can accommodate not having the public key inside the AppImage (as they could also accommodate not having the signature itself and use detached signatures), but I believe a strong point is precisely to avoid scattering different files. Using ELF sections seems like the only possible path we you ever need to re-sign AppImages due to a lost or compromised key without requiring recompilation, and also to allow CI/CD flows that could sign already-compiled AppImages as a single centralized step outside AppImage compilation.
E.g., you might want the same AppImage signed with different keys for different purposes due to different trust sources in the final deployments.
On what other content the .sha256_sig may have, I don't mind very much as long as it can be passed to gpg as if it were a PGP signature.
Tests with GPG indicate that precedent text can be admitted as long as there is a newline before -----BEGIN PGP SIGNATURE-----, but I have not reviewed if this is part of the OpenPGP spec.
From a compliance point of view, adding .sig_key would be of interest in order to assert that the digest used for the signature placed in .sha256_sig, if present, MUST assume that both .sha256_sig and .sig_key sections exist with the same length and filled with 0x00. Consider whether .upd_info should adhere or not to that same assumption. I personally don't think the update information should be immutable, and that even client-side tools or proxies might be interested in meddling with that field. If immutability of the whole archive is required, a detached signature can still be provided, but we would have no other way of providing this functionality.
Consider also that currently you are NOT signing the digest of the AppImage. You are currently signing the hex-string of the digest of the AppImage. The actual digest is a binary bytestring, not ASCII text. I don't mind very much, but you are performing a non-trivial transformation which reduces the possibilities from [0x00-0xFF] (256) to just [0x30-0x39] + [0x61-0x66] (16). The spec should be rephrased, and if a different spec is desired, an upgrade path would be desirable.
@ssaavedra welcome to the AppImage project and thank you for your remarks and suggestions. Maybe you'd also like to hang out on #AppImage on irc.freenode.net where many of the devs meet.
About the ELF section sizes, I understand we need to know the public key size before compiling in order to determine the size of .sig_key.
Please keep in mind that the runtime is pre-compiled, and the key is merely placed into it by writing at a defined offset. Essentially we need to make sure that the key that gets written into the already-compiled runtime is no larger than the padding that is inside the aready-compiled ELF. The key can be shorter, though.
I personally don't think the update information should be immutable, and that even client-side tools or proxies might be interested in meddling with that field.
Agree. The update information was put into an ELF section (rather than into the squashfs filesystem) exactly for the reason that we want to allow AppImages to be moved e.g. to a different URL without the need to re-create them entirely; and allow for just changing the update information while leaving the rest of the AppImage untouched.
Consider also that currently you are NOT signing the digest of the AppImage. You are currently signing the hex-string of the digest of the AppImage.
Looks like you are knowledgeable about the security implications here. We are interested in your suggestions how to improve, e.g., for an upcoming type-3 format which we have recently started discussing.
Thank you!
I'll try to, I currently have my znc setup a bit rotten and will try to use the pandemic situation to try to remedy that, thanks for the invite : )
Please keep in mind that the runtime is pre-compiled, and the key is merely placed into it by writing at a defined offset.
I didn't get that the pre-compilation was to happen that much early in time. It makes sense, and sorry for my misunderstanding. In that case I would advocate to pad into the section a bit more room.
Keys with subkeys are common, and should probably not be discouraged to use, because from a security perspective, having a CI system with "naked" subkeys (no main private key stored) is probably easier to revoke in more scenarios.
From a transport point-of-view, it may be a bit worse, but for http(s) webservers under Accept-Encoding: gzip could mitigate the added transport size (since compressing multiple 0x00 is trivial under that encoding) and zsync can also leverage the same properties.
In any case, the added padding is likely to be very small compared to the overall footprint.
Besides, instead of using --armored versions, we could, instead, use the binary signatures that gpg --export provides. That should reduce about ~30% the space requirements.
However, if we do so, we have to address the boundaries issue: it is no longer acceptable to break on \0 (as \0 could be a valid byte of the signature). The most straightforward solution I can think of is to include the length of the field as part of it. Since the region can be bounded by an order of magnitude, we can use a fixed number of bytes to signal the length of the field in bytes (I would suggest reserving the first 4 bytes to be safe, and force the first to be zero for future flags if the need arises). Armored signatures are nice in the sense that we can see them easily in a hexdump of the ELF. However, they are quite heavier on disk. Consider that the binaries can use among others the 0x00 byte, so it is no longer safe to assume these are "strings" in the strlen(3) sense, for example.
Agree. The update information was put into an ELF section (rather than into the squashfs filesystem) exactly [for that reason]
I know, I read your rationale, I was just stating my agreement with it. It seems @TheAssassin and you were discussing that, and I wanted to offer my opinion. However in the current implementation this means that the update information is not signed. Maybe interest in expanding the field to fit a digital signature could be explored.
Also, the system I'm designing cannot use the standardized update approach in its current state (among other reasons, canonical paths won't be https). In particular, I'm exploring offering AppImage updates over an abstraction of the dat protocol, but we can leave that discussion for a different thread. If you are talking about an upcoming type-3 format, it might be interesting to include "pluggable upgrade transports" as an option to the AppImageUpdate system.
Looks like you are knowledgeable about the security implications here.
Well, I'm not such a deep expert in cryptography to argue about the nuances of this. I have asked the question on security.SE, see if we can have more reasoned arguments than what I can offer. I was mostly stating this because of the validator that I'm putting together, which in order to implement, I have to know how to parse the different fields. Currently the only place where I have enough information to do so is by looking at the source code. This means, in my opinion, that the spec is underspecified or too ambiguous. I have proposed a more "ambitious" spec text in this PR: AppImage/AppImageSpec#30
You may not want to transition to binary digital signatures, but usual tools (e.g., GPG, OpenSSL) tend to use that approach. Besides, in the current implementation it seems you are signing the "hash of a hash", because GPG will already use your input as a "message" to be hashed in order to compute the digital signature. Instead, you would probably want to send the whole ELF content to GPG instead (swapping the appropriate headers by zeros). Changing this would mean that the updater here would need to send the ELF and not a signature to GPG. However, instead of creating the file on-disk you can also pass it to GPG as stdin (this way you don't need to copy the whole file in the disk minus a few sectors of the signature).
I do have an interest on using the AppImage format for distribution of software packages, and it would be great to be part of such discussions. I hope to get on the IRC soon-ish. Thank you for the greeting, and hope this to be a constructive exchange :)
Thank you very much @ssaavedra this is really helpful for us. Thanks for your PR as well 👍
Looking forward to meeting you on IRC.
I've just gotten bitten by this problem, too. I'm using a key with subkeys which used to work, but recently I added two more IDs to it, and that seems to have pushed its size over the limit.
Going back to unsigned images for the time being.
Or you might create a new signing key just specifically for this purpose? @mbunkus
@TheAssassin this topic is something to be considered for type-3 too.
Yeah sure, I could, but it's not really worth the effort. I'm a bit pessimistic about signed binaries. As long as systems don't forcefully require signatures (e.g. apt complaining loudly about unsigned repositories, dito rpm about unsigned packages, macOS etc.), users really don't care. I bet I won't receive a single complaint that my AppImages aren't signed anymore, especially given the fact that AppImage signature validation doesn't seem to work at all anyway (#949).
We may change that at some point in the future, but for now I guess you are correct. (It's no bad will but mainly a lack of time and contributors.)
The main reason AppImage signing isn't deployed widespreadly is its complexity. You need to understand the ELF format quite well, extract data from there, then calculate hashes but skip/zero out parts of the file, ...
For type 3 I plan to split signing into multiple parts. The runtime could be signed by us (AppImage), the payload and the metadata signed by the user. It'll be less complex, as the file format is a lot simpler. These ELF sections are cumbersome and annoying to work with.
Uh, let's not make even more complicated. Our format needs to become simpler, if anything.
go-appimaged will do all the signing automagically tor the current type-2 format.
It'll be less complex, as the file format is a lot simpler.
What in the world gives you the impression things would become more complex? The opposite is true. No more "skipping" of any kind, which is annoying anyway...
OK, I guess we should discuss it "in Ruhe" :)