Linux Cross-Platform Packaging
benhosmer opened this issue · comments
A continuation of the need for pre-built rpms that lead to exploration of creating portable linux desktop builds. #14 for historical reference.
I am happy to help set up AppImage generation. Would something like Travis CI be suitable or are you using another build pipeline?
@probonopd I believe zeal uses Travis already: https://travis-ci.org/abn
Here's the thing, I've been working on building flatpak packages for zeal. I've never built flatpak or appimage. I've consumed both. I decided on flatpak because the documentation for appimage was really confusing.
It would be nice to have a hello-world example of how to create an appimage. I do like the fact that appimage seems to be lighter weight for users, since there's nothing to install, unlike flatpak.
My perspective is that no one really cares that much as to which one this project uses. I volunteered to do flatpak since it was my original desire to have rpms that spawned this. If you're willing to help with appimage, I certainly won't turn that down.
I decided on flatpak because the documentation for appimage was really confusing.
That is very unfortunate. Can you explain exactly what you were struggling with? We think we have quite extensive documentation in https://github.com/AppImage/AppImageKit/wiki/Creating-AppImages
It would be nice to have a hello-world example of how to create an appimage
Check https://github.com/probonopd/linuxdeployqt#using-linuxdeployqt-with-travis-ci and especially the real-world examples at https://github.com/probonopd/linuxdeployqt#projects-using-linuxdeployqt
Please let me know what is unclear so that we can improve it.
I believe zeal uses Travis already: https://travis-ci.org/abn
Seems to be the private project of Arun Babu Neelicattu, is that where the official Zeal packages get built?
@probonopd that travis build config corresponds to https://github.com/abn/zeal-rpm and as of now there are no official zeal rpm packages builds. That builds an srpm in a clean room environment based on the zeal-packaging rpm spec file and then triggers a copr build that builds the packages distributed in that repository (unofficial).
@benhosmer @probonopd I believe it is okay to have PoCs of both AppImage and Flatpack versions of the application; and make a decision at a later point as to which one to use or both. Just adding my 2c there.
So would a PR for https://github.com/zealdocs/zeal that would implement building on Travis CI be considered?
@probonopd Zeal itself doesn't use Travis (https://travis-ci.org/zealdocs has no projects). And I am not sure what do you mean by "official packages". AFAIK all distros that ship Zeal have their own build services, same for our Ubuntu PPA.
There are only official builds for Windows, but they are unrelated to this discussion.
Yes, how can we get Linux builds that are done by upstream (and not by a third party like a distribution) like for Windows?
I guess that depends on actual packaging format. If Travis CI is enough for building and releasing AppImage packages, let's use it. If not, we can either use another public CI service, or our own Digital Ocean VMs. I would prefer some public CI, since it should require less maintenance going forward.
Agree, so let's try to do it on Travis CI.
Started the work here: https://github.com/probonopd/zeal/blob/patch-1/.travis.yml
The AppImage seems to be working for me:
https://github.com/probonopd/zeal/releases
Please test it and report back here. If it works for you, too, I will send a PR.
Minor issue, the correct icon is not used here:
Probably it is attempted to be loaded from /usr/share where it does not reside. Quite possibly the source code would need to be adjusted to use a relative path (relative to the main executable) instead.
@probonopd I finally got a chance to test your build, and for the most parts it works quite well, so I like the idea. Here's some observations:
- App indicator for some reason could not update its menu, and always displayed "Show Zeal" option.
- I didn't encounter your problem with the tray icon.
- Qt WebKit is super old and buggy, if we bundle all dependencies anyway, then we should include the most recent veresions.
Tested on Ubuntu 16.04 x64 with Unity.
Isn't the icon on my screenshot the "app indicator"?
Which Qt/WebKit version should we use? If I am not mistaken, Qt moved from WebKit to WebEngine (Chromium based) in newer versions.
In any case @trollixx, do you think you could take over from https://github.com/probonopd/zeal/blob/patch-1/.travis.yml and make changes as you see fit?
Isn't the icon on my screenshot the "app indicator"?
No idea. The icon is fine on Arch Linux + AwesomeWM as well.
Which Qt/WebKit version should we use? If I am not mistaken, Qt moved from WebKit to WebEngine (Chromium based) in newer versions.
Qt WebKit is being revived. See this article. Arch Linux now ships the updated version by default.
In any case @trollixx, do you think you could take over from https://github.com/probonopd/zeal/blob/patch-1/.travis.yml and make changes as you see fit?
I guess I can give it a try. Not sure on the timeline.
By the way, is there a way to bundle OpenSSL in AppImage as well? On my Arch you package couldn't connect to the Internet, because it wanted OpenSSL 1.0, and I have 1.1.
Thanks for the update on Qt WebKit. Interesting.
By the way, is there a way to bundle OpenSSL in AppImage as well?
Yes, but I'd ask you to exactly describe your system and the errors you are getting on https://github.com/probonopd/linuxdeployqt/issues/, possibly (if it looks to be related) in probonopd/linuxdeployqt#110. Thanks!
@probonopd I created a ticket explaining what's going on on Arch Linux. There's really no good workaround rather than bundling either OpenSSL 1.0 or 1.1, depending on Qt version. How do I tell linuxdeployqt to include OpenSSL? Worth noting, that it's a runtime, not a link dependency.
The wiki doesn't mention OpenSSL 1.1 at all.
Are you saying that applications compiled for OpenSSL 1.0 refuse to run on OpenSSL 1.1 systems in general? Wouldn't that be an OpenSSL bug? (In my opinition, if I compile anything against version n of a library, I expect my application to still run with version n+.1 of said library - otherwise the library cannot be relied on.)
Relevant details:
https://wiki.openssl.org/index.php/OpenSSL_1.1.0_Changes#No_longer_works
Certainly something we might want to mention on https://github.com/AppImage/AppImageKit/wiki/Desktop-Linux-Platform-Issues
@probonopd Not a bug, just confusing versioning OpenSSL uses.
So we do need to bundle OpenSSL to be able to run on systems, that don't have v1.0 or v1.1 (depending on the system AppImage is packaged on).
You mentioned above that bundling is possible, but I don't see how. There's only a switch to include system libraries, but again Qt is not linked to OpenSSL, so that would not work.
You mentioned above that bundling is possible, but I don't see how.
Just put it inside the AppDir prior to it being converted to an AppImage. You may need to modify it so that it will search for certificates in all the different places in which different Linux distributions tend to scatter them around.
On my Arch you package couldn't connect to the Internet, because it wanted OpenSSL 1.0, and I have 1.1.
I think OpenSSL 1.1 is broken if it cannot run applications that are compiled against OpenSSL 1.0.
Please add this example to https://wiki.openssl.org/index.php/OpenSSL_1.1.0_Changes#No_longer_works. They are explicitly asking for it:
If you find your library or program used to work with OpenSSL 1.0.2 but no longer works with OpenSSL 1.1.0, then please add details to discussion below at Things that no longer work.
Thank you.
Qt WebKit is being revived. See this article. Arch Linux now ships the updated version by default.
I am trying to compile Zeal with the official Qt 5.10.0 from http://download.qt.io/online/qtsdkrepository/linux_x64/desktop/qt5_5100.
I can see components called "qtwebengine", "qtwebview" and "qtwebchannel" but nothing named "qtwebkit".
The build fails with
CMake Error at /home/travis/Qt/5.10.0/gcc_64/lib/cmake/Qt5/Qt5Config.cmake:28 (find_package):
Could not find a package configuration file provided by "Qt5WebKit" with
any of the following names:
Qt5WebKitConfig.cmake
qt5webkit-config.cmake
Add the installation prefix of "Qt5WebKit" to CMAKE_PREFIX_PATH or set
"Qt5WebKit_DIR" to a directory containing one of the above files. If
"Qt5WebKit" provides a separate development package or SDK, be sure it has
been installed.
What do I need to do?
It's not yet shipped as part of the official SDK, although some distributions (Arch, Debian) already package it. So for now you need to build it yourself or use prebuilt binaries.
@benhosmer Just curious, have you had any luck with Flatpack?
@trollixx I think I started a skeleton, but abandoned it when @probonopd volunteered to do appimage.
I think it would make sense to provide all popular packaging options, just cover wider audience.
looking forward to have a Flatpak app as well :)
If someone gets this to build on Ubuntu 14.04 (trusty) on Travis CI, I'm happy to take over from there and do the AppImage.
Here is what I have so far:
https://github.com/probonopd/zeal/blob/qt5100/.travis.yml
So for now you need to build it yourself or use prebuilt binaries.
I don't know what I need to do exactly. Can you send a PR to my file above?
@probonopd Ubuntu 14.04 is too old. We need Qt 5.6 minimum. And in a couple months we'll bump to 5.9.
Another thing that should be addressed is Qt WebKit. We need to bundle with 5.212 which has A LOT of bug fixes and improvements compared to any previous version.
Ideally I would prefer to have a custom built Qt as well. We can drop a bunch of features Zeal doesn't need, e.g. QML.
@probonopd Ubuntu 14.04 is too old. We need Qt 5.6 minimum. And in a couple months we'll bump to 5.9.
Qt 5.10 is available for trusty, either from qt.io or from https://launchpad.net/~beineri/+archive/ubuntu/opt-qt591-trusty (soon).
I can look at adding Zeal to Flathub, even more so if someone from Zeal team would be interested in co-maintaining it. With KDE runtime available, it wouldn't need more attention than changing tag/commit used to build from.
While I maintain Arch package, I find any distribution specific work on leaf packages waste of time and I'd like to orphan it at some point.
@barthalion I am all for publishing Zeal to as many places as possible, I am just concerned about the cost of maintenance. Updating a PKGBUILD takes a couple of minutes, if maintaining the Flatpack is about the same effort, then I can take care of it.
@trollixx I have a prototype flatpak packaging working at https://github.com/abn/zeal-packaging/tree/flatpak/flatpak having to rebuild qtwebkit every time makes me think that should be build and managed elsewhere as an extension to the KDE runtime. Will improve on it and send a PR soon.
Looks like I missed work being done in zealdocs/zeal#911. Will review status across projects.
If someone gets this to build on Ubuntu 14.04 (trusty) on Travis CI, I'm happy to take over from there and do the AppImage.
Here is what I have so far:
https://github.com/probonopd/zeal/blob/qt5100/.travis.yml
This is the really cool implementation of what official Qt Installer does in Bash!
Zeal has been published to Flathub.
Zeal AppImage has been published to https://github.com/probonopd/zeal/releases/tag/continuous.
@probonopd It looks to me like you built v0.5.0, not the latest v0.6.1.
v0.6.1 will appear on https://github.com/probonopd/zeal/releases/tag/continuous shortly. Let me know if you would like a pull request that would produce those automatically.
I'd like to start with packaging CI builds into AppImage, so we can provide easy-to-run test binaries similarly to what we have for Windows.
I've just consolidated Linux and Windows CI's at AppVeyor, and I hope copy-pasting from @probonopd's .travis.yml will just work :)
The OpenSSL 1.0 vs 1.1 thingie is a bit of a concern though.
Might be useful, e.g., for Clear Linux OS. clearlinux/distribution#415
Is 0.6.1 in appimage released? I still see this bug zealdocs/zeal#892
I still cannot see any AppImages on https://github.com/zealdocs/zeal/releases unfortunately. What would be needed to get them there?
Closing this as we already have the Flatpak package. Progress on Snap can be tracked in zealdocs/zeal#1062.
Because we only supports Qt version included in the most recent Ubuntu LTS release, it makes Zeal incompatible with AppImage packaging tools, and their hard-coded checks for the oldest supported LTS release (based on glibc version I believe). At this time I have no desire to work around the restrictions of the tools, or depend on archaic Qt versions.
Closing this as we already have the Flatpak package. Progress on Snap can be tracked in zealdocs/zeal#1062.
This is not comparable to AppImage. It does not make portable, one-file applications.
Because we only supports Qt version included in the most recent Ubuntu LTS release, it makes Zeal incompatible with AppImage packaging tools
I think there is a big misunderstanding. Ubuntu 18.04.3 LTS is currently the latest LTS release. According to https://packages.ubuntu.com/search?keywords=qt, it comes with Qt 5.9.x.
To compile applications that run on all still-supported LTS releases of Ubuntu, we need to compile on the oldest still-supported LTS release, which currently is Ubuntu 16.04.x. This does not mean we need to use the Qt that came with it, but we can also use a newer Qt that works on Ubuntu 16.04.x. This is because a subset of Qt will be bundled and shipped along with the application.
Qt 5.11.3 for /opt Xenial is currently the latest that is available from https://launchpad.net/~beineri.
and their hard-coded checks for the oldest supported LTS release (based on glibc version I believe). At this time I have no desire to work around the restrictions of the tools, or depend on archaic Qt versions.
Again, you can use very recent versions of Qt on all still-supported LTS releases of Ubuntu.
It is general best practice to develop applications for the oldest still-supported distributions, not for the latest bleeding edge stuff. This is to ensure that all users (as opposed to just those running the latest and greatest operating systems) can use the applications.
Would you be interested in an AppImage in case I send a PR?
Actually, it's kinda funny, because I already made one some time ago, showing that it is working:
Would you be interested in an AppImage in case I send a PR?
If you are willing to maintain it, then of course! :) Almost all packages at this point are not maintained by me.
Not sure if it's a good idea to rely on 3rd party Qt builds, but at the same time that PPA have been around for quite some time.
It's unlikely that I would get to working on preparing AppImage packages myself, but if packaging can be integrated in the existing CI pipeline, and not require any work beyond then initial setup, I'm all for it.
It's unlikely that I would get to working on preparing AppImage packages myself, but if packaging can be integrated in the existing CI pipeline, and not require any work beyond then initial setup, I'm all for it.
Yes, it can be automated. I have been using Travis CI. Would that be acceptable for you or does it need to go through Shippable?
This is the file that does it:
https://github.com/probonopd/zeal/blob/patch-1/.travis.yml