AppImage for Linux

[probonopd]

Returning to an idea I posted four years ago at MrKepzie/Natron#120, I would respectfully like to suggest providing an AppImage for Linux.

This would have, among others, these advantages:

• Applications packaged as an AppImage can run on many distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)
• One app = one file = super simple for users: just download one AppImage file, make it executable, and run
• No unpacking or installation necessary
• No root needed
• No system libraries changed
• Works out of the box, no installation of runtimes needed
• Optional desktop integration with appimaged
• Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
• Can optionally GPG2-sign your AppImages (inside the file)
• Works on Live ISOs
• Can use the same AppImages when dual-booting multiple distributions
• Can be listed in the AppImageHub central directory of available AppImages
• Can double as a self-extracting compressed archive with the --appimage-extract parameter
• No repositories needed. Suitable/optimized for air-gapped (offline) machines

Here is an overview of projects that are already distributing upstream-provided, official AppImages.

If you have questions, AppImage developers are on #AppImage on irc.freenode.net.

If there is interest in this, I'd be happy to help make it happen.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[Lvaskz]

Yeah that would be great.

[probonopd]

@MrKepzie, @devernay wdyt?

[Lvaskz]

Hi, some progress about it?

[probonopd]

I can offer to help in case @MrKepzie, @devernay are interested.

[devernay]

That looks like a good idea! @Sunderland93 did the flatpak for Natron, see #258 and https://github.com/flathub/fr.natron.Natron
You can probably find inspiration there.

[probonopd]

Thanks @devernay. Would you accept a pull request that would build each git push on Travis CI and upload an AppImage to GitHub Releases? Anything special that needs to be considered?

[devernay]

you mean having snapshot builds for each git push? Will it not clutter the github releases? Will it remove old snapshot builds automatically? I would prefer an option to pull only certain builds, for example if the tag/branch being built on travis starts with the word "snapshot".
I don't think you can build the full dependencies and Natron itself in the time budget that we have on travis, so you would have to make another project to build all the dependencies, upload the full sdk somewhere, and then download this sdk in the natron build itself.

Anyway, I would encourage you to make a fully working solution on a fork, have users test the appimages it generates, then of course we can merge it.

[probonopd]

Will it remove old snapshot builds automatically?

Yes.

Anyway, I would encourage you to make a fully working solution on a fork, have users test the appimages it generates, then of course we can merge it.

Cool! 👍

[probonopd]

I don't think you can build the full dependencies and Natron itself in the time budget that we have on travis

Isn't it already being built in /home/travis/build/NatronGitHub/Natron/App?

Can we have a full usr/ structure with usr/bin/natron and usr/share/natron/... there?

If yes, then it'd be simply a matter of running the build artefacts though linuxdepoyqt. That tool can take the dependency libraries from the host (build) system and bundle them up.

[devernay]

The problem is that you have to build Qt4, a recent ffmpeg, a recent OpenImageIO, etc (see tools/jenkins/include/scripts/build-Linux-SDK.sh). And that takes a lot more than the available time on travis (50mn?).
You should experiment. If if it works, then that's fine

[probonopd]

Isn't the application already being built in the already existing Travis CI build? So it should merely be a matter of packing the build artifacts that are already there?

[devernay]

The travis-built Natron would work only on that system (Ubuntu 14.04). To make a portable binary, you have to bundle all the dependencies.

[probonopd]

Yes, which is what linuxdeployqt should be able to do rather easily.

[devernay]

except Natron has many more dependencies than just Qt, and should also work with third party OFX plugins. We already use the Qt tools to take care of the Qt stuff, but there is a lot more than that (especially in openfx-io and openfx-arena)

[devernay]

what's the difference between flatpak and appimage, BTW?

[devernay]

and how far is the appimage from the portable binaries that we ship?

[probonopd]

except Natron has many more dependencies than just Qt

Of course we need to take care of those as well. The "qt" in "linuxdeployqt" stands for the fact that it is written in Qt, not limited to Qt. Library dependencies automatically get bundled unless they are opened at runtime using dlopen(), in which case we need to tell linuxdeployqt explicitly to bundle them as well. Unless the current portable binaries have all of that already bundled and solved.

and should also work with third party OFX plugins

The best way to go about this is to bundle all high-quality third party OFX plugins along with their dependencies, so that users can download just one file and everything works without configuring, installing, "fiddling around" with anything (like a Linux Live ISO does for the OS).

We already use the Qt tools to take care of the Qt stuff, but there is a lot more than that (especially in openfx-io and openfx-arena)

I am by no means familiar with those, but are you mainly talking about .so libraries or something else?

Complex applications like GIMP, Krita, Inkskape etc. are being shipped as AppImages, along with plugins. So I'd say it's doable, and I'm happy to help.

what's the difference between flatpak and appimage, BTW?

Short story: They are the exact opposite. Flatpak = may files scattered around the filesystem, needs to be managed by a tool that needs to be installed to manage applications. AppImage = one app, one file, all done by drag-and-drop in the file manager, portable.

Long story: https://github.com/AppImage/AppImageKit/wiki/Similar-projects

and how far is the appimage from the portable binaries that we ship?

You ship portable binaries? That's awesome, because we can possibly take those as-is and put them into the AppImage to get the added benefits, like

• One app = one file = super simple for users: just download one AppImage file, make it executable, and run
• No unpacking or installation necessary
• Optional desktop integration with appimaged
• Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
• Can optionally GPG2-sign your AppImages (inside the file)
• Can be listed in the AppImageHub central directory of available AppImages
• Can double as a self-extracting compressed archive with the --appimage-extract parameter

[devernay]

Yes, our binaries are very portable, and Natron is installed in a single directory.
The build process is rather complex, but in the end we probably do something like linuxdeployqt and a lot more. I didn't know of linuxdeployqt, but we use macdeployqt on the Mac builds, and it doesn't even do half the job.
Feel free to package an AppImage based on our binary distribs for Linux. They are available from github in the "releases" tab.
For our binaries, we rely on Xorg, OpenGL, and glibc being provided by the host system. Is it the same with AppImage? Which libraries from the host system can an AppImage use? Please point me to the doc if it's relevant.

[probonopd]

Yes, our binaries are very portable, and Natron is installed in a single directory.

Cool! 👍 So basically you already have done all the work.

An AppImage is just a self-mounting filesystem that executes whatever you put inside (e.g., your already portable application). The difference to a .tar.xz is simply that the user doesn't have to extract it, but can directly run it.

The build process is rather complex, but in the end we probably do something like linuxdeployqt and a lot more. I didn't know of linuxdeployqt, but we use macdeployqt on the Mac builds, and it doesn't even do half the job.

Can you point me to the script that is doing the magic? linuxdeployqt bundles the subset of Qt and other libraries that are needed to run an application. But it looks like you are already doing that part, so you won't gain anything from using linuxdeployqt (apart from, possibly, fewer lines of code).

Feel free to package an AppImage based on our binary distribs for Linux. They are available from github in the "releases" tab.

That's rather easy to do, basically just unpack your .tar.xz, add a bit of metadata like a desktop file, and repack it as an AppImage using appimagetool.

On a deb-based distribution, this does the trick:

wget -c "https://raw.githubusercontent.com/AppImage/pkg2appimage/master/pkg2appimage"
bash -ex pkg2appimage Natron

This uses the recipe at https://github.com/AppImage/pkg2appimage/blob/master/recipes/Natron.yml.

For our binaries, we rely on Xorg, OpenGL, and glibc being provided by the host system. Is it the same with AppImage?

Basically, you decide. Remember, an AppImage is just a self-mounting filesystem that executes whatever you decide to put inside.

Which libraries from the host system can an AppImage use? Please point me to the doc if it's relevant.

Exactly the same as with your .tar.xz. Or general recommendation is to build on the oldest system you are targeting (but no newer than the oldest still-supported Ubuntu LTS, which at this time is xenial), and to bundle all dependency libraries except those listed in https://github.com/AppImage/pkg2appimage/blob/master/excludelist. You are free to deviate from this recommendation, though (it seems you have already found out what works for you).

So, if you decide to provide an official AppImage, this would close a 5-year open ticket now: MrKepzie/Natron#120 💯

[rodlie]

Can I access individual binary in the bundle through the command line (with pass-through commands) on the AppImage?

example

-/Natron.AppImage --binary NatronRenderer --args --args --args

[probonopd]

@rodlie this can be done by using a custom AppRun script. Inkscape, for example, is using this:
https://gitlab.com/inkscape/inkscape/blob/883c7bc26f6268e1a121b796c3af76b8bce315e8/packaging/appimage/AppRun#L8-29

It also allows one to symlink the AppImage under various command names (similar to how busybox works).

[rodlie]

Nice, will have a look when I get the time.

[rodlie]

One more thing. If I run a command from within the application what would the PATH be? Will applications inside the bundle have the highest PATH priority?

[probonopd]

Yes, if the AppRun script or executable sets it up accordingly. You are in total control.

[rodlie]

Ok, thanks.

[probonopd]

Check those to get a flavor for what you can do in AppRun:

• https://gitlab.com/inkscape/inkscape/blob/master/packaging/appimage/AppRun
• https://github.com/ImageMagick/ImageMagick/blob/master/AppRun

[rodlie]

So, I tested AppImage today. It will be easy to add since the build system has already done all the work, we just need to add a couple of symlinks in the portable dir and make a squashfs filesystem of the directory. I can add support in the build scripts if wanted (up to Frédéric).

I do however have some concerns regarding performance. The compression of the squashfs image impacts general performance. The more I compress the image the slower thing are.

Starting Natron (on my rather old but functional dual core 2.5ghz laptop):

• From a directory: 1 sec
• AppImage (default squashfs compression): 18 sec
• AppImage (high compression) 50 sec

[probonopd]

How did you make the squashfs image? Some slowdown is expected, but nowhere near the values you are getting. Something must be going wrong. Can you upload your AppImage or scripts for debugging please?

[probonopd]

When I do (on a deb based system)

wget -c https://raw.githubusercontent.com/AppImage/pkg2appimage/master/pkg2appimage
bash -ex pkg2appimage Natron

then the resulting AppImage launches in about 20 seconds, 18 of which are spent on creating the fontconfig cache. Subsequent launches take about 4 seconds.

[rodlie]

I'm on my way out, but can upload files later tonight (or tomorrow).

Example:

mksquashfs dir image -root-owned -noappend
cat runtime >> AppImage
cat image >> AppImage
chmod a+x AppImage

Everything works as it should, but load time is 18 secs.

I used runtime from https://github.com/AppImage/AppImageKit/releases/download/continuous/runtime-x86_64

[probonopd]

Is this true for the first launch or also for the subsequent ones? Caused by fontconfig?

[rodlie]

It's not fontconfig, I have the cache. I can see each plugin load (it's that slow).

[probonopd]

What happens if you use this to convert the AppDir into an AppImage?

wget https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage
chmod +x appimagetool-x86_64.AppImage
./appimagetool-x86_64.AppImage Natron.AppDir

[c4tom]

Wouldn't snap be another good option, instead of appImage, although I use both and like appImage a little more? Snap is not easier to create it?

[Lvaskz]

With this tools( https://github.com/AppImageCrafters/appimage-builder) it is more easy to build and with these advantages:
https://github.com/AppImageCrafters
appimage-builder documentation

appimage-builder is a novel tool for creating AppImages. It uses the system package manager to resolve the application dependencies and creates a complete bundle. It can be used to pack almost any kind of applications including those made using: C/C++, Python, and Java.

Featuring:

    Real GNU/Linux packaging (no more distro packaging)
    Simple recipes
    Simple workflow
    Backward and forward compatibility
    One binary, many target systems.

For information about the AppImage packaging format visit: https://appimage.org/
Getting help

Having trouble? We’d like to help!

Try the FAQ – it’s got answers to some common questions.
Ask or search questions in StackOverflow using the AppImage tag.
Ask or search questions in the AppImage subreddit.
Ask a question in the #appimage IRC channel.
Report bugs with appimage-builder in our issue tracker.

[Lvaskz]

here for documentation https://appimage-builder.readthedocs.io/en/latest/

[TheAssassin]

No need for anything complicated. I just spent 15 minutes writing a build script using the release tarballs (which work excellently and do not require any modifications on the binary side). Will send a PR ASAP.