Workstation/Flatpaks

Basic Idea

Flatpak (formerly xdg-app) is a way of distributing and running applications with more isolation from the system than a traditional packaging format such as RPM. Dependencies for a flatpak are either distributed as part of the runtime or are bundled into the application itself. These - the runtime and the application - are the two types of artifacts we want to build for Fedora.

runtime

A runtime contains files that multiple applications depend on. There can be any number of runtimes installed on a system: for example, there might be both a Fedora 23 and a Fedora 24 runtime installed. A runtime looks like a mini-system image with libraries in /usr/lib, data files in /usr/share, binaries in /usr/bin and so forth.

application

The application contains both the application itself and libraries it depends on that are not part of the runtime. All files in the application are under /app.

Runtimes and applications are typically distributed as ostree repositories - but also can be stored as a single-file bundle. A bundle can be an ostree delta, or an OCI Image.

When a flatpak is run, an isolated environment is created with the runtime mounted on /usr and the application mounted on /app. Depending on sandboxing, the environment might have full access to the user's home directory, or might only access to a restricted set of D-Bus portals.

Flatpaks can be built directly from upstream sources using the flatpak-builder tool, but in the Fedora context, we want to build runtimes and applications in a way that is integrated with the overall Fedora development process: when we do a security update for a library, we want to know what applications or runtimes are affected and need to be rebuilt. We don't want to create an entirely new way for sources to get into Fedora bypassing the normal Fedora procedures.

There will be a single Fedora runtime maintained on the same schedule as the Platform module for Fedora. It will be be defined as a module that includes libraries that are commonly used for graphical applications. Some of these will inherit from the Platform module. Each application has it's own module in which the relevant RPMs are rebuilt with a special RPM macros (in the flatpak-rpm-macros package) to relocate the application and bundled libraries into the /app prefix. (This is necessary, because inside an executing Flatpak, the application is mounted at /app, and the runtime at /usr)

The packages are then composed into Flatpaks by the layered image service, sharing as much of the workflow and code as possible with containers. While the native delivery mechanism for Flatpaks is as an ostree repository, they can also be distributed as OCI images, and we plan to use this format during the build process, and to distribute them to users via registry.fedoraproject.com.

Building the Flatpak Runtime

The 'flatpak-runtime' module (prototype) defines the contents of the runtime. The 'runtime' profile lists exactly what is installed into the flatpak runtime. There is also a 'buildroot' module which lists the flatpak-rpm-macros] package. This special profile is used by the modularity tools to populate the buildroot for building any module that buildrequires flatpak-runtime. The flatpak-rpm-macros package overrides the standard RPM macros so that RPMS locate their content in /app rather than /usr.

The current prototype of flatpak-runtime is based on Fedora 26 and pulls packages from the base-runtime and shared-userspace modules. It is expected that for Fedora 27 it will be switched to depend on the Platform module, and the lifetime will match the lifetime of the Platform module.

The upstream runtimes include both a runtime, and an SDK, where the SDK contains the compiler, developer headers, etc, used to build an applications from source using flatpak-builder. We do not need a SDK for the Fedora runtime, since builds will simply pull in the necessary packages to build against, but we may eventually want to produce one that can be used by third-parties who want a runtime that has the lifetime guarantees and security updates that the Fedora project, but want to build their applications (open-source or closed-source) themselves from source.

Work items

• Rebase flatpak-runtime to F27
• Extend flatpak-runtime to include additional libraries (possibly matching the org.gnome.Platform module)

Building Flatpak Modules

Each application is then defined as a module that buildrequires flatpak-runtime, and that includes the packages that should be installed into the flatpak application filesystem in its default profile. Prototype eog application module

Initially, we expect application authors to simply follow the major branch structure of Fedora and maintain the same RPMs for module use and for standard RPM installation, but in the future application authors to be able to follow upstream releases instead, and eventually bundle library versions as needed without reference to the main Fedora schedule - though they will need to keep track of the lifetime of bundled library branches. (See Changes/ArbitraryBranching)

Work Items

• Extend flatpak-module-tools to be able to autogenerate the modulemd for an application from an existing RPM in Fedora. (It already has a bunch of the necessary code but it is currently specialized to generating the flatpak-runtime modulemd)
• Create additional test applications
  • A largely actively developed application like Inkscape or the GIMP
  • An application that is sandboxed and uses the Portal infrastructure
  • A simple game using SDL

Building Flatpak Images

Flatpak applications and runtimes will be built as an extension of the Layered Build Service. By building the Flatpaks as OCI images, it will be possible to have a very similar pipeline for server-side containers and for flatpaks. Unfortunately, although having OCI image support is a desired future feature for atomic-reactor and other layered build service components, it does not yet exist, and will need to be implemented as part of the Flatpak process.

The modules that would need modification are:

• koji-containerbuild. This is the glue between koji and OSBS and also the command line client. The changes needed here would be pretty minor - it just has a bit of code that parses labels out of the Dockerfile.
• osbs-client. The headline of this module is that it is is a command-line/python client for OSBS, but it actually has a ton of code that is specific to atomic-reactor and koji and not to OSBS (OSBS in some sense doesn’t exist - image building is just part of OpenShift these days.) It sets up the configuration that gets passed to atomic-reactor. osbs-client would have to detect (or be told) that the build is a flatpak build and set up the atomic-reactor configuration differently.
• atomic-reactor. Atomic reactor is where the code that runs inside the OpenShift builder image lives. It has a plugin structure with many more-or-less independent plugins that run before and after the actual build. There’s some prototype code to make the actual build step a plugin as well, which would be one half of the flatpak equation, but many of the plugins also do Dockerfile-specific manipulation as well. If the plugin is not relevant to flatpak, then osbs-client can simply not configure that plugin to be used, but if the plugin is doing something we need, then the Dockerfile handling needs to be abstracted.

flatpak-module-tools includes flatpak-module create-flatpak which:

• Downloads modules from Koji
• Installs rpms into a Docker container to build a filesystem image
• Turns that filesystem image and turn it into a flatpak application or runtime.

This serves as an early-prototype of the build step. Instructions

Work Items

• Land support for a pluggable build step in Atomic reactor
• Update koji-cointainerbuild and obs-client to be able to submit a Flatpak build
• Implement some form of OCI image support in atomic-reactor - enough to be able to handle the output from the flatpak build and push it to registry.fedoraproject.org

Distributing Flatpaks

Flatpaks will be distributed along side containers on registry.fedoraproject.org, and whatever provisions are made for mirroring the Fedora registry will then mirror flatpaks as well.

Work Items

• Allow uploading OCI images to registry.fedoraproject.org Upstream pull request
• Create some way to get summary information for all flatpaks on registry.fedoraproject.org; this might be a patch to the codebase, a look-aside file, or a separate service.
• Modify flatpak and gnome-software so that flatpaks can be installed from registry.fedoraproject.org. (There is already support from installing from a "OCI registry" - but that's just based on alexl's guess about how OCI images would be exported via HTTP - nobody seems to be implementing such a registry.)

Packager Workflow

In order to make a Flatpak of an application, the packager needs to create two extra pieces:

• A modulemd (module metadata) file (called <modulename>.yaml)
• A JSON file describing extra-metadata for the application, such as the security permissions that it requires. (Called flatpak.json - the idea is to be similar to Dockerfile where this is a fixed name.)

With current plans, this requires three separate git repositories (rpms/eog, modules/eog, containers/eog), and separate releng steps to create each. We should consider whether it makes sense to allow, for the case where there is a clean rpm <=> module <=> container/flatpak mapping, to allow a packager to put the eog.yaml and flatpak.json files directly in rpms/ and then build a module and flatpak against that. If not, we'll need some tools and expedited process to make this convenient.

Given the name of an existing application-package in Fedora, it should be possible to write tools to generate a first version of the modulemd and JSON files, though the JSON file will need some editing.

Initially, the packager would need to manually trigger module and flatpak builds, but we'll work together with general module and container efforts to improve automation. Useful automation might include, for example:

• Building the flatpak automatically each time the module build completes
• Starting a new module build automatically when an update to a package contained within the package is filed in bodhi. (Or when changes are pushed to dist-git?)

Handling Updates

It's not clear whether we want updates for applications to be separately handled in Bodhi, or whether when package updates are made, applications are automatically rebuilt. The necessary work here still needs to be figured out, and see what can be done for F27 and F28.

Appendix: Previous Work on Building Flatpaks out of RPMS

Alex Larsson's first effort using a copr can be found at:

• https://lists.fedoraproject.org/pipermail/desktop/2015-May/012362.html
• https://github.com/alexlarsson/fedora-runtime

Owen Taylor then picked up and did some work on making this work within Koji

• https://pagure.io/fork/otaylor/koji/branch/xdg-app
• https://github.com/owtaylor/fedora-runtime