What is the Platform interface?
The platform interface is a set of components, starting with glibc, which are designed to provide a backwards-compatible API and ABI for a certain amount of time, regardless of the exact version of the package installed in the distribution.
The notion of a platform interface, and it's associated stability, already exists within Fedora and other distributions. The interface stability is achieved by freezing the version of specific components for the lifetime of a distribution release.
This component version freezing process happens during the lead up to the release and culminates in a stable API and ABI being frozen for the duration of the entire distribution release. For example glibc 2.26 is frozen for Fedora 27 as the glibc that will be used for the entire lifetime of Fedora 27.
The stability of the platform interface allows application developers to target Fedora 27, build applications, and deploy them to Fedora 27 with the knowledge that these interface will not change for the duration of the release (a major version of Fedora).
What can we make better?
The current implementation of the interface has two limitations:
- Loose validation that the interface doesn't change over the course of the major release.
- The components can't be easily upgraded to support newer features for use by applications that need those features.
First and foremost the stability of the platform API and ABI is paramount. It is the stability of this interface that provides the basis for a stable distribution. Our current interface in Fedora is maintained by a group of committed developers, and to make their job easier we must formalize that validation. The easiest way to formalize the validation is to employ a more formal notion of API and ABI and validate it after the build.
Secondly, having the ability to upgrade platform components would allow a stable distribution to upgrade these components to newer version without the need to go through a full release process. The platform components could be upgraded on a rolling basis or as needed based on other requirements e.g. new ISV application needing a new glibc and a desire from users to deploy that in the existing Fedora major version.
The difficulty with upgrading platform components has been that such an upgrade inevitably leads to a binary interface change which requires packages to either explicitly or implicitly depend upon the new component. Florian Weimer wrote about the various difficulties inherent in this upgrade. This tight coupling of implementation and interface makes it difficult to upgrade the platform components without requiring all users move to the new platform components (since implicit RPM dependencies on symbol versions would require this).
If we could split the implementation from the interface we could allow the distribution, say Fedora 27, to target a known interface for Fedora 27, and as long as the implementation was backwards compatible with this interface, then the implementation could be upgraded as required.
The benefits of fixing the above two limitations would be:
- Formal API and ABI validation allowing for measurable progress for stability guarantees.
- Separation of implementation and interface for backwards compatible packages, allowing for package upgrades or rebases to support newer software on existing releases.
What does such an implementation look like?
The technology required to solve the above limitations exists, and an implementation is as follows:
- We can use the libabigail project (https://sourceware.org/libabigail/) to serialize the ABI at the time of GA for the platform interface components. The serialized ABI is then used to validate all future builds. This would provide a level of automated assurance that has never been used before.
- We can use an alternate system root for linking all applications. Within this alternate system root we provide curated versions of the required development files for the platform interface components. These curated versions are fixed at the GA versions and define the interface at the time of GA. The compiler colludes to search the alternate system root first when linking, and so will the static linker.
From an implementation perspective it looks like this:
- We run abigail on the GA release to serialize the ABI.
- We store the abigail serialized ABI in dist-git.
- A new package named "base-glibc" is created which is a frozen version of the normal glibc package but installs the GA version of glibc into the sysroot e.g. /opt/sysroot/f27/
- The compiler colludes to search the sysroot first, along with passing the appropriate options to the static linker to make this work.
From this point onwards all applications compile as normal, and link against the curated API/ABI in the sysroot but run against the latest installed version of the platform components.
Questions?
Q: Why do we need a distinct component for the sysroot portion? Why not just install the sysroot in a distinct path as a subpackage?
A: Two reasons. First: By splitting the component into an interface and implementation component we are better able to reduce the cost of validation at each build. If the interface is rebuilt at each component build then it must also be validated and that has a non-zero cost. Instead if only the implementation is rebuilt, it needs only be verified that it matches the previous validated interface. Second: You want to only build the interface at API/ABI relevant points in time to reduce storage costs on the content delivery network. If we have just rebased glibc, that's an important time to provide a newer version of the interface package. These two issues make it desirable to split the interface and implementation into two distinct packages.
Q: How many Platform components will be able to be split like this into interface and implementation?
A: At a first analysis only stable C/C++ packages will be the most suitable for splitting into interface and implementation, and for adding validation and verification. This means we will start with glibc, and move upwards. There are some immediate problems here in that Python is not easily split like is suggested and so remains out of the initial scope, neither are C/C++ packages which need runtime introspection. In particular GObject dynamic introspection has always resisted this kind of separation, this introspection makes it difficult to cross-compile GObject code, and so it also remains out of scope. In the core runtimes though this means that the C and C++ base runtimes could immediately be decoupled as is being suggested.