- 1 Layered Docker Image Build Service
- 1.1 Summary
- 1.2 Owners
- 1.3 Current status
- 1.4 Detailed Description
- 1.5 Benefit to Fedora
- 1.6 Scope
- 1.6.1 Proposal owners
- 1.6.2 Functional Requirements
- 1.6.3 Other developers
- 1.6.4 Release engineering
- 1.6.5 Policies and guidelines
- 1.6.6 Out of Scope (Intended for Future Changes)
- 1.7 Upgrade/compatibility impact
- 1.8 How To Test
- 1.9 User Experience
- 1.10 Dependencies
- 1.11 Contingency Plan
- 1.12 Documentation
- 1.13 Release Notes
Layered Docker Image Build Service
Fedora currently ships a Docker base image, but Docker supports a layering concept. There are some applications like Cockpit which we would like to ship as layered applications.
This change will deploy the build service to support building and delivering a set of layered Docker images, and will enable Fedora contributors to create and maintain Dockerfiles from which those images will be generated.
- Release notes owner: Pete Travis
- Targeted release: Fedora 24
- Last updated: 2015-10-26
- Tracker bug: #1243736
- Ongoing Work Tracking (Taiga Kanban board): http://taiga.cloud.fedoraproject.org/project/acarter-fedora-docker-atomic-tooling/kanban
This change opens up an new type of official binary artifact produced by Fedora. Currently, we produce two main types of artifacts: RPMs, and images. The RPMs are created in Koji from specfiles in dist-git. The images come in different formats, but have in common creation in Koji from kickstart files — this includes the official Fedora Docker Base Image. This change introduces a new type of image, a Docker Layered Image, which is created from a Dockerfile and builds on top of that base image.
The system has five major parts:
- A command-line client — already integrated into rpkg; needs only minor work to enable in fedpkg
- dist-git for Dockerfiles
- A koji plugin, containerbuild
- An OpenShift 3 backend
- A distribution mechanism; a Docker Registry
For more information, see this presentation for the high level overview of the whole system.
Benefit to Fedora
What is the benefit to the platform?
Docker is a very popular way to deliver container images. This will help deliver Fedora-based containers inside the Docker ecosystem, and other applications that are part of the Fedora package collection as containers.
Fedora Server is planning to deliver Server Roles as containers; this will facilitate that. And vital parts of Fedora Atomic are now designed to be delivered in containers, but we currently have no mechanism to do that within Fedora itself.
By having our own build system that ties into a Fedora specific docker registry will lend itself to allowing for CI workflows around containerized application stacks as well as easier automated testing.
Proposal owners shall have to
- Deploy OSBS
- Deploy OpenShift v3
- Deploy container build koji plugin
- create dist-git for Dockerfiles
- Deploy OSBS
See Layered Docker Image Build Service RACI Matrix for an individual breakdown of tasks and responsibilities. This also includes a basic, task-level accounting of status, which we will keep updated.
- Builds must be integrated into koji
- This can be a plugin or built-in
- Build artifacts from koji need to be integrated into Fedora composes
- Built docker images should be in a docker registry that can be pulled immediately with the docker client for testing.
- Layered image automatic rebuilds
- In the event a layer in the stack needs an update because of an updated package, it should be rebuilt and as well as all layered images that depend on it with rebuilds cascading up as needed.
- (anything here)?
- Deploy OSBS
Policies and guidelines
- Need to determine who can submit/build images
- Determine who is responsible for building/testing images as RPMs change
- Determine policy for non-RPM content. (Initial policy will be that all content must be existing RPMs in the official released or updates repositories.)
- "Packaging guidelines" for Dockerfiles; suggest starting lightweight and building as needed. Project Atomic documentation can be a starting point.
Out of Scope (Intended for Future Changes)
There are plans right now to have ComposeDB(PDC) store information about build artifacts, integration with that system is something we will need in order to enable others in this list. ReleaseEngineering/ComposeDB
Automated QA with Taskotron
Ultimately we would like for every build of a docker layered image to trigger a test in Taskotron via fedmsg.
In the future, images could be integrated into Bodhi, just like RPMs.
It would also be nice to have a pkgdb interface, but that shouldn't be a blocker.
Content Tracking, Automatic Rebuilds
It would be nice for images to automatically rebuilt and pushed to testing when any of their contents change, or at least when any of their content changes due to a security update. But we don't want to block having the basic build service in place on that; initially, updates will need to be initiated by the container packager.
Since the world of containers is changing rapidly, it is possible that the system might need to be upgraded in future to latest stable version.
How To Test
Ideally, we have a Fedora registry. If so, then adding it and doing:
atomic run registry.fedoraproject.org/cockpit
Before we have a registry, images can be tested by downloading the images, using docker import, and docker run.
There's many potential roles interacting here - the container owner, the container user, release engineering.
From a container packager point of view, the overall experience should be very similar to that of an RPM packager. Instead of a spec file, a Dockerfile will be created, but then that will be built using fedpkg through koji. This maintains both a comfortable workflow that will be easy for contributors to adapt to, and the release engineering benefits of koji.
|dist-git repo per RPM||dist-git repo per image|
|Specfile in dist-git||Dockerfile in dist-git|
|Source code and patches||No source code — refers to existing RPMs|
|fedpkg build||fedpkg container-build|
|created in koji (koji build task)||created in koji (through containerbuild plugin)|
|pushed to mirrors||pushed to mirrors|
|available via yum/dnf||repo creation currently out of scope|
$ docker pull registry.fedoraproject.org/fedora $ docker run -ti /bin/bash registry.fedoraproject.org/fedora
Alternately, users can pull from the upstream project's Docker Hub as we will also be pushing images there.
Release Engineering & Infrastructure
The new system will need maintenance and administration. For example, the new dist-git repositories will need to be managed.
NOTE: The osbs client tooling has been packaged and is in Fedora, OpenShift is not and will potentially require a bundled library exception pending the decision on golang package guidelines and bundled libraries.
People continue to upload to the Docker Hub in an ad-hoc fashion with no integration with Fedora.
Implementation documention, user documentation, and infrastructure documentation will be developed as part of the project but does not yet exist.