From Fedora Project Wiki
 
(46 intermediate revisions by 9 users not shown)
Line 5: Line 5:
There are some applications like Cockpit which we would like to ship as layered applications.
There are some applications like Cockpit which we would like to ship as layered applications.


This change will deploy the [https://github.com/DBuildService/osbs build service] to support
This change will deploy the [https://github.com/projectatomic/osbs-client build service] to support
building and delivering a set of layered Docker images, and will enable Fedora contributors to create
building and delivering a set of layered Docker images, and will enable Fedora contributors to create
and maintain Dockerfiles from which those images will be generated.
and maintain Dockerfiles from which those images will be generated.


== Owners ==
== Owners ==
* Name: [[User:walters| Colin Walters ]]
* Name: [[User:maxamillion| Adam Miller ]]
* Name: [[User:maxamillion| Adam Miller ]]
* Name: [[User:ttomecek| Tomas Tomecek ]]
* Name: [[User:ttomecek| Tomas Tomecek ]]
* Name: [[User:twaugh|Tim Waugh]]
* Name: [[User:twaugh|Tim Waugh]]
* Name: [[User:acarter|Amanda Carter]]


* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
* Release notes owner: [[User:immanetize| Pete Travis]]
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
Line 25: Line 25:


== Current status ==
== Current status ==
* Targeted release: [[Releases/23 | Fedora 23 ]]  
* Targeted release: [[Releases/24 | Fedora 24 ]]  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
Line 35: Line 35:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug:
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1243736 #1243736]
* Ongoing Work Tracking (Taiga Kanban board): http://taiga.fedorainfracloud.org/project/acarter-fedora-docker-atomic-tooling/kanban


== Detailed Description ==
== Detailed Description ==
Line 43: Line 44:
The system has five major parts:
The system has five major parts:


* A command-line client — already integrated into rpkg; needs only minor work to enable in '''fedpkg'''
* A command-line client — already integrated into rpkg; needs only minor work to enable in '''fedpkg''' (there is discussion about either extending '''fedpkg''' or adding a new '''fedcontainer''' utility)
* '''dist-git''' for Dockerfiles
* '''dist-git''' for Dockerfiles
* A '''koji''' plugin, [https://lists.fedoraproject.org/pipermail/buildsys/2015-June/004795.html containerbuild]
* A '''koji''' plugin, [https://lists.fedoraproject.org/pipermail/buildsys/2015-June/004795.html containerbuild]
* An OpenShift 3 '''backend'''
* An [https://docs.openshift.org/latest/welcome/index.html OpenShift 3] '''backend'''
* A distribution mechanism; initially, this will be
* A distribution mechanism; a Docker Registry
# '''ftp/http''' mirror (either alt or main mirrors), and
** Currently evaluating options for this
# pushed to upstream '''Docker hub''' (running our own registry is currently out of scope; see below)
*** [https://github.com/pulp/crane Pulp Crane]
*** <strike>[https://www.docker.com/docker-registry Docker Registry]</strike>


For more information, see [https://ttomecek.fedorapeople.org/public-presentation-of-layered-image-build-service.pdf this presentation] for the high level overview of the whole system.
For more information, see [https://ttomecek.fedorapeople.org/public-presentation-of-layered-image-build-service.pdf this presentation] for the high level overview of the whole system.
=== Possible areas of concern ===
==== Docker "packaging" and DistGit ====
The topic of Fedora container image maintainers maintaining Docker images much like Fedora rpm package maintainers maintain packages in DistGit has come up recently as something we might need to consider in a more broad scope as this is likely to impact more than originally thought.
<pre>
Proposed Solution
-----------------
DistGit Restructure/Namespace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  - Move all current rpm packages that live under repositories/ to live under
    repositories/rpm/
  - Create new namespaces for other things (i.e. - repositories/docker/)
current:
/srv/git/repositories/<pkg>.git
future:
  -> /srv/git/repositories/rpms/<pkg>.git (symlinked to the old location for a little while)
    => At one point, will break `git pull` if the remote url isn't fixed
  -> /srv/git/repositories/docker/<pkg>.git
  => /srv/git/repositories/<artifact>/<pkg>.git
Artifact Abstraction
  - RPMs
  - Docker Images
  - ostrees
  - xdg-app
</pre>
==== Namespacing or disjoint DistGit for containers ====
* [https://lists.fedoraproject.org/pipermail/devel/2015-November/216683.html Discussion ongoing on devel-list]
** Namespacing is difficult, it would effectively blacklist a package name alphanumeric sequence from being used in Fedora (which may or may not matter)
* Technical concerns about disjoint DistGit for containers:
** In the event a container and a rpm package share a name, koji will potentially clobber one over the other and could "lose" rpms if a container wins the race condition at the time of a repo regen.
** If the same name exists in koji but under different tags, in the event these get crossed accidentally or in the future we could "lose" large chunks of the package set
==== pkgdb2 and fedora-packages ====
pkgdb2 ([https://admin.fedoraproject.org/pkgdb site], [https://github.com/fedora-infra/fedora-packages source]) and fedora-packages ([https://apps.fedoraproject.org/packages/ site], [https://github.com/fedora-infra/fedora-packages source]) might need patches to handle non-rpm content
* Something like pkgdb2 will be needed for "container DistGit"
** Don't know what the implications of that will be
** concerned it might disrupt some aspect of pkgdb2's data model causing a large amount of work to fix.
** what if we need a separate instance of this for a disjoint "container DistGit"?
* Concerns around these utilities being rpm-centric and assume the existence of a spec file (at least for fedora-packages web view of spec files), in the event the Dockerfile container content is to share space with DistGit then pkgdb2 and fedora-packages would need to effectively blacklist anything starting with the namespace prefix that's decided upon.
==== Container "packaging" Guidelines ====
Currently for [[Packaging:Guidelines | Fedora Packaging Guidelines]] there is the [https://fedorahosted.org/fpc/ Fedora Packaging Committee] that shepherd in best practices and guidelines for how software should be packaged into rpms. There is nothing like this for Dockerfiles (yet), however upstream Project Atomic has some [https://github.com/projectatomic/ContainerApplicationGenericLabels general guidelines for Dockerfile LABELs] which we should consider for inclusion in whatever ends up being the Fedora Container/Docker Guidelines.
Should we have a "Fedora Container Maintainers Committee" or similar that mirrors the Fedora Packaging Committee?
=== Schematic ===
[[File:LayeredImageBuildService-Overview-v2.1.png]]
Inkscape SVG source for PNG image available [https://maxamillion.fedorapeople.org/LayeredImageBuildService-Overview-v2.1.svg here]


== Benefit to Fedora ==
== Benefit to Fedora ==
Line 60: Line 121:


Fedora Server is planning to deliver Server Roles as containers; this will facilitate that. And vital parts of Fedora Atomic are now designed to be delivered in containers, but we currently have no mechanism to do that within Fedora itself.
Fedora Server is planning to deliver Server Roles as containers; this will facilitate that. And vital parts of Fedora Atomic are now designed to be delivered in containers, but we currently have no mechanism to do that within Fedora itself.
By having our own build system that ties into a Fedora specific docker registry will lend itself to allowing for CI workflows around containerized application stacks as well as easier automated testing.


== Scope ==
== Scope ==
Line 65: Line 128:


Proposal owners shall have to
Proposal owners shall have to
** Deploy OSBS
* Deploy OSBS
*** Deploy OpenShift v3
** Deploy OpenShift v3
*** Deploy container build koji plugin
** Deploy container build koji plugin
** create dist-git for Dockerfiles
* create dist-git for Dockerfiles


==== Task matrix ====
==== Task matrix ====
'''See [[Changes/Layered Docker Image Build Service/RACI|Layered Docker Image Build Service RACI Matrix]] for an individual breakdown of tasks and responsibilities.''' This also includes a basic, task-level accounting of status, which we will keep updated.
'''See [[Changes/Layered Docker Image Build Service/RACI|Layered Docker Image Build Service RACI Matrix]] for an individual breakdown of tasks and responsibilities.''' This also includes a basic, task-level accounting of status, which we will keep updated.
=== Functional Requirements ===
* Builds must be integrated into koji
** This can be a plugin or built-in
** Ensure that the sum of the things in the docker layered images come from Fedora "proper" have been built in koji.
* Build artifacts from koji need to be integrated into Fedora composes
* Built docker images should be in a docker registry that can be pulled immediately with the docker client for testing.
* Layered image automatic rebuilds
** In the event a layer in the stack needs an update because of an updated package, it should be rebuilt and as well as all layered images that depend on it with rebuilds cascading up as needed.
*** This will include the top-most layer. Example: If the container is targeting a specific package <code>foo</code> and an update makes it through [[Bodhi]] for <code>foo</code> then an automated rebuild will be triggered.
* Layered Docker image Dockerfiles should be integrated with Fedora DistGit
* Layered Docker Image build should be integrated with fedpkg
** Workflow for layered images maintainers will be similar to that of package maintainers
*** Example build command: <code>fedpkg container-build</code>
* All aspects of this pipeline shall be both human executable and able to be automated, there should be nothing bound to only human intervention that could block the processs.


=== Other developers ===  
=== Other developers ===  
Line 80: Line 158:


=== Policies and guidelines ===
=== Policies and guidelines ===
** Need to determine who can submit/build images
* Need to determine who can submit/build images
** Determine who is responsible for building/testing images as RPMs change
* Determine who is responsible for building/testing images as RPMs change
** Determine policy for non-RPM content. (Initial policy will be that all content must be existing RPMs in the official released or updates repositories.)  
* Determine policy for non-RPM content. (Initial policy will be that all content must be existing RPMs in the official released or updates repositories.)  
** "Packaging guidelines" for Dockerfiles; suggest starting lightweight and building as needed. [http://www.projectatomic.io/docs/docker-image-author-guidance/ Project Atomic documentation] can be a starting point.
* "Packaging guidelines" for Dockerfiles; suggest starting lightweight and building as needed. [http://www.projectatomic.io/docs/docker-image-author-guidance/ Project Atomic documentation] can be a starting point.
 
=== Out of Scope (Intended for Future Changes)===
 
==== ComposeDB ====


=== Out of Scope ===
There are plans right now to have ComposeDB(PDC) store information about build artifacts, integration with that system is something we will need in order to enable others in this list. [[ReleaseEngineering/ComposeDB]]


==== Registry ====
==== Automated QA with Taskotron ====


This Change does not include running a Fedora container registry. Although we might want to do so in the future, this should not be a blocker. Container images will be a) delivered to the upstream Docker Hub and b) put on the Fedora FTP/HTTP mirror network.
Ultimately we would like for every build of a docker layered image to trigger a test in Taskotron via fedmsg.


==== Other Integrations ====
==== Other Integrations ====
Line 97: Line 179:
It would also be nice to have a pkgdb interface, but that shouldn't be a blocker.
It would also be nice to have a pkgdb interface, but that shouldn't be a blocker.


CentOS has offered to let Fedora test images in their continuous integration infrastructure, and we need to figure out how we can hook that in — perhaps through bodhi.
==== Notary ====
 
https://github.com/docker/notary signing is not yet in scope for this Change.


==== Content Tracking, Automatic Rebuilds ====
==== Content Tracking, Automatic Rebuilds ====
Line 103: Line 187:
It would be nice for images to automatically rebuilt and pushed to testing when any of their contents change, or at least when any of their content changes due to a security update. But we don't want to block having the basic build service in place on that; initially, updates will need to be initiated by the container packager.
It would be nice for images to automatically rebuilt and pushed to testing when any of their contents change, or at least when any of their content changes due to a security update. But we don't want to block having the basic build service in place on that; initially, updates will need to be initiated by the container packager.


==== Bodhi Integration ====
==== Non-x86_64 Architecture Support ====


In the future, Bodhi could be used to manage the release of updated images, just as it currently is for RPM updates. However, this is ''definitely'' out of scope at this time.
Currently Docker upstream only officially supports x86_64 and we will target that for the initial delivery of the build system.


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
Line 160: Line 244:
=== Container User ===
=== Container User ===


Without a registry — a next step currently out of scope — the user experience is not quite ideal. The images will need to be downloaded manually from the mirrors and imported. For use cases like Fedora Server's, this will be scripted. Alternately, users can pull from the upstream project's Docker Hub.
 
<pre>
$ docker pull registry.fedoraproject.org/fedora
 
$ docker run -ti registry.fedoraproject.org/fedora /bin/bash
</pre>
 
 
Alternately, users can pull from the upstream project's Docker Hub as we will also be pushing images there.


=== Release Engineering & Infrastructure ===
=== Release Engineering & Infrastructure ===
Line 167: Line 259:


== Dependencies ==
== Dependencies ==
* N/A
* [https://github.com/release-engineering/koji-containerbuild koji containerbuild plugin]
** [https://github.com/projectatomic/osbs-client OpenShift Build Service]
*** [https://github.com/openshift/origin OpenShift (Architecture Version 3)]
 
NOTE: The osbs client tooling has been packaged and is in Fedora, OpenShift is not and will potentially require a bundled library exception pending the decision on golang package guidelines and bundled libraries.


== Contingency Plan ==
== Contingency Plan ==
Line 173: Line 269:


== Documentation ==
== Documentation ==
Needs filling in.
 
Implementation documention, user documentation, and infrastructure documentation will be developed as part of the project but does not yet exist.


== Release Notes ==
== Release Notes ==
Line 182: Line 279:
-->
-->


[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF24]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->

Latest revision as of 06:36, 17 November 2016

Layered Docker Image Build Service

Summary

Fedora currently ships a Docker base image, but Docker supports a layering concept. There are some applications like Cockpit which we would like to ship as layered applications.

This change will deploy the build service to support building and delivering a set of layered Docker images, and will enable Fedora contributors to create and maintain Dockerfiles from which those images will be generated.

Owners

Current status

Detailed Description

This change opens up an new type of official binary artifact produced by Fedora. Currently, we produce two main types of artifacts: RPMs, and images. The RPMs are created in Koji from specfiles in dist-git. The images come in different formats, but have in common creation in Koji from kickstart files — this includes the official Fedora Docker Base Image. This change introduces a new type of image, a Docker Layered Image, which is created from a Dockerfile and builds on top of that base image.

The system has five major parts:

  • A command-line client — already integrated into rpkg; needs only minor work to enable in fedpkg (there is discussion about either extending fedpkg or adding a new fedcontainer utility)
  • dist-git for Dockerfiles
  • A koji plugin, containerbuild
  • An OpenShift 3 backend
  • A distribution mechanism; a Docker Registry

For more information, see this presentation for the high level overview of the whole system.

Possible areas of concern

Docker "packaging" and DistGit

The topic of Fedora container image maintainers maintaining Docker images much like Fedora rpm package maintainers maintain packages in DistGit has come up recently as something we might need to consider in a more broad scope as this is likely to impact more than originally thought.

Proposed Solution
-----------------

DistGit Restructure/Namespace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  - Move all current rpm packages that live under repositories/ to live under
    repositories/rpm/
  - Create new namespaces for other things (i.e. - repositories/docker/)


current:
 /srv/git/repositories/<pkg>.git
future:
  -> /srv/git/repositories/rpms/<pkg>.git (symlinked to the old location for a little while)
     => At one point, will break `git pull` if the remote url isn't fixed
  -> /srv/git/repositories/docker/<pkg>.git
  => /srv/git/repositories/<artifact>/<pkg>.git

Artifact Abstraction
  - RPMs
  - Docker Images
  - ostrees
  - xdg-app

Namespacing or disjoint DistGit for containers

  • Discussion ongoing on devel-list
    • Namespacing is difficult, it would effectively blacklist a package name alphanumeric sequence from being used in Fedora (which may or may not matter)
  • Technical concerns about disjoint DistGit for containers:
    • In the event a container and a rpm package share a name, koji will potentially clobber one over the other and could "lose" rpms if a container wins the race condition at the time of a repo regen.
    • If the same name exists in koji but under different tags, in the event these get crossed accidentally or in the future we could "lose" large chunks of the package set

pkgdb2 and fedora-packages

pkgdb2 (site, source) and fedora-packages (site, source) might need patches to handle non-rpm content

  • Something like pkgdb2 will be needed for "container DistGit"
    • Don't know what the implications of that will be
    • concerned it might disrupt some aspect of pkgdb2's data model causing a large amount of work to fix.
    • what if we need a separate instance of this for a disjoint "container DistGit"?
  • Concerns around these utilities being rpm-centric and assume the existence of a spec file (at least for fedora-packages web view of spec files), in the event the Dockerfile container content is to share space with DistGit then pkgdb2 and fedora-packages would need to effectively blacklist anything starting with the namespace prefix that's decided upon.


Container "packaging" Guidelines

Currently for Fedora Packaging Guidelines there is the Fedora Packaging Committee that shepherd in best practices and guidelines for how software should be packaged into rpms. There is nothing like this for Dockerfiles (yet), however upstream Project Atomic has some general guidelines for Dockerfile LABELs which we should consider for inclusion in whatever ends up being the Fedora Container/Docker Guidelines.

Should we have a "Fedora Container Maintainers Committee" or similar that mirrors the Fedora Packaging Committee?

Schematic

Inkscape SVG source for PNG image available here

Benefit to Fedora

What is the benefit to the platform?

Docker is a very popular way to deliver container images. This will help deliver Fedora-based containers inside the Docker ecosystem, and other applications that are part of the Fedora package collection as containers.


Fedora Server is planning to deliver Server Roles as containers; this will facilitate that. And vital parts of Fedora Atomic are now designed to be delivered in containers, but we currently have no mechanism to do that within Fedora itself.

By having our own build system that ties into a Fedora specific docker registry will lend itself to allowing for CI workflows around containerized application stacks as well as easier automated testing.

Scope

Proposal owners

Proposal owners shall have to

  • Deploy OSBS
    • Deploy OpenShift v3
    • Deploy container build koji plugin
  • create dist-git for Dockerfiles

Task matrix

See Layered Docker Image Build Service RACI Matrix for an individual breakdown of tasks and responsibilities. This also includes a basic, task-level accounting of status, which we will keep updated.

Functional Requirements

  • Builds must be integrated into koji
    • This can be a plugin or built-in
    • Ensure that the sum of the things in the docker layered images come from Fedora "proper" have been built in koji.
  • Build artifacts from koji need to be integrated into Fedora composes
  • Built docker images should be in a docker registry that can be pulled immediately with the docker client for testing.
  • Layered image automatic rebuilds
    • In the event a layer in the stack needs an update because of an updated package, it should be rebuilt and as well as all layered images that depend on it with rebuilds cascading up as needed.
      • This will include the top-most layer. Example: If the container is targeting a specific package foo and an update makes it through Bodhi for foo then an automated rebuild will be triggered.
  • Layered Docker image Dockerfiles should be integrated with Fedora DistGit
  • Layered Docker Image build should be integrated with fedpkg
    • Workflow for layered images maintainers will be similar to that of package maintainers
      • Example build command: fedpkg container-build
  • All aspects of this pipeline shall be both human executable and able to be automated, there should be nothing bound to only human intervention that could block the processs.

Other developers

  • (anything here)?

Release engineering

  • Deploy OSBS

Policies and guidelines

  • Need to determine who can submit/build images
  • Determine who is responsible for building/testing images as RPMs change
  • Determine policy for non-RPM content. (Initial policy will be that all content must be existing RPMs in the official released or updates repositories.)
  • "Packaging guidelines" for Dockerfiles; suggest starting lightweight and building as needed. Project Atomic documentation can be a starting point.

Out of Scope (Intended for Future Changes)

ComposeDB

There are plans right now to have ComposeDB(PDC) store information about build artifacts, integration with that system is something we will need in order to enable others in this list. ReleaseEngineering/ComposeDB

Automated QA with Taskotron

Ultimately we would like for every build of a docker layered image to trigger a test in Taskotron via fedmsg.

Other Integrations

In the future, images could be integrated into Bodhi, just like RPMs.

It would also be nice to have a pkgdb interface, but that shouldn't be a blocker.

Notary

https://github.com/docker/notary signing is not yet in scope for this Change.

Content Tracking, Automatic Rebuilds

It would be nice for images to automatically rebuilt and pushed to testing when any of their contents change, or at least when any of their content changes due to a security update. But we don't want to block having the basic build service in place on that; initially, updates will need to be initiated by the container packager.

Non-x86_64 Architecture Support

Currently Docker upstream only officially supports x86_64 and we will target that for the initial delivery of the build system.

Upgrade/compatibility impact

Since the world of containers is changing rapidly, it is possible that the system might need to be upgraded in future to latest stable version.


How To Test

Ideally, we have a Fedora registry. If so, then adding it and doing:

atomic run registry.fedoraproject.org/cockpit


Before we have a registry, images can be tested by downloading the images, using docker import, and docker run.

User Experience

There's many potential roles interacting here - the container owner, the container user, release engineering.

Container Packager

From a container packager point of view, the overall experience should be very similar to that of an RPM packager. Instead of a spec file, a Dockerfile will be created, but then that will be built using fedpkg through koji. This maintains both a comfortable workflow that will be easy for contributors to adapt to, and the release engineering benefits of koji.


Workflow analogy with RPM
RPM Layered Image
dist-git repo per RPM dist-git repo per image
Specfile in dist-git Dockerfile in dist-git
Source code and patches No source code — refers to existing RPMs
fedpkg build fedpkg container-build
created in koji (koji build task) created in koji (through containerbuild plugin)
pushed to mirrors pushed to mirrors
available via yum/dnf repo creation currently out of scope

Container User

$ docker pull registry.fedoraproject.org/fedora

$ docker run -ti registry.fedoraproject.org/fedora /bin/bash


Alternately, users can pull from the upstream project's Docker Hub as we will also be pushing images there.

Release Engineering & Infrastructure

The new system will need maintenance and administration. For example, the new dist-git repositories will need to be managed.

Dependencies

NOTE: The osbs client tooling has been packaged and is in Fedora, OpenShift is not and will potentially require a bundled library exception pending the decision on golang package guidelines and bundled libraries.

Contingency Plan

People continue to upload to the Docker Hub in an ad-hoc fashion with no integration with Fedora.

Documentation

Implementation documention, user documentation, and infrastructure documentation will be developed as part of the project but does not yet exist.

Release Notes