From Fedora Project Wiki

Koji Operator Proposal

STATE: DRAFT

Abstract

A proposal for decoupling koji from the MBBOX operator and manage it in its own independent operator.

Authors

Leonardo Rossetti

Motivation

Koji is currently being handled/deployed in the MBBOX operator together with other components such as MBS.

The operator needs to manage several components such as koji-hub, kojira and also koji internal data (koji tags, usernames, certificates, users, etc) which proved to be complex enough to require its own operator.

A dedicated operator for koji means it can be used in a standalone fashion and other operators such as MBBOX can just depend on this operator instead of managing koji on its own (such dependency can be managed by operator-hub).

Proposal

The proposal is about moving koji to its own operator, repository, container images, etc.

The MBBOX operator should just create Koji custom resources and check for its status readiness.

Operator

The Koji operator will be responsible for deploying, configuring and managing koji services.

There are several operator types (operator foundations if you will) at the time of this writing: go, helm and ansible - the proposal is to stick with the ansible operator to leverage koji integration code which is written in python - the operator could also take advantadge of using an existing set of upstream koji ansible modules: https://github.com/ktdreyer/koji-ansible.

While go operators are usually faster and more fexlible, migrating to the go operator would imply writing all koji RPC integrations from scratch.

The operator won't abstract koji tags creation from a kubernetes CR since those can exponencially increase in number but the operator will enable an "admin sidecar container" instead for sysadmins to run koji admin commands (such as tag managament).

Dependencies

Koji depends on some components/services which will not be handled by the operator but are considered operator dependencies:

  • PostgreSQL
  • RabbitMQ (if running your own "fedora messaging" instance)

Secrets for both components should be provied in the same namespace koji is being deployed:

NOTE: those values were chosen for example purposes.

 #psql secret
 apiVersion: v1
 kind: Secret
 metadata:
   name: postgres
   labels:
     app: postgres
 data:
   POSTGRES_HOST: postgresql
   POSTGRES_DB: mboxdb
   POSTGRES_USER: koji
   POSTGRES_PASSWORD: mbox
 #rabbit mq secret
 apiVersion: v1
 kind: Secret
   metadata:
     name: koji-fedora-messaging-cert
   data:
     ca_cert.pem: my_ca_cert_content
     client_cert.pem: my_client_cert_content
     client_key.pem: my_client_key_content


Resources

This section describes all kubernetes resources that will be managed by the operator.

Most of those components are custom resources definitions which are implemented by the operator itself but there are a few common kubernetes resources (such as secrets and configmaps) that will also be watched by the operator.

The operator should re-run its reconcyle loop every N seconds to ensure both deployed components and koji data are in the epected state - this is automatically done by the operator reconcyle loop.

Koji Hub

Resource responsible for defining the koji-hub server config.

The operator should create an "admin" sidecar container If sidecar is set to true so sysadmins can run koji admin commands from that container.

The sidecar property should be optional and default to false.

Resource Proposal:

 apiVersion: apps.fedoraproject.org/v1alpha1
 kind: MBKojiHub
 metadata:
   name: mb-koji-hub
 spec:
   image: quay.io/fedora/koji-hub:latest
   replicas: 1
   persistent: true
   host: koji.mbox.dev # change it to match the external web url/route of koji-hub
   configmap: koji-hub
   ca_cert_secret: koji-hub-ca-cert
   service_cert_secret: koji-hub-service-cert
   postgres_secret: postgres
   http_enabled: true
   https_enabled: true
   topic_prefix: mbox_dev
   fedora_messaging_url: amqps://koji:something@rabbitmq
   messaging_cert_cm: koji-hub-msg
   ingress_backend: nginx # default
   # ingress_backend: openshift
   httpd_pvc_name: koji-hub-httpd-pvc
   httpd_pvc_size: 1Gi
   mnt_pvc_name: koji-hub-mnt-pvc
   mnt_pvc_size: 10Gi
   sidecar: true
Koji Web

Koji web is the frontend web appliction of koji to list builds, tasks, etc.

Resource Proposal:

 apiVersion: apps.fedoraproject.org/v1alpha1
 kind: KojiWeb
 metadata:
   name: koji-web
 spec:
     image: quay.io/fedora/koji-hub:latest
     replicas: 1
     client_cert_secret: koji-hub-web-client-cert
     client_username: kojiweb
     hub_host: koji_hub:8443
     cacert_secret: koji-hub-ca-cert
     client_cert_secret: koji-web-client-cert
Kojira

Kojira is a koji daemon responsible for generating build roots, repos and etc. It is a core backend component for koji.

Only one instance of kojira should be allowed per "koji deployment"/namespace.

Resource proposal:

 apiVersion: apps.fedoraproject.org/v1alpha1
 kind: MBKojira
 metadata:
   name: kojira
 spec:
   image: quay.io/fedora/kojira:latest
   configmap: kojira-config
   hub_username: kojira
   hub_host: koji-hub:8443
   src: 'no'
   max_repo_tasks: 15
   repo_tasks_limit: 15
   cacert_secret: koji-hub-ca-cert
   client_cert_secret: kojira-client-cert
   shared_pvc: koji-hub-mnt-pvc
Koji Builder

A Koji Builder (aka Koji Host) is the component which checks for new tasks in koji that are available in the pool awaiting for a builder to execute them.

Each builder supports at least one arhcitecture (x86_64, i386, etc) and can be added to channels for task context purposes.

Resource proposal:

 apiVersion: apps.fedoraproject.org/v1alpha1
 kind: KojiBuilder
 metadata:
   name: default
 spec:
   image: quay.io/fedora/koji-builder:latest
   replicas: 1
   configmap: koji-builder-configmap
   cacert_secret: koji-hub-ca-cert
   client_cert_secret: koji-builder-client-cert
   koji_hub_user: 'koji-builder.mbox.dev'
   koji_hub_url: 'https://koji-hub:8443'
   max_jobs: 5
   vendor: MBox
   host_archs: [x86_64]
   channels:[default, createrepo]
   host_name: koji-hub:8443
   ssl_verify: false
   shared_pvc: koji-hub-mnt-pvc
Koji User

Creating a KojiUser custom resource would make the operator create an user in koji and keeping it synchronized with the CR config (such as user perms) and even recreate the user if it gets deleted.

Resource Proposal:

 apiVersion: apps.fedoraproject.org/v1alpha1
 kind: KojiUser
 metadata:
   name: kojira
   labels:
     app: koji
 spec:
   permissions:
     - repo
   authentication:
     ssl: #may support others, such as krb and even multiple authentication mechanisms
       client_secret_name: kojira-cert
       ca_secret_name: ca-cert

The username field will use the value of metadata.name.

KubeCtl Plugin

A koji kubectl plugin which runs koji commands in the admin sidecar pod as a koji admin.

This plugin enables syadmins to manage koji using kubectl.

The plugin should check if the sidecar container is present otherwise fail describing the steps to set a sidecar container for koji-hub.

Kubectl plugins development are very straightforward and well documented: https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/

Example:

 kubectl koji grant-permission repo kojira


Copyright

This document has been placed in the public domain.