From Fedora Project Wiki
(Benefit to Fedora)
(Task Table)
 
(24 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{admon/warning|DRAFT|This page is a draft and still under construction.}}
 
 
<!-- Self Contained or System Wide Change Proposal?
 
<!-- Self Contained or System Wide Change Proposal?
 
Use this guide to determine to which category your proposed change belongs to.
 
Use this guide to determine to which category your proposed change belongs to.
Line 26: Line 25:
 
== Summary ==
 
== Summary ==
 
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. -->
 
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. -->
 +
 +
Centralized entry point, logging, and dash board for pre-defined Automated Workflow tasks used by the Release Engineering team with delegation and self-service tasks for members of various teams who normally depend on Release Engineering for various tasks.
  
 
== Owner ==
 
== Owner ==
Line 32: Line 33:
 
This should link to your home wiki page so we know who you are.  
 
This should link to your home wiki page so we know who you are.  
 
-->
 
-->
* Name: [[User:FASAcountName| Your Name]]
+
* Name: [[User:Maxamillion| Adam Miller]]
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
+
* Email: maxamillion@fedoraproject.org
* Email: <your email address so we can contact you, invite you to meetings, etc.>
 
 
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
 
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
 
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
 
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
Line 45: Line 45:
  
 
== Current status ==
 
== Current status ==
* Targeted release: [[Releases/<number> | Fedora <number> ]]  
+
* Targeted release: [[Releases/25 | Fedora 25 ]]  
 
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
 
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
 
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
 
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
Line 55: Line 55:
 
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 
-->
 
-->
* Tracker bug: <will be assigned by the Wrangler>
+
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1345757 #1345757]
  
 
== Detailed Description ==
 
== Detailed Description ==
  
 
<pre>
 
<pre>
                                                    +----------------+
+
+--------------+                                            +----------------+
    +------------+                                 |                |
+
|              |                +------------+             |                |
    |           +--------------------------------->+  Taskotron    |
+
| AutoCloud  |<----------------+            +------------>+  Taskotron    |
    |  fedmsg  |                                 |                |
+
|              |                |  fedmsg  |             |                |
    |           |<---------------------------------+                |
+
|             +---------------->|            |<------------+                |
    |            |                                  +----------------+
+
+--------------+                 |            |            +----------------+
    +------------+
+
                                +------------+
        ^
+
                                      ^
        |
+
                                      |
        |
+
                                      |
        |
+
                                      |
        |
+
                                      |
        |
+
                                      |
        +-----------------+
+
                                      |
                          |
+
                   +------------------+-----------------+
                          |
 
                          |
 
                          |
 
                          |
 
                   +------+-----------------------------+
 
 
                   |                                    |
 
                   |                                    |
 
                   |      Release Engineering          +----------------+
 
                   |      Release Engineering          +----------------+
Line 105: Line 100:
 
                                       +----------------------+              |
 
                                       +----------------------+              |
 
                                                               +---------------+
 
                                                               +---------------+
</pre>
 
  
  
Centralized entry point, logging, and dash board for pre-defined Automated Workflow tasks used by the Release Engineering team with delegation and self-service tasks for members of various teams who normally depend on Release Engineering for various tasks.
+
</pre>
 
 
  
 
Currently Fedora Release Engineering Automation tasks are performed by various scripts run on various machines within the Fedora Infrastructure with no real centralized logging. Some of these are automated by chron jobs and some run by hand by request of various members within the Fedora Community, normally around Fedora Test Days. Finding information about old tasks is not always the easiest of things to do and the delegation of tasks is currently not available. The goal here is to provide a solution that removes those barriers.
 
Currently Fedora Release Engineering Automation tasks are performed by various scripts run on various machines within the Fedora Infrastructure with no real centralized logging. Some of these are automated by chron jobs and some run by hand by request of various members within the Fedora Community, normally around Fedora Test Days. Finding information about old tasks is not always the easiest of things to do and the delegation of tasks is currently not available. The goal here is to provide a solution that removes those barriers.
  
=== Functional Requirements ===
+
Workflows will be executed and potentially orchestrate actions between multiple other systems or tools such as bodhi, pungi, and koji. Fedmsgs will be emitted with information about the start and completion of workflows along with metadata about them.
 
 
The following features are functional requirements
 
  
* Role Based Access Control
+
In the event of a compose, certain fedmsg output will be picked up by taskotron and autocloud to perform various levels of testing.
** Users in certain groups are allowed to execute only certain workflows)
 
** This will enable the self-service component
 
* Public central logging
 
** Workflow tasks should be logged centrally and historic runs of workflows can be publicly viewed in a central location
 
  
 
=== Technical Implementation ===
 
=== Technical Implementation ===
Line 140: Line 127:
  
 
A goal of this proposal is to have a way to execute tasks or jobs that can be centralized, role based
 
A goal of this proposal is to have a way to execute tasks or jobs that can be centralized, role based
 +
 +
UPDATE (2016-07-26): Loopabull has been chosen, eventually when Ansible Tower has been Open Sourced we would like to push features upstream to it and migrate because it also provides many features we and the Fedora Infrastructure Team would like to have.
 +
  
 
The software that becomes the "Workflow Engine" itself is currently being evaluated, this will be the thing that actually executes the Ansible playbooks. The following options are being looked at:
 
The software that becomes the "Workflow Engine" itself is currently being evaluated, this will be the thing that actually executes the Ansible playbooks. The following options are being looked at:
Line 151: Line 141:
 
** http://inaugust.com/talks/zuulv3.html#/
 
** http://inaugust.com/talks/zuulv3.html#/
 
** https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
 
** https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
 +
* Outhouse
 +
** https://pagure.io/outhouse
 +
* Taskotron (future stuff)
 +
** https://taskotron.fedoraproject.org/
 +
** https://bitbucket.org/tflink/taskotron-webtrigger/src/536f3c05072f8c6a167718d0aa6710eb38aecc1a/?at=feature%2Fnextgen-poc
 +
* Semaphore
 +
** Open Source Alternative to Tower
 +
** https://github.com/ansible-semaphore/semaphore
 +
* Loopabull
 +
** Event loop driven Ansible playbook execution engine
 +
** https://github.com/maxamillion/loopabull
  
 
== Benefit to Fedora ==
 
== Benefit to Fedora ==
Line 157: Line 158:
  
 
== Scope ==
 
== Scope ==
* Proposal owners:
+
=== Proposal owners ===
<!-- What work do the feature owners have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
+
 
 +
Proposal owners shall have to:
 +
* Determine what the "Engine" will be after evaluation and working with the Fedora RelEng and Infrastructure teams for advisement.
 +
* Deploy RelEng Automation Workflow Engine
 +
** Fully automated deployment in Fedora Infrastructure Ansible
 +
* Document Workflow Automation
 +
** How workflows are created
 +
** How to run workflows
 +
** How new contributors can get started
 +
 
 +
=== Task matrix ===
 +
 
 +
This is a [https://en.wikipedia.org/wiki/Responsibility_assignment_matrix RACI matrix] for tasks required to implement the [[Changes/ReleaseEngineeringAutomationWorkflowEngine|RelEng Automation Workflow Engine]]. Work is tracked in Taiga: https://taiga.fedorainfracloud.org/project/acarter-fedora-docker-atomic-tooling/wiki/home
 +
 
 +
 
 +
==== Is this current? ====
 +
 
 +
It is, as of <!-- this is an automatic macro — you don't need to change this line --> {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}
  
* Other developers: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+
==== Definitions ====
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
 
  
* Release engineering: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+
Here, we're using what Wikipedia calls "[https://en.wikipedia.org/wiki/Responsibility_assignment_matrix#RACI_.28alternative_scheme.29 RACI (alternative scheme)]":
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  If a rel-eng ticket exists, add a link here.  
 
Please work with releng prior to feature submission, and ensure that someone is on board to do any process development work and testing; don't just assume that a bullet point in a change puts someone else on the hook.-->
 
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
 
  
* Policies and guidelines: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
 
  
* Trademark approval: N/A (not needed for this Change)
+
:; ''Responsible''
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://fedorahosted.org/council/ ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->
+
:: The person responsible for the performance of the task. There should be exactly one person with this assignment for each task.
 +
 
 +
:; ''Assists''
 +
:: Those who assist completion of the task.
 +
 
 +
:; ''Consulted''
 +
:: Those whose opinions are sought; and with whom there is two-way communication.
 +
 
 +
:; ''Informed''
 +
:: Those who are kept up-to-date on progress; and with whom there is one-way communication.
 +
 
 +
=== Task Table ===
 +
{{admon/note|This an early cut. Please feel free to add new tasks as appropriate — just let one of the coordinators know.}}
 +
 
 +
{|
 +
! scope="col"| Task
 +
! scope="col"| Subtask
 +
! scope="col"| Responsible
 +
! scope="col"| Assists
 +
! scope="col"| Consulted
 +
! scope="col"| Informed
 +
! scope="col"| Current Status
 +
|-
 +
| <!-- task      --> Determine what the "Engine" will be
 +
| <!-- subtask    -->
 +
| <!-- responsible--> [[User:Maxamillion| Adam Miller]]
 +
| <!-- assists    -->
 +
| <!-- consulted  -->
 +
| <!-- informed  -->
 +
| <!-- status    -->100%
 +
|-
 +
| <!-- task      --> Deploy Engine solution, including ansible playbooks added for Fedora Infrastructure Ansible repo
 +
| <!-- subtask    -->
 +
| <!-- responsible--> [[User:Maxamillion| Adam Miller]]
 +
| <!-- assists    -->
 +
| <!-- consulted  -->
 +
| <!-- informed  -->
 +
| <!-- status    -->0%
 +
|-
 +
| <!-- task      --> Document the Engine solution final design
 +
| <!-- subtask    -->
 +
| <!-- responsible--> [[User:Maxamillion| Adam Miller]]
 +
| <!-- assists    -->
 +
| <!-- consulted  -->
 +
| <!-- informed  -->
 +
| <!-- status    -->0%
 +
|-
 +
| <!-- task      --> Document Automation Workflows end-to-end
 +
| <!-- subtask    -->
 +
| <!-- responsible--> [[User:Maxamillion| Adam Miller]]
 +
| <!-- assists    -->
 +
| <!-- consulted  -->
 +
| <!-- informed  -->
 +
| <!-- status    -->0%
 +
|-
 +
|-
 +
|}
 +
 
 +
==== Glossary of Nicknames ====
 +
 
 +
* maxamillion [[User:Maxamillion| Adam Miller]]
 +
 
 +
==== Various Task Notes ====
 +
 
 +
=== Functional Requirements ===
 +
 
 +
The following features are functional requirements
 +
 
 +
* Role Based Access Control
 +
** Users in certain groups are allowed to execute only certain workflows)
 +
** This will enable the self-service component
 +
* Public central logging
 +
** Workflow tasks should be logged centrally and historic runs of workflows can be publicly viewed in a central location
 +
 
 +
=== Other developers ===
 +
* (anything here)?
 +
 
 +
=== Release engineering ===
 +
* Deploy the "Engine"
 +
 
 +
=== Policies and guidelines ===
 +
* Need to determine who can create/run workflows
 +
* Define guidelines for writing workflows
  
 
== Upgrade/compatibility impact ==
 
== Upgrade/compatibility impact ==
Line 179: Line 272:
  
 
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)  
+
N/A (not a System Wide Change)
  
 
== How To Test ==
 
== How To Test ==
Line 197: Line 290:
  
 
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)  
+
N/A (not a System Wide Change)
  
 
== User Experience ==
 
== User Experience ==
Line 213: Line 306:
  
 
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
 
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+
* Contingency mechanism: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
 
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
 
* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
 
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+
* Blocks release? No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks product? product <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
+
* Blocks product? N/At <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
  
 
== Documentation ==
 
== Documentation ==
<!-- Is there upstream documentation on this change, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
 
  
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+
Documentation once written will be in the [https://docs.pagure.org/releng Fedora Release Engineering Docs site].
N/A (not a System Wide Change)
 
  
 
== Release Notes ==
 
== Release Notes ==
Line 233: Line 324:
 
-->
 
-->
  
[[Category:ChangePageIncomplete]]
+
[[Category:ChangeAcceptedF25]]
 
<!-- When your change proposal page is completed and ready for review and announcement -->
 
<!-- When your change proposal page is completed and ready for review and announcement -->
 
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
 
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
Line 241: Line 332:
 
<!-- Select proper category, default is Self Contained Change -->
 
<!-- Select proper category, default is Self Contained Change -->
 
[[Category:SelfContainedChange]]
 
[[Category:SelfContainedChange]]
<!-- [[Category:SystemWideChange]] -->
 

Latest revision as of 20:02, 17 August 2016


Release Engineering Automation Workflow Engine

Summary

Centralized entry point, logging, and dash board for pre-defined Automated Workflow tasks used by the Release Engineering team with delegation and self-service tasks for members of various teams who normally depend on Release Engineering for various tasks.

Owner

  • Name: Adam Miller
  • Email: maxamillion@fedoraproject.org
  • Release notes owner:

Current status

Detailed Description

+--------------+                                            +----------------+
|              |                 +------------+             |                |
|  AutoCloud   |<----------------+            +------------>+   Taskotron    |
|              |                 |   fedmsg   |             |                |
|              +---------------->|            |<------------+                |
+--------------+                 |            |             +----------------+
                                 +------------+
                                      ^
                                      |
                                      |
                                      |
                                      |
                                      |
                                      |
                   +------------------+-----------------+
                   |                                    |
                   |      Release Engineering           +----------------+
                   |      Workflow Automation Engine    |                |
                   |                                    |                |
                   +------------------+-----------------+                |
                         |            |                                  |
                         |            |                                  |
       +-----------------+            |                                  |
       |                              |                                  |
       |                              |                                  |
       V                              V                                  |
+-------------+                   +--------------+                       |
|             |                   |              |                       |
|    bodhi    |                   |              |                       |
|             |                   |    pungi     |                       |
+-------------+                   |              |                       |
                                  |              |                       |
                                  +----------+---+                       |
                                       ^     |                           V
                                       |     |                +---------------+
                                       |     |                |               |
                                       |     +--------------->|     koji      |
                                       |                      |               |
                                       +----------------------+               |
                                                              +---------------+


Currently Fedora Release Engineering Automation tasks are performed by various scripts run on various machines within the Fedora Infrastructure with no real centralized logging. Some of these are automated by chron jobs and some run by hand by request of various members within the Fedora Community, normally around Fedora Test Days. Finding information about old tasks is not always the easiest of things to do and the delegation of tasks is currently not available. The goal here is to provide a solution that removes those barriers.

Workflows will be executed and potentially orchestrate actions between multiple other systems or tools such as bodhi, pungi, and koji. Fedmsgs will be emitted with information about the start and completion of workflows along with metadata about them.

In the event of a compose, certain fedmsg output will be picked up by taskotron and autocloud to perform various levels of testing.

Technical Implementation

Everything will be powered by Ansible as this is a toolchain that both Fedora Infrastucture and Fedora Release Engineering is familiar with and has been using heavily for automation tasks. We are simply aiming to solve a new automation problem space with the same tool and a different set of rules/policy.

https://www.ansible.com/

The main component that will define the workflows is going to be Ansible Playbooks.

Ansible

Tasks or sets of tasks should be in an "Include Playbook" such that they are not meant to stand on their own but should be included by other Playbooks or an Ansible Role.

Workflow Playbooks should effectively be "glue" that supply necessary variables to make the "Include Playbooks" and Roles useful for the Workflow at hand.

Ansible Execution

A goal of this proposal is to have a way to execute tasks or jobs that can be centralized, role based

UPDATE (2016-07-26): Loopabull has been chosen, eventually when Ansible Tower has been Open Sourced we would like to push features upstream to it and migrate because it also provides many features we and the Fedora Infrastructure Team would like to have.


The software that becomes the "Workflow Engine" itself is currently being evaluated, this will be the thing that actually executes the Ansible playbooks. The following options are being looked at:

Benefit to Fedora

The goal here is the benefit the Fedora Contributor Community at large by making certain processes within Release Engineering be able to be more rapidly iterated upon, allowing for changes to processes to become more flexible. Another goal is to make Fedora Release Engineering more approachable by making it easier to contribute to work we d and make it easier for new members of the community to join in the Fedora Release Engineering group.

Scope

Proposal owners

Proposal owners shall have to:

  • Determine what the "Engine" will be after evaluation and working with the Fedora RelEng and Infrastructure teams for advisement.
  • Deploy RelEng Automation Workflow Engine
    • Fully automated deployment in Fedora Infrastructure Ansible
  • Document Workflow Automation
    • How workflows are created
    • How to run workflows
    • How new contributors can get started

Task matrix

This is a RACI matrix for tasks required to implement the RelEng Automation Workflow Engine. Work is tracked in Taiga: https://taiga.fedorainfracloud.org/project/acarter-fedora-docker-atomic-tooling/wiki/home


Is this current?

It is, as of 2016-08-17

Definitions

Here, we're using what Wikipedia calls "RACI (alternative scheme)":


Responsible
The person responsible for the performance of the task. There should be exactly one person with this assignment for each task.
Assists
Those who assist completion of the task.
Consulted
Those whose opinions are sought; and with whom there is two-way communication.
Informed
Those who are kept up-to-date on progress; and with whom there is one-way communication.

Task Table

Note.png
This an early cut. Please feel free to add new tasks as appropriate — just let one of the coordinators know.
Task Subtask Responsible Assists Consulted Informed Current Status
Determine what the "Engine" will be Adam Miller 100%
Deploy Engine solution, including ansible playbooks added for Fedora Infrastructure Ansible repo Adam Miller 0%
Document the Engine solution final design Adam Miller 0%
Document Automation Workflows end-to-end Adam Miller 0%

Glossary of Nicknames

Various Task Notes

Functional Requirements

The following features are functional requirements

  • Role Based Access Control
    • Users in certain groups are allowed to execute only certain workflows)
    • This will enable the self-service component
  • Public central logging
    • Workflow tasks should be logged centrally and historic runs of workflows can be publicly viewed in a central location

Other developers

  • (anything here)?

Release engineering

  • Deploy the "Engine"

Policies and guidelines

  • Need to determine who can create/run workflows
  • Define guidelines for writing workflows

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

N/A (not a System Wide Change)

User Experience

N/A (not a System Wide Change)

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? No
  • Blocks product? N/At

Documentation

Documentation once written will be in the Fedora Release Engineering Docs site.

Release Notes