Changes/ArbitraryBranching

Arbitrary Branching

Summary

Tooling changes to support the new way of branching for Fedora 27

Owner

• Name: Matt Prahl
• Email: mprahl@redhat.com
• Release notes owner:

Current status

• Targeted release: Fedora 27
• Last updated: 2017-05-26
• Tracker bug: <will be assigned by the Wrangler>

Detailed Description

To avoid maintaining multiple copies of this information, please read the Arbitrary Branching Focus Document (updated May 3rd, 2017) in the Factory2 space as a substitute for this document: https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching

For further information, view this section of Ralph Bean’s presentation at DevConf: https://youtu.be/5gqccjyjwFk?t=26m27s

Timeline

• Advertisement
  • Pending FESCo approval, we will do the following on Friday May 26th.
    • With a Freeze Break Request, patch a banner into prod pkgdb advertising the Change (here) and Infrastructure/WhatHappenedToPkgdb. Pierre (pkgdb maintainer) is on vacation this week. Ralph will send the FBR and apply the patch if approved.
    • Send an email to devel-announce linking to the Change (here) and to Infrastructure/WhatHappenedToPkgdb.
• Client tooling for new package requests, branch requests, admin actions, etc.
  • June: To be written and packaged. The FAQ will instruct people how to use these things.
• Pagure has the ability to provide user-level API tokens to do things such as create tickets on projects
  • June: To be written and deployed in pagure.io (https://pagure.io/pagure/issue/2250)
• PDC being populated
  • June. Write a script to sync existing pkgdb data to PDC, and run it. Data in PDC will not be consulted yet. Run the script again just before we turn off pkgdb in July.
• Pagure over dist-git in production (outside of Factory 2.0 control, in the hands of the Fedora Engineering team)
  • July 5th. If this slips, all other dates slip too.
• Moving over the scripts that query PkgDB to use Pagure
  • Development, done (see the infra ansible repo).
  • Perform the switchover the week of July 5th, after pagure is in place.
• Bodhi being updated and configuration changed to use Pagure and PDC
  • Development, done (see the bodhi pull request).
  • Perform the switchover (config change) the week of July 5th, after pagure is in place.
• Patch loopabull plugin to query PDC+pagure instead of pkgdb. Patch to be submitted in June.
  • https://pagure.io/releng-automation/blob/master/f/ansible/library/fedcontainer_rc.py
• Patch releng scripts that interact with pkgdb to work with the new data sources. Patches to be submitted in June.
  • https://pagure.io/releng/blob/master/f/scripts/block_retired.py
  • https://pagure.io/releng/blob/master/f/scripts/fedretire
  • https://pagure.io/releng/blob/master/f/scripts/find_unblocked_orphans.py
  • https://pagure.io/releng/blob/master/f/scripts/get-critpath
  • https://pagure.io/releng/blob/master/f/scripts/update-critpath
• We need a static JSON file generated from PDC’s data emulating PkgDB’s API, for GNOME Software.
  • Write the script to produce this file in June and sync it to the proxies after freeze.
  • Establish the HTTP redirect when we bring down PkgDB.
• PkgDB decommissioning
  • Once all other pieces above are complete, do the following (expected week of July 10th):
  • Deploy redirects to wiki page.
  • Deploy redirects to JSON file.
  • Take PkgDB out of haproxy.
  • Re-run the sync-to-pdc script to catch the last of the data.
  • Backup the PkgDB db and archive a copy for posterity.
  • Wait a week for disaster to strike.
  • Destroy PkgDB VMs.

Benefit to Fedora

See the “Background on PkgDB” and “Why Make This Change?” sections in the Arbitrary Branching Focus document linked above.

Scope

• Proposal owners: We will do all of the work listed in the linked document.

• Other developers:
  • We're relying on the infra team to get Pagure on top of dist-git.
  • We're relying on releng to find any holes in our work plan and to help us make decisions on how to solve bugs.
  • No direct work should be required from the general packaging group to support this Change.

• Release engineering: #6775

• • List of deliverables: Not affected

• Policies and guidelines: Packaging guidelines would change and would be updated after the changes are implemented in staging.

• Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

Not backwards compatible but not significantly difficult to roll back because all changes will be done in Ansible.

How To Test

• Request new packages
• Request new branches with non-standard names (i.e. not f25, f26, etc.) and build with them
• Request new branches for standard names that are still supported (i.e. f25, f26, epel6, epel7) and build with them
• Retire a package
• Orphan a package

User Experience

See the “How To Make This Change?” section in the Arbitrary Branching Focus document linked above.

Dependencies

• Pagure over dist-git will need to be updated in staging
• Pagure over dist-git will need to be deployed in production

Contingency Plan

• Contingency mechanism: Re-enable PkgDB and roll back our script changes in Ansible
• Contingency deadline: N/A
• Blocks release? Yes, it blocks Fedora 27 Modularity. They will be filing a Change document in the coming months to describe the F27 structure
• Blocks product? Fedora 27

Documentation

Documentation on new processes would be provided at a later date if there is approval on this Change Proposal.