From Fedora Project Wiki

m
(fedrepo-req is not related to Bodhi.)
 
(52 intermediate revisions by 12 users not shown)
Line 3: Line 3:
 
== Working Session Notes ==
 
== Working Session Notes ==
  
Coming soon -- winter 2017/2018.
+
[https://public.etherpad-mozilla.org/p/fedora-hack-2018 Etherpad with notes from the sessions]
  
 
=== Output and outcomes ===
 
=== Output and outcomes ===
  
This section will record output blogs following the event.
+
Blog entries:
 +
* [https://www.scrye.com/wordpress/nirik/2018/04/15/fedora-infrastructure-hackfest-2018/ Blog from [[User:Kevin|Kevin Fenzi]]]
 +
* [http://blog.pingoured.fr/index.php?post/2018/04/13/Fedora-Infrastructure-Hackathon-2018 Blog from [[User:pingou|Pierre-Yves Chibon]]]
 +
* [https://blog.electronsweatshop.com/2018-fedora-infrastructure-hackathon.html Blog from [[User:Bowlofeggs|Randy Barlow]]]
 +
* [https://sinny.io/2018/04/18/things-we-did-at-fedora-infrastructure-hackathon-2018/ Blog from [[User:sinnykumari | Sinny Kumari]]]
 +
* [https://elrod.me/posts/2018-04-20-fedora-infrastructure-hackfest-2018-recap.html Blog from [[User:Codeblock | Rick Elrod]]]
 +
* [http://smoogespace.blogspot.com/2018/04/fedora-infrastructure-hackathon-day-1-5.html Blog from [[User:Smooge | Ebeneezer Smooge]]]
 +
* [https://dustymabe.com/2018/04/23/april-fedora-infrastructure-hackfest/ Blog from [[User:dustymabe | Dusty Mabe]]]
 +
* [https://patrick.uiterwijk.org/blog/2018/4/23/infra-hackfest/ Blog from [[User:puiterwijk | Patrick Uiterwijk]]]
  
 
== Purpose ==
 
== Purpose ==
Line 18: Line 26:
 
=== Day 1 (Monday) ===
 
=== Day 1 (Monday) ===
  
* Clean up Docs, workflows and scripts for new services
+
* Go over the current lifecycle for "artifacts" and brainstorm where we can improve the developer and/or admin workflow:
 +
** Review - currently in bugzilla
 +
** New artifact request - fedpkg request-repo / fedrepo-req-admin - token issues
 +
** Commits / Building
 +
** Updates filing
 +
** CI/User feedback (waving results needs to be better)
 +
** Orphaning (orphaned.package file/listener?)
 +
** Retiring
 +
* Make such changes as we can do today, file bugs/get started on any others and note them.
 +
* Once any changes are made or planned on above:
 +
** Decide if we want to update wiki docs or move to some better place?
 +
** Identify wiki docs that need updating based on popularity.
 +
** Check db dump for strings that need indicate pages need update
 +
** Possibly decide to delete some pages
 +
** Note where things will change
 +
* Document what things we updated and changed and send to devel announce that docs/workflows are hopefully better.
  
 
=== Day 2 (Tuesday) ===
 
=== Day 2 (Tuesday) ===
  
* ELK proof of concept  
+
* Find new maintainers for jcline's applications
 +
** fedmsg
 +
** FMN
 +
** Anitya
 +
** the-new-hotness
 +
* <strike>ELK proof of concept</strike>
 
* Rawhide gating and CI
 
* Rawhide gating and CI
  
Line 46: Line 74:
  
 
{| class="wikitable"
 
{| class="wikitable"
! Deliverables: ((TOPIC 1)) !! Task Owner !! Done
+
! Deliverables: Infra !! Task Owner !! Completion
 
|-
 
|-
| Better docs, workflows and scripts for package maintainers || || (check here when done)
+
| Better docs, workflows and scripts for package maintainers || [[User:Kevin|kfenzi]] || {{progress bar|40}}
 
|-
 
|-
! Deliverables: ((TOPIC 2)) !! Task Owner !! Done
+
| Working AWX instance in Fedora Infrastructure || [[User:Kevin|kfenzi]] || {{progress bar|100}}
 
|-
 
|-
| Working AWX instance in Fedora Infrastructure || || (check here when done)
+
| More apps in openshift || [[User:Kevin|kfenzi]] || {{progress bar|30}}
 
|-
 
|-
! Deliverables: ((TOPIC 2)) !! Task Owner !! Done  
+
! Deliverables: Bodhi !! Task Owner !! Done  
 
|-
 
|-
| Working ELK search engine proof of concept || || (check here when done)
+
| Rawhide gating of some kind || [[User:Bowlofeggs|rbarlow]] || {{progress bar|50}}
 
|-
 
|-
! Deliverables: ((TOPIC 2)) !! Task Owner !! Done  
+
! Deliverables: CI !! Task Owner !! Done  
 
|-
 
|-
| Rawhide gating of some kind || || (check here when done)
+
| jenkins jobs refactored || [[User:Pingou|pingou]] || {{progress bar|100}}
|-
 
! Deliverables: ((TOPIC 2)) !! Task Owner !! Done
 
|-
 
| More apps in openshift || || (check here when done)
 
|-
 
! Deliverables: ((TOPIC 2)) !! Task Owner !! Done
 
|-
 
| jenkins jobs refactored || || (check here when done)
 
 
|-
 
|-
 
|}
 
|}
 +
 +
=== Done ===
 +
 +
==== Infra ====
 +
* Roadmap for apps -> OpenShift
 +
* Bodhi web UI running in OpenShift
 +
* OpenShift 3.9 bits pulled down for new dev instance
 +
* New templates for infra issues
 +
* Killed darkserver
 +
* Jenkins in Fedora infra now redirects from old service to CentOS
 +
* Deprecated summershum
 +
 +
==== Bodhi/gating ====
 +
* Bodhi shows missing tests
 +
* Bodhi uses waiverdb correctly
 +
* Bodhi shows waiver info
 +
* New bodhi DB model
 +
* New UI planned for Bodhi
 +
* Fixed Bodhi build testing (btest)
 +
* Bodhi can create and delete side tags
 +
* Packaged python-transitions for Bodhi side tagging transitions
 +
* Fixed cornice for testing Bodhi
 +
 +
==== CI/Atomic/Other ====
 +
* Fixed fedmsg-meta for the all-packages pipeline
 +
* New fedmsg-meta rebase/build
 +
* adamwill owns the new yak_farmers FAS group :-)
 +
* Wrote rbac-playbook to get RPM from needed systems
 +
* Hack in place for Atomic ostree mirroring
 +
* [[User:Sinnykumari|Sinny]] trained up for Atomic releases including Fedora GA
 +
* Atomic multi-arch presence for Atomic on fp.o website planned out
 +
* Cleaned sysadmin Atomic setup
  
 
=== Follow-up tickets ===
 
=== Follow-up tickets ===
 +
 +
* Bodhi
 +
** [https://github.com/fedora-infra/bodhi/projects/3 Side tag support]
 +
*** bodhi-push should not select Rawhide builds
 +
*** API changes for side tags
 +
*** can merge side tags
 +
*** can expire side tags after lifetime is up
 +
*** can edit/create tags (admin override)
 +
*** auto-push to stable on test results
 +
*** docs for Rawhide gating
 +
*** comments on update a/b results
 +
*** retest
 +
* fix docs to eliminate fedrepo-req
 +
* fedmsg
 +
** update creates
 +
* OpenShift 3.9
 +
** new Openstack cloud for dev (during freeze)
 +
** new dev 3.9 (during freeze)
 +
** new staging 3.9 (~end of freeze)
 +
** prod 3.9 (after freeze lifts)
 +
* retire hubs dev/stg
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 81: Line 154:
  
 
=== Attendees and Travel Details ===
 
=== Attendees and Travel Details ===
{{admon/note | Suggestions | These are only suggested attendees at the moment. Final roster TBD based on objectives.}}
 
 
{{admon/note | Travel details | Refer to [[Infrastructure Hackathon 2018/Travel]] for itineraries and costs.}}
 
{{admon/note | Travel details | Refer to [[Infrastructure Hackathon 2018/Travel]] for itineraries and costs.}}
 
{| class="wikitable"
 
{| class="wikitable"
! Contributor !! Arrival !! Departure !! Roommate !! Notes (airfare costs)
+
! Contributor !! Arrival !! Departure !! Roommate !! Notes (airfare costs) !! Paid/expensed? !! Ride with (arr/dep)
|-
 
| [[User:Pfrields|Paul W. Frields]] ||  ||  || N/A if FXBG || No cost if FXBG
 
|-
 
| [[User:Smooge|Stephen Smoogen]] ||  ||  || N/A if RDU || $100 (train)
 
|-
 
| [[User:Puiterwijk|Patrick Uiterwijk]] ||  ||  ||  || $1000 (handled from Platform dept)
 
 
|-
 
|-
| [[User:Codeblock|Ricky Elrod]] || ||  || || $250
+
| [[User:Smooge|Stephen Smoogen]] || Apr 8 15:03 Amtrak || Apr 13 12:20 Amtrak || [[User:Codeblock|Ricky Elrod]] || $100 (train) ||  || pfrields/pfrields
 
|-
 
|-
| [[User:Nb|Nick Bebout]] ||  ||  || || ~$400-$500
+
| [[User:Puiterwijk|Patrick Uiterwijk]] || 2018-04-08 15:35 IAD - EI0119 || 2018-04-13 17:15 IAD - EI0118 || [[User:Kevin|Kevin Fenzi]] || $1000 (Platform dept) || yes || kfenzi/kfenzi
 
|-
 
|-
| [[User:Tibbs|Jason Tibbitts]] ||  || || ||  
+
| [[User:Codeblock|Ricky Elrod]] ||  Apr 8, 20:57, IAD, UA6031 || Apr 13, 12:50, IAD, UA4995 || [[User:Smooge|Stephen Smoogen]] || $250 (Platform dept) || yes || relrod/relrod
 
|-
 
|-
| [[User:Rbarlow|Randy Barlow]] ||  || || N/A if RDU || $100 (train)
+
| [[User:Dustymabe|Dusty Mabe]] ||  Apr 8 15:03 Amtrak || Apr 13 12:20 Amtrak || || $100 (train) || || pfrields/pfrields
 
|-
 
|-
| [[User:Kevin|Kevin Fenzi]] || || || || $600 (handled from Platform dept)
+
| [[User:Rbarlow|Randy Barlow]] || Apr 8 15:03 Amtrak || Apr 13 12:20 Amtrak || [[User:Pingou|Pierre-Yves Chibon]] || $100 (train) ||  || pfrields/pfrields
 
|-
 
|-
| [[User:Pingou|Pierre-Yves Chibon]] ||  || || || $1200 (flight + train) (handled from Platform dept)
+
| [[User:Kevin|Kevin Fenzi]] || Apr 8 15:47 UA250 || Apr 13 17:05 UA251 || [[User:Puiterwijk|Patrick Uiterwijk]] || $600 (Platform dept) || yes || kfenzi/kfenzi
 
|-
 
|-
| [[User:Sinnykumari|Sinny Kumari]] || || || || $1300?
+
| [[User:Pingou|Pierre-Yves Chibon]] || Apr 7 15:50 - AF0054 || Apr 13 17:40 - AF6697 || [[User:Rbarlow|Randy Barlow]] || $1200 (flight + train) (Platform dept) || yes (flight) || kfenzi/kfenzi
 
|-
 
|-
| [[User:Ryanlerch|Ryan Lerch]] ||  || ||  || $2200 (handled from Platform dept)
+
| [[User:Sinnykumari|Sinny Kumari]] || Apr 8 15:35 - IAD(QR 707) || Apr 13 20:25 - IAD(QR 708) ||  || $1150 (flight) (Platform dept) || yes || kfenzi/kfenzi
 
|-
 
|-
| [[User:Jperrin|Jim Perrin]] || ||  || || $700 - $800
+
| [[User:Ryanlerch|Ryan Lerch]] || April 8 17:58 QF3083/AA1362 IAD ||  Apr 13 19:05 UA719 IAD || [[User:Bstinson|Brian Stinson]] || $2283 (Platform dept) || yes || relrod/kfenzi
 
|-
 
|-
| [[User:Bstinson|Brian Stinson]] ||  ||  ||  || $600-$700
+
| [[User:Jperrin|Jim Perrin]] ||  ||  ||  || $700 - $800 (handled from OSAS) || yes || jperrin/jperrin
 
|-
 
|-
| || || || ||  
+
| [[User:Bstinson|Brian Stinson]] || Apr 8 14:21 AA2354 - IAD || Apr 13 11:35 AA1556 - IAD || [[User:Ryanlerch|Ryan Lerch]] || $600-$700 (OSAS) || yes || kfenzi/relrod
 
|-
 
|-
 +
| [[User:mattdm|Matthew Miller]] || Apr 8  18:08 Amtrak ||  Apr '''12''' 8:00 Amtrak ||  || (FPL budget)|| yes || pfrields/pfrields
 
|}
 
|}
  
Line 121: Line 188:
  
 
* Work out budget
 
* Work out budget
** Travel fares -- ~$5700
+
** Travel fares -- ~$? (nb to advise)
** Lodging -- 7 rooms * $104/night + tax = approx. $4000
+
** Lodging -- 8 rooms * $129/night + tax * 5 nights = approx. $6500
 
** Meals -- est. $2500?
 
** Meals -- est. $2500?
 
** Rental cars -- $800-1000 max for two vans
 
** Rental cars -- $800-1000 max for two vans
 
** Meeting room -- $1175 for week (sponsored by Ansible)
 
** Meeting room -- $1175 for week (sponsored by Ansible)
** '''TOTAL: ~$13.5K from Fedora budget'''
+
** '''TOTAL: ~$10K from Fedora budget'''
 
* Decide on Dates and Location
 
* Decide on Dates and Location
 
** April 9-13
 
** April 9-13
Line 136: Line 203:
 
**** IAD recommended for international travelers
 
**** IAD recommended for international travelers
 
*** Workspace
 
*** Workspace
**** http://hiltongardeninn3.hilton.com/en/hotels/virginia/hilton-garden-inn-fredericksburg-DCAFHGI/popup/hotelDetails.html -- meeting room @ hotel - $400/day (total: $2000) if fewer than 10 rooms booked; 1/2 price if 10+
+
**** [http://president.umw.edu/events/campus-event-venues/stafford-campus/ University of Mary Washington - Stafford Graduate Campus] -- corporate classroom - $125/half-day
 
*** Lodging
 
*** Lodging
**** Hilton Garden Inn, ~104/night double = ~$4200 + taxes
+
**** [https://hiltongardeninn3.hilton.com/en/hotels/virginia/hilton-garden-inn-fredericksburg-DCAFHGI/index.html Hilton Garden Inn]
 
*** Rental vehicles
 
*** Rental vehicles
 
**** ~$1000 for two minivan rentals contract carrier (max)
 
**** ~$1000 for two minivan rentals contract carrier (max)
Line 152: Line 219:
 
== Plan ==
 
== Plan ==
 
# '''Location:''' Fredericksburg, VA
 
# '''Location:''' Fredericksburg, VA
#* Lodging: Hilton Garden Inn Fredericksburg
+
#* Lodging: [https://hiltongardeninn3.hilton.com/en/hotels/virginia/hilton-garden-inn-fredericksburg-DCAFHGI/index.html Hilton Garden Inn Fredericksburg]
 
# '''Date:''' April 9-13
 
# '''Date:''' April 9-13
 
#* Arrivals: Sunday April 8
 
#* Arrivals: Sunday April 8
# '''Remote Attendees:''' (include dial-in)
+
# '''Remote Attendees:''' {{fpchat|#fedora-admin}}
 
# '''Schedule'''
 
# '''Schedule'''
 
#* Event starts 09:30 daily
 
#* Event starts 09:30 daily
Line 162: Line 229:
 
== Logistics ==
 
== Logistics ==
  
'''Snacks/Beverages:'''  
+
* '''Travel to hotel:''' Travelers will group by arrival and travel via rental car from IAD (Dulles, VA) to the hotel in Fredericksburg. Travel time on Sunday afternoon is between 70-90 minutes depending on traffic conditions.
 +
* '''Snacks/Beverages:''' Limited access due to the nature of the graduate campus. Monday will give us an idea how we can accommodate during the event.
 +
* '''Meals:'''
 +
** Breakfast before event -- venues TBD, meet in lobby of hotel at 08:15 daily
 +
** Lunch -- venues TBD, 12:30-13:45 daily
 +
** Dinner -- venues TBD after event ends (17:30)
  
 
== Travel estimates ==
 
== Travel estimates ==

Latest revision as of 16:54, 26 April 2018

This is the main page for an infrastructure hackathon in spring 2018. This hackathon is intended to help the team leap ahead for several critical Fedora and CentOS initiatives.

Working Session Notes

Etherpad with notes from the sessions

Output and outcomes

Blog entries:

Purpose

Our purpose is to complete the following primary goals:

Day 0 (Sunday)

  • Travel to hackathon

Day 1 (Monday)

  • Go over the current lifecycle for "artifacts" and brainstorm where we can improve the developer and/or admin workflow:
    • Review - currently in bugzilla
    • New artifact request - fedpkg request-repo / fedrepo-req-admin - token issues
    • Commits / Building
    • Updates filing
    • CI/User feedback (waving results needs to be better)
    • Orphaning (orphaned.package file/listener?)
    • Retiring
  • Make such changes as we can do today, file bugs/get started on any others and note them.
  • Once any changes are made or planned on above:
    • Decide if we want to update wiki docs or move to some better place?
    • Identify wiki docs that need updating based on popularity.
    • Check db dump for strings that need indicate pages need update
    • Possibly decide to delete some pages
    • Note where things will change
  • Document what things we updated and changed and send to devel announce that docs/workflows are hopefully better.

Day 2 (Tuesday)

  • Find new maintainers for jcline's applications
    • fedmsg
    • FMN
    • Anitya
    • the-new-hotness
  • ELK proof of concept
  • Rawhide gating and CI

Day 3 (Wednesday)

  • AWX setup in Fedora Infrastructure
  • More Rawhide Gating and CI

Day 4 (Thursday)

  • Move some apps to OpenShift (extra goal)
  • Bodhi hackfest (extra goal)
  • Pagure hackfest (extra goal)
  • Jenkins jobs refactor (extra goal)

Day 5 (Friday)

  • Knowledge transfer and finish up

Detailed Work Items & Final Attendees

Deliverables

Deliverables: Infra Task Owner Completion
Better docs, workflows and scripts for package maintainers kfenzi
Working AWX instance in Fedora Infrastructure kfenzi
More apps in openshift kfenzi
Deliverables: Bodhi Task Owner Done
Rawhide gating of some kind rbarlow
Deliverables: CI Task Owner Done
jenkins jobs refactored pingou

Done

Infra

  • Roadmap for apps -> OpenShift
  • Bodhi web UI running in OpenShift
  • OpenShift 3.9 bits pulled down for new dev instance
  • New templates for infra issues
  • Killed darkserver
  • Jenkins in Fedora infra now redirects from old service to CentOS
  • Deprecated summershum

Bodhi/gating

  • Bodhi shows missing tests
  • Bodhi uses waiverdb correctly
  • Bodhi shows waiver info
  • New bodhi DB model
  • New UI planned for Bodhi
  • Fixed Bodhi build testing (btest)
  • Bodhi can create and delete side tags
  • Packaged python-transitions for Bodhi side tagging transitions
  • Fixed cornice for testing Bodhi

CI/Atomic/Other

  • Fixed fedmsg-meta for the all-packages pipeline
  • New fedmsg-meta rebase/build
  • adamwill owns the new yak_farmers FAS group :-)
  • Wrote rbac-playbook to get RPM from needed systems
  • Hack in place for Atomic ostree mirroring
  • Sinny trained up for Atomic releases including Fedora GA
  • Atomic multi-arch presence for Atomic on fp.o website planned out
  • Cleaned sysadmin Atomic setup

Follow-up tickets

  • Bodhi
    • Side tag support
      • bodhi-push should not select Rawhide builds
      • API changes for side tags
      • can merge side tags
      • can expire side tags after lifetime is up
      • can edit/create tags (admin override)
      • auto-push to stable on test results
      • docs for Rawhide gating
      • comments on update a/b results
      • retest
  • fix docs to eliminate fedrepo-req
  • fedmsg
    • update creates
  • OpenShift 3.9
    • new Openstack cloud for dev (during freeze)
    • new dev 3.9 (during freeze)
    • new staging 3.9 (~end of freeze)
    • prod 3.9 (after freeze lifts)
  • retire hubs dev/stg
Follow-up: Ticket
(Describe item that still needs to be completed) (link to ticket)

Attendees and Travel Details

Note.png
Travel details
Refer to Infrastructure Hackathon 2018/Travel for itineraries and costs.
Contributor Arrival Departure Roommate Notes (airfare costs) Paid/expensed? Ride with (arr/dep)
Stephen Smoogen Apr 8 15:03 Amtrak Apr 13 12:20 Amtrak Ricky Elrod $100 (train) pfrields/pfrields
Patrick Uiterwijk 2018-04-08 15:35 IAD - EI0119 2018-04-13 17:15 IAD - EI0118 Kevin Fenzi $1000 (Platform dept) yes kfenzi/kfenzi
Ricky Elrod Apr 8, 20:57, IAD, UA6031 Apr 13, 12:50, IAD, UA4995 Stephen Smoogen $250 (Platform dept) yes relrod/relrod
Dusty Mabe Apr 8 15:03 Amtrak Apr 13 12:20 Amtrak $100 (train) pfrields/pfrields
Randy Barlow Apr 8 15:03 Amtrak Apr 13 12:20 Amtrak Pierre-Yves Chibon $100 (train) pfrields/pfrields
Kevin Fenzi Apr 8 15:47 UA250 Apr 13 17:05 UA251 Patrick Uiterwijk $600 (Platform dept) yes kfenzi/kfenzi
Pierre-Yves Chibon Apr 7 15:50 - AF0054 Apr 13 17:40 - AF6697 Randy Barlow $1200 (flight + train) (Platform dept) yes (flight) kfenzi/kfenzi
Sinny Kumari Apr 8 15:35 - IAD(QR 707) Apr 13 20:25 - IAD(QR 708) $1150 (flight) (Platform dept) yes kfenzi/kfenzi
Ryan Lerch April 8 17:58 QF3083/AA1362 IAD Apr 13 19:05 UA719 IAD Brian Stinson $2283 (Platform dept) yes relrod/kfenzi
Jim Perrin $700 - $800 (handled from OSAS) yes jperrin/jperrin
Brian Stinson Apr 8 14:21 AA2354 - IAD Apr 13 11:35 AA1556 - IAD Ryan Lerch $600-$700 (OSAS) yes kfenzi/relrod
Matthew Miller Apr 8 18:08 Amtrak Apr 12 8:00 Amtrak (FPL budget) yes pfrields/pfrields

Planning Prerequisites

See the How to organize a FAD list; you can keep your to-do list here.

  • Work out budget
    • Travel fares -- ~$? (nb to advise)
    • Lodging -- 8 rooms * $129/night + tax * 5 nights = approx. $6500
    • Meals -- est. $2500?
    • Rental cars -- $800-1000 max for two vans
    • Meeting room -- $1175 for week (sponsored by Ansible)
    • TOTAL: ~$10K from Fedora budget
  • Decide on Dates and Location
    • April 9-13
    • Arrivals: Sunday April 8; departures: Friday April 13
  • Arrange Facilities
  • List Resources
  • Be Somewhat Structured
  • Arrange Lodging
  • Arrange Refreshments
  • Arrange a Social Event -- N/A

Plan

  1. Location: Fredericksburg, VA
  2. Date: April 9-13
    • Arrivals: Sunday April 8
  3. Remote Attendees: #fedora-admin[?]
  4. Schedule
    • Event starts 09:30 daily
    • Event ends by 12:00pm Friday April 13, to allow for travel outbound

Logistics

  • Travel to hotel: Travelers will group by arrival and travel via rental car from IAD (Dulles, VA) to the hotel in Fredericksburg. Travel time on Sunday afternoon is between 70-90 minutes depending on traffic conditions.
  • Snacks/Beverages: Limited access due to the nature of the graduate campus. Monday will give us an idea how we can accommodate during the event.
  • Meals:
    • Breakfast before event -- venues TBD, meet in lobby of hotel at 08:15 daily
    • Lunch -- venues TBD, 12:30-13:45 daily
    • Dinner -- venues TBD after event ends (17:30)

Travel estimates

Contributor Taxi/transport (to/from home) Airfare Taxi/transport (to/from site) Parking Other
  1. Travel: (TOTAL) (est)
  2. Housing: (TOTAL) (est)
  3. Space:
    • (COST) - Supplied by (LOCATION)
  4. Supplies:
    • N/A
  5. Food: (TOTAL) (est -- to be paid by Fedora Engineering budget)

Total budget: (TOTAL) estimated