From Fedora Project Wiki

(Fix NVR_LIST)
 
(109 intermediate revisions by 33 users not shown)
Line 1: Line 1:
{{Draft|This page is under re-construction.  It covers NFR only for developer Users. To complete for Tester, Mirror, & Other}}
+
This document shows how to submit an update for a package you maintain in Fedora.  It assumes you already have a package in the Fedora repositories. It is not a guide to using the Fedora package source control system: see the [[Package maintenance guide]] for that.
  
This document shows how to update a package you maintain in Fedora.  It assumes you already have a package in the Fedora repositories.
+
* For details of the policy on requirements for updates at various stages of the [[Fedora Release Life Cycle]], refer to [[Updates Policy]].
This document is divided in three sections to give Developers, Testers, and Mirror Admins some guidelines on how to submit packages for [[Releases/Rawhide | <code>rawhide]], development</code> and <code>pending</code>.  
 
  
* For more information on becoming a package maintainer, refer to [[PackageMaintainers/Join]]
+
== Overview ==
* For more guidance on package updates, refer to [[PackageMaintainers/Package update guidelines]].
+
 
 +
This page is intended for new and existing package maintainers. Testers and regular users may be interested in the [[QA:Updates_Testing|updates-testing]] repository and the [[QA:Update_feedback_guidelines|update feedback guidelines]]. This page specifically covers the update submission process.
 +
 
 +
There are two significantly different package update submission workflows in Fedora:
 +
 
 +
* [[Releases/Rawhide|Rawhide]], and [[Releases/Branched|Branched]] up to the [[Updates Policy#Bodhi enabling|Bodhi enabling point]]
 +
* [[Releases/Branched|Branched]] releases after the Alpha change deadline, and stable releases
 +
 
 +
The repository layouts differ somewhat for Rawhide, Branched and stable releases, but the update workflows split up as described above.
 +
 
 +
== Rawhide and early Branched ==
 +
 
 +
The package update workflow for Rawhide and Branched before the ''Bodhi enabling point'' is simple:
 +
 
 +
# Build the package with {{command|fedpkg build}} (see the [[Package maintenance guide]] for more details)
 +
 
 +
This is all you need to do. Your package will appear in the next daily compose of Rawhide or Branched and will be used in any image composes built from that tree.
 +
 
 +
== Later Branched and stable releases ==
 +
 
 +
At the [[Updates Policy#Bodhi enabling|Bodhi enabling point]], the [[Bodhi]] update feedback system is enabled by [[ReleaseEngineering|Release Engineering]] and builds submitted with {{command|fedpkg build}} are no longer automatically sent to any official [[Repositories|repository]]. The update workflow for releases of this type is:
 +
 
 +
{{admon/tip|Fedora account name|{{command|fedpkg}} should be able to discover your [[Account_System|Fedora account system]] user name from the {{filename|~/.fedora.cert}} file set up by {{command|fedora-packager-setup}} when you first [[Join_the_package_collection_maintainers#Install_the_client_tools_.28Koji.29_and_set_up_your_certificate|configured your system for packaging]]. If this fails for any reason, you can specify it with {{command|--user (username)}}. For the {{command|bodhi}} command line tool, you may need to specify your Fedora user name with {{command|-u (username)}} if it differs from your system user name.}}
 +
 
 +
# Build the package with {{command|fedpkg build}}
 +
# Submit an update for the package with {{command|fedpkg update}}, the [https://admin.fedoraproject.org/updates/ Bodhi web interface], or the [https://fedorahosted.org/bodhi/wiki/CLI Bodhi CLI tool]. This causes the package to be sent to the [[Repositories#updates-testing|''updates-testing'']] repository
 +
# Monitor the update's status and the feedback you receive via the web interface or the emails that are sent to you, and modify it with updated or additional builds if necessary
 +
# After the update meets the criteria in the [[Updates Policy]] and you are satisfied it should be released as a stable update, submit the update to ''[[Repositories#stable|stable]]'' with {{command|bodhi -R stable}} or the web interface
 +
 
 +
{{admon/important|Updating inter-dependent packages|If a package you wish to update requires other package(s) to be rebuilt before it or they will work properly, you '''must''' submit the builds together as a multi-package update. See [[#multi|below]] for more details on this.}}
 +
 
 +
=== Update attributes ===
 +
 
 +
At the time you submit the update, you will be asked for several attributes. The type of the update should be fairly self-explanatory: either it fixes bugs, adds new features, or is a new package.
 +
 
 +
If you are asked whether you want to send the update to ''updates-testing'' or ''stable'', this is a no-op: all updates now go through ''updates-testing''. It does not matter what you choose.
 +
 
 +
There are several schools of thought on filling out the update description. Some would suggest you consider the target audience: for a stable release, in particular, many Fedora users will see this text, and many of them may not be particularly familiar with your package. Consider not simply describing literally the changes in the update, but explaining as if to an outsider why your are updating the package, what benefits it will bring to them (if any), and anything they may want to note in order to have a smooth update experience.
 +
 
 +
If you associate one or more bug reports with your update, Bodhi will post comments into Bugzilla to alert those following the bug reports that an update is available. If you mark your update as fixing the bug(s), Bodhi will move the report(s) through the '''MODIFIED''', '''ON_QA''' and '''CLOSED ERRATA''' states of the [[BugZappers/BugStatusWorkFlow|bug workflow]] as your update reaches various points in the process. Using this mechanism can be very useful both for you and for users of your package.
 +
 
 +
You may set a ''karma'' (feedback) level at which the update will automatically be submitted to ''stable''. This is optional. If you choose to use it, please carefully consider an appropriate feedback level. For a relatively obscure package which is quite stable, 1 or 2 may be an appropriate value. For a popular, sensitive and complex package such as {{package|firefox}} or the {{package|kernel}}, the default of 3 may be insufficient and a choice of 5 or even 10 may be appropriate.
 +
 
 +
=== Who will receive your update, when? ===
  
== Overview ==
+
When a release is in Branched state, the ''updates-testing'' repository is enabled by default so most users will see the package, but only packages from the stable ''fedora'' repository are used in building milestone releases (Alpha, Beta and Final) and nightly images.
Here you can find some preliminary information about the new process of Package Management. If you want to know the difference between [[Releases/Rawhide | <code>rawhide]], development</code> and <code>pending</code> , and which one is suitable for you, along with an overall understanding of the release naming and repos, visit our new and improved [[ReleaseEngineering/Overview | Fedora Development Process Overview]] page.  
 
  
New contributors (mandatory reading), new Testers (highly suggested reading), new Consumers (useful reading), or anybody interested in how Fedora is developed would find this page useful.
+
Where a package goes when it is marked as ''stable'' differs between Branched and stable releases. In Branched releases, ''stable'' packages are pushed to the base ''fedora'' repository. In stable releases, ''stable'' packages are pushed to the ''updates'' repository. However, from the point of view of the packager, this is an insignificant implementation detail. For more details, see [[Repositories]].
  
In particular, these are the '''new paths on mirrors''':
+
When a release is in stable state, the ''updates-testing'' repository is disabled by default, but [[QA]] team members and others run with it enabled in order to provide testing and Bodhi feedback. The main user population will see your update only when it passes Bodhi, is marked as ''stable'' and reaches the ''updates'' repository.
* <code> /pub/fedora/linux/development/rawhide/ </code> will become the new path of Rawhide. It will continue to not have install images, and will be the place where builds from the <code> "devel" </code> branch in CVS go to. It'll be {{FedoraVersion|long|next}} intended content.
 
* <code> /pub/fedora/linux/development/13/ </code> will become the new path of the branched Fedora 13 content. This is where builds from the F-13/ branch in CVS will go after they pass through Bodhi as "stable".
 
* <code> /pub/fedora/linux/updates/testing/13/ </code> will be where potential Fedora 13 builds go after passing through bodhi as "testing". This is where you'll find the latest stuff proposed for ''freeze break'' and where testing and peer review of these freeze breaks will happen. When a maintainer feels enough testing has happened, or enough karma triggers the Bodhi auto request, the build will be marked "stable" and show up in the development/13 tree at the next nightly compose.  
 
(View [[No_Frozen_Rawhide_coming_soon | NFR coming soon! New paths on mirrors!]])
 
  
== For Developers ==
+
{{anchor|multi}}
 +
=== Updating inter-dependent packages ===
  
{{admon/note|
+
If an update you wish to submit would cause a dependency issue of any kind (a strict package dependency error, or simply another package failing to operate correctly) if updated alone, you must not submit the package as a single-package update. You must always collect all inter-dependent or related packages together into a single multi-package update, such that no user will face problems if they install all the packages in the update together.
If you want to build a new package, but you aren't sure if it should go to Rawhide or {{FedoraVersion|long|next}}, then:
 
# New packages should always be built at least for Rawhide
 
# New packages can be built for Pending and existing Fedora Releases, however they should go through updates-testing first. If the new package is critical-path it will require net positive karma from [[ReleaseEngineering | Releng]] / [[QA]] and peers as outlined below.}}
 
<!------------------------------------------------------ FIRST TABLE -------------------------------------------------------------->
 
{|
 
|- style="background-color:#DEF3FE"
 
||  ||valign=top| Check out and build from the devel/branch ||valign=top| No action required. Happens [[Releases/Rawhide#Nightly_live_builds | nightly automatically]] ||valign=top| Check out and build from the [https://admin.fedoraproject.org/updates/{{FedoraVersion|short|next}} {{FedoraVersion|short|next}}/branch]  || Request a testing update in [https://admin.fedoraproject.org/updates/ Bodhi] for {{FedoraVersion|short|next}}. Bodhi ''admins push'' it.  ||valign=top| Peers test the update and provide karma feedback via [https://admin.fedoraproject.org/updates/ Bodhi]  ||valign=top| Requests a push to ''stable'' within [https://admin.fedoraproject.org/updates/ Bodhi]. Bodhi admins ''push'' it.
 
|-  
 
|style="background-color:#DEF3FE"| To build a package for rawhide ||style="background-color:#DEF3FE"|  ||  ||  ||  ||  ||
 
|-
 
|style="background-color:#DEF3FE"| To build package for Rawhide available for testing || ||style="background-color:#DEF3FE"|  ||  ||  ||  ||
 
|-
 
|style="background-color:#DEF3FE"| To build a non-critical path package for {{FedoraVersion|short|next}}  (branched content that is {{FedoraVersion|short|next}} in development)  || || ||style="background-color:#DEF3FE"|  ||  ||  || 
 
|-
 
|style="background-color:#DEF3FE"| For a package, not in the critical path, in the "pending" updates-testing repo for a week that hasn't received karma feedback ||  || || ||style="background-color:#DEF3FE" align=center| Note<sup>(1)</sup>  ||  ||style="background-color:#DEF3FE" align=center| Note<sup>(1)</sup>
 
|-
 
|style="background-color:#DEF3FE"| To build a critical path package for {{FedoraVersion|short|next}} (Pending) ||  ||  ||style="background-color:#DEF3FE"|  ||  ||  ||
 
|-
 
|style="background-color:#DEF3FE"| For a package, in critical path, in the "pending" updates-testing repo for a week that hasn't received karma feedback ||  ||  || ||  ||style="background-color:#DEF3FE" align=center| Note<sup>(2)</sup>  ||
 
|-
 
|style="background-color:#DEF3FE"| To mark, built package for {{FedoraVersion|short|next}}, available for testing || || ||  ||style="background-color:#DEF3FE"|  ||style="background-color:#DEF3FE"|  ||
 
|-
 
|style="background-color:#DEF3FE"| To mark ''stable and tagged'' for {{FedoraVersion|short|next}} tested update || || ||  ||  ||  ||style="background-color:#DEF3FE" align=center| Note<sup>(3)</sup>
 
|-
 
|style="background-color:#DEF3FE"| Built a package to handle an urgent issue (e.g. security problem, non-functioning system, etc.) and want to push <sup>'''(4)'''</sup> it to the {{FedoraVersion|short|next}} branch || || || ||  ||  ||style="background-color:#DEF3FE" align=center|  Notes<sup>(5),(6),(7)</sup> 
 
|}<BR>
 
  
Notes
+
For example: if you maintain a package ''libfoo'' which the package ''bar'' depends on, and you need to update ''libfoo'', you should check that ''bar'' continues to function correctly with the updated version of ''libfoo''. If it does not, you must ensure the appropriate changes are made to ''bar'', and include the updated ''bar'' in your update along with the updated ''libfoo''.
----
 
<sup>(1)</sup> Mutually exclusive<BR>
 
<sup>(2)</sup> Query [[QA|QA]]/[[ReleaseEngineering|Release Engineering]], too.<BR>
 
<sup>(3)</sup> Provided the package has net positive Karma from QA or releng and at least one more net positive karma point. If karma autopush is checked submit to stable is done automatically<BR>
 
<sup>(4)</sup> In all cases, it's ''necessary'' to be very very very sure the update will not cause additional problems'''<BR>
 
<sup>(5)</sup> if the package is not in the critical path, nor addressing a security problem, then it can be requested a push to stable.<BR>
 
<sup>(6)</sup> if the package addresses a security issue then it should be marked as security. Wait for the Security team to sign off on the update (not sure how this happens right now) before requesting the package be pushed to stable.<BR>
 
<sup>(7)</sup> if the package is in the critical path, then wait for a QA/Release Engineering/Peer net positive karma vote in Bodhi before requesting the package be pushed to stable. <BR>
 
  
----
+
The ''fedpkg'' tool does not handle multi-package updates. You can add multiple packages to an update using the [https://admin.fedoraproject.org/updates/ Bodhi web application], or the {{command|bodhi}} command line tool. You can pass as many package names as you like to the {{command|bodhi --new}} to create a new multi-package update, or use {{command|bodhi --edit}} to edit an existing update.
<BR>
 
<!------------------------------------------------------ SECOND TABLE -------------------------------------------------------------->
 
{|
 
|- style="background-color:#DEF3FE"
 
|| || File a buildroot override tag request as outlined in the [[Alpha_Freeze_Policy | Policy page]], and proceed to build the package ||valign=top| Issue the [https://admin.fedoraproject.org/updates/ Bodhi] request  || || || ||
 
|-
 
|style="background-color:#DEF3FE"| To build a package for the ''Pending'' {{FedoraVersion|short|next}} which requires a package not yet pushed ''stable'' for {{FedoraVersion|short|next}}||style="background-color:#DEF3FE"| ||style="background-color:#DEF3FE"| ||width="100px"| ||width="100px"| ||width="100px"| ||width="100px"|
 
|}
 
  
 +
It is possible you will run into problems with permissions when trying to add builds of packages you do not have commit privileges for to an update, or trying to add a build for a package you do have privileges for to someone else's update. If you encounter a situation like this, you should contact the [[ReleaseEngineering|release engineering]] team or a proven packager for help.
  
 +
You may need a ''buildroot override'' to complete a multi-package update successfully. For instance in the case described above, you may need to rebuild ''bar'' against the new ''libfoo'' package and submit both packages together as a multi-package update. However, in the normal course of events, you would not be able to build another package against your new ''libfoo'' build until it reached the [[Repositories#stable|''stable'']] state. To resolve this dilemma, you can request a buildroot override, which causes the ''libfoo'' build to be included in the buildroot for a short time in order to get the ''bar'' package build done.
  
{{admon/tip|
+
You can request a buildroot override with bodhi: {{command|bodhi --buildroot-override<nowiki>=</nowiki>(name-version-release) --duration<nowiki>=</nowiki>2 --notes<nowiki>=</nowiki>"Useful details."}} This would submit a buildroot override with a duration of two days. Buildroot overrides are usually granted within 15-30 minutes of submission. If you submit an override request with the bodhi tool, it will suggest a command that will let you monitor when the package appears in the buildroot, so you can fire your dependent build at the appropriate time.
To check and see if the build will be in the {{FedoraVersion|long|next}}, as they will be tagged with ''"dist-f1?"'', you can run ''koji latest-pkg dist-f1?'' to see what the latest build of your package is for {{FedoraVersion|long|next}}.}}
 
  
== For Testers ==
+
You can also request buildroot overrides from the [https://admin.fedoraproject.org/updates/ Bodhi web application].
{|
 
||  || Read the [[Releases/Rawhide | Rawhide]] page and  follow the instructions to get rawhide installed.|| Leave the ''rawhide'' yum repo enabled and keep ''fedora'', ''updates'', and ''updates-testing'' repos disabled. || Consume the rawhide firehose and report issues as you find them.|| Install from Alpha, Beta, the Last Known Good ''Pending'' snapshot or from a ''Pending'' nightly live image || <code>yum update</code> to the latest ''pending'' content || Update your Fedora 12 system by reading instructions at [[FIXME]] || Check the [[QA/Join]] page that describes the different testable repos, skill level and investment involved. || Read the [[QA:Package Acceptance Test Plan]] and use it as a guideline for testing
 
  
|-
+
The [[Bodhi/BuildRootOverrides|buildroot override instructions]] explain the buildroot override process in more detail.
|| If you want to be a tester and want to install rawhide on your system to test the latest and greatest packages at all times || X || X || X || || ||  ||  ||
 
|-
 
|| If you want to install and run the ''pending'' Fedora release (aka {{FedoraVersion|long|next}}) as your desktop and to participate in test days ||  ||  || || X || X || ||  ||
 
|-
 
|| If you want to update your Fedora 12 system to the ''Pending'' Fedora 13 and start testing it || || || || || ||X ||  ||
 
|-
 
|| If you want to pitch in and provide test feedback for new packages ('Rawhide', 'pending', 'stable' updates-testing, )  || || || || || ||  || X ||
 
|-
 
|| If you want to become a member of the [[QA]] team to represent it in providing positive feedback on critical path package updates on the 'stable' and 'pending' fedora releases || || || || || ||  || X  || X
 
|-
 
|}
 
[[Category:Package Maintainers]][[Category:How to]]
 
  
{{admon/note| |
+
=== Handling feedback from automated tests ===
* If you are a member of the QA FAS group, and provided positive feedback on a 'pending' or 'stable' package update.  The package update is released and includes a major regression.  What now?
 
# Review in weekly QA meeting.  It's ok, mistakes happen.
 
# Accident/omission.
 
# Misuse
 
}}
 
  
== For Mirror Admins ==
+
Fedora's automated testing systems, [[Taskotron]] and [[OpenQA]], may run automated tests on your update. Several of the [[Taskotron/Tasks]] are documented. The openQA tests are functional tests of some critical [[Workstation]] and [[Server]] features.
* If you are a mirror admin and want to prepare for additional repos coming to your mirror :
 
** Read <code>mirror-list(-d)</code> and watch for announcements regarding new paths being added to the Master Mirror.
 
** Check your ''sync exclusion settings'' to ensure you either get, or don't get the new path (depending on your needs).
 
  
== Build a package for Rawhide ==
+
In the Bodhi web interface, updates have a ''Automated Tests'' tab which displays the results of all automated tests. Tests shown with a red background failed. The tests are not all 100% accurate, but they are fairly often correct. If you see a failure, it is a very good idea to click on the result (which will take you to a detailed log) and investigate the issue. If you are unsure what the test indicates, you can contact the [[QA]] team for help.
To build a package for Rawhide, check it out from the CVS devel/branch.
 
  
Install fedora-packager if not already installed.
+
==== Waive a result ====
  
Check out a local working copy of the CVS module you plan to edit, e.g. (for a description of the directory layout, see the [[PackageMaintainers/Anatomy| Anatomy]]  page:
+
At present, a failure of the ''dist.rpmdeplint'', ''dist.abicheck'', or ''org.centos.prod.ci.pipeline.complete'' tests will prevent your update from being released. On the update's ''Details'' page in the Bodhi web interface, the '''Test Gating Status''' will be shown as ''N of N required tests failed''. If you are sure such a failure is a false one, you can 'waive' the result using the {{code|waiverdb-cli}} tool, from the {{package|waiverdb-cli}} package (there is [https://github.com/fedora-infra/bodhi/pull/2095 work pending] to be able to waiver more easily, directly from the Bodhi UI).
<pre>fedora-cvs <package_name>
 
</pre>
 
  
* If you update to a new upstream version, you have to upload the tarball to an external lookaside cache. Operations on the lookaside cache require a client-side certificate, to get one, run
+
You can submit a waiver for a failing result with {{pkg|waiverdb-cli}} specifying the {{code|subject}} and the {{code|testcase}}:
  
<pre>fedora-cert -n</pre>
+
  waiverdb-cli -t YOUR_TESTCASE_HERE -s '{"item": "this-is-the-subject", "type": "also-this-is-part-of-the-subject"}' -p "fedora-26" -c "This is fine"
  
To upload a new source tarball and replace an older one, run
+
Example:
  
<pre>make new-sources FILES="/path/to/yournewtarball.tar.gz"
+
  waiverdb-cli -t dist.rpmdeplint -s '{"item": "python-requests-1.2.3-1.fc26", "type": "koji_build"}' -p "fedora-26" -c "This is fine"
</pre>
 
  
You can omit the path if the source is the current directory.
+
You can also waive a failing result by result's id, which you can retrieve from resultsdb with curl.  To do that, you'll need the {{code|testcase}} name and the {{code|nvr}}. For example:
  
In the branch directory (i.e. devel in this case) or for multiple files:
+
  curl "https://taskotron.fedoraproject.org/resultsdb_api/api/v2.0/results?testcases=dist.python-versions&item=python-alembic-0.9.7-1.fc27" | jq ".data[0].id"
<pre>make new-sources FILES="yournewtarball.tar.gz yourdata.tar.gz"
 
</pre>
 
  
This also updates your local copy of the '''.cvsignore''' and '''sources''' files. You will need to do this for each branch  that you will be building the new version for.  The new tarball will be uploaded only once and the rest will be md5sum checked and not uploaded, only the '''.cvsignore''' and '''sources''' files will be updated.
+
This should print out the {{code|result_id}} of the failing result. You can then submit a waiver for this failing result with {{pkg|waiverdb-cli}}
  
If you just want to add another tarball (e.g. a big gzipped patch or a documentation tarball), use:
+
  waiverdb-cli -p fedora-27 -r YOUR_ID_HERE -c "This is fine."
<pre>make upload FILES="somefile.tar.gz"
 
</pre>
 
Contrary to <code>make new-sources</code>, this does not purge old files from the <code>.cvsignore</code> and <code>sources</code> files.
 
  
If you just have a small patch, initscript, or otherwise plain text file, you can commit those directly to CVS.  This can be done with the <code>cvs add</code> command, e.g.:
+
Also, if you enabled automatic stable push at a karma threshold, this will be disabled if any automated test fails. If you have examined the result and you are sure it is a false one and there is no problem with the package, you may re-enable the automatic push mechanism or submit the package to ''stable'' manually once it meets the other requirements of the [[Updates Policy]].
<pre>cvs add packagename-fix-the-foobar.patch
 
</pre>
 
  
* Use the command <code>make i686</code> or <code>make x86_64</code> to test a package build on your local system.  Then install and test the package.  If something doesn't work, fix it and repeat this step. You can also use the koji build system to do a scratch build perhaps for some arch you don't have locally. For example, to build just for x86_64:
+
==== Waive the absence of a result ====
<pre>make srpm; koji build --scratch --arch-override x86_64 dist-f14 packagename-version-release.src.rpm
 
</pre>
 
  
* Check if everything that has changed is correct with
+
Submitting a waiver using subject/testcase allows to waive the absence of a result (eg. the test never ran for some reason, so there is no result item).
<pre>cvs diff -u
 
</pre>
 
  
* Commit the verified changes to the <code>devel/</code> branch.
+
If which testcase you should be specified it is not clear/known, it is possible to run this python script, chainging it with the corrects parameter:
<pre>cvs commit
 
</pre>
 
  
As a test whether the full commit was fine, you can check out a fresh working copy into a different directory. It should succeed in fetching the binaries from lookaside cache and also pass simple build tests such as '''make i686''' or '''make srpm''' at least.
+
#!/usr/bin/env python
* Tag your package:
+
""" Ask a question of greenwave. """
<pre>make tag
+
# Usage: either modify and set PRODUCT_VERSION and NVR_LIST and run, or pass version as first arg and then NVRs as further args
</pre>
+
import pprint
 +
import requests
 +
import sys
 +
PRODUCT_VERSION = 'fedora-27' if len(sys.argv) == 1 else sys.argv[1]
 +
NVR_LIST = [] or sys.argv[2:]  # Insert your NVRs here, or pass them via command line args
 +
for nvr in NVR_LIST:
 +
    url = (
 +
        'https://greenwave-web-greenwave.app.os.fedoraproject.org/'
 +
        'api/v1.0/decision')
 +
    payload = dict(
 +
        #verbose=True,
 +
        decision_context='bodhi_update_push_stable',
 +
        product_version=PRODUCT_VERSION,
 +
        subject=[{'item': nvr, 'type': 'koji_build'}],
 +
    )
 +
    response = requests.post(url, json=payload)
 +
    print("-" * 40)
 +
    print(nvr, response, response.status_code)
 +
    data = response.json()
 +
    print(pprint.pformat(data))
  
* Instruct the builders to build your package:
+
The output will show that Greenwave is requiring a specific testcase to run, but it cannot find a result for it (neither pass nor failure). So now it is possible to submit a waiver with the specified testcase in the output and the subject already known.
<pre>make build
 
</pre>
 
  
* Check the koji build page at http://koji.fedoraproject.org/koji/ for the build process.
+
==== Troubleshooting ====
  
=== Example ===
+
If you run the {{code|waiverdb-cli}} tool and it gives you the following error:
  
<pre>
+
  Error: The config option "resultsdb_api_url" is required
fedora-cvs foo
 
cd foo/devel
 
wget -N http://dl.sf.net/foo/foo-0.0.2.tar.bz2
 
make new-sources FILES="foo-0.0.2.tar.bz2"
 
gedit foo.spec
 
# change the required things in the specfile
 
make i686
 
# check that the changes you made are correct
 
cvs diff -u
 
# commit
 
cvs commit -m "Update to 0.0.2"
 
# request build
 
make tag build
 
</pre>
 
  
The package builders publish your package in the <code>development</code> tree, also called "Rawhide."  If the package is a stable update, you may also provide it to users of the currently-maintained stable, or branched Fedora release. To make it available to F-11 or F-12 users, or testers of the branched F-13 for example, use the procedure outlined in the [[#Packages_in_the_stable_branches |next chapter]].
+
Edit {{code|/etc/waiverdb/client.conf}} and add the following line:
  
An alternative may be used, [[PackageMaintainers/UsingCvsFaq#Import_of_complete_src.rpm_packages| the import of a complete src.rpm]].
+
  resultsdb_api_url=https://taskotron.fedoraproject.org/resultsdb_api/api/v2.0
  
More in-depth information on the build system is at
+
=== Branched milestone freezes ===
[[PackageMaintainers/UsingKoji| UsingKoji]].
 
  
=== Removing a package build from the devel branch ===
+
For a short period before each milestone release, the stable [[Repositories#fedora|''fedora'']] repository is frozen. These periods are shown as the [[Milestone freezes]] (Beta Freeze, Final Freeze) on schedules. During these periods, builds will not be marked ''stable'' and pushed from [[Repositories#updates-testing|''updates-testing'']] to ''fedora'' even after being submitted manually or automatically. In the normal course of events, they will be pushed after the milestone release is approved at a [[Go_No_Go_Meeting]]. If you believe your update deserves to break a milestone freeze, a ''freeze exception'' may be granted through the [[QA:SOP_freeze_exception_bug_process|freeze exception process]]. Accepted release blocking bugs are granted the same status through the [[QA:SOP_blocker_bug_process|blocker bug process]].
  
From time to time you may want to remove a package build you pushed to the devel branch (rawhide).  This could happen in a situation where a bug
+
For more on the Fedora development process, see [[Fedora Release Life Cycle]].
or issue is found in your package that will be resolved upstream in the next release.  You may want to wait for this release instead
 
of back-porting a fix yourself, so pulling the broken package from rawhide makes sense.
 
{{admon/caution|Use this carefully!|This should only be done on the same day of the build. If your build has already been published in rawhide you must not untag it!}}
 
  
You can remove the package from rawhide by using koji as follows:
+
{{admon/tip|
 +
If you are unsure whether your build is currently considered ''stable'' for a given release, you can check with {{command|koji latest-pkg fXX}} (where XX is the release).}}
  
<pre>
+
== Security updates ==
koji untag-pkg dist-f13 foo-1.1.3-1.fc13
 
</pre>
 
  
Where <code>foo-1.1.3-1.fc13</code> is replaced with the name of your package build. See <code>koji help</code> or [[PackageMaintainers/UsingKoji | UsingKoji]] for more information.
+
There is an additional process that layers over the regular update process for bugs identified as security issues. If a bug is assigned to you that blocks a [[Security Tracking Bugs|security tracking bug]], you must follow that process in addition to this one.
  
= Working with packages in the stable branches =
+
== New package submissions ==
Stable branches are branches within CVS for either released Fedoras, or a branched Fedora that is still in bugfix/polish mode but has not yet been released.
 
  
# Make any required changes in the <code>F-11</code>, <code>F-12</code> or <code>F-13</code>, etc.. directory.  In many cases, you can apply the same changes from the <code>devel/</code> branch to the other branches.  Use the <code>diff</code> and <code>patch</code> utilities for this purpose.
+
If you want to build a new package, but you aren't sure which releases to send it to:
# Use the command <code>make i686</code> or <code>make x86_64</code> to test a package build on your local system.  Then install and test the package.  If something doesn't work, fix it and repeat this step.
 
# Commit the verified changes to the branch you are working on: <pre>cvs commit</pre>
 
# Tag your package: <pre>make tag</pre>
 
# Instruct the builders to build your package:<pre>make build</pre>
 
# Check the koji build page at http://koji.fedoraproject.org/koji/ for the build process.
 
  
* If you want to build a package for the Pending {{FedoraVersion|long|next}} but it requires package that is not yet pushed "stable" for Fedora 13.
+
* New packages should always be built for Rawhide
# You would need to file a buildroot override tag request as outlined in the policy page Alpha_Freeze_Policy
+
* New packages can be built for Branched and stable releases if adding them would provide value to users of those releases without significant risk of causing harm
# Once tagged, you can proceed to build her package and issue the [[Package_update_HOWTO#Submit_your_update_to_Bodhi | Bodhi]] request
 
  
== Submit your update to Bodhi ==
+
The submission process for new packages, after they have passed the [[Package_Review_Process]] and been [[Package_SCM_admin_requests|given an SCM repository]], is exactly the same as that for package updates.
  
# This can be accomplished in a few different ways.  The easiest being: <pre>make update</pre>  If your local username differs from that of your Fedora account, you will need to specify it with the following command: <pre>make update BODHI_USER=foo</pre> Or you add <code>BODHI_USER=foo</code> to the file  <code>~/.cvspkgsrc</code>.
+
== Consider creating a package test plan ==
## Alternatively, you can also submit your update using the [https://fedorahosted.org/bodhi/wiki/CLI bodhi-client]
 
## or the [https://admin.fedoraproject.org/updates web interface] .
 
# Once submitted, bodhi will automatically request that your update be pushed to updates-testing.  If you feel that community testing is unnecessary for your update, you can choose to push it straight to the stable fedora-updates repository instead. '''Pushing directly to stable skips peer review and is strongly discouraged!!''' Note that security updates follow a [[Security/TrackingBugs |slightly different process]] .
 
# A Release Engineer then signs and pushes out your updates.  The signing step is currently a manual process, so your updates will not be instantly released once submitted to bodhi.
 
# Once pushed to testing, people are able to +1/-1 the updates "karma", based on whether or not it seems to be functional for them.  If your update reaches a karma of 3, it will automatically be pushed to stable.  Likewise, if it reaches -3, it will be automatically unpushed.  If your update does not receive enough feedback to automatically push it to stable, you will have to submit it as a final update yourself.  This can easily be done with the command-line tool, or with the web interface.
 
#  You will then be notified when your update has been pushed to stable.  Bodhi will close all associated bugs and send an announcement to fedora-package-announce.  At this point, your update has been officially released!
 
  
== Get Automatically Notified on New Upstream Releases ==
+
If you [[QA:SOP_test_case_creation|create test cases]] for your package, and [[QA:SOP_package_test_plan_creation|categorize them appropriately]], they will be automatically linked in Bodhi, so that testers will have some guidance for planned update testing.
To automatically get notifications via bugzilla whenever upstream has a new release, refer to [[Upstream_Release_Monitoring | upstream release monitoring]] page.
 
  
== Reference ==
+
[[Category:Package Maintainers]]
http://fedoraproject.org/wiki/PackageMaintainers/UsingCvsFaq
 

Latest revision as of 14:23, 30 March 2018

This document shows how to submit an update for a package you maintain in Fedora. It assumes you already have a package in the Fedora repositories. It is not a guide to using the Fedora package source control system: see the Package maintenance guide for that.

Overview

This page is intended for new and existing package maintainers. Testers and regular users may be interested in the updates-testing repository and the update feedback guidelines. This page specifically covers the update submission process.

There are two significantly different package update submission workflows in Fedora:

The repository layouts differ somewhat for Rawhide, Branched and stable releases, but the update workflows split up as described above.

Rawhide and early Branched

The package update workflow for Rawhide and Branched before the Bodhi enabling point is simple:

  1. Build the package with fedpkg build (see the Package maintenance guide for more details)

This is all you need to do. Your package will appear in the next daily compose of Rawhide or Branched and will be used in any image composes built from that tree.

Later Branched and stable releases

At the Bodhi enabling point, the Bodhi update feedback system is enabled by Release Engineering and builds submitted with fedpkg build are no longer automatically sent to any official repository. The update workflow for releases of this type is:

Idea.png
Fedora account name
fedpkg should be able to discover your Fedora account system user name from the ~/.fedora.cert file set up by fedora-packager-setup when you first configured your system for packaging. If this fails for any reason, you can specify it with --user (username). For the bodhi command line tool, you may need to specify your Fedora user name with -u (username) if it differs from your system user name.
  1. Build the package with fedpkg build
  2. Submit an update for the package with fedpkg update, the Bodhi web interface, or the Bodhi CLI tool. This causes the package to be sent to the updates-testing repository
  3. Monitor the update's status and the feedback you receive via the web interface or the emails that are sent to you, and modify it with updated or additional builds if necessary
  4. After the update meets the criteria in the Updates Policy and you are satisfied it should be released as a stable update, submit the update to stable with bodhi -R stable or the web interface
Important.png
Updating inter-dependent packages
If a package you wish to update requires other package(s) to be rebuilt before it or they will work properly, you must submit the builds together as a multi-package update. See below for more details on this.

Update attributes

At the time you submit the update, you will be asked for several attributes. The type of the update should be fairly self-explanatory: either it fixes bugs, adds new features, or is a new package.

If you are asked whether you want to send the update to updates-testing or stable, this is a no-op: all updates now go through updates-testing. It does not matter what you choose.

There are several schools of thought on filling out the update description. Some would suggest you consider the target audience: for a stable release, in particular, many Fedora users will see this text, and many of them may not be particularly familiar with your package. Consider not simply describing literally the changes in the update, but explaining as if to an outsider why your are updating the package, what benefits it will bring to them (if any), and anything they may want to note in order to have a smooth update experience.

If you associate one or more bug reports with your update, Bodhi will post comments into Bugzilla to alert those following the bug reports that an update is available. If you mark your update as fixing the bug(s), Bodhi will move the report(s) through the MODIFIED, ON_QA and CLOSED ERRATA states of the bug workflow as your update reaches various points in the process. Using this mechanism can be very useful both for you and for users of your package.

You may set a karma (feedback) level at which the update will automatically be submitted to stable. This is optional. If you choose to use it, please carefully consider an appropriate feedback level. For a relatively obscure package which is quite stable, 1 or 2 may be an appropriate value. For a popular, sensitive and complex package such as Package-x-generic-16.pngfirefox or the Package-x-generic-16.pngkernel, the default of 3 may be insufficient and a choice of 5 or even 10 may be appropriate.

Who will receive your update, when?

When a release is in Branched state, the updates-testing repository is enabled by default so most users will see the package, but only packages from the stable fedora repository are used in building milestone releases (Alpha, Beta and Final) and nightly images.

Where a package goes when it is marked as stable differs between Branched and stable releases. In Branched releases, stable packages are pushed to the base fedora repository. In stable releases, stable packages are pushed to the updates repository. However, from the point of view of the packager, this is an insignificant implementation detail. For more details, see Repositories.

When a release is in stable state, the updates-testing repository is disabled by default, but QA team members and others run with it enabled in order to provide testing and Bodhi feedback. The main user population will see your update only when it passes Bodhi, is marked as stable and reaches the updates repository.

Updating inter-dependent packages

If an update you wish to submit would cause a dependency issue of any kind (a strict package dependency error, or simply another package failing to operate correctly) if updated alone, you must not submit the package as a single-package update. You must always collect all inter-dependent or related packages together into a single multi-package update, such that no user will face problems if they install all the packages in the update together.

For example: if you maintain a package libfoo which the package bar depends on, and you need to update libfoo, you should check that bar continues to function correctly with the updated version of libfoo. If it does not, you must ensure the appropriate changes are made to bar, and include the updated bar in your update along with the updated libfoo.

The fedpkg tool does not handle multi-package updates. You can add multiple packages to an update using the Bodhi web application, or the bodhi command line tool. You can pass as many package names as you like to the bodhi --new to create a new multi-package update, or use bodhi --edit to edit an existing update.

It is possible you will run into problems with permissions when trying to add builds of packages you do not have commit privileges for to an update, or trying to add a build for a package you do have privileges for to someone else's update. If you encounter a situation like this, you should contact the release engineering team or a proven packager for help.

You may need a buildroot override to complete a multi-package update successfully. For instance in the case described above, you may need to rebuild bar against the new libfoo package and submit both packages together as a multi-package update. However, in the normal course of events, you would not be able to build another package against your new libfoo build until it reached the stable state. To resolve this dilemma, you can request a buildroot override, which causes the libfoo build to be included in the buildroot for a short time in order to get the bar package build done.

You can request a buildroot override with bodhi: bodhi --buildroot-override=(name-version-release) --duration=2 --notes="Useful details." This would submit a buildroot override with a duration of two days. Buildroot overrides are usually granted within 15-30 minutes of submission. If you submit an override request with the bodhi tool, it will suggest a command that will let you monitor when the package appears in the buildroot, so you can fire your dependent build at the appropriate time.

You can also request buildroot overrides from the Bodhi web application.

The buildroot override instructions explain the buildroot override process in more detail.

Handling feedback from automated tests

Fedora's automated testing systems, Taskotron and OpenQA, may run automated tests on your update. Several of the Taskotron/Tasks are documented. The openQA tests are functional tests of some critical Workstation and Server features.

In the Bodhi web interface, updates have a Automated Tests tab which displays the results of all automated tests. Tests shown with a red background failed. The tests are not all 100% accurate, but they are fairly often correct. If you see a failure, it is a very good idea to click on the result (which will take you to a detailed log) and investigate the issue. If you are unsure what the test indicates, you can contact the QA team for help.

Waive a result

At present, a failure of the dist.rpmdeplint, dist.abicheck, or org.centos.prod.ci.pipeline.complete tests will prevent your update from being released. On the update's Details page in the Bodhi web interface, the Test Gating Status will be shown as N of N required tests failed. If you are sure such a failure is a false one, you can 'waive' the result using the waiverdb-cli tool, from the Package-x-generic-16.pngwaiverdb-cli package (there is work pending to be able to waiver more easily, directly from the Bodhi UI).

You can submit a waiver for a failing result with waiverdb-cli specifying the subject and the testcase:

 waiverdb-cli -t YOUR_TESTCASE_HERE -s '{"item": "this-is-the-subject", "type": "also-this-is-part-of-the-subject"}' -p "fedora-26" -c "This is fine"

Example:

 waiverdb-cli -t dist.rpmdeplint -s '{"item": "python-requests-1.2.3-1.fc26", "type": "koji_build"}' -p "fedora-26" -c "This is fine"

You can also waive a failing result by result's id, which you can retrieve from resultsdb with curl. To do that, you'll need the testcase name and the nvr. For example:

 curl "https://taskotron.fedoraproject.org/resultsdb_api/api/v2.0/results?testcases=dist.python-versions&item=python-alembic-0.9.7-1.fc27" | jq ".data[0].id"

This should print out the result_id of the failing result. You can then submit a waiver for this failing result with waiverdb-cli

 waiverdb-cli -p fedora-27 -r YOUR_ID_HERE -c "This is fine."

Also, if you enabled automatic stable push at a karma threshold, this will be disabled if any automated test fails. If you have examined the result and you are sure it is a false one and there is no problem with the package, you may re-enable the automatic push mechanism or submit the package to stable manually once it meets the other requirements of the Updates Policy.

Waive the absence of a result

Submitting a waiver using subject/testcase allows to waive the absence of a result (eg. the test never ran for some reason, so there is no result item).

If which testcase you should be specified it is not clear/known, it is possible to run this python script, chainging it with the corrects parameter:

#!/usr/bin/env python
""" Ask a question of greenwave.  """
# Usage: either modify and set PRODUCT_VERSION and NVR_LIST and run, or pass version as first arg and then NVRs as further args
import pprint
import requests
import sys
PRODUCT_VERSION = 'fedora-27' if len(sys.argv) == 1 else sys.argv[1]
NVR_LIST = [] or sys.argv[2:]  # Insert your NVRs here, or pass them via command line args
for nvr in NVR_LIST:
    url = (
        'https://greenwave-web-greenwave.app.os.fedoraproject.org/'
        'api/v1.0/decision')
    payload = dict(
        #verbose=True,
        decision_context='bodhi_update_push_stable',
        product_version=PRODUCT_VERSION,
        subject=[{'item': nvr, 'type': 'koji_build'}],
    )
    response = requests.post(url, json=payload)
    print("-" * 40)
    print(nvr, response, response.status_code)
    data = response.json()
    print(pprint.pformat(data))

The output will show that Greenwave is requiring a specific testcase to run, but it cannot find a result for it (neither pass nor failure). So now it is possible to submit a waiver with the specified testcase in the output and the subject already known.

Troubleshooting

If you run the waiverdb-cli tool and it gives you the following error:

 Error: The config option "resultsdb_api_url" is required

Edit /etc/waiverdb/client.conf and add the following line:

 resultsdb_api_url=https://taskotron.fedoraproject.org/resultsdb_api/api/v2.0

Branched milestone freezes

For a short period before each milestone release, the stable fedora repository is frozen. These periods are shown as the Milestone freezes (Beta Freeze, Final Freeze) on schedules. During these periods, builds will not be marked stable and pushed from updates-testing to fedora even after being submitted manually or automatically. In the normal course of events, they will be pushed after the milestone release is approved at a Go_No_Go_Meeting. If you believe your update deserves to break a milestone freeze, a freeze exception may be granted through the freeze exception process. Accepted release blocking bugs are granted the same status through the blocker bug process.

For more on the Fedora development process, see Fedora Release Life Cycle.

Idea.png
If you are unsure whether your build is currently considered stable for a given release, you can check with koji latest-pkg fXX (where XX is the release).

Security updates

There is an additional process that layers over the regular update process for bugs identified as security issues. If a bug is assigned to you that blocks a security tracking bug, you must follow that process in addition to this one.

New package submissions

If you want to build a new package, but you aren't sure which releases to send it to:

  • New packages should always be built for Rawhide
  • New packages can be built for Branched and stable releases if adding them would provide value to users of those releases without significant risk of causing harm

The submission process for new packages, after they have passed the Package_Review_Process and been given an SCM repository, is exactly the same as that for package updates.

Consider creating a package test plan

If you create test cases for your package, and categorize them appropriately, they will be automatically linked in Bodhi, so that testers will have some guidance for planned update testing.