From Fedora Project Wiki
(fork)
 
(delete)
 
Line 1: Line 1:
== Background ==
* Most Fedora events should be ''self-hosting'' and capable of being led by non-subject matter experts.  This SOP sets forth how to arrange and lead a Blocker Bug Meeting.
* These meetings are a part of the [[QA:SOP_blocker_bug_process]] and the [[QA:SOP_freeze_exception_bug_process]], and their purpose is to review proposed blocker and freeze exception bugs and decide whether to accept them, and to monitor the progress of fixing existing accepted blocker and freeze exception bugs.
* Proactive review and handling of blocker bug lists is the one of the best ways to make sure our releases ship as closely on time as possible.
* Blocker Bug Meetings are not owned by any one team in Fedora.  They are a collaborative effort between Release Engineering, Quality Assurance, Development, and Project Management.


== Meeting Setup ==
=== Announcement ===
Announce the Blocker Bug Meeting.  Include the following important information in the announcement:
<ul>
<li>Meeting time in the format: {{#time: H:i | 16:00 UTC}} UTC ({{#timel: H:i | 16:00 UTC - 4 hours}} EDT, {{#timel: H:i | 16:00 UTC + 1 hours}} CET).  To help participants convert UTC to their local time, the following text can be used.
<pre>
To convert UTC to your local time, take a look at
  http://fedoraproject.org/wiki/Infrastructure/UTCHowto
or run:
  date -d '2010-03-19 16:00 UTC'</pre>
</li><li>Place: [irc://irc.freenode.net/fedora-blocker-review #fedora-blocker-review] on irc.freenode.net
</li><li>Link to [http://qa.fedoraproject.org/blockerbugs/current list of bugs to be reviewed] from the blocker bug tracking webapp
</li><li>Link to the appropriate [[Fedora Release Criteria]] (e.g. [[Fedora_{{FedoraVersionNumber|next}}_Alpha_Release_Criteria|Alpha]], [[Fedora_{{FedoraVersionNumber|next}}_Beta_Release_Criteria|Beta]] or [[Fedora_{{FedoraVersionNumber|next}}_Final_Release_Criteria|Final]]) to guide decision making during the meeting
</li><li>Link to the [[QA:SOP_blocker_bug_process]] and [[QA:SOP_freeze_exception_bug_process]]
</li><li>Link to this page
</li></ul>
=== Reminders ===
* Two days before the meeting, send reminder to [https://admin.fedoraproject.org/mailman/listinfo/test-announce test-announce@lists.fedoraproject.org].  Both the [https://admin.fedoraproject.org/mailman/listinfo/devel devel] and [https://admin.fedoraproject.org/mailman/listinfo/test test] mailing lists are subscribed to [https://admin.fedoraproject.org/mailman/listinfo/test-announce test-announce], so no additional announcements are needed.
=== Requesting Status Before The Meeting ===
The meeting is most productive when we have the latest information to work with. Often this requires specifically requesting more information in the bug.  Review each of the open blocker bugs and review the comments to see if the bug is progressing. 
If the bug does not appear to be making progress, request more information.  Here are some examples text that might help solicit information:
<ul><li> Set the bug to ''needinfo-assignee'' ...
<pre>
Please help us save time at this week's {{FedoraVersion|long|next}} Alpha Blocker Bug review
meeting by adding your comments on the assessment of this bug.  If this bug is
still unresolved by Friday, July 23, 2010, we would appreciate your attendance
at the {{FedoraVersion|long|next}} Alpha blocker meeting on freenode in the #fedora-blocker-review
channel at 16:00 UTC.
The following information would be very helpful to have prior to Friday:
a) whether you believe it is truly a blocker bug, and if so, which of the
https://fedoraproject.org/wiki/Fedora_Release_Criteria it infringes
b) what additional information you need to troubleshoot or fix this bug
c) When you estimate having a fix ready
Thank you,
your name
</pre>
</li><li>Set the bug to ''needinfo-reporter''
<pre>
Dear Bug Reporter,
This blocker bug has been marked as fixed.  If possible could you verify that
this problem is in fact fixed in the latest build?  Your help in completing
this task and adding a comment to this bug prior to the next {{FedoraVersion|long|next}} Alpha
Blocker meeting on Friday, July 23, 2010, would be most appreciated.
Thank you,
your name
</pre>
</li></ul>
== In Action ==
=== Quorum ===
Minimum active attendance for a meeting is three people, ideally one representing each of the stakeholder groups. If active attendance at the meeting drops below three people at any point, it must be ended or suspended until more people join.
=== Three hour time limit ===
Blocker meeting should usually run for a maximum of three hours. Experience indicates that beyond this amount of time, participation and attention levels drop significantly. If a meeting has run for three hours and all bugs have not yet been covered, the meeting leader should [[#Tear_Down|wind up the meeting and organize a follow-up meeting to complete the work]], usually to take place the following day.
=== Introduction ===
The meeting leader should open [http://qa.fedoraproject.org/blockerbugs/current/irc the blocker bug webapp's IRC page] in a text editor. This link will give you a template for running the meeting: introduction text and text for each bug which you can copy and paste straight into the meeting channel. It will be up to date as of the time you access the page.
The meeting leader should join [irc://irc.freenode.net/fedora-meeting #fedora-blocker-review] on irc.freenode.net and get the meeting started.  Use the [[Zodbot#Meeting_Functions|meetbot]] commands below to help capture minutes and direct the flow of the meeting.
<ol>
<li>The meeting leader should start the meeting using the following lines:
{{#tag:pre|
#startmeeting F{{FedoraVersion|number|next}}-blocker-review
#meetingname F{{FedoraVersion|number|next}}-blocker-review
#topic Roll Call
}}</li>
<li>After it's clear who's available, give meeting privileges to some other attendees, so that they can take it over in case you lose your connection:
<pre>#chair person1 person2</pre></li>
<li>Then continue with a boilerplate:
{{#tag:pre|
#topic Introduction
Why are we here?
#info Our purpose in this meeting is to review proposed blocker and nice-to-have bugs and decide whether to accept them, and to monitor the progress of fixing existing accepted blocker and nice-to-have bugs.
#info We'll be following the process outlined at:
#link https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting
#info The bugs up for review today are available at:
#link http://qa.fedoraproject.org/blockerbugs/current
#info The criteria for release blocking bugs can be found at:
#link https://fedoraproject.org/wiki/Fedora_{{FedoraVersion|number|next}}_Alpha_Release_Criteria
#link https://fedoraproject.org/wiki/Fedora_{{FedoraVersion|number|next}}_Beta_Release_Criteria
#link https://fedoraproject.org/wiki/Fedora_{{FedoraVersion|number|next}}_Final_Release_Criteria
}}</li>
<li>The meeting leader should then paste the counts of proposed and accepted blocker and freeze exception bugs from the top of the IRC template.</li>
<li>The meeting leader should find someone who is willing to act as a ''bug secretary'', making changes to bugs in Bugzilla as necessary. This role can be shared around during the meeting if desired.</li>
</ol>
=== Review Proposed Blocker Bugs ===
Next, proceed through the list of bugs.  First, review the proposed [[QA:SOP_blocker_bug_process|blocker bugs]]. For each bug, this is the process:
# The meeting leader pastes the #topic, #link and #info lines from the IRC template
# The group then decides whether the bug is accepted as a blocker; if not, the group can discuss whether to accept it as a blocker for a later milestone, or accept it as a freeze exception issue instead, or reject it entirely. This decision should be based on the [[Fedora_Release_Criteria|Release Criteria]] for the release in question: no bug should be accepted as a blocker unless it violates the release criteria (or has been designated as a blocker by [[Fedora_Engineering_Steering_Committee|FESCo]])
#* All meeting attendees can vote +1 or -1 to blocker status during discussion, and change their vote at any time
#* The meeting leader should use every effort to try and reach strong consensus on the bug's status
#* If all such efforts fail, the meeting leader should tally the votes and propose an action based on the majority opinion. The meeting leader should consider only votes from members of the stakeholder groups (development, release engineering, QA and project management) in such a tally
#* Use plenty of <code>#info ...</code> tags to record additional notes on the bug
#* After discussion, the leader will seek feedback on a proposed action <code>proposed #agreed 123456 - accepted|rejected as a release blocker</code>
#* If the group votes to accept the bug as a blocker, the leader should determine whether it is a normal blocker, 0-day blocker, or previous release blocker, and include the appropriate tag (<code>AcceptedBlocker</code>, <code>Accepted0Day</code>, or <code>AcceptedPreviousRelease</code>) in the proposal
#* Attendees can 'ack', 'nack' or 'patch' the proposed action, as a mechanism for catching typos or suggesting refinements. The meeting leader can adjust the proposed action at their discretion
# The group should also consider the status of work on fixing the bug. It should be clear who currently has responsibility for each blocker bug and what the next required action is on each bug
#* <code>#action tester - Will attempt to reproduce the issue reported  in 123456</code>
#* <code>#action developer - Expects to have a patch available for testing later today</code>
# The meeting leader should summarize the discussion with <code>#agreed</code> and <code>#info</code> before moving on to the next bug.  If specific action beyond the immediate update of the bug report is needed, it should be noted with a <code>#action</code> item before moving on.
#* <code>#agreed 123456 - accepted|rejected as a release blocker</code>
#* In some cases it may be impossible to make a definite decision until more information is available. In this case, use <code>#agreed 123456 - unable to determine status with currently available information</code>, and ensure there is an action item for someone to make sure the necessary information is available for the next meeting
# Lastly, as directed in the [[QA:SOP_blocker_bug_process]], the bug secretary should update the bug report ''Comments'', ''Blocks:'' and ''Whiteboard'' fields as appropriate
#* Summarize the meeting discussion in a new comment (see suggested stock responses for [[BugZappers/StockBugzillaResponses#Blocker_Bug_Removal|removing blocker status]] and [[BugZappers/StockBugzillaResponses#Blocker_Bug_Downgrade_to_Target|downgrading to Target]]).
#* If the bug is agreed to be a blocker, add the text <code>AcceptedBlocker</code>, <code>Accepted0Day</code>, or <code>AcceptedPreviousRelease</code> - as agreed - with that exact spelling and case to the ''Whiteboard'' field
#* If the bug is agreed to '''not''' be a blocker, add the text <code>RejectedBlocker</code> (with that exact spelling and case) to the ''Whiteboard'' field, and remove the blocker tracker bug from the Blocks: field
#* If the decision could not be made, do not add any text to the ''Whiteboard'' field
{{admon/tip|Need more time?|Sometimes it is difficult to reach consensus on whether a bug should block a particular milestone.  Given that there may be many bugs to review, it is recommended that no more than 5 minutes be spent reviewing a bug.  If additional time is needed, note who has the action items to follow-up on the issue after the meeting, and move on to the next bug in the list.}}
=== Review Accepted Blocker Bugs ===
The meeting should also review all existing accepted blocker bugs to ensure work is progressing on fixing them. Blocker status can be re-discussed and re-voted on if there is a reason to do so, otherwise the meeting should focus on ensuring the next steps in fixing the bug are clarified and the meeting leader should ensure appropriate #info and #action tags are added.
=== Review Proposed Freeze Exception Bugs ===
Using the same procedure outlined above, review the [[QA:SOP_freeze_exception_bug_process|freeze exception bugs]]. It is not necessary to review previously accepted freeze exception bugs, in most cases. The Whiteboard fields for freeze exception bugs are ''AcceptedFreezeException'' and ''RejectedFreezeException''.
=== Review <code>CLOSED</code>, but not <code>VERIFIED</code> Blocker Bugs ===
If time and energy remain, the meeting leader may quickly review the list of previously accepted blocker bugs that have been <code>CLOSED</code>, but were '''not''' <code>VERIFIED</code>.  This is intended as a quick review to highlight any critical bugs where additional testing is desired.  Use the queries below to find applicable bugs.
* [https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&type0-1-0=equals&order=Importance&value0-1-0=Tracking&field0-1-0=keywords&field0-0-0=alias&type0-0-0=regexp&value0-0-0=^{{FedoraVersion|short|next}} List of all {{FedoraVersion|short|next}} Tracking bugs].  Use this query to update the following queries with any new Tracking bugs.
* List of <code>CLOSED</code>, but not <code>VERIFIED</code>, Blocker bugs:
** [https://bugzilla.redhat.com/buglist.cgi?negate1=1&type1-0-0=changedto&type0-1-0=equals&field0-1-0=bug_status&field0-0-0=blocked&value2-0-0=Tracking&field2-0-0=keywords&type2-0-0=nowordssubstr&query_format=advanced&value0-1-0=CLOSED&value1-0-0=VERIFIED&type0-0-0=equals&value0-0-0=657616&field1-0-0=bug_status {{FedoraVersion|short|next}}Alpha]
** [https://bugzilla.redhat.com/buglist.cgi?negate1=1&type1-0-0=changedto&type0-1-0=equals&field0-1-0=bug_status&field0-0-0=blocked&value2-0-0=Tracking&field2-0-0=keywords&type2-0-0=nowordssubstr&query_format=advanced&value0-1-0=CLOSED&value1-0-0=VERIFIED&type0-0-0=equals&value0-0-0=657618&field1-0-0=bug_status {{FedoraVersion|short|next}}Beta]
** [https://bugzilla.redhat.com/buglist.cgi?negate1=1&type1-0-0=changedto&type0-1-0=equals&field0-1-0=bug_status&field0-0-0=blocked&value2-0-0=Tracking&field2-0-0=keywords&type2-0-0=nowordssubstr&query_format=advanced&value0-1-0=CLOSED&value1-0-0=VERIFIED&type0-0-0=equals&value0-0-0=617261&field1-0-0=bug_status {{FedoraVersion|short|next}}Blocker]
* List of <code>CLOSED</code>, but not <code>VERIFIED</code>, freeze exception bugs:
** [https://bugzilla.redhat.com/buglist.cgi?negate1=1&type1-0-0=changedto&type0-1-0=equals&field0-1-0=bug_status&field0-0-0=blocked&value2-0-0=Tracking&field2-0-0=keywords&type2-0-0=nowordssubstr&query_format=advanced&value0-1-0=CLOSED&value1-0-0=VERIFIED&type0-0-0=equals&value0-0-0=657617&field1-0-0=bug_status {{FedoraVersion|short|next}}Alpha Freeze Exception]
** [https://bugzilla.redhat.com/buglist.cgi?negate1=1&type1-0-0=changedto&type0-1-0=equals&field0-1-0=bug_status&field0-0-0=blocked&value2-0-0=Tracking&field2-0-0=keywords&type2-0-0=nowordssubstr&query_format=advanced&value0-1-0=CLOSED&value1-0-0=VERIFIED&type0-0-0=equals&value0-0-0=657619&field1-0-0=bug_status {{FedoraVersion|short|next}}Beta Freeze Exception]
** [https://bugzilla.redhat.com/buglist.cgi?negate1=1&type1-0-0=changedto&type0-1-0=equals&field0-1-0=bug_status&field0-0-0=blocked&value2-0-0=Tracking&field2-0-0=keywords&type2-0-0=nowordssubstr&query_format=advanced&value0-1-0=CLOSED&value1-0-0=VERIFIED&type0-0-0=equals&value0-0-0=657621&field1-0-0=bug_status {{FedoraVersion|short|next}} Freeze Exception]
=== Open discussion ===
While the meeting can run long, if time remains, it is nice to open up the meeting for any feedback or bugs not already discussed.
* <code>#topic Open Discussion <your bugs here></code>
== Tear Down ==
Finally, after the list of bugs has been reviewed, identify who is responsible for the [[#Tear_Down]] process.  Typically, this is the meeting leader's responsibility, but it doesn't hurt to clearly identify who will be
# Confirm the next meeting time and date
#* <code>#info Next meeting time - XX:XX UTC on $date</code>
# Thank participants, and close the meeting
#* <code>#endmeeting</code>
# Send a summary of the meeting's findings and action items to [https://admin.fedoraproject.org/mailman/listinfo/test test@lists.fedoraproject.org].  As part of the email include information about the date and time of the next meeting.
== Secretary Duty ==
=== Introduction ===
Secretary duties can be done 'on the fly' as the meeting proceeds, or after the meeting is complete. The secretary's duty is to update the bugs themselves to reflect the discussion held and statuses agreed at the meeting. The high-level goal of this is to ensure that anyone who looks at the bug report for a bug that was proposed as a blocker or freeze exception issue can easily find out what decisions were taken regarding its status, when, and why, and easily find the logs of the original discussion if so desired.
If you find yourself unsure what to do when performing secretary duties, try to place yourself in the shoes of someone coming to the bug report two years later with no knowledge of the Fedora blocker review process, and ask yourself what such a person would find it helpful to read. Also consider that it is almost always safer to err on the side of providing too much rather than too little information.
=== Explanatory comment format ===
All changes resulting from a blocker / freeze exception review meeting must be accompanied by an explanatory comment. The comment must state that the bug was discussed at the blocker / freeze exception review meeting held on a specific date, and link to the Meetbot directory containing the minutes and logs of the meeting. Here is a template that can be used to begin the comment:
<pre>
Discussed at the YYYY-MM-DD (blocker / freeze exception) review meeting:
http://meetbot.fedoraproject.org/fedora-blocker-review/YYYY-MM-DD/
</pre>
Choose from "blocker review meeting" or "freeze exception review meeting" depending on whether the bug was considered as a blocker or freeze exception issue. The rest of the comment's content will depend on what was actually discussed decided about the bug: see below.
=== Newly-accepted blocker / freeze exception issues ===
If the bug was discussed as a ''proposed'' blocker or freeze exception issue and was '''accepted''', the secretary must add the text <code>AcceptedBlocker</code>, <code>Accepted0Day</code>, <code>AcceptedPreviousRelease</code> or <code>AcceptedFreezeException</code> to the Whiteboard field, and double check that the bug is set as blocking the correct tracker.
In the explanatory comment, the secretary must state that the bug was accepted as a blocker or freeze exception issue, and explain why. If the bug was accepted as a blocker, the secretary must link to the relevant release criterion and include its text. If a previous comment has already included this information, it is sufficient to refer to that comment.
Always try to ensure that the comment alone serves as a complete and comprehensible explanation of why the bug was accepted: if it is not very clear to a casual reader why the bug constitutes a violation of the cited criterion, provide an explanation.
Example: [https://bugzilla.redhat.com/show_bug.cgi?id=1040650#c13]
=== Newly-rejected blocker / freeze exception issues ===
If the bug was discussed as a ''proposed'' blocker or freeze exception issue and was '''rejected''', the secretary must add the text ''RejectedBlocker'' or ''RejectedFreezeException'' to the Whiteboard field, and remove the relevant tracker bug from the Blocks: field.
In the explanatory comment, the secretary must state that the bug was rejected as a blocker or freeze exception issue, and explain why. For clarity it is a good idea to include a link to the relevant release criteria and a specific explanation as to why the bug was not considered to be a violation.
Always try to ensure that the comment alone serves as a complete and comprehensible explanation of why the bug was rejected.
Example: [https://bugzilla.redhat.com/show_bug.cgi?id=1040438#c7]
=== Proposed blocker / freeze exception issue not accepted or rejected ('punted') ===
Sometimes, a proposed blocker or freeze exception issue will not be accepted or rejected during a given meeting - instead, the group will agree that it is not yet possible to decide its status definitely, and agree to re-consider it at a later date. This is often referred to informally as ''punting'' (an American term, derived from American football). Often, the discussion will highlight the particular information that is needed before the bug's status can be properly determined.
The secretary must include an explanatory comment on each bug explicitly considered and ''punted'', starting in the usual format (see above). It must explain clearly that the blocker or freeze exception status was not yet conclusively determined, and summarize any agreement from the meeting on what information and/or action is needed before the status can be determined and any agreed timetable for its re-consideration.
If the necessary information or action is clearly the responsibility of a particular party, the secretary should set the ''needinfo'' flag on that party.
Ultimately, the goal is that the comment should make it clear why the status could not yet be determined, and what has to happen before it can be determined.
Example: [https://bugzilla.redhat.com/show_bug.cgi?id=994180#c12]
=== Accepted blocker / freeze exception status report ===
Where an already-accepted blocker or freeze exception issue is discussed, various results are possible. If any concrete action on the bug report is agreed upon as part of the discussion, it will usually be the secretary's responsibility to implement it, accompanied by the usual explanatory comment.
If the discussion does not result in any concrete action but does result in what looks like useful and substantive new analysis of the issue, the secretary should add a comment in the usual format, summarizing this analysis. This includes adding a note whose main purpose is to remind a maintainer that a deadline is impending and the issue should be treated as a high priority, when appropriate.
If the meeting discussion does not add anything substantive to what is already covered in the bug report, it is not necessary for the secretary to make any change or add any comment to the bug report.
=== Change in status of a previously-accepted blocker / freeze exception issue ===
If a discussion of a previously accepted blocker or freeze exception issue leads to an agreement to change its status - for instance, to reject it, or upgrade a freeze exception issue to a blocker issue, downgrade a blocker issue to a freeze exception issue, or change the milestone an issue blocks or has a freeze exception for - it is the secretary's responsibility to update the bug report to reflect the new status.
This will usually be similar to marking acceptance or rejection of a proposed blocker or freeze exception issue: ensure the ''Blocks:'' field is set correctly and the ''Whiteboard:'' field contains only the appropriate magic words, and add a comment explaining the change, with the usual references to release criteria where necessary.
=== Complex rejection / acceptance cases ===
There are some reasonably commonly-encountered 'complex' rejection/acceptance cases which it will be useful to describe.
One case involves a bug is rejected as a blocker or freeze exception for the next milestone but accepted for a subsequent milestone. For instance: at a blocker review meeting for Fedora 20 Alpha, a proposed blocker is discussed. The group agrees that it is not an ''Alpha'' blocker, but is a ''Beta'' blocker. In this case, the secretary must change the ''Blocks:'' field appropriately (in the example, from ''F20AlphaBlocker'' to ''F20BetaBlocker'') and set the ''Whiteboard:'' as for a normal acceptance case (so, in the example, add ''AcceptedBlocker'').
Another case involves a bug being rejected as a blocker issue, but accepted as a freeze exception issue. In this case, the secretary must add both ''RejectedBlocker'' and ''AcceptedFreezeException'' to the ''Whiteboard:'' field, and set the bug to block only the relevant freeze exception tracker. If a bug was rejected as a blocker but accepted as a freeze exception issue for Fedora 20 Alpha, the secretary would add both ''RejectedBlocker'' and ''AcceptedFreezeException'' to the ''Whiteboard:'' field and ensure the bug blocked ''F20AlphaFreezeException'' but not ''F20AlphaBlocker''.
A bug can also be accepted as a freeze exception issue for one milestone, and a blocker issue for a later milestone. In this case, the secretary must add both ''AcceptedBlocker'' and ''AcceptedFreezeException'' to the ''Whiteboard:'' field, and ensure the bug blocks both relevant trackers. For instance, if a bug was accepted as a freeze exception issue for Fedora 20 Alpha and a blocker issue for Fedora 20 Beta, the secretary would add both ''AcceptedBlocker'' and ''AcceptedFreezeException'' to the ''Whiteboard:'' field and ensure the bug was set to block both ''F20AlphaFreezeException'' and ''F20BetaBlocker'' (but not ''F20AlphaBlocker'' or ''F20BetaFreezeException'').
In all complex cases, the explanatory comment should explicitly note each acceptance and rejection, and explain the reasons in the usual manner.

Latest revision as of 12:25, 3 March 2016