ReleaseEngineering/Meetings/2008-apr-21

Fedora Release Engineering Meeting :: Monday 2000-04-21

Preview Release

• PR was made available via torrents on Friday
• Attempting to sync release trees to master mirrors so that we can offer direct (and jigdo) downloads
• upload is mostly done--waiting for snapmirror to sync up the various RH netapps
• after sync is complete will flip a bit and let the mirrors get it

CVS Branching

• 36+ hours of outage for branching
• cannot do this again--must to figure out a way to do this without an outage
• the second batch of cvs branches is about 50% of the way through which started 11~ hours ago
• part of the delay is the pkgdb updates--a different SCM wouldn't necessarily fix the problem

Release Candidate

1. They're real RCs, as in "we think we fixed all we're going to take, so now we're going to do the composes that, if they turn out right, are what we ship"

• not "hey that RC passed, now go compose the final"

1. RC will not be a public release

• lead time to get bits moved around just takes too long for it to be really worth it

1. Everyone can do composes leading up to RC for testing various fixes 1. Using pungi people outside of Red Hat can create their own RC using rawhide

Tagging SOP

• need writeup on Release Engineering process and steps to tag packages at freeze
• add something to http://fedoraproject.org/wiki/ReleaseEngineering/SOP/FreezingRawhide

2008-04-22-14:30-meeting-snapshot

-!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering Meeting - Roll Call		<a href="#t13:11" class="time">13:11</a>
* poelcat here :)		<a href="#t13:11" class="time">13:11</a>
f13	ping: jwb jeremy spot warren wwoods poelcat rdieter notting lmacken	<a href="#t13:11" class="time">13:11</a>
* f13 wonders if he forgot anybody		<a href="#t13:12" class="time">13:12</a>
notting	not really here	<a href="#t13:12" class="time">13:12</a>
jwb	1/2 here	<a href="#t13:12" class="time">13:12</a>
* warren here		<a href="#t13:12" class="time">13:12</a>
* rdieter here		<a href="#t13:12" class="time">13:12</a>
* lmacken is kind of here		<a href="#t13:12" class="time">13:12</a>
-!- f13 changed the topic of #fedora-meeting to: releng - Preview Release		<a href="#t13:15" class="time">13:15</a>
f13	Sorry for not being good with agenda creation, I suck.	<a href="#t13:15" class="time">13:15</a>
jwb	meh	<a href="#t13:15" class="time">13:15</a>
f13	PR was made available via torrents on Friday, and a number of people have them.	<a href="#t13:15" class="time">13:15</a>
f13	I've also been trying to get the release trees synced to the master mirrors so that we can offer direct (and jigdo) downloads	<a href="#t13:15" class="time">13:15</a>
f13	that upload is mostly done, now we're just waiting for snapmirror to sync up the various RH netapps	<a href="#t13:16" class="time">13:16</a>
f13	then I'll flip a bit and let the mirrors get it	<a href="#t13:16" class="time">13:16</a>
f13	then PR will be out there, and everybody will be happy	<a href="#t13:19" class="time">13:19</a>
-!- f13 changed the topic of #fedora-meeting to: releng - CVS branching		<a href="#t13:19" class="time">13:19</a>
f13	this is a disaster.	<a href="#t13:20" class="time">13:20</a>
f13	looking like 36+ hours of outage for branching	<a href="#t13:21" class="time">13:21</a>
f13	cannot do that again.  We /have/ to figure out a way to do this without an outage.	<a href="#t13:21" class="time">13:21</a>
f13	and I think all we do an outage for is to turn off mail, and short circuit some slow things.  If we just make our mail senders smart enough to opt-out given certain options, we can live with teh other slowness because there won't be an outage.	<a href="#t13:21" class="time">13:21</a>
f13	I haven't filed tickets on it, but toshio and I have talked a bit	<a href="#t13:22" class="time">13:22</a>
f13	as it stands, the second batch of cvs branches is about 50% of the way through, and I started it 11~ hours ago	<a href="#t13:25" class="time">13:25</a>
poelcat	f13: w/o intending to start a flamewar on source control	<a href="#t13:26" class="time">13:26</a>
poelcat	would git or something else make this easier?	<a href="#t13:26" class="time">13:26</a>
f13	poelcat: tis the season.	<a href="#t13:26" class="time">13:26</a>
f13	poelcat: that's really ahrd to say, as we can't just drop in replace our system with git	<a href="#t13:26" class="time">13:26</a>
jwb	poelcat, it would from the mass-<doing something> perspective.  dunno about the rest of the VCS uses	<a href="#t13:26" class="time">13:26</a>
f13	poelcat: theoretically it would get better, but a good chunk of the delay was doing all the branch requests in pkgdb	<a href="#t13:27" class="time">13:27</a>
f13	we have to tell pkgdb that all these packages now have an F-9 branch, and to clone the permissions from devel/ over to the F-9 version	<a href="#t13:27" class="time">13:27</a>
f13	the good news is that we had very few failures thus far, and it's gone smoother (although longer) than last time	<a href="#t13:28" class="time">13:28</a>
jwb	f13, abadger1999 said that took 15s/package	<a href="#t13:30" class="time">13:30</a>
jwb	whereas CVS took 3min?	<a href="#t13:30" class="time">13:30</a>
f13	jwb: 3 min seems high.	<a href="#t13:30" class="time">13:30</a>
jwb	those were the numbers i saw yesterday	<a href="#t13:31" class="time">13:31</a>
abadger1999	jwb: 3pkgs /minute	<a href="#t13:31" class="time">13:31</a>
jwb	oh, sorry	<a href="#t13:31" class="time">13:31</a>
jwb	i suck	<a href="#t13:31" class="time">13:31</a>
abadger1999	jwb: pkgdb-client 4 pkgs / minute; pkgdb2branch 3 pkgs/minute	<a href="#t13:32" class="time">13:32</a>
jwb	ok.	<a href="#t13:32" class="time">13:32</a>
f13	plus or minus the spinup time of the task	<a href="#t13:33" class="time">13:33</a>
f13	the cvs branching can be done somewhat parallel, I wasn't sure if the pkgdb-client stuff could or not	<a href="#t13:34" class="time">13:34</a>
f13	moving on.	<a href="#t13:37" class="time">13:37</a>
-!- f13 changed the topic of #fedora-meeting to: releng - Release Candidates		<a href="#t13:37" class="time">13:37</a>
abadger1999	It should be fine to do in parallel.  There's three app servers so using three parallel processes ought to be reasonable.	<a href="#t13:37" class="time">13:37</a>
f13	abadger1999: ok, we'll have to make note of doing that in the future	<a href="#t13:37" class="time">13:37</a>
abadger1999	<nod>	<a href="#t13:37" class="time">13:37</a>
f13	Couple things with RC.  1) they're real RCs, as in "we think we fixed all we're going to take, so now we're going to do the composes that, if they turn out right, are what we ship"	<a href="#t13:38" class="time">13:38</a>
f13	not "hey that RC passed, now go compose the final"	<a href="#t13:38" class="time">13:38</a>
f13	2) they likely won't be very public.  Lead time to get bits moved around just takes too long for it to be really worth it.	<a href="#t13:39" class="time">13:39</a>
f13	3) we can do all the composes we want leading up to then for testing various fixes, and with pungi other folks can too, plus rawhide.	<a href="#t13:39" class="time">13:39</a>
jwb	so if the RCs aren't public...	<a href="#t13:39" class="time">13:39</a>
jwb	or could you define "very"?	<a href="#t13:39" class="time">13:39</a>
f13	4) I'm somewhat not available for a week starting this coming Thursday.	<a href="#t13:39" class="time">13:39</a>
f13	jwb: until we get more infrastructure in place, the release composes happen on my workstation. They're as public as my workstation is.	<a href="#t13:40" class="time">13:40</a>
f13	which sucks, but so does the tubes.	<a href="#t13:40" class="time">13:40</a>
poelcat	f13: could they be put on download.fedora.r.c but not sync'd to the mirrors?	<a href="#t13:40" class="time">13:40</a>
jwb	poelcat, so people will swamp d.f.r.c?	<a href="#t13:40" class="time">13:40</a>
f13	poelcat: given that it takes about a day to sync stuff out there, they'd be quite quickly obsoleted, and cause a lot of churn to hte mirrors.	<a href="#t13:41" class="time">13:41</a>
rdieter	or torrents? (or that probably means even more work)	<a href="#t13:41" class="time">13:41</a>
f13	and yes, swamp d.f.r.c which right now == PHX	<a href="#t13:41" class="time">13:41</a>
poelcat	it seems funny that everyone can participate in the whole process until the end :)	<a href="#t13:41" class="time">13:41</a>
f13	poelcat: it's far better than other places where you can't participate at all	<a href="#t13:41" class="time">13:41</a>
f13	poelcat: the turn around time we have with RCs just doesn't work with waiting for N folks around the world to download it and give their approval	<a href="#t13:41" class="time">13:41</a>
poelcat	but we say 'all our bits are free' ;-) jk	<a href="#t13:42" class="time">13:42</a>
f13	unless we insert week delays for each RC and change absolutely nothing between it and the next RC unless a blocker comes up.	<a href="#t13:42" class="time">13:42</a>
poelcat	it's not a big deal, it just struck me	<a href="#t13:42" class="time">13:42</a>
jwb	we can still direct people to do network installs of rawhide	<a href="#t13:43" class="time">13:43</a>
f13	sure, we won't even wait for other "RH" folks in other offices to sign off on it.	<a href="#t13:43" class="time">13:43</a>
f13	really, the folks around where the compose are happening will be doing most hte smoke testing, but everything should be known fixed or passed on at that point, so really we're just sanity checking the compose results itself.	<a href="#t13:43" class="time">13:43</a>
f13	poelcat: for lolz, read the FreeBSD release process sometime.  It's cute (:	<a href="#t13:44" class="time">13:44</a>
jwb	f13, i will note that with 2ndary arches we might have to think harder	<a href="#t13:44" class="time">13:44</a>
jwb	or care less	<a href="#t13:44" class="time">13:44</a>
wwoods	the difference between the RC composes and rawhide will usually be one or two packages, and frequently nil	<a href="#t13:45" class="time">13:45</a>
wwoods	given that all the bits are public, and the code to produce the RC from the bits is also public	<a href="#t13:45" class="time">13:45</a>
wwoods	the fact that we don't invite people to totally frag our tubes just to get already-public bits	<a href="#t13:46" class="time">13:46</a>
wwoods	is Not A Big Deal	<a href="#t13:46" class="time">13:46</a>
poelcat	wwoods: yep, i said that above :)	<a href="#t13:46" class="time">13:46</a>
wwoods	esp. given the average time between RC composes versus turnaround time on community testing	<a href="#t13:46" class="time">13:46</a>
wwoods	just reaffirmign	<a href="#t13:46" class="time">13:46</a>
rdieter	I wantz my fedoraz, now now now	<a href="#t13:46" class="time">13:46</a>
wwoods	reaffirmignon	<a href="#t13:46" class="time">13:46</a>
rdieter	mmm... tasty	<a href="#t13:46" class="time">13:46</a>
jwb	wwoods, but... making shiny discs is hard	<a href="#t13:47" class="time">13:47</a>
wwoods	we aren't *hiding* anything from anyone, this *isn't* a loss of transparency	<a href="#t13:47" class="time">13:47</a>
f13	anywho, the week I'm gone should only need tag requesting and signing done as far as F9 is concerned	<a href="#t13:49" class="time">13:49</a>
f13	many people can tag, bill can sign.	<a href="#t13:49" class="time">13:49</a>
f13	I'll probably poke online once a day or so, and make sure F7/8 updates keep flowing	<a href="#t13:49" class="time">13:49</a>
poelcat	f13: i've been watching all the mail go by on releng list about tagging... do we have an SOP page that explains how it all works?	<a href="#t13:50" class="time">13:50</a>
poelcat	i'd consider helping, but don't know the process or what to do :)	<a href="#t13:51" class="time">13:51</a>
poelcat	or maybe I'd make things worse and that is by design ;-)	<a href="#t13:51" class="time">13:51</a>
f13	poelcat: we have the FinalFreeze policy page, but that's mostly a developer POV.  I don't think I have anything written up from the releng pov	<a href="#t13:51" class="time">13:51</a>
jwb	poelcat, you just wait for 2 +1s	<a href="#t13:51" class="time">13:51</a>
rdieter	nod, the only thing missing there is how to actually do the tagging.	<a href="#t13:52" class="time">13:52</a>
poelcat	jwb: and then what?	<a href="#t13:52" class="time">13:52</a>
f13	Could probably add something to <a href="http://fedoraproject.org/wiki/ReleaseEngineering/SOP/FreezingRawhide">http://fedoraproject.org/wiki/ReleaseEngineering/SOP/FreezingRawhide</a>	<a href="#t13:52" class="time">13:52</a>
jwb	then you run: koji tag-pkg --force f9-final <package>	<a href="#t13:52" class="time">13:52</a>
f13	poelcat: but to run --force, you have to have super cow releng powers	<a href="#t13:52" class="time">13:52</a>
-!- f13 changed the topic of #fedora-meeting to: releng - open floor		<a href="#t13:54" class="time">13:54</a>
rdieter	(selfishly) this blocker scares the bejezus out of me:  <a href="https://bugzilla.redhat.com/show_bug.cgi?id=443320">https://bugzilla.redhat.com/show_bug.cgi?id=443320</a> , no idea what's going on, only started happening recently. ??	<a href="#t13:57" class="time">13:57</a>
buggbot	Bug 443320: low, low, ---, X/OpenGL Maintenance List, NEW , Cannot enter keyboard input into kdm after logout	<a href="#t13:57" class="time">13:57</a>
poelcat	rdieter: i saw that issue after an initial install trying to log in for the first time	<a href="#t13:59" class="time">13:59</a>
poelcat	but couldn't reproduce it	<a href="#t13:59" class="time">13:59</a>
rdieter	not that I expect miracles here/now, just wanted to raise awareness.	<a href="#t13:59" class="time">13:59</a>
rdieter	I can (semi?) reliably get my laptop into a state to reproduce on every logout.  (suspend->resume seems to be the trigger for me) ??	<a href="#t14:00" class="time">14:00</a>
f13	well, if there is anything else...	<a href="#t14:00" class="time">14:00</a>
rdieter	nothin here	<a href="#t14:01" class="time">14:01</a>
jwb	f13, random question	<a href="#t14:02" class="time">14:02</a>
jwb	f13, RHEL5 ppc box (aka quad) for developers to muck with... after F9 goes out?	<a href="#t14:02" class="time">14:02</a>
f13	yes, I need to mail whatshisname to ask about apple store details.	<a href="#t14:03" class="time">14:03</a>
f13	although we still need a place to host it I think.  OLPC may be hostile	<a href="#t14:03" class="time">14:03</a>
jwb	why?	<a href="#t14:03" class="time">14:03</a>
f13	dgilmore was saying they didn't want any non-olpc machines in the racks anymore	<a href="#t14:04" class="time">14:04</a>
rdieter	cooties?	<a href="#t14:04" class="time">14:04</a>
f13	rdieter: I have no clue.	<a href="#t14:05" class="time">14:05</a>

Generated by irclog2html.py 2.3 by Marius Gedminas - find it at mg.pov.lt!