From FedoraProject

Jump to: navigation, search

Fedora Release Engineering Meeting :: Monday 2007-08-27

F8 Test 2

  • Freeze 2008-08-28
  • < f13> $ grep ppc32 really-need-to-rebuild |wc -l == 262
  • < f13> $ grep BuildID really-need-to-rebuild |wc -l == 1232
  • start fixing broken deps and broken upgrade paths ourselves instead of waiting for maintainers to do it


  • Do a scripted rebuild of remaining ppc32 build targets for test2.
  • Allow community to continue rebuilding on their own for BuildID but don't hold up test2 for it.

IRC Transcript

f13releng meeting<a href="#t13:07" class="time">13:07</a>
Idea.png f13 changed the topic of #fedora-meeting to: Fedora Release Engineering Meeting<a href="#t13:07" class="time">13:07</a>
f13notting: jeremy: warren: spot: rdieter: wwoods: jwb: ping<a href="#t13:08" class="time">13:08</a>
f13poelcat: ping you too<a href="#t13:08" class="time">13:08</a>
nottingf13: here, but completely drowning in mail and out of the loop<a href="#t13:08" class="time">13:08</a>
jwbhere<a href="#t13:08" class="time">13:08</a>
* wwoods here<a href="#t13:09" class="time">13:09</a>
* spot is here<a href="#t13:09" class="time">13:09</a>
* poelcat here<a href="#t13:09" class="time">13:09</a>
* jeremy is here<a href="#t13:09" class="time">13:09</a>
wwoodsI'll be in-and-out of the office today and gone Friday through Monday, by the way<a href="#t13:10" class="time">13:10</a>
f13wwoods: fun!<a href="#t13:12" class="time">13:12</a>
f13wwoods: you have an uncanny knack of being out of the office right around freeze/release times.<a href="#t13:12" class="time">13:12</a>
Idea.png f13 changed the topic of #fedora-meeting to: Fedora Release Engineering Meeting - Test 2<a href="#t13:13" class="time">13:13</a>
wwoodsTest 2 has an uncanny knack for landing right during things like my honeymoon<a href="#t13:13" class="time">13:13</a>
wwoodsor, this time, my anniversary<a href="#t13:13" class="time">13:13</a>
f13so the freeze is tomorrow.  I'm thinking sometime in the evening.<a href="#t13:13" class="time">13:13</a>
f13well, that is unless we want to extend the freeze until we complete all the necessary rebuilds.<a href="#t13:13" class="time">13:13</a>
f13$ grep ppc32 really-need-to-rebuild |wc -l<a href="#t13:14" class="time">13:14</a>
f13262<a href="#t13:14" class="time">13:14</a>
wwoodsI was thinking - given that the test2 freeze is the feature freeze, people seem to be throwing in a lot of last-minute stuff<a href="#t13:14" class="time">13:14</a>
f13$ grep BuildID really-need-to-rebuild |wc -l<a href="#t13:14" class="time">13:14</a>
f131232<a href="#t13:14" class="time">13:14</a>
spotf13: how freshly regenerated is that?<a href="#t13:14" class="time">13:14</a>
wwoodsso it might make sense to consider an extended freeze for test2 in general<a href="#t13:14" class="time">13:14</a>
* rdieter pops in.<a href="#t13:16" class="time">13:16</a>
f13spot: I did a new one this morning.<a href="#t13:16" class="time">13:16</a>
spotokay<a href="#t13:16" class="time">13:16</a>
f13wwoods: if we followed that logic, we'd be delaying every test2 from here on out<a href="#t13:16" class="time">13:16</a>
f13wwoods: and we'd just be giving people another week to rush more last-minute things in<a href="#t13:16" class="time">13:16</a>
wwoodsf13: yes, longer freeze for all test2 releases was my suggestion<a href="#t13:17" class="time">13:17</a>
spotdo we want to freeze before we script the rebuild or the reverse?<a href="#t13:17" class="time">13:17</a>
wwoodsI thought the freeze was a hard cutoff, though?<a href="#t13:17" class="time">13:17</a>
wwoodsit just seems like there's a lot more churn leading up to the test2 freeze<a href="#t13:17" class="time">13:17</a>
wwoodsso it's harder to stabilize<a href="#t13:18" class="time">13:18</a>
jeremywwoods: there's always the "ask fesco for an exception" process<a href="#t13:18" class="time">13:18</a>
f13wwoods: oh, a longer /freeze/.  I thought you meant delay the freeze.<a href="#t13:18" class="time">13:18</a>
wwoodsbut there's no more time in the schedule than any other release.<a href="#t13:18" class="time">13:18</a>
wwoodsoh! yes, sorry, that's what I meant, a longer freeze<a href="#t13:18" class="time">13:18</a>
f13but as long as we're in "freeze" and haven't cut gold isos, people will ask/beg/yell/scream for their changes to get snuck in.<a href="#t13:18" class="time">13:18</a>
wwoodsThis is just a discussion topic, not a QA Demand<a href="#t13:18" class="time">13:18</a>
nottingmore likely, they'll also ask for rawhide to be pushed forward so random fixes can get out<a href="#t13:19" class="time">13:19</a>
f13and the longer we're in freeze the longer rawhide stagnates, or the more irrelevant the release becomes if we allow rawhide to truck on.<a href="#t13:19" class="time">13:19</a>
f13spot: I'm not really sure.  I don't want to do any signing until we finish up the builds.<a href="#t13:19" class="time">13:19</a>
f13personally, I'd be OK with just getting the ppc32 ones knocked out so that selinux works there again, and extend the BuildID/Licensing rebuild stuff for a bit.<a href="#t13:19" class="time">13:19</a>
f13and allow maintainers to get to them on their own instead of a script.<a href="#t13:20" class="time">13:20</a>
spotseems reasonable<a href="#t13:20" class="time">13:20</a>
rdieterditto (did we just lose f13/jeremy?)<a href="#t13:22" class="time">13:22</a>
wwoodswe did. better wait for the split to end<a href="#t13:22" class="time">13:22</a>
rdieterparty off.<a href="#t13:22" class="time">13:22</a>
rdieter:)<a href="#t13:22" class="time">13:22</a>
spotwell, now that we've decided that, meeting adjourned?<a href="#t13:23" class="time">13:23</a>
spot;)<a href="#t13:23" class="time">13:23</a>
rdieterponies!<a href="#t13:23" class="time">13:23</a>
wwoodsha<a href="#t13:23" class="time">13:23</a>
jwbi wanted to bug jeremy about presto<a href="#t13:24" class="time">13:24</a>
jwbmmcgrath, ping<a href="#t13:24" class="time">13:24</a>
mmcgrathjwb: pong<a href="#t13:25" class="time">13:25</a>
jwbmmcgrath, has jeremy been working with you on the deltarpm/presto stuff?<a href="#t13:25" class="time">13:25</a>
poelcatjwb: jeremy changed the feature page to target F9<a href="#t13:26" class="time">13:26</a>
jwbah, bummer<a href="#t13:26" class="time">13:26</a>
mmcgrathnot me other then verifying space on the mirrors and such<a href="#t13:26" class="time">13:26</a>
jwbok thanks poelcat<a href="#t13:26" class="time">13:26</a>
mmcgrathIIRC the major blocker was getting koji integration though I could be wrong on that.<a href="#t13:26" class="time">13:26</a>
jwbmmcgrath, that is true.  jeremy you back now?<a href="#t13:27" class="time">13:27</a>
wwoodsjeremy and f13 should be back<a href="#t13:29" class="time">13:29</a>
spotno, they're not<a href="#t13:30" class="time">13:30</a>
spoti just asked f13 via AIM<a href="#t13:30" class="time">13:30</a>
spot<f13> what do you all think about that?<a href="#t13:30" class="time">13:30</a>
spot<f13> 250~ builds to plow through in a night is a lot more reasonable than 1500<a href="#t13:30" class="time">13:30</a>
spot<jeremy> sounds okay<a href="#t13:30" class="time">13:30</a>
spot<f13> lets be all red tapy about it.<a href="#t13:30" class="time">13:30</a>
spot<f13> Proposal: Script rebuild the remaining ppc32 builds for Test2 (can happen both before and after freeze with manual tagging).  Don't delay test2 for BuildID and License rebuilds.<a href="#t13:30" class="time">13:30</a>
spot+1 to the proposal<a href="#t13:31" class="time">13:31</a>
mdomsch1500 won't complete in a night<a href="#t13:31" class="time">13:31</a>
mdomschif my builders are any indication<a href="#t13:31" class="time">13:31</a>
* f13_ taps his foot waiting for the other server to figure things out.<a href="#t13:31" class="time">13:31</a>
f13_mdomsch: not by far.<a href="#t13:32" class="time">13:32</a>
f13_1500 would take a bit over 24 hours.<a href="#t13:32" class="time">13:32</a>
spotmdomsch: some of my packages build fine in koji but fail for your builders...<a href="#t13:32" class="time">13:32</a>
f13_our builds take longer due to all the post build processing by koji<a href="#t13:32" class="time">13:32</a>
f13_spot: his builders started a week ago... (:<a href="#t13:32" class="time">13:32</a>
f13_so changes between now and then lead to differences.<a href="#t13:32" class="time">13:32</a>
f13oh hey, I'm back.<a href="#t13:32" class="time">13:32</a>
spotf13: even without any changes from me<a href="#t13:32" class="time">13:32</a>
f13_hrm.<a href="#t13:33" class="time">13:33</a>
f13_that's cute.  I can see what I type, but not what other people are saying.<a href="#t13:33" class="time">13:33</a>
spothaha<a href="#t13:33" class="time">13:33</a>
f13_spot: changes to the build environment I mean.<a href="#t13:33" class="time">13:33</a>
wwoodsheh<a href="#t13:33" class="time">13:33</a>
spotyay freenode!<a href="#t13:33" class="time">13:33</a>
mdomschI restarted after last week Monday's meeting, but it took nearly the full week in 7 builders<a href="#t13:35" class="time">13:35</a>
wwoodsoi<a href="#t13:35" class="time">13:35</a>
mdomschat least 3 days, not counting the downtime between failed runs<a href="#t13:35" class="time">13:35</a>
f13_so, jeremy and I both voted +1 on the proposal, I only caught a spot vote +1, where there any other votes?<a href="#t13:37" class="time">13:37</a>
spotf13: not on this side, no<a href="#t13:37" class="time">13:37</a>
drago01which proposal? (joined later)<a href="#t13:37" class="time">13:37</a>
spot <f13> Proposal: Script rebuild the remaining ppc32 builds for Test2 (can happen both before and after freeze with manual tagging).  Don't delay test2 for BuildID and License rebuilds.<a href="#t13:38" class="time">13:38</a>
drago01ok<a href="#t13:38" class="time">13:38</a>
jwbi have automated rebuilds that way, but +1<a href="#t13:38" class="time">13:38</a>
jwber, s/have/hate<a href="#t13:38" class="time">13:38</a>
spotjwb: people were given lots of warning to do it manually<a href="#t13:39" class="time">13:39</a>
f13jwb: I do too, which is why I want to do only as few as necessary.<a href="#t13:39" class="time">13:39</a>
jwbspot, yes i know<a href="#t13:39" class="time">13:39</a>
wwoodsso - what's the deadline for the rebuilds? at what point/date will we say "anything left on the list will rebuilt this day"<a href="#t13:40" class="time">13:40</a>
f13wwoods: I'm thinking a week or two weeks before test3 freeze.<a href="#t13:40" class="time">13:40</a>
f13notting: you still around?<a href="#t13:41" class="time">13:41</a>
f13wwoods: vote?<a href="#t13:41" class="time">13:41</a>
nottingf13: +0 from me<a href="#t13:42" class="time">13:42</a>
wwoodsI'm not sure I fully understand the proposal, but if this means that ppc rebuilds will happen Soon then +1<a href="#t13:43" class="time">13:43</a>
f13well, I don't see anybody really opposed to this, so bam, proposal approved.<a href="#t13:43" class="time">13:43</a>
jwbf13, does it need to run through FESCo?<a href="#t13:43" class="time">13:43</a>
* jwb hopes not<a href="#t13:43" class="time">13:43</a>
f13jwb: I hope not as well.  I can mention it on the list but I don't expect any push back there.<a href="#t13:44" class="time">13:44</a>
wwoodsalso we need to be sure that people know when the rebuild deadline (and inevitable subsequent mass-rebuild) is<a href="#t13:44" class="time">13:44</a>
f13I'll blurb before kicking off the script.<a href="#t13:44" class="time">13:44</a>
wwoodsso we don't have to deal with the constant "omg why's koji broken" questions<a href="#t13:44" class="time">13:44</a>
f13wwoods: yah, that's a bigger subject that I want to deal with not today.<a href="#t13:44" class="time">13:44</a>
wwoodsfair 'nuff<a href="#t13:44" class="time">13:44</a>
f13poelcat: decision, do a scripted rebuild of remaining ppc32 build targets for test2.  Allow community to continue rebuilding on their own for BuildID but don't hold up test2 for it.<a href="#t13:45" class="time">13:45</a>
f13so for the rest of test2....<a href="#t13:45" class="time">13:45</a>
f13same thing as before, going to start signing soon (maybe tonight), making sure that compose tools work fully<a href="#t13:46" class="time">13:46</a>
f13chasing down broken deps and upgrade paths<a href="#t13:46" class="time">13:46</a>
jwbf13, i had another question on that...<a href="#t13:46" class="time">13:46</a>
f13coaxing rawhide to keep going.<a href="#t13:46" class="time">13:46</a>
f13jwb: shoot<a href="#t13:46" class="time">13:46</a>
wwoodsI've created the F8Test2 tracker<a href="#t13:46" class="time">13:46</a>
jwbyou told ChitleshGoorah that his FEL spin had to be done by test2... why is that?<a href="#t13:46" class="time">13:46</a>
wwoodsI'll make the TreeTesting p age in a bit<a href="#t13:46" class="time">13:46</a>
f13wwoods: notice that, thanks.<a href="#t13:46" class="time">13:46</a>
f13jwb: a new spin is a Feature, and Features need to be testable by test2.<a href="#t13:46" class="time">13:46</a>
f13jwb: rather a new spin that we're going to host and put along side the other spins, like the Desktop and the KDE one.<a href="#t13:47" class="time">13:47</a>
jwbf13, hm... k.  so that begs the question of where the KDE spin stands<a href="#t13:47" class="time">13:47</a>
jwbrdieter, ?<a href="#t13:47" class="time">13:47</a>
f13jwb: kde spin has been there since last release.<a href="#t13:47" class="time">13:47</a>
f13the config for it is shipped in livecd-tools<a href="#t13:47" class="time">13:47</a>
rdieterjwb: not much new, certainly no kde4 bits, if that's what you're asking.<a href="#t13:47" class="time">13:47</a>
drago01f13: upgrade paths from rawhide->test2 or f7->test2 ?<a href="#t13:48" class="time">13:48</a>
jwbf13, so you're just talking about the config?<a href="#t13:48" class="time">13:48</a>
f13jwb: well a config that leads to a working Live image.<a href="#t13:48" class="time">13:48</a>
f13jwb: we had a KDE Live for test1 so they're covered.<a href="#t13:48" class="time">13:48</a>
f13I don't really want to introduce a new Live image after the feature freeze.<a href="#t13:48" class="time">13:48</a>
jwbok.  i don't forsee the FEL spin being overly complicated<a href="#t13:48" class="time">13:48</a>
poelcatf13: thanks for recap... pls recap others... I need to step away briefly<a href="#t13:48" class="time">13:48</a>
f13poelcat: will do.<a href="#t13:48" class="time">13:48</a>
jwbperhaps ChitleshGoorah and jeremy can bang that out quickly enough<a href="#t13:49" class="time">13:49</a>
f13jwb: if not, there is always hosting it on the side and waiting for F9<a href="#t13:49" class="time">13:49</a>
drago01f13: because if its f7->test2 we need a hack for compiz in anaconda<a href="#t13:49" class="time">13:49</a>
f13drago01: mostly f7-test2<a href="#t13:49" class="time">13:49</a>
f13er f7->test2<a href="#t13:50" class="time">13:50</a>
drago01f13: the control center stuff will come with the next gnome release because krh wants to work with upstream (=f9)<a href="#t13:50" class="time">13:50</a>
f13drago01: is there a bug filed against anaconda for tihs?<a href="#t13:50" class="time">13:50</a>
f13this?<a href="#t13:50" class="time">13:50</a>
drago01f13: no but can fill it<a href="#t13:50" class="time">13:50</a>
f13that would help<a href="#t13:51" class="time">13:51</a>
drago01ok<a href="#t13:51" class="time">13:51</a>
jeremyjwb: FEL should be pretty straight-forward.  it's almost there<a href="#t13:51" class="time">13:51</a>
jwbgrea<a href="#t13:51" class="time">13:51</a>
jwbt<a href="#t13:51" class="time">13:51</a>
f13so it may be a rough couple of days<a href="#t13:52" class="time">13:52</a>
f13and there may be slips involved.<a href="#t13:52" class="time">13:52</a>
f13if anybody wants to work on the broken deps + upgrade paths, that would be neat, whilst I work on testing spin tools and scripting the rebuilds.<a href="#t13:53" class="time">13:53</a>
f13jeremy: we should get a set of rawhide isos into torrent if possible<a href="#t13:53" class="time">13:53</a>
jeremyf13: I'm trying to get something<a href="#t13:53" class="time">13:53</a>
f13(I think his machine was getting taken out via livecd-creator + auditd)<a href="#t13:53" class="time">13:53</a>
jwbf13, i'll try and dust off my scripts for the upgrade path stuff soon<a href="#t13:53" class="time">13:53</a>
f13in other news, the next rpm of pungi I make will be a pretty significant one.<a href="#t13:54" class="time">13:54</a>
jwbwhy's that?<a href="#t13:54" class="time">13:54</a>
f13it will now take kickstart files as input, has a few more cli options, and a lot more things hardcoded.<a href="#t13:54" class="time">13:54</a>
jwbhave docs to go with it?<a href="#t13:54" class="time">13:54</a>
f13but it also handles $basearch and $releasever values.<a href="#t13:54" class="time">13:54</a>
f13jwb: no, that would just be silly (:<a href="#t13:54" class="time">13:54</a>
jwbi was going to start poking at a XFCE spin...<a href="#t13:55" class="time">13:55</a>
f13k<a href="#t13:55" class="time">13:55</a>
jwbexpect many questiosn<a href="#t13:55" class="time">13:55</a>
f13what this does mean is that a single .ks file is usable for spinning i386, x86_64, ppc, etc...<a href="#t13:55" class="time">13:55</a>
f13it also has much better logging support, including full logging should you short cut it (*cough* rawhide)<a href="#t13:55" class="time">13:55</a>
f13doesn't need a comps file anymore (snakes and combines one out of groupdata in repos you provide)<a href="#t13:56" class="time">13:56</a>
f13and a few other goodies that I can't remember.  It's been a busy weekend.<a href="#t13:56" class="time">13:56</a>
jwbdoes it have "include" support?<a href="#t13:56" class="time">13:56</a>
f13notting: I need to get with you before I build this, or at least before the next automated rawhide run, as mash will need changes.<a href="#t13:56" class="time">13:56</a>
f13jwb: not yet, or at least untested.<a href="#t13:56" class="time">13:56</a>
jwbshoot, ok<a href="#t13:56" class="time">13:56</a>
f13jwb: it uses pykickstart, so if pykickstart does that for me behind the scenes then perhaps yes.<a href="#t13:57" class="time">13:57</a>
nottingf13: ok<a href="#t13:57" class="time">13:57</a>
wwoodspykickstart has flags for handling includes<a href="#t13:57" class="time">13:57</a>
jeremyf13: %includes just work without you doing anything.  so untested, not "not there"<a href="#t13:57" class="time">13:57</a>
f13jwb: there are at most 3 things you define in your kickstart file.  %packages, repo(s), and optionally part iso to define iso size you want.<a href="#t13:57" class="time">13:57</a>
f13everything else is done via cli flags.<a href="#t13:58" class="time">13:58</a>
jwbi was wondering if you could %include the gnome ks file and just add some more %packages after that<a href="#t13:58" class="time">13:58</a>
f13jwb: you may be able to do that.<a href="#t13:58" class="time">13:58</a>
f13I just haven't tested it.<a href="#t13:58" class="time">13:58</a>
jwbwhich would make FEL pretty trivial<a href="#t13:58" class="time">13:58</a>
f13that's livecd-creator though, not pungi (:<a href="#t13:58" class="time">13:58</a>
f13'and that /does/ work.<a href="#t13:59" class="time">13:59</a>
jwboh<a href="#t13:59" class="time">13:59</a>
nirikjwb: I'd be happy to help with an Xfce spin. I did play some already with a Xfce livecd...<a href="#t13:59" class="time">13:59</a>
jwbyes, sorry<a href="#t13:59" class="time">13:59</a>
jeremyjwb: FEL is a live image.  and yes, it needs to switch to using %include; that's one of the things I told chitlesh this morning<a href="#t13:59" class="time">13:59</a>
f13anywho, we're out of time.  Anything else pressing?<a href="#t13:59" class="time">13:59</a>
jwbf13, yes<a href="#t13:59" class="time">13:59</a>
f13should we try to meet again later this week?<a href="#t13:59" class="time">13:59</a>
jwbf13, F8 naming<a href="#t13:59" class="time">13:59</a>
f13oh yeah!<a href="#t13:59" class="time">13:59</a>
wwoodsshould we file bugs to track broken deps for f8t2?<a href="#t13:59" class="time">13:59</a>
f13got me a list to feed into Legal?<a href="#t13:59" class="time">13:59</a>
jwbf13, i have a list at home of all the submissions.  i'll filter it some and send it to you<a href="#t13:59" class="time">13:59</a>
wwoodsor just keep track on a wiki page?<a href="#t13:59" class="time">13:59</a>
jwbf13, look for an email tonight, tomorrow morning<a href="#t13:59" class="time">13:59</a>
f13wwoods: at this point, BOFH mode is engaged and we should just fix them.<a href="#t14:00" class="time">14:00</a>
f13jwb: perfect.  I appreciate that.<a href="#t14:00" class="time">14:00</a>
f13wwoods: bugs are fine though if you want to track that way<a href="#t14:00" class="time">14:00</a>
jwbnirik, if you've already started, then send me what you have.  i can test stuff in KVM :)<a href="#t14:00" class="time">14:00</a>
wwoodsnah. nobody likes dealing with bugzilla unless they have to<a href="#t14:00" class="time">14:00</a>
wwoodswe can just use a wiki page for coordination<a href="#t14:00" class="time">14:00</a>
f13(my life for a bugzilla cli tool.  bugzilla --new --product Fedora --component frobitz --summary "Broken deps <blah>" --description "Your shit is broken, fix it." --blocks F8Test2)<a href="#t14:01" class="time">14:01</a>
wwoodsf13: definitely on my TODO list<a href="#t14:01" class="time">14:01</a>
f13anything else?<a href="#t14:01" class="time">14:01</a>
f13Ok, meeting over.  Cheers all.<a href="#t14:02" class="time">14:02</a>
f13working on it<a href="#t14:04" class="time">14:04</a>
Idea.png f13 changed the topic of #fedora-meeting to: Channel is used by various Fedora groups and committees for their regular meetings | Note that meetings often get logged | For questions about using Fedora please ask in #fedora | See <a href=""></a> for meeting schedule<a href="#t14:05" class="time">14:05</a>
f13poelcat: no further "decisions", just some statements of what we need to work on for test2.  We are going to start fixing broken deps and broken upgrade paths ourselves instead of waiting for maintainers to do it.<a href="#t14:13" class="time">14:13</a>

Generated by 2.3 by Marius Gedminas - find it at!