From Fedora Project Wiki

< QA‎ | Meetings

Revision as of 15:37, 21 February 2014 by Holmja (talk | contribs) (Removed some HTML markup.)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

IRC Transcript

-!- wwoods changed the topic of #fedora-meeting to: Fedora QA Meeting | init<a href="#t10:04" class="time">10:04</a>
wwoods<a href=""></a><a href="#t10:04" class="time">10:04</a>
* mmcgrath wonders if this is related to the other outages going on. Investigating.<a href="#t10:05" class="time">10:05</a>
* wwoods begins the log<a href="#t10:06" class="time">10:06</a>
wwoodsokay. I don't have a big agenda to get through today<a href="#t10:06" class="time">10:06</a>
wwoodsI've been working on rewriting the QA pages on the wiki to help people get started in QA<a href="#t10:06" class="time">10:06</a>
wwoods(unfortunately, no wiki right now, so you'll have to take my word for it)<a href="#t10:07" class="time">10:07</a>
f13I believe you<a href="#t10:07" class="time">10:07</a>
rickyI got to it, just really slowly.<a href="#t10:07" class="time">10:07</a>
wwoodsI've also been talking with Len DiMaggio about QA plans for Fedora - he's got decades of experience in the industry with this stuff so it's good to see what he thinks of our efforts<a href="#t10:08" class="time">10:08</a>
wwoods(<a href=""></a>)<a href="#t10:08" class="time">10:08</a>
wwoodsanyway, quick roll call: Lovechild, f13, mether, rdieter, nirik, mkanat: who's here?<a href="#t10:09" class="time">10:09</a>
mkanatI am.<a href="#t10:09" class="time">10:09</a>
metherI am<a href="#t10:10" class="time">10:10</a>
mkanatKind of randomly, really. :-)<a href="#t10:10" class="time">10:10</a>
wwoodsheh!<a href="#t10:10" class="time">10:10</a>
f13wwoods: wonk.<a href="#t10:10" class="time">10:10</a>
wwoodsLovechild: I finally put links to the Fedora QA schedule on the QA page<a href="#t10:10" class="time">10:10</a>
wwoods(see near the bottom, under Regular Meetings)<a href="#t10:10" class="time">10:10</a>
wwoodsin theory we had a Bug Day on Monday the 19th. did that happen?<a href="#t10:11" class="time">10:11</a>
metherno<a href="#t10:11" class="time">10:11</a>
rdieterhere<a href="#t10:11" class="time">10:11</a>
wwoodsmether: ah well<a href="#t10:11" class="time">10:11</a>
wwoodsWell. I have another one scheduled for Monday the 3rd<a href="#t10:12" class="time">10:12</a>
wwoods(it's even on the calendar)<a href="#t10:12" class="time">10:12</a>
wwoodsI'd like to go through some new F8 bugs, but there's a lot. probably we should choose a couple of subsystems<a href="#t10:12" class="time">10:12</a>
wwoodssound and kernel, for instance<a href="#t10:12" class="time">10:12</a>
wwoodsAnyway, I'll post to test-list after this about it<a href="#t10:14" class="time">10:14</a>
wwoodsanyone who has a suggestion for things to examine can speak up there<a href="#t10:14" class="time">10:14</a>
metheris there important bugs filed against Fedora 8 that we should look at?<a href="#t10:15" class="time">10:15</a>
f13that requires triaging<a href="#t10:16" class="time">10:16</a>
wwoodsThere's nothing I know of that's not on Bugs/F8Common<a href="#t10:16" class="time">10:16</a>
f13which is some of the point of bug days<a href="#t10:16" class="time">10:16</a>
metherIn a number of systems, it seems firstboot doesnt show up but I havent seen any specific bug reports on that for instance<a href="#t10:16" class="time">10:16</a>
wwoodsbut.. yes, f13's got it<a href="#t10:16" class="time">10:16</a>
mkanatmether: Yes, I filed the bug that's probably the source of most of that.<a href="#t10:16" class="time">10:16</a>
methermkanat: great. whats the bug number?<a href="#t10:16" class="time">10:16</a>
mkanatmether: I'll go find it.<a href="#t10:17" class="time">10:17</a>
metherI will add that to the common bugs list<a href="#t10:17" class="time">10:17</a>
* mkanat wishes was new enough that buggbot could query it.<a href="#t10:17" class="time">10:17</a>
rdieterfwiw, I triaged a whole bunch of kde bugs on the 19th, resulting in closing a  whole slew of ancient bugs, and marking many newer ones NEEDINFO.<a href="#t10:18" class="time">10:18</a>
mkanatbug 364951<a href="#t10:18" class="time">10:18</a>
buggbotBug <a href=""></a> high, high, ---, Chris Lumens, ASSIGNED , firstboot doesn't start if network connection fails<a href="#t10:18" class="time">10:18</a>
wwoodsinteresting<a href="#t10:19" class="time">10:19</a>
metheri have seen atleast a dozen different posts<a href="#t10:19" class="time">10:19</a>
metherand a few people in my office have hit this bug<a href="#t10:19" class="time">10:19</a>
mkanatIt's most likely that.<a href="#t10:19" class="time">10:19</a>
methervery likely<a href="#t10:20" class="time">10:20</a>
metherironically, I spend more time in this release checking out how yum upgrade works and didnt test Anaconda much<a href="#t10:20" class="time">10:20</a>
mkanatHeh. :-)<a href="#t10:21" class="time">10:21</a>
mkanatWell, I have to say, yum upgrade worked really well for me, so... :-D<a href="#t10:21" class="time">10:21</a>
Lovechildfor what it's worth, here - I just zoned out for a bit<a href="#t10:21" class="time">10:21</a>
methermkanat: yes. I think it has worked pretty well this release<a href="#t10:21" class="time">10:21</a>
metherwwoods: there was some discussions in fedora-devel list about testing anaconda<a href="#t10:21" class="time">10:21</a>
metherwwoods: in the context of the upgrade problems (actually a yum issue)<a href="#t10:22" class="time">10:22</a>
mkanatmether: Yeah. :-)<a href="#t10:22" class="time">10:22</a>
wwoodsmether: yeah, I saw that there was a discussion but didn't read it<a href="#t10:22" class="time">10:22</a>
LovechildI thought SNAKE was going to be our savior on anaconda issues<a href="#t10:22" class="time">10:22</a>
metherLovechild: it was Kate before. I dont know if SNAKE is any better<a href="#t10:22" class="time">10:22</a>
metherhas anyone worked with it?<a href="#t10:22" class="time">10:22</a>
wwoodsunfortunately snake only helps when we have good inputs<a href="#t10:22" class="time">10:22</a>
wwoodslike package lists<a href="#t10:22" class="time">10:22</a>
f13really what would help more than anything is getting more and more people to test upgrades during the development cycle.<a href="#t10:23" class="time">10:23</a>
wwoodsunless we knew in advance what package sets to try, we wouldn't have hit this<a href="#t10:23" class="time">10:23</a>
* poelcat is here<a href="#t10:23" class="time">10:23</a>
f13if we had better utilities to say snapshot your system, attempt an upgrade, note failures, restore to old snapshot this would be much more compelling<a href="#t10:23" class="time">10:23</a>
f13We can do some of that with virt, but...<a href="#t10:24" class="time">10:24</a>
poelcatmether: we attempted to have a bug day... a few people stopped by<a href="#t10:24" class="time">10:24</a>
metherI think the release criteria should include testing without a network connection<a href="#t10:24" class="time">10:24</a>
f13the real value is in testing this stuff on people's day to day machines.<a href="#t10:24" class="time">10:24</a>
f13mether: feel free to add it.<a href="#t10:24" class="time">10:24</a>
wwoodsmether: that's reasonable.<a href="#t10:24" class="time">10:24</a>
metherpoelcat: yes we *attempted*. It didnt really happen<a href="#t10:24" class="time">10:24</a>
methernot blaming anyone. it just didnt<a href="#t10:24" class="time">10:24</a>
Lovechildf13: that sounds like a compelling idea, some kind of tool to build an image pre update would seem to be a good idea<a href="#t10:24" class="time">10:24</a>
f13Lovechild: we need such tools anyway, and the ability to restore to said image outside the system (shoudl the system become hosed) from like a rescue kit.<a href="#t10:25" class="time">10:25</a>
f13Lovechild: the OS X time machine stuff is extremely neat<a href="#t10:25" class="time">10:25</a>
Lovechildf13: could we build it into the rescue cd, it would be a simple boot - the only problem is really restoring the system.<a href="#t10:26" class="time">10:26</a>
* wwoods bought a mac mini this week, didn't get Leopard on it (!@%!@##&)<a href="#t10:26" class="time">10:26</a>
wwoodsthere's vestigial snapshot/rollback support stuff in yum and rpm<a href="#t10:27" class="time">10:27</a>
f13wwoods: take it back!<a href="#t10:27" class="time">10:27</a>
Lovechildso what we really need is a filesystem that natively supports that kind of snapshotting such as btrfs<a href="#t10:27" class="time">10:27</a>
f13wwoods: we want it outside rpm/yum<a href="#t10:27" class="time">10:27</a>
wwoodsf13: nah, I'll just give 'em $10 and make 'em send me a copy<a href="#t10:27" class="time">10:27</a>
f13wwoods: it needs to be general purpose so that it catches silly "I just deleted my home directory" stuff.<a href="#t10:27" class="time">10:27</a>
wwoodsand/or "borrow" a copy in the meantime<a href="#t10:27" class="time">10:27</a>
wwoodsf13: seriously? that's *our* problem?<a href="#t10:27" class="time">10:27</a>
f13Lovechild: LVM can easily do snapshots, regardless of file system.<a href="#t10:27" class="time">10:27</a>
f13wwoods: it's Fedora's problem.<a href="#t10:27" class="time">10:27</a>
Lovechildf13: that's not silly, that's tragic.. think of all porn<a href="#t10:28" class="time">10:28</a>
f13wwoods: we seriously need something like time machine, or XP's snapshot system.<a href="#t10:28" class="time">10:28</a>
metherLovechild: that might take a year or two if we are relying on that specific filesystem. LVM's seems to be a better answer to me<a href="#t10:28" class="time">10:28</a>
f13Lovechild: there is a Feature for RescueKit, a greatly improved rescue image.<a href="#t10:28" class="time">10:28</a>
wwoodsyes, it's a need, but it's not a QA problem<a href="#t10:28" class="time">10:28</a>
f13wwoods: sure, it would just help make QA more compelling if one could reliably recover from a failed attempt<a href="#t10:28" class="time">10:28</a>
wwoodsoh absolutely<a href="#t10:29" class="time">10:29</a>
Lovechildmether: lvm is appealing but it's definitely not without it's downfalls either the tools to use it are abominable<a href="#t10:29" class="time">10:29</a>
metherf13: btw what happened to the idea of dropping the boot.iso and just using rescue iso images for everything?<a href="#t10:29" class="time">10:29</a>
mkanatEven just doing, "Here's a list of packages installed, and then do this, and then try an upgrade" would probably work.<a href="#t10:29" class="time">10:29</a>
metherLovechild: idk. if you understand the concepts, the tools seem fairly logical to me<a href="#t10:30" class="time">10:30</a>
f13mether: that's in the Anaconda feature pages.<a href="#t10:30" class="time">10:30</a>
Lovechildalso lvm has some known performance issues on raid setups, cutting my throughput in half thanks to a bolt on solution<a href="#t10:30" class="time">10:30</a>
metheranyway back to topic<a href="#t10:30" class="time">10:30</a>
metheris there anything specific to discuss?<a href="#t10:30" class="time">10:30</a>
f13Lovechild: to be fair, system-config-lvm has improved over time.<a href="#t10:30" class="time">10:30</a>
wwoodsmkanat: right<a href="#t10:30" class="time">10:30</a>
Lovechildmether: we are talking about a time machine like things here.. that actual users can operate, lvm is far to complex currently it needs serious sugar<a href="#t10:30" class="time">10:30</a>
f13bugzilla change is upcoming, I really think somebody needs to make more noise about this.<a href="#t10:30" class="time">10:30</a>
poelcatf13 noise about what?<a href="#t10:31" class="time">10:31</a>
rdieteroh?<a href="#t10:31" class="time">10:31</a>
wwoodswhich bugzilla change?<a href="#t10:31" class="time">10:31</a>
metherLovechild: The Apple underlying technologies are similar too. If you think we need better gui's, I wont argue with that<a href="#t10:31" class="time">10:31</a>
f13wwoods: devel -> rawhide<a href="#t10:32" class="time">10:32</a>
f13and removing the test releases.<a href="#t10:32" class="time">10:32</a>
metherLovechild: any idea what was the live cd installation problem linux action show kept cribbing about ?<a href="#t10:32" class="time">10:32</a>
wwoodsf13: ah. yes.<a href="#t10:32" class="time">10:32</a>
f13mether: Apple also doesn't really give you much of a choice on using a filesystem that their tools wouldn't support.<a href="#t10:32" class="time">10:32</a>
wwoodsI'll make some noise on test-list<a href="#t10:32" class="time">10:32</a>
Lovechildmether: I'm just saying before we have said tools in a shape users can operate and that integrates with the system. No users are realistically using it so defacto we don't have that capability<a href="#t10:32" class="time">10:32</a>
f13mether: We have far too many choices that makes trying to create easy to use tools a nightmare as they would only work part of the time.<a href="#t10:32" class="time">10:32</a>
metherf13: I suggested long back to drop the rest of the filesystems we kind of allow<a href="#t10:33" class="time">10:33</a>
wwoodsf13: they used to let you install to UFS<a href="#t10:34" class="time">10:34</a>
Lovechildmether: They didn't give enough information to actually triage the problem nor even exclude user error.. attempts to get more information is so far unsuccessful<a href="#t10:34" class="time">10:34</a>
wwoodsexcept if you did, it totally screwed your system<a href="#t10:34" class="time">10:34</a>
wwoodsahh, 10.0<a href="#t10:34" class="time">10:34</a>
wwoodsfun times<a href="#t10:34" class="time">10:34</a>
metherLovechild: I send them a mail<a href="#t10:35" class="time">10:35</a>
wwoodsthere are other bugzilla changes (3.0 wooo) but those are a while off<a href="#t10:35" class="time">10:35</a>
metherLovechild: err, I will send them<a href="#t10:35" class="time">10:35</a>
wwoodsoh! if there are bugzilla interfaces (XMLRPC etc) that you need, *please* send me an email<a href="#t10:35" class="time">10:35</a>
wwoodsso I can let dkl know what interfaces we use, so we can be sure those get ported<a href="#t10:35" class="time">10:35</a>
Lovechildmether: well maybe they are just ignoring me so maybe if you tried<a href="#t10:36" class="time">10:36</a>
mkanatwwoods: I was just reading the spec for all that, the other day. :-)<a href="#t10:37" class="time">10:37</a>
metherLovechild: live cd installations have all worked successfully for me. So I was just wondering if there was a problem that commonly occured that I didnt know about.<a href="#t10:37" class="time">10:37</a>
mkanatwwoods: I was chatting with dkl about it since I'm in charge of the WebService, upstream.<a href="#t10:37" class="time">10:37</a>
wwoodsmkanat: oh excellent!<a href="#t10:37" class="time">10:37</a>
mkanatWell, kind of in charge of it.<a href="#t10:37" class="time">10:37</a>
wwoods"in charge" like "you were the last one to touch it! it's yours now!"<a href="#t10:37" class="time">10:37</a>
wwoods?<a href="#t10:37" class="time">10:37</a>
mkanatwwoods: LOL. Well, like I wrote a lot of it so it's mine by default.<a href="#t10:38" class="time">10:38</a>
mkanatwwoods: A lot of Bugzilla is that way, for me. :-D<a href="#t10:39" class="time">10:39</a>
wwoodsI know how that goes. Well, awesome. Glad to know you'll be involved in that process.<a href="#t10:39" class="time">10:39</a>
mkanatwwoods: Indeed. :-) You can feel free to ask me about things or suggest things to me, also.<a href="#t10:40" class="time">10:40</a>
wwoodsBut yeah, I know FESCo and the pkgdb and such use some xmlrpc stuff, so I'll need to mention this to them as well<a href="#t10:40" class="time">10:40</a>
mkanatAny Fedora QA person is free to ask me about Bugzilla, as long as it's not like, some hours-long explanation thing. :-)<a href="#t10:40" class="time">10:40</a>
f13I don't know what xmlrpc calls I need to make, I just know tasks I wish to complete<a href="#t10:40" class="time">10:40</a>
wwoodsf13: right, well, you're OK then. I just need to make sure that the interfaces that some of the tools use stick around<a href="#t10:41" class="time">10:41</a>
f13like package reassignment, that would do a bugzilla query, find all instances of bugs of a package assigned ot the previous owner, to reassign them to the new owner.<a href="#t10:41" class="time">10:41</a>
wwoods(or the tools get change right quick)<a href="#t10:41" class="time">10:41</a>
wwoodsI'm trying to write a wiki page about using Bugzilla in Fedora<a href="#t10:41" class="time">10:41</a>
wwoodsthere's some stuff that we haven't written down that people should know<a href="#t10:42" class="time">10:42</a>
mkanatwwoods: You could link that from the New Bug page, too.<a href="#t10:42" class="time">10:42</a>
wwoodse.g. we ignore pri/sev, we only pay attention to a few bug states, etc.<a href="#t10:42" class="time">10:42</a>
wwoodsoh good call.<a href="#t10:42" class="time">10:42</a>
mkanatwwoods: It's just [% IF product = 'Fedora' %] I think, in 2.18.<a href="#t10:42" class="time">10:42</a>
mkanatIn 3.0 it'd probably be [% IF = 'Fedora' %] .<a href="#t10:43" class="time">10:43</a>
wwoodsVery cool. I'll try to get that set up once we have a page to refer to<a href="#t10:43" class="time">10:43</a>
wwoodsgoing back to the rollback discussion for a second<a href="#t10:44" class="time">10:44</a>
wwoodsfor the F9 cycle I want to focus more on helping people use the tools we've got<a href="#t10:44" class="time">10:44</a>
wwoodsrather than making new tools<a href="#t10:44" class="time">10:44</a>
wwoodssomething that does full filesystem rollbacks is a really good example - we already have lvm support for that, just need a good (or merely acceptable) interface<a href="#t10:45" class="time">10:45</a>
wwoodssame for, like, a tester tool that shows what you've installed recently, lets you roll back to the stable/previous version, etc<a href="#t10:46" class="time">10:46</a>
wwoodson a package level<a href="#t10:46" class="time">10:46</a>
wwoodswe already have support for all that stuff in the tools (bodhi, yum, etc) but it needs an interface and docs<a href="#t10:46" class="time">10:46</a>
wwoodsso I expect to spend most of the next six months working on docs on how we do stuff<a href="#t10:46" class="time">10:46</a>
wwoodsand tools that improve those processes<a href="#t10:46" class="time">10:46</a>
wwoodsstuff like SNAKE and Beaker and Apport will have to wait (or get picked up by other interested parties)<a href="#t10:47" class="time">10:47</a>
wwoodsso there's the Grand Vision. what that means in concrete terms is probably something like: more bug days and working on stuff we've got<a href="#t10:48" class="time">10:48</a>
wwoodsand if we have time, making little GUIs for the things we do all the time anyway<a href="#t10:48" class="time">10:48</a>
wwoodslike yum upgrades and downgrading packages and showing what the common bugs are<a href="#t10:48" class="time">10:48</a>
wwoodsokay, enough vague philosophy talk<a href="#t10:49" class="time">10:49</a>
wwoodsyawn city. getting near the end of the hour.<a href="#t10:49" class="time">10:49</a>
LovechildI'd ve very sad to see apport fall to the wayside, sadly I'm not capable of picking it up<a href="#t10:49" class="time">10:49</a>
wwoodsquick recap: I'm mailing test-list about a Bug Day on Dec. 3 and upcoming changes to bugzilla (devel->rawhide, removing test releases)<a href="#t10:50" class="time">10:50</a>
wwoodsLovechild: yeah, although I think there's a lot of potential for someone to merge apport + packagekit + bugbuddy<a href="#t10:50" class="time">10:50</a>
wwoodscombine that with standardized bugzilla 3.0 web services and you'd have a sweet tool that works for everyone evverywhere<a href="#t10:50" class="time">10:50</a>
Lovechildwwoods: oh please don't end the meeting like that, now my floor will be all covered in drool<a href="#t10:50" class="time">10:50</a>
wwoodswithout so much distro-specific cruft stuffed into it<a href="#t10:51" class="time">10:51</a>
wwoodsbasically I'd like to see bugbuddy use PK to get debuginfo, and use some of the ideas from apport about bug retracing and making hashes of tracebacks to auto-find dupes<a href="#t10:52" class="time">10:52</a>
wwoodsso if someone wants to talk to the bug-buddy maintainer and drive that project, I'd be happy to provide advice<a href="#t10:52" class="time">10:52</a>
wwoodsbut mostly we need to work on getting more people testing updates and rawhide<a href="#t10:53" class="time">10:53</a>
wwoodsand triaging bugs<a href="#t10:53" class="time">10:53</a>
wwoodsanyway. sorry. oh, qa-list is basically closed<a href="#t10:53" class="time">10:53</a>
wwoods(it's strange that mailman doesn't have a "close list temporarily" checkbox)<a href="#t10:54" class="time">10:54</a>
wwoodsyou can use bodhi's RSS feeds to track new package updates<a href="#t10:54" class="time">10:54</a>
lmackentest notifications will go away soon<a href="#t10:54" class="time">10:54</a>
lmackeni'm almost done with the captcha code to allow anonymous feedback<a href="#t10:54" class="time">10:54</a>
wwoodscool!<a href="#t10:54" class="time">10:54</a>
wwoodsI'm working on adding info about bodhi to the QA docs<a href="#t10:55" class="time">10:55</a>
wwoodsmether: are you going to track that firstboot bug?<a href="#t10:55" class="time">10:55</a>
mkanatwwoods: I think bug-buddy already supports standard Bugzilla webservices, too.<a href="#t10:55" class="time">10:55</a>
mkanatwwoods: I know that GNOME was working on it, at least.<a href="#t10:55" class="time">10:55</a>
wwoodsmkanat: niiiice.<a href="#t10:55" class="time">10:55</a>
wwoodsso yeah, teach it to install debuginfo packages (where applicable) and we'd be most of the way here<a href="#t10:56" class="time">10:56</a>
mkanatwwoods: Yeah, GNOME is totally interested in that, too.<a href="#t10:56" class="time">10:56</a>
mkanatwwoods: They had some discussion about it, maybe you could grab somebody from there about it.<a href="#t10:56" class="time">10:56</a>
mkanatwwoods: Or I could grab somebody for you.<a href="#t10:56" class="time">10:56</a>
wwoodsthat would be very cool<a href="#t10:56" class="time">10:56</a>
wwoodsin theory it's totally dead simple with BuildIDs in Fedora<a href="#t10:57" class="time">10:57</a>
mkanatwwoods: Cool. I've pinged Olav Vitters, we'll see if he's around.<a href="#t10:57" class="time">10:57</a>
wwoodsyou can just look at a header in the core and it'll tell you all the debuginfo you need<a href="#t10:57" class="time">10:57</a>
wwoodsif we set up the debuginfo building properly you'd just have to do yum install [list of buildids straight from corefile] <a href="#t10:58" class="time">10:58</a>
wwoodsbut yeah, distro-specific hooks for installing debuginfo would be completely grand<a href="#t10:59" class="time">10:59</a>
poelcatwwoods: is this in the plans for F9? <a href=""></a><a href="#t10:59" class="time">10:59</a>
wwoodspoelcat: no<a href="#t10:59" class="time">10:59</a>
poelcatwwoods: i'll remove the target release if you like :)<a href="#t10:59" class="time">10:59</a>
wwoodsUnless someone else wants to own it<a href="#t10:59" class="time">10:59</a>
wwoodsBasically I think the ideas in Apport are sound, but it's something that should go into upstream tools<a href="#t11:01" class="time">11:01</a>
wwoodsthe current implementation is very ubuntu-y. very brown.<a href="#t11:01" class="time">11:01</a>
wwoodsokay. is there anything else people wanted to discuss?<a href="#t11:02" class="time">11:02</a>
wwoodslet's consider this meeting over, then. thanks for your time, everyone<a href="#t11:03" class="time">11:03</a>
wwoodsBug Day Monday! Don't forget!<a href="#t11:04" class="time">11:04</a>
wwoodsooh maybe we should do pulseaudio bugs<a href="#t11:04" class="time">11:04</a>
* joejaxx is happy he was able to attend one :D<a href="#t11:04" class="time">11:04</a>
joejaxxpartially*<a href="#t11:04" class="time">11:04</a>
wwoodsjoejaxx: we're glad to have ya!<a href="#t11:04" class="time">11:04</a>
wwoodsheh<a href="#t11:04" class="time">11:04</a>
joejaxx:)<a href="#t11:04" class="time">11:04</a>

Generated by 2.3 by Marius Gedminas - find it at!