FWN/Beats/Developments

New RPM Sparks Exploded Source Debate

The announcement[1] of a sparkling new alpha-version of RPM by PanuMatilainen was greeted with congratulations and applause and later some passionate argument. It has been approximately one year since Panu solicited (see FWN#98 "Panu Opens Pandora's Box"[2] and FWN#99 "RPM Roadmap (Cont.)"[3]) suggestions from those not intimately involved in RPM development as to which problems should be fixed. This initiative was taken after deciding to move RPM-4.4 to bugfix maintenance due to artistic differences with the current RPM coder (who then led a very public fork named RPM5[4]). Panu's new RPM-4.6.0 implements many of those suggestions as detailed[5] in the release notes and many of those involved in the initial roadmap process (such as RalfCorsepius who cleaned[6] up the autotool stuff also helped to implement the desired changes. By Panu's estimate over 2300 commits were made[7] to the source since the initiative to get RPM development back on schedule began, and although the wiki provides[8] essential details of what has been implemented there is still a good deal of information lacking.

[1] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00477.html

[2]http://fedoraproject.org/wiki/FWN/Issue98#RPM.Roadmap.....Panu.Opens.Pandora.27s.Box

[3] http://fedoraproject.org/wiki/FWN/Issue99#RPM.Roadmap..28Cont..29

[4] http://rpm5.org/

[5] http://wiki.rpm.org/Releases/4.5.90

[6] http://www.mail-archive.com/rpm-maint@lists.rpm.org/msg00386.html

[7] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00538.html

[8] http://wiki.rpm.org/Releases/4.5.90

The announcement contained suggestions for users (of Rawhide where the alpha is available) about how to trouble-proof themselves and a more extensive list of notes for packagers. Of note are the changes to the macros to eliminate the old buildroot directory defaults and ignore[9] the BuildRoot in an rpm's spec file, and the addition of support for LZMA[10] compression. Another cool new feature is the addition of a macro to allow iteration over all patches, something which was welcomed[11] by JarodWilson, who noted that RHEL5 needed 1800 lines in the kernel specfile solely to mention each patch.

[9] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00531.html

[10] http://en.wikipedia.org/wiki/LZMA

[11] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00502.html

It seems that a massive amount of work has gone into API changes and internal cleanup of the code in order to set up a framework for the addition of new features in the future.

ThorstenLeemhuis expressed[12] happiness with Panu's contribution but wondered whether the FESCo Feature process[13] had been shown to be unnecessarily bureaucratic by the manner in which this change had occurred. JoshBoyer and JeffSpaleta drew[14] a slightly different lesson and suggested that it ought to be made easier for a developer to determine whether their package upgrade should be filed as a feature. Panu also agreed[15] that Thorsten's points were fair but excused himself on the grounds of concentrating on upstream RPM development and not being sure what the demaraction between feature and non-feature was.

[12] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00549.html

[13] https://fedoraproject.org/wiki/Features/Policy

[14] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00562.html

[15] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00565.html

PaulFrields suggested[16] that it might be useful to think of the "Fedora feature process as leveraging what Fedora can provide for an upstream community. Two things that come to mind immediately are QA/testing and widespread publicizing of the feature." JohnPoelstra also drew attention[17] to the synergistic advantages of the Features process resulting from its public communication of what is being worked on currently. CallumLerwick, responding to Thorsten, gave his understanding of the Features process as "a conduit for the Engineering side of Fedora to collaborate with the Marketing side of Fedora, to allow the Marketing people to build up pre-release hype for new features without having to second-guess us notoriously busy, and quiet, engineering types. It allows the Marketing people to keep tabs on engineering activities and have reasonable certainty as to the status of the feature, specifically whether or not it is going to be finished in time for the final release." He emphasized the voluntary participation of developers and software engineers in the process and the benefit resulting from having marketing clued-in to interesting changes. JesseKeating responded[18] that the process was "way more than just marketing fluff. Features have very real schedule impact, just consider this time around, RPM with a bunch of new features, and a new gcc coming at some point soon. Usually we want to rebuild for both of those. Without some high level coordination, how do we schedule so that we rebuild once for all of the right reasons instead of multiple times individually?" The marketing advantages of the Feature process were confirmed[19] by PaulFrields. MatthiasClasen made[20] some concrete suggestions on how to improve the Feature process. They included the addition of definitions or explanations for each section and the preception that the review of his feature pages felt a bit like getting homework graded. [16] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00580.html

[17] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00581.html

[18] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00626.html

[19] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00582.html

[20] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00645.html

The point about co-ordination of activities was highlighted when DougLedford confessed[21] that his first reaction had been "Oh hell...what a colossal waste of time" when he realized that he had spent a week studying what was now obsolete RPM source code. Panu's friendly response that Doug "could've just asked" drew out the central problem: "Yeah, I know, I just didn't know a big update like this was in the works." Doug's interest lay[22] in extending rpmdb to add fields to allow interaction with SCMs[23] mostly "to be able to support exploded source repos and usage of exploded source repos as canonical source versions of binary packages." Panu answered[24] that these sorts of changes were probably post-Fedora 10 and that he too was keen to integrate with SCM tools. He pleaded for some more patience to settle this clean, new codebase down before implementing such changes: "I know. People have been waiting SO long for various things to happen in RPM that everybody's out of patience and wants their stuff in NOW. Please try to be patient a little bit longer: once this release stabilizes, RPM can move to a "normal" development-release cycle where folks will not have to wait 5+ years to get their changes in :)" SethVidal and ToshioKuratomi were impatient with Doug's impatience[25,26] with the latter noting that Fedora Policy "to allow using source control repos interchangeably with tarballs would [not] be approved in time for F10 either."

[21] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00683.html

[22] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00694.html

[23] http://en.wikipedia.org/wiki/Software.cono/guration.management

[24] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00696.html

[25] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00702.html

[26] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00699.html

Later Panu requested[27] that packagers "refrain from using the new spec features in Fedora to minimize the fuss in case disaster strikes and we need to go back to rpm 4.4.x. The new rpm is on probation for a while ;) Please do test and use the new things as much as possible in private, just not yet in Fedora CVS. A further notification will be sent when the probation is over."

[27] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00703.html

Doug expended a good deal of effort both trying to get an answer as to whether there was any point in him trying to go ahead and add some of the basic features which he thought were necessary, and also explaining why the ability to interact directly with a distributed SCM/VCS instead of through the middle-man of a tarball was a good idea. Among the advantages Doug described were "since you built the binary packages from this exploded source repo, then in order to give people the exact sources you built from, you need to make the repo available for clone/checkout by people. You need never once build an srpm or tarball from this repo if you don't want to [...] the first advantage to this type of setup is that every SCM worth a pile of dog poo will store the different versions of software in some form of change related format that keeps you from duplicating the same things over and over again like tarball after tarball does. You generally take a hit in size versus a single tarball, but end up saving quite a lot in the long run [...] you get to work on the code in native format, try things out, run build tests, and all the while the pain of repetitive rpm source processing is reduced[...]" Doug went on to explain that in the case where the upstream also uses a distributed SCM then things become even easier. He attached his notes (in tomboy format) with yet more detail.

Doug was obviously brimming with ideas about how this would make Fedora development easier and reacted[28] with a certain amount of frustration to Panu's and Seth's assumption that he was asking them to do something which they could not get around to until Fedora 11. Doug also pointed[29] out that the problem of forcing the creators of spins to distribute their own sources was also possibly solved by using distributed SCMs and that he had discussed this with JesseKeating at the recent FUDCon. His perception was that the Fedora Project was actively blocking Red Hat's needs. JesseKeating later returned[30] to the problem of compliance with the distribution requirements of the GPL: I either have to offer you a CD/DVD of corresponding sources in <insert vague nonlegal terms here> format, or provide you a written offer to provide the above that is good for the next 3 years, or pass along such written offer that I myself may have gotten. Nobody has confirmed nor denied what that <vague nonlegal terms here> means, nor how long the 3year clock ticks on those formats, and whether or not directions on how to get the source from our public source repo is OK.

[28] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00729.htm

[29] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00776.html

[30] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00793.html

This has been an inadequate summary of a complex topic. If you are interested in it you are well advised to read the thread especially Doug's posts and the responses to them. They start here[31].

[31] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00683.html

Fedora EDU Spin Preview Temporarily Pulled

A spin targeted at educational environments, "Fedora EDU"[1], was announced [2] as available in preview by SebastianDzillias. He explained that its focus was on mathematical applications with a KDE-4.1 desktop environment and that it was currently x86 only.

[1] http://fedoraproject.org/wiki/SIGs/Education/Roadmap

[2] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00628.html

JesseKeating was quick to point out[3] that the use of the KDE-4.1 preview provided by the "kde-redhat" repositories meant that the spin could not use the "Fedora" trademarks. JoshBoyer amplified[4] on this with the information that "you need to get Board/Spin SIG/Rel Eng approval to call a spin a Fedora spin" and that there was no need to use the kde-redhat repositories as Rawhide already had the KDE preview packaged up. ChristopherAillon also noted the existence of the SpinSubmissionProcess and LukeMacken wondered [5] whether anyone wanted to help out in creating a "fedora-spins" mailing list to help unblocked that process.

[3] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00629.html

[4] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00632.html

[5] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00651.html

Further discussion between RexDieter and JoshBoyer revealed[6] that the spin was also based on Fedora 9, which led Josh to suggest that new spins should dovetail into the release process and specifically that this one should be based off Fedora 10. Rex agreed that this had been the plan, but that this was simply a preview to obtain feedback. JeffSpaleta thought[7] that "Preview binaries are great, because it shows that these particular Spin developers are making their best effort to get this working and tested[.]" He asked for confirmation that there was no actual policy preventing such preview spins being built against the current (as opposed to rawhide) release and suggested that the main problems were "they pushed ahead and used non-fedora binaries in what they published [and] [w]e don't want anyone out in the wild to get the idea that this is a baked concept. The generic logos are there specifically so we can do preview spins like this." Jose'Matos thought[8] that the question should be extended beyond spins to consider the general case of "non-official" repositories such as the TEXLive, Python-2.5 and other repositories. Although he was aware of the inherent possibility of diluting rawhide testing due to such repositories "I would have expected that by now we had some kind of mechanism to deal with such cases other than the non-official stance of every of those repositories." CharlesDostale thought[9] that an ultra-rawhide as hinted at by Jose' would be interesting.

[6] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00637.html

[7] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00644.html

[8] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00650.html

[9] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00690.html

JeroenvanMeeuwen agreed[10] with Jeff that there was no current hard policy against working off the current release and decided to "propose to the spin sig to have the spin-kickstarts master branch use generic-logos (master branch is for development so basically anyone can do anything there)." RexDieter took responsibility and announced on his blog[11] that the spin was pulled until the aforementioned problems had been resolved.

[10] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00653.html

[11] http://rdieter.livejournal.com/2008/07/10/

Later Jeroen posted[12] an explanation and request for how those wishing to produce official Fedora spins should proceed. An exchange[13] between JesseKeating and RahulSundaram focused on the question of whether FESCo should, as the designated body, be the one to decide whether the "desktop" and KDE variants were not spins. Jesse argued that because they were produced as part of the distribution they should be treated as "in essence the non-contrib part" and thus not to be treated as spins. Jesse argued strongly that it was "ill advised [for FESCo to have voted that spins are not features and] Releng and the Spins SIG want them to be features, and I'll use my powers in FESCo now and the board as well if necessary to push that agenda." He returned to the theme that Features were essential to the process of co-ordinating the production of a release. Rahul agreed[14] with the logic of Jesse's argument but disagreed with the over-riding of FESCo's decision "[..] if FESCo makes a decision, it should be the same group reversing it instead of any of us arbitrarily deciding otherwise. There is no point in FESCo making such decisions otherwise." JeffSpaleta also appeared[15] to believe that the decision-making process was slightly off-kilter.

[12] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00686.html

[13] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00720.html

[14] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00724.html

[15] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00721.html

Fedora, Meet OLPC. OLPC, Meet Fedora

GregdeKoenigsberg asked[1] Fedora packagers to help out the OLPC project by taking up the reins as far as package maintenance goes. He noted that contrary to some press reports the OLPC project has not died and quoted some surprising statistics: "OLPC has shipped over 300,000 units to kids around the world. They plan to ship at least another 50,000 more each month, and very likely more than that. It's entirely possible that by the end of 2008, there will be a million OLPC systems deployed worldwide. Of those systems, 100% of them currently run Fedora, and 0% of them currently run Windows despite the press clippings you may have read." Greg argued that this made OLPC Fedora's single largest customer and that the community was exceptionally well placed to help this continue. Some of the tasks were "simple issues that even novice packagers could handle." JeffSpaleta suggested that a "Sugar Desktop Spin" for standard PC hardware with a SIG to help organize around would improve efficiency.

[1] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00433.html

DebarshiRay wondered[2] "Can someone running vanilla Fedora (8/9/etc.) without any physical access to the XO hardware maintain/use OLPC packages?" and DennisGilmore answered[3] unequivocally "Yes, there is only a tiny handful of packages that are specific to the hardware/setup of the XO. the rest should always be applicable for use outside of the XO, sugar is in F-9 there is still some kinks in regards to the packaging that needs fixed. but if it doesnt work right on a normal fedora desktop its a bug and needs fixing." ChristopherAillon pointed out[4] that plenty of packagers maintain their packages for architecturs, such as PPC, for which they do not have physical hardware and that XO was no different.

[2] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00461.html

[3] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00462.html

[4] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00463.html

ColinWalters thought[5] that simple manual fixing of problems missed the opportunity to automate the process where appropriate. DanielDrake raised[6] a related point, based on his experience of working on upgrading the OLPC from Fedora 7 to Fedora 9, which was that there appeared to be ever increasing bloat as a result of dependency chains. Due to the limited space on the XO Daniel requested help in slimming things down somehow. MatthiasClasen remembered[7] that when he had been involved with OLPC it had been necessary to do a lot of "dependency pruning." He advised that the best course of action was to "keep fighting this by filing bugs and pointing out package split candidates, since these deps have the tendency to grow back." RahulSundaram noted[8] that the need to produce LiveCDs helped combat the bloat tendency and RichardJones separately mentioned[9] oVirt[10] in the same context.

[5] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00439.html

[6] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00490.html

[7] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00503.html

[8] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00512.html

[9] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00529.html

[10] http://ovirt.org/

After MatthiasClasen told Daniel that gvfs would possibly be split-up for Fedora 10, but that it was unlikely without a fork for Fedora 9, DennisGilmore added that he had been working on always tracking Rawhide for the olpc and that a Fedora 10-based build was possible. DanielBerrange spoke for the oVirt project when he expressed[11] a desire for tools to produce nightly reports on statistics such as the "disk footprint of the chain starting from package 'X', or list of dependancies from package 'X', or perhaps something that given a kickstart o/le can report the total size of the package set listed in the kickstart without actually going through the full livecd (or equiv) build process." SethVidal offered to take care of this and Daniel added[12] some further desiderata. Shortly afterwards Seth whipped-up an implementation to which JeremyKatz commented "The thing which becomes important to see is growth (or shrinkage) in packages as well as what new packages/removed packages there are. Which involves fiddly questions of growth thresholds and human analysis of the output." DavidTimms was excited[13] by the output, including what it appeared to reveal about the minimal set of install packages. Seth ended up modifying the output to a simple format which allowed[14] the use of standard UNIX text-processing commands to do fun things with the output.

[11] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00519.html

[12] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00525.html

[13] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00649.html

[14] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00685.html

Getting Rid of pam.console for Fedora 10 ?

Currently Fedora sports both pam.console and HAL-based ACL support and BillNottingham posted[1] that it was "time to cut the cord and remove pam.console, so we only have one way of setting device permissions to worry about." He attached a list of affected packages.

[1] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00555.html

ChrisAdams wondered[2] how he would implement HAL-based ACLs for his serial ports to access other consoles. Following Bill's request for an lshal output and a pointer to an example HAL policy Chris's own stab at producing the policy seemed to pass muster. He added "I have another system where I have multiple USB-to-RS232 adapters; one is used for outbound terminal sessions (console user gets access) and one for a modem (no console access). I dioeerentiate between the two with a udev rule that adds a symlink (e.g. "term" and "modem") and then set the permissions with a pam.console match on the symlink. Is it possible to match something set from udev like that (so I don't have two places to keep track of hardare serial numbers and such for matching)?" Bill's reply suggested examining /usr/share/hal/fdi/information/10freedesktop/10-usb-pda.fdi and /usr/share/hal/fdi/policy/10osvendor/20-acl-management.fdi in order to see respectively how varying information in HAL is handled and then ACL management is applied. This led a happy JeffSpaleta to exclaim[3] "that was the first explanation of how to do this sort of thing on how to generate new hardwar access control rules that I've actually followed."

[2] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00567.html

[3] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00570.html

DavidZeuthen corrected[4] Bill's description of the surgery to "the plan is actually to move this to ConsoleKit (HAL is going away and all that etc. etc.) but that's most likely F11 material. So suggest to hold ooe this feature for now." This might reassure DmitryButskoy who commented[5] that pam.console's "auth" features were useful.

[4] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00621.html

[5] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00608.html

New PackageKit and GNOME-packagekit in Fedora 9

RichardHughes drew attention[1] to the availability of API-breaking update of PackageKit and gnome-packagekit in the Fedora 9 "updates-testing" repository. He requested testing and bug reporting via email.

[1] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00304.html

Richard noted that speed-wise there would be a major improvement coming much later based upon his work on profiling yum and working around "slow paths in the API [...] For instance, the group list used to take 14 seconds on my machine, and now completes in less than a tenth of a second using master." SethVidal cautioned[2] against Richard's approach of accessing the SQLite databases directly instead of going through YUM's layers and suggested that instead "I've implemented a searchNames() method to pkgSack in yum which will let you search very quickly for multiple package names."

[2] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00305.html

Most of the other responses reported no significant issues after several days of testing, except that MartinSourada found[3] some non-intuitive behavior when installing local rpms amd a potential security hole.

[3] https://www.redhat.com/archives/fedora-devel-list/2008-July/msg00309.html