Testing
From FedoraProject
Fedora Testing Guide
Preface
Let us take a quick moment to describe to you the goals of this document. This document is here to provide you, the novice but enthusiastic Fedora user, with recommendations on how you can become an important part of the development process of Fedora as a tester. We are not going to lie to you, stepping up to become a part of the development process can be frustrating for some people, especially if they are unprepared for the experience. The key to having a worthwhile experience is starting off with the right expectations and a minimal set of tools you can use to do commonly needed troubleshooting. This document is our attempt to provide both.
Join the Bug Triaging Team
Fedora Proposed Updates
- Fedora updates-testing repository contains updates scheduled to be released for the maintained releases of Fedora. Testing them and providing feedback in the fedora-test mailing list and the relevant bugzilla reports helps developers to fix any potential regressions in them. To test this, enable updates-testing repository in /etc/yum.repos.d and run 'yum update' to get them. A dedicated test system is more useful for these checks. If you want to test specific packages or enable the testing repository on the fly, you can do the following:
yum update --enablerepo=updates-testing
yum install <foo> --enablerepo=updates-testing
Fedora Development Repository (codename: Rawhide)
- The Fedora Development branch (named
Rawhide) is an ongoing daily release of new development updates in Fedora. Its purpose is to act as the foundation for Fedora's progress. It often contains bleeding-edge and potentially unstable software. It is intended for developers and testers only, as any package can easily break at any time and without warning. The best way to test it is to have a dedicated partition or system to do installs on. More information on rawhide is available at http://fedoraproject.org/wiki/Releases/Rawhide
- Periodically, there will also be Fedora 'test' releases made. If you are not interested in testing Rawhide, test releases are slightly more stable (they are at least tested to install successfully on a few machines). There are typically three test releases before every final relese of Fedora.
Qemu
If you want to use Qemu emulator for testing, look at qemu page
Fedora Release Schedule
Feedback and Communication
Once you have started using the development packages, it's important to give developers feedback on how they are doing. The development tree has constant changes that might affect the way applications behave in your system. It is important that developers understand what works and what doesn't in order to improve Fedora. Do not assume that a problem is known.
- Make sure you are subscribed to the fedora-test and fedora-devel mailing lists for receiving daily rawhide reports about changes in the development packages and other related announcements and discussions. Feel free to post in the fedora-test and fedora-devel lists if you need any help or to initiate discussions about recent development changes or bugs. See the Communicating and getting help page for more information. If you are new to the list it would also be prudent to read the recent archives.
- Report bugs and suggest enhancements to the Fedora Bugzilla. Use the BugsAndFeatureRequests guidelines to report them effectively
Initial Expectations
- A dedicated test system
- Interest in Fedora development
Here are some important initial expectations I think every testers should be aware of before participating in the development process. If these expectations don't suit you, you will most likely have a more frustrating experience later on in the process. As much as testing efforts are appreciated, no one wants to make the process unduly frustrating, so its important to come into the process with your head calibrated with what to expect.
- Be prepared to do a fresh install of the final release. By installing a test release you are essentially agreeing to doing a fresh install of a stable release when you are ready to stop participating in the development process. Please remember that the Fedora development process is an on-going affair and the test releases are just more easily digestable versions of the current development tree. Once you install the test release you are essentially running a version of the development tree. Once you start installing updates you are syncing with the development tree. Moving off the development tree to any stable release could be considered downgrading in a sense.
- Be prepared to file bug reports and follow up as needed with developers. The bulk of communication with developers happens through bugzilla. The fedora-test-list mailinglist is great for discussion between testers for things like bug confirmation or help trying to troubleshoot a bug you don't have enough experience with. But at the end of the day, important issues must get filed in bugzilla to make sure the right developer sees the issue in time to do something about it for the final release. Other people in the community can help you file a useful bugreport, but you must be prepared to file. That means at a minimum registering a bugzilla account at bugzilla.redhat.com and making some effort to get use to the bugzilla interface.
- Be prepared to see daily package update failures. Test releases get updates directly from the development tree. The development tree is not guaranteed to be self-consistent every day. You will see somewhat randomly occuring update problems simply because of how the set of development packages are built. Its impossible to rebuild all the development packages overnight. So daily package updates will result in dependency problems on occasion, which the developers fix in one or two days simply by requesting further package rebuilds.
There is a daily build report of the development tree sent to the fedora-test-list every morning as part of the automated push of packages out to the publicly accessible trees. The daily report contains information about new,removed and updated packages. It also contains a summary of known dependency problems for each arch for which the development tree is built for. Please, if you experience any problem updating against the development tree the first thing you should review is the last 2 or 3 build reports. If you are seeing a dependency problem summarized in the latest build report, you can be sure the developers are aware of the problem. Below I give you some helpful troubleshooting tips to pinpoint exactly whats going on when you see unexpected problems.
- Approach the test release as a valuable chance to learn more about your system. There is a good chance you will run into some bugs in subsystems or components that you are very unfamiliar with as part of the testing process. Use this an opportunity to learn more about that particular subsystem and get familiar with its documentation. Even documentation has bugs, by following up and trying to learn from the documentation you might be able to help clean up badly worded or out of date documentation as well. The more you learn, the more effective you will be in the future if you participate in the development process again. Be as proactive as you can about reading up on how things work and you will have a much more valuable experience overall.
Successful strategies for keeping up with package updates
The test releases receive their updates from the development tree, affectionately known as rawhide.
raw: because the packages there are untested and freshly built
hide: because its your hide that's going to get burned if there is a problem with them.
Once you have lived through one testing cycle, you can probably deal with rawhide's more characteristic characteristics without much drama. But if you are new to the testing process, you can get yourself into trouble very quickly. Even worse, you can get in trouble in a way that makes it difficult for other testers to help you track down enough information after the fact to file a useful bugreport. Please remember the goal is to get issues identified for the developers to fix. Inexperienced tester needs to take more care to keep notes and avoid using shortcuts when updating or installing packages. Here are my personal suggestions as to what you can do to make it easier to diagnose problems seen after package updates.
- when using yum take the time to review the list of package actions before you proceed, don't disable the review step
- get familiar with /var/log/rpmpkgs log file and the /var/log/yum.log file.
- get a notebook and make notes about system configuration changes you make. Many problems can be traced to simple configuration errors.. but can appear as package update bugs. When working with other testers to confirm the problem, notes as to the other changes you have made since last update/reboot can be invaluable in tracing the problem down accurately.
- keep at least one older kernel around that you are confident works as expected
- reboot daily, to test to see if any of your updates have affected startup. Its much more difficult to track down a boot up problem that was caused by a week+ old update, if you are updating daily but have not rebooted.
- get familiar with useful grub features for troubleshooting boot up failures. covered below in tips and tricks
- don't force or nodeps any package to work around dependency problems. These sorts of shortcuts make it very difficult for other testers to compare their system behavior to yours.
Commonly experienced problems
yum update failures
Because the development tree is not guaranteed to be self-consistent everyday, you will frequently see yum update fail with errors. Don't Panic. Most dependency problems will be fixed by the developers in one or two days. But in the meantime you can still continue to do most of the yum updates by use of yum's exclude options. Let's work with an example from the timeframe between fc4t1 and fc4t2 test releases:
yum update
... Error: Missing Dependency: libwnck-1.so.4 is needed by package gnome-applets Error: Missing Dependency: libpisock.so.8 is needed by package gnome-pilot
First, identify what packages are providing libwnck-1.so.4 and libpisock.so.8 using yum's provides command
yum provides libwnck-1.so.4
... libwnck.i386 X.Y.Z-A installed yum provides libpisock.so.8 ... pilot-link.i386 B:X.Y.Z-A.pre0.0 installed
Note the name of the package that provides those libraries and note whether a version of the package is installed. Now you can attempt to exclude packages that were part of the dependancy error message:
yum --exclude=pilot-link --exclude=libwnck --exclude=gnome-applets --exclude=gnome-pilot update
Repeat the above steps if the exclude options expose more dependency errors. The exclude process runs a very good chance of letting you proceed with the other package updates until the developers rebuild the necessary packages.
Normally developers are made aware of these issues by the build system and will have these issues fixed in a couple of days. If however the problem lingers longer than that on your system, that could be an indication that you have run into a situational bug that not everyone is seeing. This is when you can spring into action as a tester and make a difference. But, before you file a new bugreport there is a short recipe you can follow to avoid filing unnecessarily. Please remember that test releases exist primarily to help the developers identify problems so they can be fixed in time for release. Unfortunately, reactionary bug filing of duplicate or well known issues can take developer time away from actually fixing issues.
1. read fedora-test-list: Go back into your archives or the web archives for fedora-test-list and read over the threads in the last 48 hours and see if there has been any discussion about the specific update errors you have been seeing. Generally, these sorts of errors are seen by most everyone with similar hardware, so its a very good chance that other testers are already discussing it. Please don't just post a new post to fedora-test-list until after you have reviewed the last 48 hours worth of posts. Having multiple discussions about the same issue is a drain on the time of other testers and developers. 2. search bugzlla.redhat.com: Search to see if there are any reports about the update issue you have seen 3. drop a note into fedora-test-list: Please only start a new thread only after you attempted to find previous discussion of this problem in the test-list or in bugzilla. Other testers can help you confirm the problem, or if they can't confirm it they can help you determine if its a configuration problem or user error on your part. The test-list is a great way to assistance from other more experienced testers, but please do what you can to use the archives responsibility to avoid duplication of information and discussion. 4. File a new bug report: If the exact nature of the dependency problem during updating is lingering for several days or if the problem seems specialized to your situation and it doesn't appear that the developer is aware of this problem.... file a new bug. If you are unsure how to file, experienced testers in fedora-test-list can make suggestions. Please don't assume its a yum bug. Most dependency issues are packaging bugs in one of the packages detailed in the error messages.
Tips and Tricks
- You might need to disable GPG check in /etc/yum.conf or the fedora-devel repository in /etc/yum.repos.d if packages are incorrectly signed
- The broken dependency list which is part of the daily rawhide reports only provides the first layer of dependencies and not the entire list to save build time. If you would like to get updates excluding all the potentially broken dependencies in Fedora development tree, use the yum-skip-broken plugin.
Installation
There are two ways to install Rawhide
- Install a Fedora release, then upgrade to Rawhide.
- Install Rawhide directly from a development tree.
Install a Fedora release
For details and information on installing a Fedora release (includes Alpha and Beta), please see the Installation Guide for the appropriate release.
Upgrading from a previous release
Once your system is installed, you can upgrade to the rawhide repository one of two ways.
This can be accomplished using the following graphical applications:
- First, modify your software sources using:
gpk-repo- Leave only the Fedora - Rawhide software source enabled
- Next, update your system using:
gpk-application
Alternatively, you may upgrade using the command-line:
- Type:
# yum --disablerepo=* --enablerepo=fedora-devel update
Installing Rawhide Using CD/DVD
The same process used to install an Fedora release can often be used to install rawhide. For an overview of the process, please review the Installation Guide.
- First, determine your system architecture (see http://docs.fedoraproject.org/install-guide/f9/en_US/sn-which-arch.html).
- Next, locate a nearby mirror where you can download install media (see http://mirrors.fedoraproject.org/publiclist/Fedora/development/).
- Next, download a Rawhide Fedora CD, DVD, or
boot.isofor your architecture from a (see http://docs.fedoraproject.org/install-guide/f9/en_US/sn-expert-download.html) - Burn the downloaded image to a CD/DVD
- Boot from the CD/DVD
For tips on installing without media, checkout how to install your system using your network interface http://docs.fedoraproject.org/install-guide/f9/en_US/ap-medialess-install.html.
Follow the on-screen instructions from Anaconda, the graphical installer. The installation is very straightforward.
