From Fedora Project Wiki
(TEST CASE refactoring: moved test cases from test plan to separate wikis, added links to those)
Line 93: Line 93:


Now select the crash, click "report" to report it, or click "delete" to delete it.
Now select the crash, click "report" to report it, or click "delete" to delete it.
== Test Plan ==
First let's go thru the config file and explain purpose of every option
one can find there.....
Let start with common section:
<pre>
[ Common ]
# Enable GPG check
# this option is to ensure, that we primary report crashes only in fedora signed packages
# if you disable this abrt will report crashes in customized unsigned packages
EnableOpenGPG = no
</pre>
How to test it:
Set this to yes, create some package (or download 3rd party one), install it and crash the program from that package - ABRT should ignore it. Set this to no, crash some program from the unsigned package, this time ABRT should notice it. Test this with signed Fedora package, it should report the crash in both cases.
Using GPG Keys:
<pre>
# this is the place, where user can add other trusted gpg keys used to sign packages
# from some repository (e.g. rpm fusion)
# GPG keys
OpenGPGPublicKeys = /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora
</pre>
How to test this:
Set gpg check to yes and instal some package from fusion rpm (or any other repo). Crash an application from this package - abrt should ignore it this time. Leave gpg check on and import the gpg key from the repo (if you haven't already). Add the gpg key into abrt config, crash semething from the package - and this time it should be reported.
Blacklisting packages:
<pre>
# blacklisted packages
BlackList = bash, bind, apache2
</pre>
How to test this: what's in blacklist is ignored, I think this is pretty obvious how to test it
ABRT Plugins:
<pre>
# enabled plugins
EnabledPlugins = SQLite3, CCpp, Logger
</pre>
How to test this: only plugins in EnabledPlugins are loaded, so try to run <code>$ abrt -d </code> and
you should see what is being loaded and try to fidle with enabled plugins -- if you remove SQLite3 plugin it should complain that it needs some DB plugin.
Database Plugin:
<pre>
# selected DB plugin
Database = SQLite3
# this option is to ensure, that crashdumps won't take the whole HDD space
MaxCrashReportsSize = 1000;
</pre>
How to test this: set this to some lower value and try to generate some crashdumps with abrt and
exceed this limit - the last dump shouldn't be saved
note - the last saved dump can exceed this value, but the next one won't
be saved
Actions and reporters:
<pre>
# list of actions and reporters which are activated immediately after a
crash
# occurs
ActionsAndReporters =
</pre>
If you add here some reporter or action plugin, then it shout be run at crash time. So for example:
<pre>ActionsAndReporters = RunApp("ls". "ls-output")</pre>
Save output from ls command into the file called ls-output. This file will be used in summary report. Try Reporter plugin Mailx(<subject>) or other...,
Now let's go through second section. There is saved association among analyzers and actions and/or reporter plugins:
<pre>
[ AnalyzerActionsAndReporters ]
# all C/C++ crashers will be reporter via Logger
CCpp = Logger
# if bind crash. then an email will be send and Logger should be silent
CCpp:bind = Mailx("[abrt] Bind crashed")
# all kernel crashers will be reporter via Logger
Kerneloops = KerneloopsReporter
</pre>
How to test this: try to change the analyzer - reporter association to smth else and report some crash.
it should reported somewhere else (Logger logs to /var/log/abrt-logger, mailx sends email to the configured email address)
Last section is:
<pre>
[ Cron ]
# h:m - at h:m an action plugin is activated
# s - every s seconds is an action plugin activated
10 = KerneloopsScanner # every 10 second the KerneloopsScanner is activated
</pre>
How to test: Try to add activation of other action plugin in particular time. Cron should activate it.


=== ABRT Test Cases to be tested ===
=== ABRT Test Cases to be tested ===
Line 198: Line 100:


Available tests include:
Available tests include:
* [[QA:Testcase_abrt]] - default crash report
* [[QA:Testcase_ABRT]] - default crash report
* [[QA:Testcase_abrt_Logger]] - Logger plugin
* [[QA:Testcase_ABRT_Logger]] - Logger plugin
* [[QA:Testcase_abrt_Mailx]] - Mailx plugin
* [[QA:Testcase_ABRT_Mailx]] - Mailx plugin
 
* [[QA:TestCase ABRT BlackList]] - package blacklist
* [[QA:Testcase ABRT GPG check]] --reporting crashes of signed packages only
* [[QA:TestCase ABRT GPG Keys]] -- additional keys for signing packages
* [[QA:Testcase ABRT Actions and Reporters]] -- testing of action/reporting plugins
* [[QA:Testcase ABRT Cron]] -- periodical plugins using cron
* [[QA:Testcase ABRT Plugins]] -- configuring plugins


Some more cases might be in [https://fedoraproject.org/wiki/Category:ABRT_Test_Cases  ABRT_Test_Cases] Category.
[https://fedoraproject.org/wiki/Category:ABRT_Test_Cases  ABRT_Test_Cases] might have more TestCases in the future.
'''<<FIXME: move all tests from this page into this,, update existing ABRT (or CrachCather) Test Cases>>'''




Revision as of 21:14, 17 August 2009

Warning.png
This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.
Important.png
If you can somehow improve quality of this document, please join the fight and help


DATE TIME WHERE
2009-08-20 From 12:00 to 21:00 UTC (8am - 5pm EDT, 14:00 - 23:00 CET) #fedora-qa (webirc)

Testing ABRT

Today's installment of Fedora Test Day will focus on Automated Bug Reporting Tool (ABRT). This tool should help non-power users with bug reporting, making it as easy as a few mouse clicks. ABRT is a daemon that watches for application crashes. When a crash occurs, it collects the crash data (core file, application's command line etc.) and takes action according to the type of application that crashed and according to the configuration in the abrt.conf configuration file. Bottom line: do not hunt the bugs with a pitchforks, rather use bugzappers/big light source to draw them from the dark and kill them easily at close range.

ABRT should be easy for users and very useful for developers and admins. Ease of crash/bug reporting and quick response from maintainers based on info from ABRT should make Fedora more stable and thus more attractive for users.

We will test:

  • ABRT GUI;
  • compiled (C/C++) programs support;
  • Python programs support;
  • detection if apps are from supported packages;
  • using configured plugins:
    • database plugin;
    • bugzilla plugin (report crash to bugzilla);
    • file transfer plugin (transfer crash data via network);
    • mailx plugin (send mail when crash happens);
    • kernel oops plugin;
    • RunApp plugin (run arbitrary application when crash happens).

Who's available

The following people will be available for testing, workarounds, bug fixes, and general discussion:

Prerequisite for Test Day

You will need to get current ABRT (select your method):

  1. install latest ABRT from Rawhide
  2. install latest ABRT from its development repository on Fedora 11

<<FIXME: for now we have only internal RedHat IP server, setup public repository!>>. For Redhat internal abrt development server create /etc/yum.repos.d/abrt.repo containing: 

[abrt]
name=ABRT
baseurl=http://10.34.33.153/$releasever/$basearch/
gpgcheck=0
enabled=1
  1. download and burn ABRT Testing CD (ISO images will be available at least 24 hours before the test day starts)
Important.png
Note: for reporting to work well, you should have a fully-updated system (Fedora 11 or Rawhide). Reporting uses -debuginfo packages, which have to match the version of the corresponding crashed program's packages. This also avoids reporting a bug which has already been fixed in a later version of the software.


ABRT Documentation

How to start using ABRT

Install abrt and plugins: 

# yum install abrt-desktop

This is not needed when you will use ABRT TestDay Live CD.

Important.png
After installation of ABRT you should reboot your computer to have everything working properly.
# chkconfig abrt on 
# reboot

Test basic functionality

Now if something crashes, ABRT will handle the crash. If you want to test this, crash something, for example: 

$ xfontsel &
$ killall -6 xfontsel

An icon will appear in the notification area. You can now run the GUI: 

$ abrt-gui &

Now select the crash, click "report" to report it, or click "delete" to delete it.

ABRT Test Cases to be tested

Available tests include:

ABRT_Test_Cases might have more TestCases in the future.


Known issues

Consult list of active tickets before reporting an issue. The most common issues are listed below:

  • Recreate Report for same crash - if you want to recreate a report of same crash, you have to remove ABRT's db and cache (rm -r /var/cache/abrt/*) and make a crash gain
  • Nothing happened? - if it seems to do nothing, try to disable PGP checking and restart daemon
  • Graphical warnings - gui writes some gtk warnings
  • rpm %requires - it needs rpm >= 4.7.0-1
  • you cannot use abrt to catch crashes of OpenOffice, KDE and Java apps -- they are catching those crashes themselves.
  • if you're updating from version <= 0.0.5-6 make sure you remove /var/abrt/cache/abrt-db (we've changed the db fomrat and not yet implemented automatic updating mechanism)
  • GUI doesn't inform you about success of failure of a reporting. But you can find results in ABRT's db (default path /var/cache/abrt, it is sqlite3 db, table name is "abrt")

Test Results

Issues that were identified

Tester Description Bug references
User:FasUser ..... #xxx

Things that work alright

Tester Description
User:FasUser ..............


Warning.png
TODO: some tools/scripts/icons for users to help with testing. Let the user easily monitor what is going on, or give him some tools so he can test ABRT even without command line tools. Or is this a bad idea? Please help us with this!
Warning.png
TODO: should users during ABRT test day report bugs directly into Bugzilla? How can we delete all such "bugs" later if they will eat too much space? Or will we use another Bugzilla? Where? How? ? Who can help?
Retrieved from "https://fedoraproject.org/w/index.php?title=Test_Day:2009-08-20_ABRT&oldid=119328"