From Fedora Project Wiki

(Created page with "{{Infobox_group | name = Fedora Test Days | image = link=QA/Fedora_18_test_days | caption = FIXME | date = <<FIXME>> | time = all day ...")
 
(Drop direct Test Days category membership)
 
(113 intermediate revisions by 11 users not shown)
Line 1: Line 1:
{{autolang|base=yes}}
{{Infobox_group
{{Infobox_group
| name = Fedora Test Days
| name = Fedora Test Days
| image = [[File:Echo-testing-48px.png|link=QA/Fedora_18_test_days]]
| caption = [[https://fedorahosted.org/abrt/wiki ABRT & libreport]]
| caption = [[FIXME|FIXME]]
| date = 2012-10-01
| date = <<FIXME>>
| time = 6:00 UTC - 16:00 UTC
| time = all day
| website = [https://fedorahosted.org/abrt/wiki ABRT homepage]
| website = [[QA/Fedora_18_test_days]]
| irc = [irc://irc.freenode.net/#fedora-test-day #fedora-test-day] ([http://webchat.freenode.net/?channels=fedora-test-day webirc])
| irc = [irc://irc.freenode.net/#fedora-test-day #fedora-test-day] ([http://webchat.freenode.net/?channels=fedora-test-day webirc])
| fedora_mailing_list = test
| fedora_mailing_list = test
Line 14: Line 17:
== What to test? ==
== What to test? ==


Today's instalment of Fedora Test Day will focus on '''<<FIXME>>'''
Today's instalment of Fedora Test Day will focus on:
* '''Automated Bug Reporting Tool ([https://fedorahosted.org/abrt ABRT])'''
* '''Unified Problem Reporting UI ([[Features/Unified Problem Reporting UI|libreport]])'''
* '''Simplified crash reporting via ABRT Server ([[Features/SimplifiedCrashReporting|Fedora 18 Feature]])'''
* '''Gnome-abrt'''


== Who's available ==
== Who's available ==


The following cast of characters will be available testing, workarounds, bug fixes, and general discussion ...
The following cast of characters will be available testing, workarounds, bug fixes, and general discussion ...
* Development - [[User:Developer1|Developer1]] (irc_nick1), [[User:Developer2|Developer2]] (irc_nick2)
* Development - Jiří Moskovčák (jmoskovc), Denys Vlasenko (dvlasenk), Karel Klíč (kklic), Michal Toman (mtoman), [[User:Rmarko|Richard Marko]] (rmarko), Filák Jakub (jfilak), Lichvár Miroslav (mlichvar)
* Quality Assurance - [[User:Tester1|Tester1]] (irc_nick3), [[User:Tester2|Tester2]] (irc_nick4), [[User:Tester3|Tester3]] (irc_nick5)
* Quality Assurance - David Kutálek (dkutalek), Martin Kyral (mkyral)


== Prerequisite for Test Day ==  
== Prerequisite for Test Day ==  


List any prerequisite needs for the test event.  A fresh system, virtualized guest, a blank DVD ... a desire to break software?
* Physical machine or a virtual machine
* An updated [http://alt.fedoraproject.org/pub/alt/nightly-composes/ Fedora 18 pre-release]
 
== How to test? ==
 
=== Update your machine ===
 
If you're running Fedora 18, make sure you have all the current updates for it installed, using the update manager.


* Usb key
=== Live image ===
* Usb externally connected HD IDE/SATA
* Empty HD IDE/SATA/SCSI
* Free space on HD


Here's a chunk which is commonly used for most Test Days. Replace XX with whatever Fedora release is pending:
You may download a non-destructive Fedora 18 live image for your architecture. Tips on using a live image are available at [[FedoraLiveCD]].


* An updated [http://fedoraproject.org/get-prerelease Fedora XX pre-release], [[Releases/Rawhide|Rawhide]] (tips on installing Rawhide below), or a [http://alt.fedoraproject.org/pub/alt/nightly-composes/desktop/ nightly live image]
Download the [http://alt.fedoraproject.org/pub/alt/nightly-composes/ nightly Fedora 18 build] for your architecture. Choose the mainstream 'desktop' spin or any other desktop-like spin (kde, lxde, xfce).
* Your hardware profile uploaded to [http://www.smolts.org Smolt] according to [http://smolts.org/smolt-wiki/Main_Page#Usage these instructions]


== How to test? ==
==== Get latest abrt/libreport packages ====


High level details on how a contributor can get involved.  This can include (but not limited to):
The latest versions are:


* Areas to target with exploratory testing
abrt-2.0.13-1.fc18<br/>
* A list of pre-defined test cases to execute
libreport-2.0.14-1.fc18<br/>
* How to report back results
btparser-0.19-1.fc18<br>
gnome-abrt-0.2.2-1.fc18


Here's another common chunk (again, replace XX as above):
Use the following command to install/update all of them:


=== '''Update your machine''' ===
<pre>
yum -y install --enablerepo=updates-testing \
  abrt abrt-addon-ccpp abrt-addon-kerneloops abrt-addon-python abrt-addon-vmcore abrt-addon-xorg \
  abrt-desktop abrt-dbus abrt-gui abrt-libs abrt-cli abrt-retrace-client abrt-plugin-bodhi abrt-tui \
  libreport libreport-compat libreport-filesystem libreport-cli libreport-gtk libreport-newt libreport-web \
  libreport-python libreport-plugin-mailx libreport-plugin-logger libreport-plugin-bugzilla \
  libreport-plugin-kerneloops libreport-plugin-reportuploader libreport-plugin-rhtsupport libreport-plugin-ureport \
  btparser btparser-python \
  gnome-abrt elfutils
</pre>


If you're running Fedora XX, make sure you have all the current updates for it installed, using the update manager. If you want to try Rawhide, see the instructions on the [[Releases/Rawhide|Rawhide]] page on the various ways in which you can install or update to Rawhide. Or:
{{admon/note|Elfutils needed|Please make sure you have elfutils installed - abrt-ccpp is missing requires - [[https://bugzilla.redhat.com/show_bug.cgi?id=859674 Bug 859674]] }}


=== '''Live image''' ===
Make sure that:
* '''abrtd''' and '''abrt-ccpp''' services are running:
<pre>
# systemctl start abrtd.service
# systemctl status abrtd.service


Optionally, you may download a non-destructive Rawhide live image for your architecture. Tips on using a live image are available at [[FedoraLiveCD]]. Live images can be found [http://alt.fedoraproject.org/pub/alt/nightly-composes/desktop/ here].
# systemctl start abrt-ccpp.service
# systemctl status abrt-ccpp.service
</pre>
* file {{filename|/proc/sys/kernel/core_pattern}} contains following output:
<pre>
# cat /proc/sys/kernel/core_pattern
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e
</pre>


== Test Cases ==
== Test Cases ==


Provide a list of test areas or test cases that you'd like contributors to execute. For other examples, see [[:Category:Test_Cases]].
 
For better orientation there are several groups of test cases, top priority first.
Follow each of these test cases, or just some group(s) of them in case of limited time for testing. 
 
=== New features or test cases ===
 
# [[Features/SimplifiedCrashReporting]] - Simplified Crash Reporting (F18 Feature)
# [[QA:Testcase ABRT Reporting Known Crash]] - reporting of known crash
# [[QA:Testcase ABRT ruby gem]] - ruby exceptions
# [[QA:Testcase GNOME ABRT MAIN]] - gnome-abrt
# [[QA:Testcase ABRT Configuration Storage]] - configuration storage
 
=== Basic functionality ===
 
# [[QA:Testcase_ABRT]] - Basic test
# [[QA:Testcase_ABRT_Logger]] - Logger plugin
# [[QA:Testcase_ABRT_Bugzilla]] - Bugzilla plugin
# [[QA:Testcase_ABRT_Mailx]] - Mailx plugin
# [[QA:Testcase ABRT CLI]] - CLI interface
# [[QA:Testcase ABRT kernel]] - kernel oops
# [[QA:Testcase ABRT python]] - python tracebacks
# [[QA:Testcase ABRT vmcore]] - kernel oops
 
=== Integration ===
 
# [[QA:Testcase Libreport sealert]] - sealert integration
# [[QA:Testcase Libreport anaconda]] - anaconda integration
# [[QA:Testcase Libreport Anaconda Install]] - anaconda integration (directly in install)
# [[QA:Testcase Libreport firstboot]] - firstboot integration
# [[QA:Testcase Retrace Server CLI]] - Retrace Server CLI
# [[QA:Testcase Retrace Server GUI]] - Retrace Server in ABRT GUI
 
{{admon/important|Close all fake bugs you create!|During certain test cases you will be required to simulate application crashes and report these crashes into Bugzilla. It is '''very important''' that after every such fake report you open the link provided by ABRT and '''close that bug''' as NOTABUG with comment like ''"ABRT testing bug"''. Otherwise you would overload package developers with fake bug reports. Thank you.}}
 
{{admon/note|Report SELinux denials against ABRT|If you encounter some SELinux (AVC) denials during testing, please report them against the '''abrt''' package, not '''selinux''' package. Thanks.}}
 
 
{{admon/note|Try to break Abrt|Besides above test cases, fell free to experiment and try to break Abrt in your own ways. We will be happy with that :-).}}


== Test Results ==
== Test Results ==


Construct a table or list to allow testers to post results.  Each column should be a test case or configuration, and each row should consist of test results. Include some instructions on how to report bugs, and any special instructions. Here's an example, from a Palimpsest test day:


If you have problems with any of the tests, report a bug to [https://bugzilla.redhat.com Bugzilla] usually for the component [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&version=13&component=udisks udisks], or [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&version=13&component=gnome-disk-utility gnome-disk-utility] for bugs in the Palimpsest graphical front end itself. If you are unsure about exactly how to file the report or what other information to include, just ask on IRC and we will help you. Once you have completed the tests, add your results to the Results table below, following the example results from the first line as a template. The first column should be your name with a link to your User page in the Wiki if you have one, and the second should be a link to the Smolt profile of the system you tested. For each test case, use the [[Template:result|result template]] to enter your result, as shown in the example result line.
If you have problems with any of the tests, report a bug to [https://bugzilla.redhat.com Bugzilla] usually for the component [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&version=18&component=abrt abrt], or [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&version=18&component=libreport libreport] for bugs in the libreport library, or [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&version=18&component=btparser btparser] for backtrace parser related bugs, or [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&version=18&component=gnome-abrt gnome-abrt] for new Gnome Abrt GUI. If you are unsure about exactly how to file the report or what other information to include, just ask on IRC and we will help you. Once you have completed the tests, add your results to the Results table below, following the example results from the first line as a template. The first column should be your name with a link to your User page in the Wiki if you have one. For each test case, use the [[Template:result|result template]] to enter your result, as shown in the example result line.
 
 
==== New features or test cases ====


{|
{|
! User
! User
! Smolt Profile
! [[QA:Testcase_ABRT|Simplified crash reporting]]
! [[QA:Testcase_sample_1|Sample test 1]]
! [[QA:Testcase_ABRT_Logger|Reporting known crash]]
! [[QA:Testcase_sample_2|Sample test 2]]
! [[QA:Testcase ABRT ruby gem|Ruby exceptions]]
! [[QA:Testcase_sample_3|Sample test 3]]
! [[QA:Testcase GNOME ABRT MAIN|Gnome ABRT]]
! [[QA:Testcase_sample_4|Sample test 4]]
! [[QA:Testcase_ABRT_Configuration_Storage|Configuration storage]]
! References
! References
|-
|-
| [[User:SampleUser|Sample User]]
| [[User:SampleUser|Sample User]]
| [http://www.smolts.org/client/show/pub_84465125-1350-4f83-87b9-5f16f7430eb8 HW]
| {{result|none}}
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref>Test pass, but also encountered {{bz|54321}}</ref>
| {{result|warn}} <ref>Test pass, but also encountered {{bz|54321}}</ref>
| {{result|fail}} <ref>{{bz|12345}}</ref>
| {{result|fail}} <ref>{{bz|12345}}</ref>
| {{result|inprogress}}
| {{result|inprogress}}
| <references/>
|-
| [[User:dkutalek|David Kutalek]]
| {{result|inprogress}}
| {{result|pass}}
| {{result|inprogress}}
| {{result|inprogress}}
| {{result|pass}} <ref>Tested with gnome shell</ref>
| <references/>
|-
| [[User:mtoman|Michal Toman]]
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref>will-crash package is not built for F18, you need to install the F17 version from koji</ref>
| {{result|pass}}
| {{result|pass}}
| <references/>
|-
| [[User:mkyral|Martin Kyral]]
| {{result|warn}} <ref>Wrong BZ configuration leads to processing failure with no possibility of recovery {{bz|861899}}</ref>
| {{result|pass}}
| {{result|none}}
| {{result|pass}}
| {{result|pass}}
| <references/>
|-
| [[User:isenfeld|Iveta Senfeldova]]
| {{result|pass}} <ref>Problem with tabs highlighting, confusing warning message: {{bz|861880}}</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}} <ref>Same as Brano Nater, popup does not show up.</ref>
| <references/>
|-
| [[User:bnater|Brano Nater]]
| {{result|pass}} <ref>note added to {{bz|861880}}</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}} <ref group="long">No popup (step 7.) for gnome-keyring appear. A new record for report_Bugzilla appears in "Password and Keys" application, but it's incorrectly populated</ref>
| <references/>
|-
| [[User:jskarvad|Jaroslav Skarvada]]
| {{result|pass}}
| {{result|warn}} <ref>Processing failed. ---Running report_uReport --- (exited with 1), on second try it worked, thus no reproducer.</ref>
| {{result|warn}} <ref>No will-crash package, had to install from koji.</ref>
| {{result|pass}}
| {{result|fail}} <ref>Secret Service is not available, your settings won't be saved!</ref>
| <references/>
|-
| [[User:Adamjosephcook|Adam Joseph Cook]]
| {{result|pass}}
| {{result|pass}}
| {{result|pass}} <ref>All tests passed.</ref>
| {{result|pass}}
| {{result|pass}} <ref>All tests passed. Test instructions not clear. All password keyrings must be deleted before test for the popup to appear.</ref>
| <references/>
|}
==== Basic functionality ====
{|
! User
! [[QA:Testcase_ABRT|Basic test]]
! [[QA:Testcase_ABRT_Logger|Logger reporting]]
! [[QA:Testcase_ABRT_Bugzilla|Bugzilla reporting]]
! [[QA:Testcase_ABRT_Mailx|Mailx plugin]]
! [[QA:Testcase ABRT CLI|CLI interface]]
! [[QA:Testcase ABRT kernel|Kernel oops]]
! [[QA:Testcase ABRT python|Python crash]]
! [[QA:Testcase ABRT vmcore|Kernel panic(vmcore)]]
! References
|-
| [[User:dkutalek|David Kutalek]]
| {{result|warn}} <ref>Test passed, but processing window is wider then screen {{bz|861872}}</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}} <ref>Not able to report issue because of {{bz|854266}}</ref>
| {{result|pass}}
| {{result|inprogress}}
| <references/>
|-
| [[User:mtoman|Michal Toman]]
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}} <ref>AVC {{bz|854266}}</ref><ref>In permissive doesn't work for the oops from the testcase, but works for all random oopses on my system.</ref>
| {{result|pass}}
| {{result|none}}
| <references/>
|-
| [[User:mtoman|Martin Kyral]]
| {{result|pass}}
| {{result|fail}}  <ref>Needed to repeat the procedure for the issue to be written in the log {{bz|861973}}</ref>
| {{result|warn}} <ref>Wrong BZ configuration leads to processing failure with no possibility of recovery {{bz|861899}}</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|inprogress}}
| {{result|inprogress}}
| {{result|fail}} <ref>No kernel dump created (five tries in VirtualBox)</ref>
| <references/>
|-
| [[User:isenfeld|Iveta Senfeldova]]
| {{result|pass}}
| {{result|fail}} <ref>If I closed the Problem data window report-gtk got killed: {{bz|861989}}</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}}<ref>AVC {{bz|854266}}</ref>
| {{result|pass}}
| {{result|none}}
| <references/>
|-
| [[User:jskarvad|Jaroslav Skarvada]]
| {{result|pass}}
| {{result|pass}}
| {{result|fail}} <ref>No bugzilla in the menu.</ref>
| {{result|pass}}
| {{result|pass}} <ref>Instructions are not clear about DIR location.</ref>
| {{result|warn}} <ref>'Can't find kernel package corresponding to '2.6.27.9-159.fc10.i686', but at least catched.</ref>
| {{result|pass}}
| N/A
| <references/>
|-
| [[User:Adamjosephcook|Adam Joseph Cook]]
| {{result|fail||863813}}
| {{result|warn||861973}}
| {{result|warn||861973}} <ref>Test instructions not clear. Bugzilla reporting under report-gui->Preferences.</ref>
| {{result|warn||861973}}
| {{result|pass}} <ref>Test instructions not clear. DIR is 'Directory' field in command 'abrt-cli list'.</ref>
| {{result|fail}} <ref>Crash notification did not appear per expected results of test.</ref>
| {{result|fail}} <ref>Python exception did not invoke abrt-gui per expected results of test. Many SELinux denials on python may have effected test.</ref>
| {{result|fail}} <ref>Kernel panic not caught by abrt. This could be due to significant SELinux denials. Kdump was difficult to configure on FC18.</ref>
| <references/>
|-
|}
==== Integration ====
{|
! User
! [[QA:Testcase Libreport sealert|sealert]]
! [[QA:Testcase Libreport anaconda|anaconda]]
! [[QA:Testcase Libreport Anaconda Install|anaconda install]]
! [[QA:Testcase Libreport firstboot|firstboot]]
! [[QA:Testcase Retrace Server CLI|Retrace cli]]
! [[QA:Testcase Retrace Server GUI|Retrace gui]]
! References
|-
| [[User:dkutalek|David Kutalek]]
| {{result|pass}} <ref>Tested appending to existing bug - works with minor issue {{bz|864891}}</ref>
| {{result|inprogress}}
| {{result|inprogress}}
| {{result|inprogress}}
| {{result|pass}} <ref>client could handle corrupted problem dir better {{bz|861996}}</ref>
| {{result|inprogress}}
| <references/>
|-
| [[User:jskarvad|Jaroslav Skarvada]]
| {{result|pass}}
| {{result|fail}} <ref>Error not catched.</ref>
| N/A
| {{result|fail}} <ref>No crash but following error message: only root can do this</ref>
| {{result|pass}} <ref>Probably not a good practice to notify user about wrong password.</ref>
| {{result|pass}}
| <references/>
|-
| [[User:Adamjosephcook|Adam Joseph Cook]]
| {{result|pass}}
| {{result|warn||861973}} <ref>Passes test if abrt reporting is run twice per bug report.</ref>
| {{result|fail}} <ref>Ran test in VM. anaconda crashed with 'kill -USR1 <pid>', however, no mechanism to report problem available.</ref>
| {{result|fail}} <ref>firstboot will not crash - 'only root can do this' is returned to console.</ref>
| {{result|pass}}
| {{result|pass}}
| <references/>
| <references/>
|-
|-
|}
|}


[[Category:Test Days]]
== Long comments ==
[[Category:QA Templates]]
<references group="long" />
 
  [[Category:Fedora 18 Test Days]]

Latest revision as of 19:33, 26 June 2015


Fedora Test Days
}}}
[ABRT & libreport]
Date 2012-10-01
Time 6:00 UTC - 16:00 UTC
Website ABRT homepage
IRC #fedora-test-day (webirc)
Mailing list test


Note.png
Can't make the date?
If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find at Bugzilla, and add your results to the results section. If this page is more than a month old when you arrive here, please check the current schedule and see if a similar but more recent Test Day is planned or has already happened.

What to test?[edit]

Today's instalment of Fedora Test Day will focus on:

  • Automated Bug Reporting Tool (ABRT)
  • Unified Problem Reporting UI (libreport)
  • Simplified crash reporting via ABRT Server (Fedora 18 Feature)
  • Gnome-abrt

Who's available[edit]

The following cast of characters will be available testing, workarounds, bug fixes, and general discussion ...

  • Development - Jiří Moskovčák (jmoskovc), Denys Vlasenko (dvlasenk), Karel Klíč (kklic), Michal Toman (mtoman), Richard Marko (rmarko), Filák Jakub (jfilak), Lichvár Miroslav (mlichvar)
  • Quality Assurance - David Kutálek (dkutalek), Martin Kyral (mkyral)

Prerequisite for Test Day[edit]

How to test?[edit]

Update your machine[edit]

If you're running Fedora 18, make sure you have all the current updates for it installed, using the update manager.

Live image[edit]

You may download a non-destructive Fedora 18 live image for your architecture. Tips on using a live image are available at FedoraLiveCD.

Download the nightly Fedora 18 build for your architecture. Choose the mainstream 'desktop' spin or any other desktop-like spin (kde, lxde, xfce).

Get latest abrt/libreport packages[edit]

The latest versions are:

abrt-2.0.13-1.fc18
libreport-2.0.14-1.fc18
btparser-0.19-1.fc18
gnome-abrt-0.2.2-1.fc18

Use the following command to install/update all of them: 

yum -y install --enablerepo=updates-testing \
  abrt abrt-addon-ccpp abrt-addon-kerneloops abrt-addon-python abrt-addon-vmcore abrt-addon-xorg \
  abrt-desktop abrt-dbus abrt-gui abrt-libs abrt-cli abrt-retrace-client abrt-plugin-bodhi abrt-tui \
  libreport libreport-compat libreport-filesystem libreport-cli libreport-gtk libreport-newt libreport-web \
  libreport-python libreport-plugin-mailx libreport-plugin-logger libreport-plugin-bugzilla \
  libreport-plugin-kerneloops libreport-plugin-reportuploader libreport-plugin-rhtsupport libreport-plugin-ureport \
  btparser btparser-python \
  gnome-abrt elfutils
Note.png
Elfutils needed
Please make sure you have elfutils installed - abrt-ccpp is missing requires - [Bug 859674]

Make sure that:

  • abrtd and abrt-ccpp services are running:
# systemctl start abrtd.service
# systemctl status abrtd.service

# systemctl start abrt-ccpp.service
# systemctl status abrt-ccpp.service
  • file /proc/sys/kernel/core_pattern contains following output:
# cat /proc/sys/kernel/core_pattern
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e

Test Cases[edit]

For better orientation there are several groups of test cases, top priority first. Follow each of these test cases, or just some group(s) of them in case of limited time for testing.

New features or test cases[edit]

  1. Features/SimplifiedCrashReporting - Simplified Crash Reporting (F18 Feature)
  2. QA:Testcase ABRT Reporting Known Crash - reporting of known crash
  3. QA:Testcase ABRT ruby gem - ruby exceptions
  4. QA:Testcase GNOME ABRT MAIN - gnome-abrt
  5. QA:Testcase ABRT Configuration Storage - configuration storage

Basic functionality[edit]

  1. QA:Testcase_ABRT - Basic test
  2. QA:Testcase_ABRT_Logger - Logger plugin
  3. QA:Testcase_ABRT_Bugzilla - Bugzilla plugin
  4. QA:Testcase_ABRT_Mailx - Mailx plugin
  5. QA:Testcase ABRT CLI - CLI interface
  6. QA:Testcase ABRT kernel - kernel oops
  7. QA:Testcase ABRT python - python tracebacks
  8. QA:Testcase ABRT vmcore - kernel oops

Integration[edit]

  1. QA:Testcase Libreport sealert - sealert integration
  2. QA:Testcase Libreport anaconda - anaconda integration
  3. QA:Testcase Libreport Anaconda Install - anaconda integration (directly in install)
  4. QA:Testcase Libreport firstboot - firstboot integration
  5. QA:Testcase Retrace Server CLI - Retrace Server CLI
  6. QA:Testcase Retrace Server GUI - Retrace Server in ABRT GUI
Important.png
Close all fake bugs you create!
During certain test cases you will be required to simulate application crashes and report these crashes into Bugzilla. It is very important that after every such fake report you open the link provided by ABRT and close that bug as NOTABUG with comment like "ABRT testing bug". Otherwise you would overload package developers with fake bug reports. Thank you.
Note.png
Report SELinux denials against ABRT
If you encounter some SELinux (AVC) denials during testing, please report them against the abrt package, not selinux package. Thanks.


Note.png
Try to break Abrt
Besides above test cases, fell free to experiment and try to break Abrt in your own ways. We will be happy with that :-).

Test Results[edit]

If you have problems with any of the tests, report a bug to Bugzilla usually for the component abrt, or libreport for bugs in the libreport library, or btparser for backtrace parser related bugs, or gnome-abrt for new Gnome Abrt GUI. If you are unsure about exactly how to file the report or what other information to include, just ask on IRC and we will help you. Once you have completed the tests, add your results to the Results table below, following the example results from the first line as a template. The first column should be your name with a link to your User page in the Wiki if you have one. For each test case, use the result template to enter your result, as shown in the example result line.


New features or test cases[edit]

User Simplified crash reporting Reporting known crash Ruby exceptions Gnome ABRT Configuration storage References
Sample User
Pass pass
Warning warn
[1]
Fail fail
[2]
Inprogress inprogress
Inprogress inprogress
  1. ↑ Test pass, but also encountered RHBZ #54321
  2. ↑ RHBZ #12345
David Kutalek
Inprogress inprogress
Pass pass
Inprogress inprogress
Inprogress inprogress
Pass pass
[1]
  1. ↑ Tested with gnome shell
Michal Toman
Pass pass
Pass pass
Warning warn
[1]
Pass pass
Pass pass
  1. ↑ will-crash package is not built for F18, you need to install the F17 version from koji
Martin Kyral
Warning warn
[1]
Pass pass
none
Pass pass
Pass pass
  1. ↑ Wrong BZ configuration leads to processing failure with no possibility of recovery RHBZ #861899
Iveta Senfeldova
Pass pass
[1]
Pass pass
Pass pass
Pass pass
Fail fail
[2]
  1. ↑ Problem with tabs highlighting, confusing warning message: RHBZ #861880
  2. ↑ Same as Brano Nater, popup does not show up.
Brano Nater
Pass pass
[1]
Pass pass
Pass pass
Pass pass
Fail fail
[long 1]
  1. ↑ note added to RHBZ #861880
Jaroslav Skarvada
Pass pass
Warning warn
[1]
Warning warn
[2]
Pass pass
Fail fail
[3]
  1. ↑ Processing failed. ---Running report_uReport --- (exited with 1), on second try it worked, thus no reproducer.
  2. ↑ No will-crash package, had to install from koji.
  3. ↑ Secret Service is not available, your settings won't be saved!
Adam Joseph Cook
Pass pass
Pass pass
Pass pass
[1]
Pass pass
Pass pass
[2]
  1. ↑ All tests passed.
  2. ↑ All tests passed. Test instructions not clear. All password keyrings must be deleted before test for the popup to appear.

Basic functionality[edit]

User Basic test Logger reporting Bugzilla reporting Mailx plugin CLI interface Kernel oops Python crash Kernel panic(vmcore) References
David Kutalek
Warning warn
[1]
Pass pass
Pass pass
Pass pass
Pass pass
Fail fail
[2]
Pass pass
Inprogress inprogress
  1. ↑ Test passed, but processing window is wider then screen RHBZ #861872
  2. ↑ Not able to report issue because of RHBZ #854266
Michal Toman
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Fail fail
[1][2]
Pass pass
none
  1. ↑ AVC RHBZ #854266
  2. ↑ In permissive doesn't work for the oops from the testcase, but works for all random oopses on my system.
Martin Kyral
Pass pass
Fail fail
[1]
Warning warn
[2]
Pass pass
Pass pass
Inprogress inprogress
Inprogress inprogress
Fail fail
[3]
  1. ↑ Needed to repeat the procedure for the issue to be written in the log RHBZ #861973
  2. ↑ Wrong BZ configuration leads to processing failure with no possibility of recovery RHBZ #861899
  3. ↑ No kernel dump created (five tries in VirtualBox)
Iveta Senfeldova
Pass pass
Fail fail
[1]
Pass pass
Pass pass
Pass pass
Fail fail
[2]
Pass pass
none
  1. ↑ If I closed the Problem data window report-gtk got killed: RHBZ #861989
  2. ↑ AVC RHBZ #854266
Jaroslav Skarvada
Pass pass
Pass pass
Fail fail
[1]
Pass pass
Pass pass
[2]
Warning warn
[3]
Pass pass
N/A
  1. ↑ No bugzilla in the menu.
  2. ↑ Instructions are not clear about DIR location.
  3. ↑ 'Can't find kernel package corresponding to '2.6.27.9-159.fc10.i686', but at least catched.
Adam Joseph Cook
Fail fail [1]
Warning warn [2]
Warning warn [2]
[3]
Warning warn [2]
Pass pass
[4]
Fail fail
[5]
Fail fail
[6]
Fail fail
[7]
  1. ↑ RHBZ #863813
  2. ↑ 2.0 2.1 2.2 RHBZ #861973
  3. ↑ Test instructions not clear. Bugzilla reporting under report-gui->Preferences.
  4. ↑ Test instructions not clear. DIR is 'Directory' field in command 'abrt-cli list'.
  5. ↑ Crash notification did not appear per expected results of test.
  6. ↑ Python exception did not invoke abrt-gui per expected results of test. Many SELinux denials on python may have effected test.
  7. ↑ Kernel panic not caught by abrt. This could be due to significant SELinux denials. Kdump was difficult to configure on FC18.

Integration[edit]

User sealert anaconda anaconda install firstboot Retrace cli Retrace gui References
David Kutalek
Pass pass
[1]
Inprogress inprogress
Inprogress inprogress
Inprogress inprogress
Pass pass
[2]
Inprogress inprogress
  1. ↑ Tested appending to existing bug - works with minor issue RHBZ #864891
  2. ↑ client could handle corrupted problem dir better RHBZ #861996
Jaroslav Skarvada
Pass pass
Fail fail
[1] 		N/A
Fail fail
[2]
Pass pass
[3]
Pass pass
  1. ↑ Error not catched.
  2. ↑ No crash but following error message: only root can do this
  3. ↑ Probably not a good practice to notify user about wrong password.
Adam Joseph Cook
Pass pass
Warning warn [1]
[2]
Fail fail
[3]
Fail fail
[4]
Pass pass
Pass pass
  1. ↑ RHBZ #861973
  2. ↑ Passes test if abrt reporting is run twice per bug report.
  3. ↑ Ran test in VM. anaconda crashed with 'kill -USR1 <pid>', however, no mechanism to report problem available.
  4. ↑ firstboot will not crash - 'only root can do this' is returned to console.

Long comments[edit]

  1. ↑ No popup (step 7.) for gnome-keyring appear. A new record for report_Bugzilla appears in "Password and Keys" application, but it's incorrectly populated
Retrieved from "https://fedoraproject.org/w/index.php?title=Test_Day:2012-10-01_ABRT&oldid=416617"