Features/RetraceServer

From FedoraProject

< Features(Difference between revisions)
Jump to: navigation, search
m (Co-workers)
(update)
Line 1: Line 1:
 
= Retrace Server =
 
= Retrace Server =
 
== Summary ==
 
== Summary ==
Application that provides the possibility of remote coredump analysis.
+
Retrace server allows ABRT users to get better backtraces from their crashes by retracing coredumps remotely, on a server owned by Fedora Project.
  
 
== Owner ==
 
== Owner ==
Line 22: Line 22:
  
 
== Detailed description ==
 
== Detailed description ==
At the moment of backtrace creation user needs to have all relevant debuginfos installed locally. Downloading and installing them may require a lot of time and storage space depending on the package and its dependencies. The idea of Retrace Server is to liberate users from this: User sends the coredump packed along with some other required information (e.g. packages list) to the Retrace Server, the Retrace Server installs required debuginfo files, creates backtrace using standard debugging tools and provides it to the user for further processing.
+
When ABRT generates a backtrace from a coredump, it needs debuginfo data to be available for the binary and all libraries involved in the crash. Debuginfo packages require a lot of storage space, and sometimes they are not available at all -- package update causes removal of older update of the same package and it's debuginfo.
 +
 
 +
Other problem is that GDB (which generates the backtrace from the coredump) needs data from the binary and libraries that were involved in the crash. If user updated some relevant package between observing the crash and reporting it, he might be unable to generate good quality backtrace because of updates. This happens often, because Fedora is updated frequently.
 +
 
 +
Retrace server is one possibility how to solve these issues. ABRT offers user to upload her/his coredump to a remote server, and the retracing step happens there. The server creates an environment identical with what was on user's computer at the time of the crash, by installing all the required packages and their debuginfo. The retrace server is able to do that because it keeps all the older packages from updates, and relevant part of updates-testing locally on the server.
 +
 
 +
After creating the backtrace, only the submitter is allowed to download/view it.
  
 
== Benefit to Fedora ==
 
== Benefit to Fedora ==
No need to download debuginfo packages while creating backtrace from various binary crashes.
+
Users:
 +
# Less disk space and processing time needed to use ABRT to report crashes.
 +
# Possibility to report older crashes.
 +
# Lower chance of failure, less time spent on crashes which cannot be retraced because debuginfo is no longer available.
 +
 
 +
Developers:
 +
# Higher quality of ABRT reports.
 +
# Possibility to quickly get a backtrace from any/random Fedora coredump.
  
 
== Scope ==
 
== Scope ==
The project itself consists of 3 parts:
+
ABRT is extended to support Retrace Server.
* '''Interface''': Receives the archive from user, unpacks all files, puts new task into the queue.
+
* '''Analyzer''': Takes the task from queue, for each creates a virtual root with all required packages and debuginfos installed and creates the backtrace within.
+
* '''Repo Synchronizer''': Downloads packages to a local repository containing all versions of all packages, to be able to handle crashes from older versions.
+
  
[[Features/ABRT|ABRT]] plugin that packs required files and sends the archive to the Retrace Server must be implemented.
+
Retrace server is installed on https://retrace01.fedoraproject.org.
  
== How To Test ==
+
The server implementation consists of three parts:
The communication between client and server will be handled by future ABRT plugin. As soon as it is implemented, the testing script will be removed.
+
* '''HTTP Interface''': Receives the archive with a coredump from user, unpacks all files, puts new task into the queue.
 +
* '''Analyzer''': Takes a task from queue, creates a virtual root with all required packages and debuginfos installed and runs GDB to create the backtrace.
 +
* '''Repository Synchronizer''': Downloads packages to a local repository containing all versions of all packages (no removal of older updates).
  
 +
== How To Test ==
 +
Testing will be possible using a new version ABRT with the retrace server support, which will land in Rawhide when stabilised.
  
At the moment, testing is possible using "test-uploader" from "retrace" branch of [http://git.fedorahosted.org/git/?p=abrt.git ABRT Git repository]. The script is only designed for test purposes and does not handle all errors possible.
+
At the moment, testing is possible using the [http://git.fedorahosted.org/git/?p=abrt.git;a=blob;f=retrace/test-uploader/upload.py;h=db75da16d9e07d82168162065f1e2b74c07286e0;hb=refs/heads/retrace upload.py] script from the retrace branch of [http://git.fedorahosted.org/git/?p=abrt.git ABRT Git repository]. The script is only designed for test purposes and does not handle all possible errors.
  
 
'''Usage:'''
 
'''Usage:'''
Line 50: Line 64:
  
 
Running ''upload.py'' with different number of arguments will display short help message.
 
Running ''upload.py'' with different number of arguments will display short help message.
 
  
 
retrace01.fedoraproject.org machine should handle Fedora 13 x86_64, Fedora 14 i686 and Fedora 14 x86_64 crashes.
 
retrace01.fedoraproject.org machine should handle Fedora 13 x86_64, Fedora 14 i686 and Fedora 14 x86_64 crashes.
Line 67: Line 80:
  
 
== User Experience ==
 
== User Experience ==
If ABRT catches a crash that needs to be retraced, user will be given option to either generate backtrace locally (no change) or use Retrace Server.
+
ABRT reporting wizard offers a possibility to use retrace server instead of local retracing to generate a backtrace. Local retracing is still the default action.
  
 
== Dependencies ==
 
== Dependencies ==
 +
* new version of ABRT, supporting the retrace server option
 +
* a server with the retrace server application up and running
 +
 +
Server code dependencies:
 
* mock
 
* mock
 
* xz
 
* xz
Line 76: Line 93:
  
 
== Contingency Plan ==
 
== Contingency Plan ==
Not required, feature can easily be disabled.
+
Hide the possibility of using the retrace server from users in ABRT's graphical user interface.
  
 
== Documentation ==
 
== Documentation ==
* https://fedorahosted.org/abrt/
+
* [https://fedorahosted.org/abrt/attachment/wiki/AbrtRetraceServer/retrace-server Design document]
 +
* [https://fedorahosted.org/abrt/wiki/AbrtRetraceServer Retrace server wiki page on ABRT wiki]
  
 
== Release Notes ==
 
== Release Notes ==
* The user has to trust Retrace Server's administrator because of sensitive data that may be contained in the coredump.
+
ABRT, a crash reporting tool in Fedora, now allows to prepare a part of crash processing remotely, on a server owned by Fedora Project. Remote coredump retracing leads to better quality of reports. Retrace server can generate good backtraces with much higher success rate than local retracing.
  
 
== Comments and Discussion ==
 
== Comments and Discussion ==

Revision as of 14:07, 24 January 2011

Contents

Retrace Server

Summary

Retrace server allows ABRT users to get better backtraces from their crashes by retracing coredumps remotely, on a server owned by Fedora Project.

Owner

  • Email: mtoman@redhat.com

Co-workers

Current Status

  • Targeted release: Fedora 15
  • Last updated: 2011-01-12
  • Percentage of completion: 80%

Detailed description

When ABRT generates a backtrace from a coredump, it needs debuginfo data to be available for the binary and all libraries involved in the crash. Debuginfo packages require a lot of storage space, and sometimes they are not available at all -- package update causes removal of older update of the same package and it's debuginfo.

Other problem is that GDB (which generates the backtrace from the coredump) needs data from the binary and libraries that were involved in the crash. If user updated some relevant package between observing the crash and reporting it, he might be unable to generate good quality backtrace because of updates. This happens often, because Fedora is updated frequently.

Retrace server is one possibility how to solve these issues. ABRT offers user to upload her/his coredump to a remote server, and the retracing step happens there. The server creates an environment identical with what was on user's computer at the time of the crash, by installing all the required packages and their debuginfo. The retrace server is able to do that because it keeps all the older packages from updates, and relevant part of updates-testing locally on the server.

After creating the backtrace, only the submitter is allowed to download/view it.

Benefit to Fedora

Users:

  1. Less disk space and processing time needed to use ABRT to report crashes.
  2. Possibility to report older crashes.
  3. Lower chance of failure, less time spent on crashes which cannot be retraced because debuginfo is no longer available.

Developers:

  1. Higher quality of ABRT reports.
  2. Possibility to quickly get a backtrace from any/random Fedora coredump.

Scope

ABRT is extended to support Retrace Server.

Retrace server is installed on https://retrace01.fedoraproject.org.

The server implementation consists of three parts:

  • HTTP Interface: Receives the archive with a coredump from user, unpacks all files, puts new task into the queue.
  • Analyzer: Takes a task from queue, creates a virtual root with all required packages and debuginfos installed and runs GDB to create the backtrace.
  • Repository Synchronizer: Downloads packages to a local repository containing all versions of all packages (no removal of older updates).

How To Test

Testing will be possible using a new version ABRT with the retrace server support, which will land in Rawhide when stabilised.

At the moment, testing is possible using the upload.py script from the retrace branch of ABRT Git repository. The script is only designed for test purposes and does not handle all possible errors.

Usage:

The script upload.py takes two arguments:

First (mandatory) - ABRT crash directory, by default found in the /var/spool/abrt/ directory.

Second (optional) - Retrace server address (not important at the moment, there is only one testing machine running retrace server - retrace01.fedoraproject.org).

Running upload.py with different number of arguments will display short help message.

retrace01.fedoraproject.org machine should handle Fedora 13 x86_64, Fedora 14 i686 and Fedora 14 x86_64 crashes.


Script's output includes raw HTTP response containig X-Task-Id and X-Task-Password headers. You may ask the retrace server about three things using these headers:

  • Status - HTTP response contains X-Task-Status header with one of three values 'PENDING', 'FINISHED_SUCCESS', 'FINISHED_FAILURE'. wget may be used to show the output (the HTTP response body is the same as X-Task-Status header's value):

wget -S --no-check-certificate -O /dev/null --header="X-Task-Password: <Task-Password>" "https://retrace01.fedoraproject.org/<Task-Id>/"

  • Log - Afrer retrace is finished (FINISHED_SUCCESS or FINISHED_FAILURE) the log is available. wget may be used to download the log:

wget -S --no-check-certificate --header="X-Task-Password: <Task-Password>" "https://retrace01.fedoraproject.org/<Task-Id>/log"

  • Backtrace - Afrer successful retrace (FINISHED_SUCCESS) the backtrace is available. wget may be used to download the backtrace:

wget -S --no-check-certificate --header="X-Task-Password: <Task-Password>" "https://retrace01.fedoraproject.org/<Task-Id>/backtrace"


Each Task-Password is for single use. After every status / log / backtrace request, the response contains X-Task-Password header with new Task-Password.

User Experience

ABRT reporting wizard offers a possibility to use retrace server instead of local retracing to generate a backtrace. Local retracing is still the default action.

Dependencies

  • new version of ABRT, supporting the retrace server option
  • a server with the retrace server application up and running

Server code dependencies:

  • mock
  • xz
  • mod_wsgi
  • python-webob

Contingency Plan

Hide the possibility of using the retrace server from users in ABRT's graphical user interface.

Documentation

Release Notes

ABRT, a crash reporting tool in Fedora, now allows to prepare a part of crash processing remotely, on a server owned by Fedora Project. Remote coredump retracing leads to better quality of reports. Retrace server can generate good backtraces with much higher success rate than local retracing.

Comments and Discussion