Features/DebuginfoFS

= DebuginfoFS =

Summary
A read-only, internet-mountable filesystem that provides debuginfo (for use with and other debugging tools), and a client service to mount the debuginfo filesystem when needed.

Owner

 * Name: Will Woods
 * email: [mailto:wwoods@redhat.com wwoods@redhat.com]

Current status

 * Targeted release: Fedora ?
 * Last updated: 2009-04-06
 * Percentage of completion: 85%

Detailed Description
When a program crashes or is otherwise misbehaving, one of the most helpful things to put in a bug report is a traceback of the offending process, gathered by using. GNOME's does this automatically. But without debugging info, the tracebacks are incomplete.

can be used to install the needed debuginfo packages, but this takes a long time (2-5 minutes) and 95% of the installed data is unused.

This Feature will provide scripts and tools to provide a WebDAV filesystem containing all the debuginfo needed for everything in Fedora, and a service to allow users (or automatic crash-reporting processes - like Features/CrashCatcher) to mount the filesystem when needed.

Benefit to Fedora
By simplifying the process of getting debugging info, we can improve the quality of bug reports, which should make it easier (and faster) to get bugs fixed.

Scope
There will be two main parts to this feature:
 * 1) A   package, which provides
 * 2) * A  service,
 * 3) * Configuration files to specify which distributions (version/arch) to support, and
 * 4) * Some documentation on setting up / maintaining the server.
 * 5) A   package, which provides
 * 6) * A  service,
 * 7) * Configuration files to specify the location of the server

If possible, we should also provide a public debuginfofs server for Fedora users, and pre-configure the service to use that.

The debuginfofs package has been submitted for review: https://bugzilla.redhat.com/show_bug.cgi?id=487587

We are currently planning to make some extensions to to create metadata that can be used to export the contents of a package repo as a filesystem.

How To Test
1. Prepare a debuginfo server curl -O http://wwoods.fedorapeople.org/review/debuginfofs-server-0.2-1.fc10.noarch.rpm yum localinstall --nogpgcheck debuginfofs-server-0.2-1.fc10.noarch.rpm debuginfofs-mirror --releasever=CLIENTDISTRO --basearch=CLIENTARCH fedora fedora-updates service debuginfofs-server start
 * public instance coming soon (?)

2. Install / configure debuginfofs on the client curl -O http://wwoods.fedorapeople.org/review/debuginfofs-0.2-1.fc10.noarch.rpm yum localinstall --nogpgcheck debuginfofs-0.2-1.fc10.noarch.rpm vi /etc/sysconfig/debuginfofs # change DEBUGINFOFS_SERVER

3. Mount debuginfofs service debuginfofs start

4. Crash a program in gdb [wwoods@metroid debuginfofs]$ gdb $(which sleep) GNU gdb Fedora (6.8-29.fc10) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later  This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (gdb) set args 999 (gdb) run Starting program: /bin/sleep 999 [Hit Ctrl-C] Program received signal SIGINT, Interrupt. __kernel_vsyscall at arch/x86/vdso/vdso32/int80.S:16 16	arch/x86/vdso/vdso32/int80.S: No such file or directory. in arch/x86/vdso/vdso32/int80.S Current language: auto; currently asm (gdb) bt (gdb)
 * 1) 0 __kernel_vsyscall  at arch/x86/vdso/vdso32/int80.S:16
 * 2) 1 0x008ac430 in __nanosleep_nocancel  from /lib/libc.so.6
 * 3) 2 0x0804abd3 in xnanosleep (seconds=999) at xnanosleep.c:112
 * 4) 3 0x080490f6 in main (argc=2, argv=0xbffff514) at sleep.c:150

If everything is working as expected, you will see the function/file names, as above.

User Experience
Getting debuginfo to get a backtrace for a bug report is faster and easier when the  service is installed and enabled.

Server preparation

 * A public debuginfofs server currently requires somewhere around 25GB of disk per Fedora version/arch - a total of ~300GB for Fedora (9,10,11,Rawhide) * (i386,x86_64,ppc) - plus a lot of bandwidth and disk I/O to keep the files up to date.
 * This has proven to be very hard to maintain in our testing.
 * Some further work could be done to drastically reduce the amount of disk space needed by providing a virtual filesystem of the debuginfo repo contents, and only unpacking debuginfo when it is needed
 * This would also reduce setup time to near-zero, since we don't have the overhead of unpacking all the RPMs.
 * This would best be implemented using extensions to to add extra metadata to make FUSE exporting of a package repo's contents easier.

Network filesystem

 * We've chosen WebDAV as the network filesystem for this project, so:
 * The server-side requires 's Dav module.
 * Client-side requires a FUSE implementation of WebDAV.
 * is already packaged, but turns out to be unsuitable for this task.
 * davfs2 has therefore been packaged for Fedora - the review request is here.

Client-side Handling

 * If we need access to the source files as well as the debugging symbols, there will need to be an additional layer on the client-side to find the source files and put them in a place where  can find them.
 * For most purposes, these files are not necessary, so the current implementation does not provide them.


 * , KCrash, and other crash handlers may need modifications to recognize and use .build-id files.
 * handles this properly in Fedora as of  - see bug 490778

Contingency Plan
None needed - things stay as they are.

Documentation
The gdb info pages and Releases/FeatureBuildId give some background on the Build-Id feature that makes this possible.

http://fedorapeople.org/gitweb?p=wwoods/public_git/debuginfofs.git;a=tree

Release Notes
There is a new  service, which provides debuginfo for all Fedora packages over the Internet.

Comments and Discussion

 * See Talk:Features/DebuginfoFS