Features/DebuginfoFS

From FedoraProject

< Features(Difference between revisions)
Jump to: navigation, search
m (bad hr!)
(Documentation)
(22 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Summary ==
+
= DebuginfoFS =
A read-only, internet-mountable filesystem that provides debuginfo (for use with {{package|gdb}} and other debugging tools), and
+
  
a client service to mount the debuginfo filesystem when needed.
+
== Summary ==
 +
A read-only, internet-mountable filesystem that provides debuginfo (for use with {{package|gdb}} and other debugging tools), and a client service to mount the debuginfo filesystem when needed.
  
 
== Owner ==
 
== Owner ==
Line 9: Line 9:
  
 
== Current status ==
 
== Current status ==
* Targeted release: [[Releases/{{FedoraVersion||next}} | {{FedoraVersion|long|next}} ]]  
+
* Targeted release: [[Releases/?|Fedora ?]]
* Last updated: January 22, 2009
+
* Last updated: 2009-04-06
* Percentage of completion: 10%
+
* Percentage of completion: 85%
  
 
== Detailed Description ==
 
== Detailed Description ==
Line 25: Line 25:
 
== Scope ==
 
== Scope ==
 
<!-- What work do the developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
 
<!-- What work do the developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
 +
 +
There will be two main parts to this feature:
 +
# A <code>debuginfofs-server</code> package, which provides
 +
#* A <code>debuginfofs-server</code> service,
 +
#* Configuration files to specify which distributions (version/arch) to support, and
 +
#* Some documentation on setting up / maintaining the server.
 +
# A <code>debuginfofs</code> package, which provides
 +
#* A <code>debuginfofs</code> service,
 +
#* Configuration files to specify the location of the server
 +
 +
If possible, we should also provide a public debuginfofs server for Fedora users, and pre-configure the service to use that.
 +
 +
The debuginfofs package has been submitted for review: https://bugzilla.redhat.com/show_bug.cgi?id=487587
 +
 +
We are currently planning to make some extensions to {{package|createrepo}} to create metadata that can be used to export the contents of a package repo as a filesystem.
  
 
== How To Test ==
 
== How To Test ==
<!-- This does not need to be a full-fledged document.  Describe the dimensions of tests that this feature is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.
 
  
Remember that you are writing this how to for interested testers to use to check out your feature - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your feature.
+
1. Prepare a debuginfo server
 +
* ''public instance coming soon (?)''
 +
curl -O http://wwoods.fedorapeople.org/review/debuginfofs-server-0.2-1.fc10.noarch.rpm
 +
yum localinstall --nogpgcheck debuginfofs-server-0.2-1.fc10.noarch.rpm
 +
debuginfofs-mirror --releasever=CLIENTDISTRO --basearch=CLIENTARCH fedora fedora-updates
 +
service debuginfofs-server start
  
A good "how to test" should answer these four questions:
+
2. Install / configure debuginfofs on the client
 +
curl -O http://wwoods.fedorapeople.org/review/debuginfofs-0.2-1.fc10.noarch.rpm
 +
yum localinstall --nogpgcheck debuginfofs-0.2-1.fc10.noarch.rpm
 +
vi /etc/sysconfig/debuginfofs ''# change DEBUGINFOFS_SERVER''
  
0. What special hardware / data / etc. is needed (if any)?
+
3. Mount debuginfofs
1. How do I prepare my system to test this feature? What packages
+
service debuginfofs start
need to be installed, config files edited, etc.?
+
2. What specific actions do I perform to check that the feature is
+
working like it's supposed to?
+
3. What are the expected results of those actions?
+
  
-->
+
4. Crash a program in gdb
 +
[wwoods@metroid debuginfofs]$ '''gdb $(which sleep)'''
 +
GNU gdb Fedora (6.8-29.fc10)
 +
Copyright (C) 2008 Free Software Foundation, Inc.
 +
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 +
This is free software: you are free to change and redistribute it.
 +
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
 +
and "show warranty" for details.
 +
This GDB was configured as "i386-redhat-linux-gnu"...
 +
(gdb) '''set args 999'''
 +
(gdb) '''run'''
 +
Starting program: /bin/sleep 999
 +
'''[Hit Ctrl-C]'''
 +
Program received signal SIGINT, Interrupt.
 +
__kernel_vsyscall () at arch/x86/vdso/vdso32/int80.S:16
 +
16 arch/x86/vdso/vdso32/int80.S: No such file or directory.
 +
in arch/x86/vdso/vdso32/int80.S
 +
Current language:  auto; currently asm
 +
(gdb) '''bt'''
 +
#0  __kernel_vsyscall () at arch/x86/vdso/vdso32/int80.S:16
 +
#1  0x008ac430 in __nanosleep_nocancel () from /lib/libc.so.6
 +
#2  0x0804abd3 in xnanosleep (seconds=999) at xnanosleep.c:112
 +
#3  0x080490f6 in main (argc=2, argv=0xbffff514) at sleep.c:150
 +
(gdb)
 +
 
 +
If everything is working as expected, you will see the function/file names, as above.
  
 
== User Experience ==
 
== User Experience ==
<!-- If this feature is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
+
 
 +
Getting debuginfo to get a backtrace for a bug report is faster and easier when the <code>debuginfofs</code> service is installed and enabled.
  
 
== Dependencies ==
 
== Dependencies ==
<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this feature depends?  In other words, completion of another feature owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel feature)? -->
+
 
 +
=== Server preparation ===
 +
* A public debuginfofs server currently requires somewhere around 25GB of disk per Fedora version/arch - a total of ~300GB for Fedora (9,10,11,Rawhide) * (i386,x86_64,ppc) - plus a lot of bandwidth and disk I/O to keep the files up to date.
 +
** ''This has proven to be very hard to maintain in our testing.''
 +
** Some further work could be done to drastically reduce the amount of disk space needed by providing a virtual filesystem of the debuginfo repo contents, and only unpacking debuginfo ''when it is needed''
 +
*** This would also reduce setup time to near-zero, since we don't have the overhead of unpacking all the RPMs.
 +
** This would best be implemented using extensions to {{package|createrepo}} to add extra metadata to make FUSE exporting of a package repo's contents easier.
 +
 
 +
=== Network filesystem ===
 +
* We've chosen WebDAV as the network filesystem for this project, so:
 +
** The server-side requires {{package|httpd}}'s Dav module.
 +
** Client-side requires a FUSE implementation of WebDAV.
 +
*** {{package|wdfs}} is already packaged, but turns out to be unsuitable for this task.
 +
*** [http://sourceforge.net/projects/dav/ davfs2] has therefore been packaged for Fedora - the review request is [https://bugzilla.redhat.com/show_bug.cgi?id=488858 here].
 +
 
 +
=== Client-side Handling ===
 +
* If we need access to the source files as well as the debugging symbols, there will need to be an additional layer on the client-side to find the source files and put them in a place where <code>gdb</code> can find them.
 +
** For most purposes, these files are not necessary, so the current implementation does not provide them.
 +
 
 +
* {{package|bug-buddy}}, KCrash, and other crash handlers may need modifications to recognize and use .build-id files.
 +
** {{package|bug-buddy}} handles this properly in Fedora as of <code>bug-buddy-2.26.0-2</code> - see [https://bugzilla.redhat.com/show_bug.cgi?id=490778 bug 490778]
  
 
== Contingency Plan ==
 
== Contingency Plan ==
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "None necessary, revert to previous release behaviour."  Or it might not.  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
+
 
 +
None needed - things stay as they are.
  
 
== Documentation ==
 
== Documentation ==
<!-- Is there upstream documentation on this feature, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
 
  
 +
The gdb info pages and [[Releases/FeatureBuildId]] give some background on the Build-Id feature that makes this possible.
 +
 +
[http://fedorapeople.org/gitweb?p=wwoods/public_git/debuginfofs.git;a=tree http://fedorapeople.org/gitweb?p=wwoods/public_git/debuginfofs.git;a=tree]
  
 
== Release Notes ==
 
== Release Notes ==
  
<!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
+
There is a new <code>debuginfofs</code> service, which provides debuginfo for all Fedora packages over the Internet.
<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this feature, indicate them here.  You can also link to upstream documentation if it satisfies this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release. -->
+
  
 
== Comments and Discussion ==
 
== Comments and Discussion ==
Line 65: Line 131:
  
 
[[Category:FeaturePageIncomplete]]
 
[[Category:FeaturePageIncomplete]]
 +
 
<!-- When your feature page is completed and ready for review -->
 
<!-- When your feature page is completed and ready for review -->
 
<!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler -->
 
<!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler -->
 
<!-- After review, the feature wrangler will move your page to Category:FeatureReadyForFesco... if it still needs more work it will move back to Category:FeaturePageIncomplete-->
 
<!-- After review, the feature wrangler will move your page to Category:FeatureReadyForFesco... if it still needs more work it will move back to Category:FeaturePageIncomplete-->
 
<!-- A pretty picture of the page category usage is at: https://fedoraproject.org/wiki/Features/Policy/Process -->
 
<!-- A pretty picture of the page category usage is at: https://fedoraproject.org/wiki/Features/Policy/Process -->

Revision as of 15:44, 29 July 2009

Contents

DebuginfoFS

Summary

A read-only, internet-mountable filesystem that provides debuginfo (for use with Package-x-generic-16.pnggdb and other debugging tools), and a client service to mount the debuginfo filesystem when needed.

Owner

Current status

  • Targeted release: Fedora ?
  • Last updated: 2009-04-06
  • Percentage of completion: 85%

Detailed Description

When a program crashes or is otherwise misbehaving, one of the most helpful things to put in a bug report is a traceback of the offending process, gathered by using Package-x-generic-16.pnggdb. GNOME's Package-x-generic-16.pngbug-buddy does this automatically. But without debugging info, the tracebacks are incomplete.

debuginfo-install can be used to install the needed debuginfo packages, but this takes a long time (2-5 minutes) and 95% of the installed data is unused.

This Feature will provide scripts and tools to provide a WebDAV filesystem containing all the debuginfo needed for everything in Fedora, and a service to allow users (or automatic crash-reporting processes - like Features/CrashCatcher) to mount the filesystem when needed.

Benefit to Fedora

By simplifying the process of getting debugging info, we can improve the quality of bug reports, which should make it easier (and faster) to get bugs fixed.

Scope

There will be two main parts to this feature:

  1. A debuginfofs-server package, which provides
    • A debuginfofs-server service,
    • Configuration files to specify which distributions (version/arch) to support, and
    • Some documentation on setting up / maintaining the server.
  2. A debuginfofs package, which provides
    • A debuginfofs service,
    • Configuration files to specify the location of the server

If possible, we should also provide a public debuginfofs server for Fedora users, and pre-configure the service to use that.

The debuginfofs package has been submitted for review: https://bugzilla.redhat.com/show_bug.cgi?id=487587

We are currently planning to make some extensions to Package-x-generic-16.pngcreaterepo to create metadata that can be used to export the contents of a package repo as a filesystem.

How To Test

1. Prepare a debuginfo server

  • public instance coming soon (?)
curl -O http://wwoods.fedorapeople.org/review/debuginfofs-server-0.2-1.fc10.noarch.rpm
yum localinstall --nogpgcheck debuginfofs-server-0.2-1.fc10.noarch.rpm
debuginfofs-mirror --releasever=CLIENTDISTRO --basearch=CLIENTARCH fedora fedora-updates
service debuginfofs-server start

2. Install / configure debuginfofs on the client

curl -O http://wwoods.fedorapeople.org/review/debuginfofs-0.2-1.fc10.noarch.rpm
yum localinstall --nogpgcheck debuginfofs-0.2-1.fc10.noarch.rpm
vi /etc/sysconfig/debuginfofs # change DEBUGINFOFS_SERVER

3. Mount debuginfofs

service debuginfofs start

4. Crash a program in gdb

[wwoods@metroid debuginfofs]$ gdb $(which sleep)
GNU gdb Fedora (6.8-29.fc10)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...
(gdb) set args 999
(gdb) run
Starting program: /bin/sleep 999
[Hit Ctrl-C]
Program received signal SIGINT, Interrupt.
__kernel_vsyscall () at arch/x86/vdso/vdso32/int80.S:16
16	arch/x86/vdso/vdso32/int80.S: No such file or directory.
	in arch/x86/vdso/vdso32/int80.S
Current language:  auto; currently asm
(gdb) bt
#0  __kernel_vsyscall () at arch/x86/vdso/vdso32/int80.S:16
#1  0x008ac430 in __nanosleep_nocancel () from /lib/libc.so.6
#2  0x0804abd3 in xnanosleep (seconds=999) at xnanosleep.c:112
#3  0x080490f6 in main (argc=2, argv=0xbffff514) at sleep.c:150
(gdb)

If everything is working as expected, you will see the function/file names, as above.

User Experience

Getting debuginfo to get a backtrace for a bug report is faster and easier when the debuginfofs service is installed and enabled.

Dependencies

Server preparation

  • A public debuginfofs server currently requires somewhere around 25GB of disk per Fedora version/arch - a total of ~300GB for Fedora (9,10,11,Rawhide) * (i386,x86_64,ppc) - plus a lot of bandwidth and disk I/O to keep the files up to date.
    • This has proven to be very hard to maintain in our testing.
    • Some further work could be done to drastically reduce the amount of disk space needed by providing a virtual filesystem of the debuginfo repo contents, and only unpacking debuginfo when it is needed
      • This would also reduce setup time to near-zero, since we don't have the overhead of unpacking all the RPMs.
    • This would best be implemented using extensions to Package-x-generic-16.pngcreaterepo to add extra metadata to make FUSE exporting of a package repo's contents easier.

Network filesystem

  • We've chosen WebDAV as the network filesystem for this project, so:
    • The server-side requires Package-x-generic-16.pnghttpd's Dav module.
    • Client-side requires a FUSE implementation of WebDAV.
      • Package-x-generic-16.pngwdfs is already packaged, but turns out to be unsuitable for this task.
      • davfs2 has therefore been packaged for Fedora - the review request is here.

Client-side Handling

  • If we need access to the source files as well as the debugging symbols, there will need to be an additional layer on the client-side to find the source files and put them in a place where gdb can find them.
    • For most purposes, these files are not necessary, so the current implementation does not provide them.
  • Package-x-generic-16.pngbug-buddy, KCrash, and other crash handlers may need modifications to recognize and use .build-id files.

Contingency Plan

None needed - things stay as they are.

Documentation

The gdb info pages and Releases/FeatureBuildId give some background on the Build-Id feature that makes this possible.

http://fedorapeople.org/gitweb?p=wwoods/public_git/debuginfofs.git;a=tree

Release Notes

There is a new debuginfofs service, which provides debuginfo for all Fedora packages over the Internet.

Comments and Discussion