Darkserver

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(Fill in Why we need the service?)
Line 7: Line 7:
 
Source code is under github and can be found [https://github.com/kushaldas/darkserver here].
 
Source code is under github and can be found [https://github.com/kushaldas/darkserver here].
  
'''Why we need the service ?'''
+
'''Why we need the service?'''
  
 
To enable developer tools to identify exact package builds from which process images (e.g. core dumps) come.
 
To enable developer tools to identify exact package builds from which process images (e.g. core dumps) come.
 
This can enable their analysis, debugging profiling, by finding out where the rpm / elf / dwarf files may be
 
This can enable their analysis, debugging profiling, by finding out where the rpm / elf / dwarf files may be
 
found, so they can download them.  (This is even better than abrt-action-install-debuginfo-to-abrt-cache because that apparently cannot query files no longer indexed by repodata.)
 
found, so they can download them.  (This is even better than abrt-action-install-debuginfo-to-abrt-cache because that apparently cannot query files no longer indexed by repodata.)
 +
 +
<code>darkclient</code> may look like the same what can be done with <code>repoquery</code>:
 +
<pre>
 +
$ repoquery -q --enablerepo='*-debuginfo' -f /usr/lib/debug/.build-id/0d/aa18d6291a5d0d174556c8c46f1345eba61a65
 +
bash-debuginfo-0:4.2.10-5.fc16.x86_64
 +
</pre>
 +
 +
Just with <code>repoquery</code> one has only available:
 +
* GA release from repository fedora
 +
* the very latest update from repository updates
 +
* possibly the very latest update from repository updates-testing
 +
 +
One does not have available:
 +
* any update which has been already obsoleted by a more recent update
 +
* any release from Koji not submitted to Bodhi
 +
 +
This is a real problem making bugreports investigations difficult or even lost in some cases:
 +
* https://bugzilla.redhat.com/show_bug.cgi?id=629966#c4
 +
* https://bugzilla.redhat.com/show_bug.cgi?id=556227#c2
 +
* RHEL Bug 739685 - not public
 +
It is more common but I have spent more time searching for it.
 +
 +
Also the <code>repoquery -qf</code> way is too slow as it has to download <code>*-filelists.sqlite.bz2</code> files for all the repositories first.
 +
 +
Requests for this feature of build-id to NVRA queries implemented some other ways have been declined before:
 +
* [https://fedorahosted.org/fedora-infrastructure/ticket/2387 #2387: all-versions Koji repository]
 +
* [https://fedorahosted.org/fesco/ticket/470 #470: buildid repo request]
  
 
'''Architecture of the project'''
 
'''Architecture of the project'''

Revision as of 21:45, 6 December 2011

Darkserver is a service written to help people finding details of build-id(s). People will be able query the service based on build-id(s) or rpm package names. The service will provide output in JSON format as it will be easier for other tools to parse the output.

The whole idea is based on a feature of BUILD-ID which was completed in Fedora 8 time, you can read about it here.

Source Code

Source code is under github and can be found here.

Why we need the service?

To enable developer tools to identify exact package builds from which process images (e.g. core dumps) come. This can enable their analysis, debugging profiling, by finding out where the rpm / elf / dwarf files may be found, so they can download them. (This is even better than abrt-action-install-debuginfo-to-abrt-cache because that apparently cannot query files no longer indexed by repodata.)

darkclient may look like the same what can be done with repoquery:

$ repoquery -q --enablerepo='*-debuginfo' -f /usr/lib/debug/.build-id/0d/aa18d6291a5d0d174556c8c46f1345eba61a65
bash-debuginfo-0:4.2.10-5.fc16.x86_64

Just with repoquery one has only available:

  • GA release from repository fedora
  • the very latest update from repository updates
  • possibly the very latest update from repository updates-testing

One does not have available:

  • any update which has been already obsoleted by a more recent update
  • any release from Koji not submitted to Bodhi

This is a real problem making bugreports investigations difficult or even lost in some cases:

It is more common but I have spent more time searching for it.

Also the repoquery -qf way is too slow as it has to download *-filelists.sqlite.bz2 files for all the repositories first.

Requests for this feature of build-id to NVRA queries implemented some other ways have been declined before:

Architecture of the project

View this image.

Requirements

 * Django >= 1.2.5
 * MySQL server

The web service does not need any authentication. It will require read only access to the database, the users will only require to query the service.

The information on the database will be filled up by darkserver-import command, which will only execute through bodhi on every update pushed through.

How to setup?

Install the darkserver and darkserver-import rpms. Run the createtable.sql under /usr/share/darkserver to create table structure in the database.

$ mysql -u user -p -Ddatabasename < /usr/share/darkserver/createtable.sql

Edit the configuration file at /etc/darkserver/darkserverweb.conf as

[darkserverweb]
host=MySQL host name
user=MySQL user (read only access)
password=password
database=database name

Remember that the webserver only requires read only access to MySQL.

To use the darkserver-import tool you will have to fill up /etc/darkserver/darkserver.conf in the same way.

[darkserver]
host=MySQL host name
user=MySQL user (read/write access)
password=password
database=database name