From Fedora Project Wiki

< Features
Revision as of 17:16, 6 January 2009 by Poelstra (talk | contribs) (release notes section is incomplete)

ReviewOMatic

Review-o-matic is an infrastructure feature. Aim is making an application which gets up as a service and can also be set up locally.

Summary

The aim of the project includes the creation of a application that would review the packages as per the Package Review Guidelines and generate a report. It will also include automating all common steps which every reviewer performs for each review and building guideline checks for all Package guidelines which can be automated.

Owner

Current status

  • Targeted release: Fedora 11
  • Last updated: 2009-01-06
  • Percentage of completion: 15%

Detailed Description

Review-o-matic would be an application which takes a Spec or source RPM through its feeder and does a review and posts its report via its reporter interface. Feeder will have different interfaces to get input from Bugzilla or any generic URI. Similarly, reports can be provided as Bugzilla comments or as a local file. Our process also includes(very much obviously) running rpmlint on input files. We plan to push all guidelines checks to upstream rpmlint. But there are some hybrid checks which:

  • are fedora specific
  • require input from logs generated via builders or multiple file inspection and are very much out of scope of rpmlint

will need to be kept in review-o-matic.

(Check table) is were inspecting each checks one by one and decide on which are programmable and which are not, takes place.

Plan is to speed up automating guidelines once structural architecture get working with very basic capabilities and later on concentrate on automating guidelines.

Structural Architecture of whole design for now is:

Source: https://fedoraproject.org/wiki/Image:Rom1.dia

Benefit to Fedora

Fedora is known for setting high standards in RPM packaging. With a repository that is growing each day, and our emphasis on working upstream we have a collection of packages that makes us proud, our derived distributors happy and our competitors envious. However to maintain high standards we need to make sure that package review requests get the necessary time and attention to meet the large queue of new submissions.

  • This is an effort to take away the grunt work from the review process and make it easier for both the reviewer and the submitter to get an initial idea about the quality of their packages by taking a Spec/SRPM pair and running it through Review-O-Matic and getting a report.
  • It will be possible to do periodic audits of the entire tree, similar to what Matt Domsch's is currently doing to uncover FTBFS packages. With Review-O-Matic it will be possible go beyond detecting build failures and uncover a whole bunch of other violations as well. This might be lucrative to third parties, having a stake in the quality of the distribution (eg., derived distributors like CentOS, academic or enterprise users, etc.), to carry out regular quality analysis on our repositories. Currently it is difficult, albeit possible, to do this using a mix of RPMLint, Mock and some ad-hoc scripts. Since Review-O-Matic ties together these utilities, and adds a few customised Fedora specific checks written by the Fedora developers themselves, one can rest assured that the generated reports are in accordance with the latest guidelines approved by the Fedora Packaging Committee.

Scope

Requires completing the structural design and writing checks or guideline programs for those packaging guidelines which can be automated.

How To Test

  • Making test specs, rpms and srpms and analyszing the reports generated and comparing them to expected output. Unit tests or doc tests are in plan.

User Experience

  • Application can be initiated locally, via Bugzilla, via web service.
  • Developers/Maintainers input files (spec, srpm, rpm) to be reviewed.
  • Generates reports and presents via Bugzilla, web service or output file.

Dependencies

  • rpmlint
  • rpm-python
  • python-bugzilla
  • koji
  • mock

Contingency Plan

Not Needed.

Documentation

Right now wiki has some bare minimum documentation: https://fedorahosted.org/review-o-matic/wiki

Release Notes

To be filled up later.

Use Cases

  • A newbie submitting a package wants to do an automated review on the Spec/SRPM that he/she is going to submit.
  • A reviewer wants to run a Spec/SRPM through the mundane steps of running them through RPMLint, building it in Mock/Koji, etc..
  • Maintainers, release engineers, derived distributors wants to audit an entire repository for packaging guideline violations, build failures, etc..

What RoM is not ?

  • It does no change to present workflow for reviews.
  • It is not a replacement for reviewers.
  • It is not a replacement for rpmlint.
  • Every package which goes into fedora 11 or is already there need not pass through it.

Comments and Discussion

See Talk:Features/ReviewOMatic

Retrieved from "https://fedoraproject.org/w/index.php?title=Features/ReviewOMatic&oldid=68658"