Features/ReviewOMatic

From FedoraProject

< Features(Difference between revisions)
Jump to: navigation, search
(Benefit to Fedora: re-written)
m (bad hr!)
 
(13 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
= ReviewOMatic =
 
= ReviewOMatic =
  
* Home Page https://fedorahosted.org/review-o-matic/wiki
+
== Summary ==
* Browse Source Code https://fedorahosted.org/review-o-matic/browser
+
* Mailing List https://fedorahosted.org/mailman/listinfo/review-o-matic
+
  
Review-o-matic is an infrastructure feature. Aim is making an application which gets up as a service and can also be set up locally.
+
An application that reviews packages according to the Package Review Guidelines and generate a report. It also include automates all common steps which every reviewer performs for each review and building guideline checks for all Package guidelines which can be automated.
 
+
== Summary ==
+
  
The aim of the project includes the creation of a application that would review the packages as per the Package Review Guidelines and generate a report. It will also include automating all common steps which every reviewer performs for each review and building guideline checks for all Package guidelines which can be automated.
+
Review-o-matic is an application which will be a feature of the infrastructure. It will be provided as a hosted services and can also be set up locally.
  
 
== Owner ==
 
== Owner ==
Line 21: Line 17:
 
== Detailed Description ==
 
== Detailed Description ==
  
Review-o-matic would be an application which takes a spec or srpm or just rpms through its feeder and does a review and posts its report via its reporter interface. Feeder will have different interfaces to get input e.g bugzilla, local host m/c (as files) or web interface or service. Reports will also be provided to different interfaces depending upon needs e.g Bugzilla, local m/c (as file) or web service. Our process also includes(very much obviously) running rpmlint on input files. We plan to push all guidelines checks to upstream rpmlint. But there are some hybrid checks which:
+
Review-o-matic would be an application which takes a Spec or source RPM through its feeder and does a review and posts its report via its reporter interface. Feeder will have different interfaces to get input from Bugzilla or any generic URI. Similarly, reports can be provided as Bugzilla comments or as a local file. Our process also includes(very much obviously) running rpmlint on input files. We plan to push all guidelines checks to upstream rpmlint. But there are some hybrid checks which:
  
 
* are fedora specific
 
* are fedora specific
Line 40: Line 36:
 
    
 
    
 
----
 
----
 +
 +
* Home Page https://fedorahosted.org/review-o-matic/wiki
 +
* Browse Source Code https://fedorahosted.org/review-o-matic/browser
 +
* Mailing List https://fedorahosted.org/mailman/listinfo/review-o-matic
 +
  
 
== Benefit to Fedora ==
 
== Benefit to Fedora ==
Line 45: Line 46:
 
Fedora is known for setting high standards in RPM packaging. With a repository that is growing each day, and our emphasis on working upstream we have a collection of packages that makes us proud, our derived distributors happy and our competitors envious. However to maintain high standards we need to make sure that package review requests get the necessary time and attention to meet the large queue of new submissions.
 
Fedora is known for setting high standards in RPM packaging. With a repository that is growing each day, and our emphasis on working upstream we have a collection of packages that makes us proud, our derived distributors happy and our competitors envious. However to maintain high standards we need to make sure that package review requests get the necessary time and attention to meet the large queue of new submissions.
  
This is an effort to take away the grunt work from the review process and make it easier for both the reviewer and the submitter to get an initial idea about the quality of their packages by taking a Spec/SRPM pair and running it through Review-O-Matic and getting a report.
+
* This is an effort to take away the grunt work from the review process and make it easier for both the reviewer and the submitter to get an initial idea about the quality of their packages by taking a Spec/SRPM pair and running it through Review-O-Matic and getting a report.
  
It will be possible to do periodic audits of the entire tree, similar to what Matt Domsch's is currently doing to uncover FTBFS packages. With Review-O-Matic it will be possible go beyond detecting build failures and uncover a whole bunch of other violations as well. This might be lucrative to third parties, having a stake in the quality of the distribution (eg., derived distributors like CentOS, academic or enterprise users, etc.), to carry out regular quality analysis on our repositories. Currently it is difficult, albeit possible, to do this using a mix of RPMLint, Mock and some ad-hoc scripts. Since Review-O-Matic ties together these utilities, and adds a few customised Fedora specific checks written by the Fedora developers themselves, one can rest assured that the generated reports are in accordance with the latest guidelines approved by the Fedora Packaging Committee.
+
* It will be possible to do periodic audits of the entire tree, similar to what Matt Domsch's is currently doing to uncover FTBFS packages. With Review-O-Matic it will be possible go beyond detecting build failures and uncover a whole bunch of other violations as well. This might be lucrative to third parties, having a stake in the quality of the distribution (eg., derived distributors like CentOS, academic or enterprise users, etc.), to carry out regular quality analysis on our repositories. Currently it is difficult, albeit possible, to do this using a mix of RPMLint, Mock and some ad-hoc scripts. Since Review-O-Matic ties together these utilities, and adds a few customised Fedora specific checks written by the Fedora developers themselves, one can rest assured that the generated reports are in accordance with the latest guidelines approved by the Fedora Packaging Committee.
  
 
== Scope ==
 
== Scope ==
  
 
Requires completing the structural design and writing checks or guideline programs for those packaging guidelines which can be automated.
 
Requires completing the structural design and writing checks or guideline programs for those packaging guidelines which can be automated.
 +
 +
== Use Cases ==
 +
 +
* A newbie submitting a package wants to do an automated review on the Spec/SRPM that he/she is going to submit.
 +
 +
* A reviewer wants to run a Spec/SRPM through the mundane steps of running them through RPMLint, building it in Mock/Koji, etc..
 +
 +
* Maintainers, release engineers, derived distributors wants to audit an entire repository for packaging guideline violations, build failures, etc..
 +
 +
== What RoM is not ? ==
 +
 +
* It does no change to present workflow for reviews.
 +
* It is not a replacement for reviewers.
 +
* It is not a replacement for rpmlint.
 +
* Every package which goes into fedora 11 or is already there need not pass through it.
 +
  
 
== How To Test ==
 
== How To Test ==
Line 69: Line 86:
 
* python-bugzilla
 
* python-bugzilla
 
* koji
 
* koji
 +
* mock
  
 
== Contingency Plan ==
 
== Contingency Plan ==
Line 81: Line 99:
 
== Release Notes ==
 
== Release Notes ==
  
To be filled up later.
+
* <s>Irrelevant since it will not directly impact the end-user</s>
 
+
* '''DRAFT''': During the Fedora 11 development cycle a new tool was developed called ReviewOMatic. ReviewOMatic reviews packages according to the Package Review Guidelines and generates a report. It also include automates all common steps which every reviewer performs for each review and building guideline checks for all Package guidelines which can be automated. A hosted instance of Review-o-matic provided to developers in the Fedora infrastructure.
== Who are target users ? ==
+
 
+
* Reporters for new packages
+
* Reviewers.
+
* Maintainers for already existing packages (indirectly it can be run by them periodically on packages they are maintaining).
+
 
+
== What RoM is not ? ==
+
 
+
* It does no change to present workflow for reviews.
+
* It is not a replacement for reviewers.
+
* It is not a replacement for rpmlint.
+
* Every package which goes into fedora 11 or is already there need not pass through it.
+
  
 
== Comments and Discussion ==
 
== Comments and Discussion ==
Line 100: Line 106:
 
See [[Talk:Features/ReviewOMatic]]
 
See [[Talk:Features/ReviewOMatic]]
  
----
+
[[Category:FeaturePageIncomplete]]
 
+
[[Category:FeatureReadyForWrangler]]
+

Latest revision as of 01:57, 22 January 2009

Contents

[edit] ReviewOMatic

[edit] Summary

An application that reviews packages according to the Package Review Guidelines and generate a report. It also include automates all common steps which every reviewer performs for each review and building guideline checks for all Package guidelines which can be automated.

Review-o-matic is an application which will be a feature of the infrastructure. It will be provided as a hosted services and can also be set up locally.

[edit] Owner

[edit] Current status

  • Targeted release: Fedora 11
  • Last updated: 2009-01-06
  • Percentage of completion: 15%

[edit] Detailed Description

Review-o-matic would be an application which takes a Spec or source RPM through its feeder and does a review and posts its report via its reporter interface. Feeder will have different interfaces to get input from Bugzilla or any generic URI. Similarly, reports can be provided as Bugzilla comments or as a local file. Our process also includes(very much obviously) running rpmlint on input files. We plan to push all guidelines checks to upstream rpmlint. But there are some hybrid checks which:

  • are fedora specific
  • require input from logs generated via builders or multiple file inspection and are very much out of scope of rpmlint

will need to be kept in review-o-matic.

(Check table) is were inspecting each checks one by one and decide on which are programmable and which are not, takes place.

Plan is to speed up automating guidelines once structural architecture get working with very basic capabilities and later on concentrate on automating guidelines.

Structural Architecture of whole design for now is:

Rom1.png

Source: https://fedoraproject.org/wiki/Image:Rom1.dia



[edit] Benefit to Fedora

Fedora is known for setting high standards in RPM packaging. With a repository that is growing each day, and our emphasis on working upstream we have a collection of packages that makes us proud, our derived distributors happy and our competitors envious. However to maintain high standards we need to make sure that package review requests get the necessary time and attention to meet the large queue of new submissions.

  • This is an effort to take away the grunt work from the review process and make it easier for both the reviewer and the submitter to get an initial idea about the quality of their packages by taking a Spec/SRPM pair and running it through Review-O-Matic and getting a report.
  • It will be possible to do periodic audits of the entire tree, similar to what Matt Domsch's is currently doing to uncover FTBFS packages. With Review-O-Matic it will be possible go beyond detecting build failures and uncover a whole bunch of other violations as well. This might be lucrative to third parties, having a stake in the quality of the distribution (eg., derived distributors like CentOS, academic or enterprise users, etc.), to carry out regular quality analysis on our repositories. Currently it is difficult, albeit possible, to do this using a mix of RPMLint, Mock and some ad-hoc scripts. Since Review-O-Matic ties together these utilities, and adds a few customised Fedora specific checks written by the Fedora developers themselves, one can rest assured that the generated reports are in accordance with the latest guidelines approved by the Fedora Packaging Committee.

[edit] Scope

Requires completing the structural design and writing checks or guideline programs for those packaging guidelines which can be automated.

[edit] Use Cases

  • A newbie submitting a package wants to do an automated review on the Spec/SRPM that he/she is going to submit.
  • A reviewer wants to run a Spec/SRPM through the mundane steps of running them through RPMLint, building it in Mock/Koji, etc..
  • Maintainers, release engineers, derived distributors wants to audit an entire repository for packaging guideline violations, build failures, etc..

[edit] What RoM is not ?

  • It does no change to present workflow for reviews.
  • It is not a replacement for reviewers.
  • It is not a replacement for rpmlint.
  • Every package which goes into fedora 11 or is already there need not pass through it.


[edit] How To Test

  • Making test specs, rpms and srpms and analyszing the reports generated and comparing them to expected output. Unit tests or doc tests are in plan.

[edit] User Experience

  • Application can be initiated locally, via Bugzilla, via web service.
  • Developers/Maintainers input files (spec, srpm, rpm) to be reviewed.
  • Generates reports and presents via Bugzilla, web service or output file.

[edit] Dependencies

  • rpmlint
  • rpm-python
  • python-bugzilla
  • koji
  • mock

[edit] Contingency Plan

Not Needed.

[edit] Documentation

Right now wiki has some bare minimum documentation: https://fedorahosted.org/review-o-matic/wiki

[edit] Release Notes

  • Irrelevant since it will not directly impact the end-user
  • DRAFT: During the Fedora 11 development cycle a new tool was developed called ReviewOMatic. ReviewOMatic reviews packages according to the Package Review Guidelines and generates a report. It also include automates all common steps which every reviewer performs for each review and building guideline checks for all Package guidelines which can be automated. A hosted instance of Review-o-matic provided to developers in the Fedora infrastructure.

[edit] Comments and Discussion

See Talk:Features/ReviewOMatic