GSOC 2012/Student Application nehaljwani/OnDemandBuildService

=On-Demand Fedora Build Service=

Proposal Description
This page shall describe the proposed project.

An overview of your proposal
On Demand Build Service: This project is intended to help the developers, testers, etc who need to custom-install Fedora for specific purposes by creating a web interface capable of:


 * Building and hosting live-images of Fedora [Desktop/KDESpin/LXDESpin/XfceSpin]
 * Retrieving packages from the following repos : release,updates,update-testing,koji
 * User will be able to specify the version of the package required * Pull these newer builds (that wouldn't otherwise be pulled in to the .iso) in to the image build.
 * Estabilishing a Client-Login system via open-id.

and also provide an API as well as CLI for easy-to select, grab and push.

The need you believe it fulfills
This will help clients in the following manner:


 * First, testers will easily be able to judge the performance of unstable programs on the TC/RC builds.
 * The project will resolve the problem of 'yum install' this, 'yum install' that for each program that a client of a specific organisation/institution requires for his work. Admins/Server People will find this tool very helpful as they can build an image in custom mode and use it on several workstations. Also, people running slow internet connections will find this beneficial.
 * Having version of the packet specified by the user will enable the him to work with the version in which he/she feels comfortable and also the developer will be able to check compatibility issues.

Any relevant experience you have
Well, I have experience in automation using scripts and I have the knowledge of creating LiveCDs. Currently, I am exploring the packaging services of Fedora which include building rpms and pushing them into repositories.

How do you intend to implement your proposal

 * Capturing packages :

-Creation of a pseudo-repo using metadata which will just contain the list of packages that are available and can be pulled from repos: updates, update-testing, koji and release. This pseudo-repo will be updated regularly so that development versions of various packages are easily available. This repo will not have physical copies of the packages, instead it will just have links to the packages kept in the original repositories.

-Upon the selection of client, a temporary side-repository will be built and the required packages for the LiveCD will be kept there till the finish of the build.


 * Development of web-based UI:

1. The client will be able to create an account which will store all of his previous requests. So that if he wants to make minor changes to previous ones, it will be easy.

2. Since the requests will be pushed into a job queue and will be taken care of by the master node, status of the ongoing process will be available to the client. [In case of multiple requests, or request in queue, the build process might take some time]

3. Once the image is made, it will be available for download from the image-hosting service for a short span of time, say 5 days. Client will be notified and the script used for generating will be stored in for future use.

Key Points:
 * Implementation of kickstart file
 * Implementation of job queue management system
 * Handling the master node
 * Implementation of the web-interface

A Concise Walkthrough Of The Entire Process:


 * Client selects the required pacakages from the pseudo-repo.
 * A temporary side-repo is generated, which holds all the required packages.
 * The KickStart file is generated as per the request.
 * The Job Queue system handles the remaining process of builing the live image.
 * After lifecycle of 5 days, the image is deleted and requests are logged.

=*= Further, I would like to add that since the build tools are going to change after F18, we'll be maintaining a parallel code and keep inculcating changes by following the process going on at the development level of the buid tools. Needless to say, this will help in easily shifting from old to new base as soon as newer versions of the build tools are released.

Final deliverable of the proposal at the end of the period

 * Build Services being processed through web2py on the master node
 * Client build services running on designated build nodes
 * A REST API to the build service
 * Sqlite/MySql storing old requests along with the scripts generated

A rough timeline for your progress

 * Till May 15: Developing an automated script like this [Made By Mentor] for retrieving data from repos and creating pseudo-repos. Note: This is just a basic model. The script needs to be modified to suite the needs of the project and make it error-free.


 * May 16 - June 15: Testing Scripts and implementing the build services, implementation of client services


 * June 16 - July 15: Testing of http/ftp interface where the image will be hosted, development of RESTful API and job queueing system


 * Mid-Term Evaluation: Web-UI will be running and image building services will be available with primitive features. Parallel hosting will have to wait.


 * July 16 - August 5: Testing of API and queueing system, improving the user interface for normal users, adding xtra features [if time permits]


 * August 5 - August 10: Final testing and documentation

Any other details you feel we should consider
1. Since this is my first project, I'll be happy to handle the part concerning scripting and development of web-ui. I'll explore the implementation of RESTful API and build servcies on the go.

2. I would like to add-in sone extra features to the main idea which can be implemented after the base of the project has finished, for eg:
 * Adding/Removing packages from official releases.
 * Specifying Image Size priorities [optional feature for those who have slow connections], so that if a new version of a particular package has more dependencies, then older one will be taken and size will be controlled under specified limits.

Have you communicated with a potential mentor? If so, who?

 * I have communicated with Tim Flink, who provided me the main idea behind this project and helped me make this proposal.

Other Contact:

 * Potential Student Partner :Amit K Saha