Summer Coding 2010 proposal - Copr TG app

Description
The Copr TG front-end is part of the Cool Other Package Repo (Copr) Fedora Project to build and manage third-party user repositories. It is a TurboGears 2 web application that handles communication with the end-user on one side and with the buildsystem on the other side. It gets commands from the users (packages to build, repos to create etc.) and gives them links to builds/repos it has. Together with the website and the command-line client, it is the main way for users to interact with Copr and their repos. It passes the commands it gets from the users to the buildsystem.

The app will have to interact with the buildsystem's API and also provide a JSON API to be used by the different clients (website and command line).

Package stuff
packages.py

User wants a package added to one of his coprs so he sends a SRPM to the app. The following information must also be present in the query:
 * arches to be built on
 * copr to be built into
 * coprs that will be included in the build environment.

User can also:
 * query a package for different information
 * delete a package from one of her repos.
 * disable a package from one of the repos, so that future builds can be done against a previous version of the package in case of brokage
 * grant rights to one of her repos to other users.

def newpkg(self, srpm, archs='all', repo=None, deprepos=None) """Receive a SRPM and send it to the buildsystem after a check-up.

The SRPM will have to be checked for license conformance etc.   Returns: - lots of different informative errors about why the srpm is wrong or an OK message and a redirect to the package page if the request came from the webui

Keyword arguments: :srpm: a real and (hopefully) Fedora-conformant SRPM file :archs: one of the supported Fedora arches: i686, ppc, ppc64 etc.   :copr: a unique identifier of a copr, could default to           the user`s default copr :dep_collections: a list of unique ids of Coprs """

def info(self, copr, pkgname) """Get metadata and other information about a package.

Information about a package includes: name, version, release, license, summary, state, builds (list of unique ids)

Returns: :package: a Package object

Keyword args uniquely identify a package: :copr: name of a copr :pkgname: name of a package """

def disable(self, copr, pkgname): """Disable a package, but don't delete it from copr.

Keyword args uniquely identify a package: :copr: name of a copr :pkgname: name of a package """

def delete(self, copr, pkgname): """Delete a package from a user-controlled copr.

Also deletes all its associated builds in that copr.

Returns some OK/not OK message.

Keyword args uniquely identify a package: :copr: name of a copr :pkgname: name of a package """

def grant(self, copr, username, rights): """Change rights of a user on a user owned copr.

Keyword arguments: :copr: name of a copr :username: username of a Copr user :rights: a list of rights such as ['push', 'delete', 'admin' etc.   """

def builds(self, copr, pkgname): """Return a list of builds for the package.

Returns: :builds: list of Builds

Keyword arguments: :copr: name of a copr :pkgname: name of the package """

Coprs stuff
coprs.py

User can:
 * create a new copr, specifying a name and one of the supported architectures + fedora/epel release.
 * add/remove dependency coprs.
 * delete one of her coprs.
 * list owned coprs and copr she has writing rights to.
 * request a .repo file from any copr in Copr.

def create(self, name, release): """Create a new copr.

Redirects to the repo information page if successful. Keyword arguments: :name: name of the new copr :release: one of the existing Fedora/EPEL releases """

def list(self, username=current_username): """List all the coprs that this user has any privileges to.

Returns a list of copr labels.

"""

def delete(self, copr): """Delete a given user-owned repo.

"""

def get_repo(self, copr, arch): """Retrieves a .repo file of a specific copr for the given arch.

Returns a .repo file :kwarg copr: a copr label :kwarg arch: one of the supported architectures """

def add_dependency(self, copr, dependency): """Add another copr as a dependency of the current copr.

All the packages in a dependency copr are included in the current copr's   buildroot.

:kwarg dependency: another copr label """

def remove_dependency(self, copr, dependency): """Remove one of the copr's current dependencies.

:kwarg dependency: another copr label """

The technologies that will be used: TurboGears2, postgreSQL, Copr

Timeline

 * 1st Week - Since the buildsystem part of Copr is still being built, I'll need to take some time at the beginning of the project to understand what has been built. This time will also be used for setting up the TG app.
 * 2nd Week - work on building the DB schema for Coprs (collections of repositories)
 * 3rd Week - code CRUD operations on coprs
 * 4th Week - code copr dependency management
 * 5th Week - build database schema for packages and builds
 * 6th Week - code packages and build related methods

The TG app should now be fully functional, but the code will still be subject to change, including the API.

- midterm evaluation due -


 * 7th Week - deploy the TG app on a public Fedora Infrastructure testing server and engage the community for feed-back. All the time afterwards will probably be spent on improving the API from the feedback in addition to whatever else must be done that week.
 * 8-9th Week - write a simple Copr command line client. This will be a working prototype, but will likely have some rough edges.
 * 10th Week - write wiki documentation for API methods with examples
 * 11th Week - time that will probably be taken up by unforeseen occurrences like the buildsystem API changing during coding.

Convincing
I've been a Fedora Project contributor since May 2008 when I began working on the Fedora PackageDB with Toshio Kuratomi. I worked on small features, visible to the general Fedora user. One of the features I'm most proud of that I worked on in that period was an advanced package search functionality.

Last summer I participated in GSOC working on a bunch of features for the pkgdb aimed at making it more user-centric, rather than developer-centric. These included importing metadata from yum, a new package page with commenting and tags and exporting those tags for future use in yum via sqlite. This work was later adapted and included into the 0.5.x series of the PackageDB after some design changes to PackageDB itself.

At the end of the summer I began contributing to the Fedora Project as a Package Maintainer and have contributed this way since then. I've also begun a personal project of a Romanian FOSS advertising network. More info on the project on its webpage (Romanian) or on github (code).

Impact
If your project is successfully completed, what will its impact be on the Fedora community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Fedora community, at least one of whom should be a Fedora Summer Coding mentor. Provide email contact information for non-Summer Coding mentors.

Ionuț's answer
This is a part of the larger Copr project of creating third-party Fedora user repositories. These would empower users to easily distribute their own versions of floss applications. More customized packages can be used to cover a larger pool of user needs. Having all the different variations in one place also reduces redoing work that has already been done by others.

The TG app part of Copr is a central part of the system, being the middle-layer between the different Copr clients and the actual buildsystem.

Toshio's answer
Having this done would be very beneficial for the Copr project. There's three main pieces to Copr and this would be writing the necessary functionality for one of those pieces -- the one that actually lets packagers submit their builds to the buildsystem. Talking with Ionut, if he has time after this is done, he'd be able to contribute to other pieces of Copr as well.

Copr itself helps out the Fedora Community by giving us a means to deploy packages that aren't in Fedora. I see a few major use cases there:


 * 1) Packages that are under review -- I often have several packages in review and having these packages in a repository would make it easier for others to use the packages until they get into Fedora proper
 * 2) Packages that are at different versions than the Fedora package -- Sometimes we either want to test out new features on a stable release before building them for everyone to consume or we want to build new versions on a stable release for people who are early adopters but not push it to a released Fedora.  This gives us a place to do that
 * 3) Packages that are too hard to package for Fedora -- Although we would ideally have all packages in Fedora, there are certain pieces of software that are hard to package and that people want to use despite that.  Chromium and zimbra spring to mind.  Copr provides a way to host these.
 * 4) Alternate versions of packages run at a particular site -- I know in Fedora Infrastructure we have a few packages that we've had to build locally due to needing newer versions or simultaneous, parallel installable versions, or, in a few cases, the package not being applicable to people outside of the Infrastructure group.  Having a public repository for those packages is great -- having an easy way to manage the repositories would be even beter.

skvidal's answer
coprs being a relatively new and relatively young project affords a contributor the opportunity to be involved without having to trudge through an immense amount of startup.

If we get these sections done they help us drive ahead with the project and, hopefully, have a working coprs system by the time summer coding is over.

Sooner is always better than later.

Since coprs impacts a lot of users and developers in fedora having things done quicker is going to earn a good number of points.

Other questions

 * 1) What will you do if you get stuck on your project and your mentor isn't around?
 * 2) * I generally find my way out of most technical problems by googling, going deeper into the source code or just asking on the relevant IRC channel (#turbogears, #python etc.)
 * 3) *For design problems, though these are less likely to happen, I know my way around the community mailing lists and IRC channels and generally know who's who. If my mentor will be missing for several days I'll probably start asking for advice on #fedora-admin and #fedora-devel.
 * 4) In addition to the required blogging minimum of twice per week, how do you propose to keep the community informed of your progress and any problems or questions you might have over the course of the project?
 * 5) *I'm constantly lurking on IRC in #fedora-devel and #fedora-admin and am also subscribed to the relevant mailing lists.
 * 6) Optional coding test: https://fedoraproject.org/wiki/Summer_Coding_pre-qualifying_ideas#Write_a_func_module_to_check_license_tags
 * 7) http://github.com/mapleoin/flice/blob/master/flice.py

Miscellaneous

 * 1) We want to make sure that you are prepared before the project starts
 * 2) * Can you set up an appropriate development environment?
 * 3) **sure
 * 4) * Have you met your proposed mentor and members of the associated community?
 * 5) **Yes, I've worked with my mentor before and have been part of the Fedora community for 2 years.
 * 6) What is your t-shirt size?
 * M


 * 1) Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?
 * 2) * I like peppermint tea.