Summer Coding 2010 proposal - Copr TG app

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(answer 3)
(Timeline: wrote a more detailed timeline)
Line 169: Line 169:
  
 
=== Timeline ===
 
=== Timeline ===
* one week - look into the buildsystem API and setup the TG skeleton
+
 
* first half - write coprs related methods
+
* 1st Week - Since the buildsystem part of Copr is still being built, I'll need to take some time at the beginning of the project to understand what has been built. This time will also be used for setting up the TG app.
* second half - write package related methods
+
* 2nd Week - work on building the DB schema for Coprs (collections of repositories)
* one week - write wiki page describing the API, write tutorials, examples etc.
+
* 3rd Week - code CRUD operations on coprs
 +
* 4th Week - code copr dependency management
 +
* 5th Week - build database schema for packages and builds
 +
* 6th Week - code packages and build related methods
 +
 
 +
The TG app should now be fully functional, but the code will still be subject to change, including the API.
 +
 
 +
''- midterm evaluation due -''
 +
 
 +
* 7th Week - deploy the TG app on a public Fedora Infrastructure testing server and engage the community for feed-back. All the time afterwards will probably be spent on improving the API from the feedback in addition to whatever else must be done that week.
 +
* 8-9th Week - write wiki documentation for all the API methods with examples
 +
* 10th Week - write a simple copr command line client just to showcase the basic features
 +
* 11th Week - time that will probably be taken up by unforeseen occurences like the buildsystem API changing during coding.
  
 
=== Convincing ===
 
=== Convincing ===

Revision as of 21:21, 24 May 2010

Contents

About me

  1. What is your name?
    • Ionuț Arțăriși
  2. What is your email address?
    • mapleoin@fedoraproject.org
  3. What is your wiki username?
  4. What is your IRC nickname?
    • maploin
  5. What is your primary language? (We have mentors who may speak your preferred languages and can match you with one of them if you request.)
    • Romanian. English is fine, thank you.
  6. Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)
    • Romania. My working hours vary a lot. Anytime between 6AM and 12PM GMT.
  7. Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?
  8. Bonus level: What's your schedule and why?
    • I'm a terminal year college student in Romania where school officially ends in July. Since this is also the year when I have to write my thesis I'd like to deviate from the standard Summer Coding schedule. My first term will be the standard second term and I'll add another term starting from the standard midterm evaluation date. I'm aligning with a few standard dates to try and keep things as simple as possible for others. I can and will probably start working a bit earlier. So it would look like this (11 weeks in total):
      1. First term - 12 July - 16 August (my midterm evaluation date)
      2. Second term - 17 August - 27 September (student final report etc.)

Copr TG front-end

Description

The Copr TG front-end is part of the Cool Other Package Repo (Copr) Fedora Project to build and manage third-party user repositories. It is a TurboGears 2 web application that handles communication with the end-user on one side and with the buildsystem on the other side. It gets commands from the users (packages to build, repos to create etc.) and gives them links to builds/repos it has. Together with the website and the command-line client, it is the main way for users to interact with Copr and their repos. It passes the commands it gets from the users to the buildsystem.

The app will have to interact with the buildsystem's API and also provide a JSON API to be used by the different clients (website and command line).

Package stuff

packages.py

User wants a package added to one of his coprs so he sends a SRPM to the app. The following information must also be present in the query:

  • arches to be built on
  • copr to be built into
  • coprs that will be included in the build environment.

User can also:

  • query a package for different information
  • delete a package from one of her repos.
  • disable a package from one of the repos, so that future builds can be done against a previous version of the package in case of brokage
  • grant rights to one of her repos to other users.
def newpkg(self, srpm, archs='all', repo=None, deprepos=None)
    """Receive a SRPM and send it to the buildsystem after a check-up.

    The SRPM will have to be checked for license conformance etc.
    Returns:
    - lots of different informative errors about why the srpm is wrong
    or an OK message and a redirect to the package page if the request came
    from the webui

    Keyword arguments:
    :srpm: a real and (hopefully) Fedora-conformant SRPM file
    :archs: one of the supported Fedora arches: i686, ppc, ppc64 etc.
    :copr: a unique identifier of a copr, could default to
           the user`s default copr
    :dep_collections: a list of unique ids of Coprs
    """

def info(self, copr, pkgname)
    """Get metadata and other information about a package.

    Information about a package includes: name, version, release, license,
    summary, state, builds (list of unique ids)

    Returns:
    :package: a Package object

    Keyword args uniquely identify a package:
    :copr: name of a copr
    :pkgname: name of a package
    """

def disable(self, copr, pkgname):
    """Disable a package, but don't delete it from copr.

    Keyword args uniquely identify a package:
    :copr: name of a copr
    :pkgname: name of a package
    """


def delete(self, copr, pkgname):
    """Delete a package from a user-controlled copr.

    Also deletes all its associated builds in that copr.

    Returns some OK/not OK message.

    Keyword args uniquely identify a package:
    :copr: name of a copr
    :pkgname: name of a package
    """

def grant(self, copr, username, rights):
    """Change rights of a user on a user owned copr.

    Keyword arguments:
    :copr: name of a copr
    :username: username of a Copr user
    :rights: a list of rights such as ['push', 'delete', 'admin' etc.
    """

def builds(self, copr, pkgname):
    """Return a list of builds for the package.

    Returns:
    :builds: list of Builds

    Keyword arguments:
    :copr: name of a copr
    :pkgname: name of the package
    """

Coprs stuff

coprs.py

User can:

  • create a new copr, specifying a name and one of the supported architectures + fedora/epel release.
  • add/remove dependency coprs.
  • delete one of her coprs.
  • list owned coprs and copr she has writing rights to.
  • request a .repo file from any copr in Copr.
    
def create(self, name, release):
    """Create a new copr.

    Redirects to the repo information page if successful.
    
    Keyword arguments:
    :name: name of the new copr
    :release: one of the existing Fedora/EPEL releases
    """

def list(self, username=current_username):
    """List all the coprs that this user has any privileges to.

    Returns a list of copr labels.

    :kwarg username: username of a Copr user2
    """

def delete(self, copr):
    """Delete a given user-owned repo.

    :kwarg copr: a copr label
    """

def get_repo(self, copr, arch):
    """Retrieves a .repo file of a specific copr for the given arch.

    Returns a .repo file
    
    :kwarg copr: a copr label
    :kwarg arch: one of the supported architectures
    """

def add_dependency(self, copr, dependency):
    """Add another copr as a dependency of the current copr.

    All the packages in a dependency copr are included in the current copr's
    buildroot.

    :kwarg copr: a copr label
    :kwarg dependency: another copr label
    """

def remove_dependency(self, copr, dependency):
    """Remove one of the copr's current dependencies.

    :kwarg copr: a copr label
    :kwarg dependency: another copr label
    """


The technologies that will be used: TurboGears2, postgreSQL, Copr

Timeline

  • 1st Week - Since the buildsystem part of Copr is still being built, I'll need to take some time at the beginning of the project to understand what has been built. This time will also be used for setting up the TG app.
  • 2nd Week - work on building the DB schema for Coprs (collections of repositories)
  • 3rd Week - code CRUD operations on coprs
  • 4th Week - code copr dependency management
  • 5th Week - build database schema for packages and builds
  • 6th Week - code packages and build related methods

The TG app should now be fully functional, but the code will still be subject to change, including the API.

- midterm evaluation due -

  • 7th Week - deploy the TG app on a public Fedora Infrastructure testing server and engage the community for feed-back. All the time afterwards will probably be spent on improving the API from the feedback in addition to whatever else must be done that week.
  • 8-9th Week - write wiki documentation for all the API methods with examples
  • 10th Week - write a simple copr command line client just to showcase the basic features
  • 11th Week - time that will probably be taken up by unforeseen occurences like the buildsystem API changing during coding.

Convincing

I've been a Fedora Project contributor since May 2008 when I began working on the Fedora PackageDB with Toshio Kuratomi. I worked on small features, visible to the general Fedora user. One of the features I'm most proud of that I worked on in that period was an advanced package search functionality.

Last summer I participated in GSOC working on a bunch of features for the pkgdb aimed at making it more user-centric, rather than developer-centric. These included importing metadata from yum, a new package page with commenting and tags and exporting those tags for future use in yum via sqlite. This work was later adapted and included into the 0.5.x series of the PackageDB after some design changes to PackageDB itself.

At the end of the summer I began contributing to the Fedora Project as a Package Maintainer and have contributed this way since then. I've also begun a personal project of a Romanian FOSS advertising network. More info on the project on its webpage (Romanian) or on github (code).

Me and the community

Impact

If your project is successfully completed, what will its impact be on the Fedora community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Fedora community, at least one of whom should be a Fedora Summer Coding mentor. Provide email contact information for non-Summer Coding mentors.

Ionuț's answer

This is a part of the larger Copr project of creating third-party Fedora user repositories. These would empower users to easily distribute their own versions of floss applications. More customized packages can be used to cover a larger pool of user needs. Having all the different variations in one place also reduces redoing work that has already been done by others.

The TG app part of Copr is a central part of the system, being the middle-layer between the different Copr clients and the actual buildsystem.

Toshio's answer

Having this done would be very beneficial for the Copr project. There's three main pieces to Copr and this would be writing the necessary functionality for one of those pieces -- the one that actually lets packagers submit their builds to the buildsystem. Talking with Ionut, if he has time after this is done, he'd be able to contribute to other pieces of Copr as well.

Copr itself helps out the Fedora Community by giving us a means to deploy packages that aren't in Fedora. I see a few major use cases there:

  1. Packages that are under review -- I often have several packages in review and having these packages in a repository would make it easier for others to use the packages until they get into Fedora proper
  2. Packages that are at different versions than the Fedora package -- Sometimes we either want to test out new features on a stable release before building them for everyone to consume or we want to build new versions on a stable release for people who are early adopters but not push it to a released Fedora. This gives us a place to do that
  3. Packages that are too hard to package for Fedora -- Although we would ideally have all packages in Fedora, there are certain pieces of software that are hard to package and that people want to use despite that. Chromium and zimbra spring to mind. Copr provides a way to host these.
  4. Alternate versions of packages run at a particular site -- I know in Fedora Infrastructure we have a few packages that we've had to build locally due to needing newer versions or simultaneous, parallel installable versions, or, in a few cases, the package not being applicable to people outside of the Infrastructure group. Having a public repository for those packages is great -- having an easy way to manage the repositories would be even beter.

skvidal's answer

coprs being a relatively new and relatively young project affords a contributor the opportunity to be involved without having to trudge through an immense amount of startup.

If we get these sections done they help us drive ahead with the project and, hopefully, have a working coprs system by the time summer coding is over.

Sooner is always better than later.

Since coprs impacts a lot of users and developers in fedora having things done quicker is going to earn a good number of points.

Other questions

  1. What will you do if you get stuck on your project and your mentor isn't around?
    • I generally find my way out of most technical problems by googling, going deeper into the source code or just asking on the relevant IRC channel (#turbogears, #python etc.)
    • For design problems, though these are less likely to happen, I know my way around the community mailing lists and IRC channels and generally know who's who. If my mentor will be missing for several days I'll probably start asking for advice on #fedora-admin and #fedora-devel.
  2. In addition to the required blogging minimum of twice per week, how do you propose to keep the community informed of your progress and any problems or questions you might have over the course of the project?
    • I'm constantly lurking on IRC in #fedora-devel and #fedora-admin and am also subscribed to the relevant mailing lists.
  3. Optional coding test: https://fedoraproject.org/wiki/Summer_Coding_pre-qualifying_ideas#Write_a_func_module_to_check_license_tags
    1. http://github.com/mapleoin/flice/blob/master/flice.py

Miscellaneous

  1. We want to make sure that you are prepared before the project starts
    • Can you set up an appropriate development environment?
      • sure
    • Have you met your proposed mentor and members of the associated community?
      • Yes, I've worked with my mentor before and have been part of the Fedora community for 2 years.
  2. What is your t-shirt size?
    • M
  3. Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?
    • I like peppermint tea.