Devshell API

The following document explains the key components in Fedora Devshell, and how to extend it for your own nefarious purposes. The Modules section addresses the kinds of actions Fedora Devshell can do, and how they are implemented. The Utility functions highlights some of the key components used as building blocks. The Coding Guidelines are critically important, as they must be followed to in order to submit patches. The Submission Guidelines explain how to get your patches in.

Directory
is a Base Class for pretty much any class needing persistence on the filesystem. It manages storing metadata attributes in a simple INI-style file under .devshell in each directory. It establishes the link between some directory on the file system and the operations available from the command line, inside the source code, and from any IDE. The initializer takes one optional parameter, a path to the directory on the file system, and it loads up the  from there. If not given,  uses the current working directory.

It exposes these methods:
 * is_sysdir_dir(dir) - given some directory, it determines if this is a  managed by Fedora Devshell
 * load_dir(dir) - a hook to be run when loading some directory that has already been created. do not call this directly, although feel free to override it in subclasses. make sure to call the parent method
 * make_dir(dir) - a hook to be run when creating a  out of some new directory. do not call this directly, although feel free to override it in subclasses. make sure to call the parent method
 * close - a hook to save persisted information when Fedora Devshell exits. safe to call directly, the results should be idempotent. feel free to override it in subclasses, make sure to call the parent method
 * rename(new_name) - a hook that renames the directory, assuming it's been changed on the file system to match already. feel free to override it in subclasses, make sure to call the parent method
 * move(self, new_loc) - moves the directory to a new location, and calls rename
 * copy(self, new_loc): - copies the directory to a new location and calls rename, returns the new  object

And properties:
 * name - the name of the directory on the file system
 * dir - the absolute path to the directory on the file system
 * parent - the absolute path to the parent of the directory on the file system

MetaDirectory
This metaclass registers all directory objects with DirFactory

DirFactory
A subclass of  that will return the correct subclass of   for a given directory on the filesystem, rather than the base class.

Package
is an overarching manager for packages that will eventually become RPMs. There should be a one-to-one correspondence with  and a source RPM. There will also theoretically be a one-to-one correspondence between  and entries in the Fedora CVS repository. It manages a spec file, a list of sources, patches for the actively chosen s and other administrative details.

It exposes the following methods:


 * add_spec(spec_file) - given some spec file, set the current spec file to be this one.
 * get_srpm_name(profile) - given a profile, determine the name of the source RPM
 * ver(profile=None) - given a possible profile, determine the upstream version from the current spec file
 * copy_source(source_dir) - given some, copy it into this   and add it as a source
 * move_source(source_dir) - given some, move it into this   and add it as a source
 * add_source(source_dir) - given some  directory, assuming it is already inside the   directory, add it as a source
 * rem_source(source) - remove a  from the source list, without deleting it from the file system
 * del_source(source) - delete a  from the file system and remove it from this
 * fetch_sourceballs(profile=None) - with an optional profile for version information, fetch all the source tarballs from the s belonging to this package. To do so, it creates symlinks to save space

The following properties:
 * spec_file - the name of the current spec file
 * pkg_name - the canonical name of the package in the Fedora CVS repository
 * sources - a list of the s
 * sourceballen - a list of all the source tarballs that have been fetched at some point

Possible Package Types
s can be managed in all sorts of ways. Fedora currently uses CVS to manage the workflow. Other people might want to use. In the future, we will need modules to manage these different workflows

BuildSystem
is a Python class of type. s are registerd with a, which can load any   loaded in the run time space. It's initializer takes one argument, a  either on the file system or a Python object. is an interface on top of common build systems such as autotools, cmake, and cabal.

It exposes five methods.


 * configure(target, *args) - runs the configure stage of the build system - can be a noop. target points to where to install it on the system, *args are passed to the underlying package source to work on a specific branch.
 * build(*args) - runs the build stage - should not be a noop. *args are passed to the underlying package soruce to work on a specific branch.
 * install(*args) - runs the install stage - cannot be a noop. *args are passed to the underlying package soruce to work on a specific branch.
 * install_source(target, *args) - configures, builds, and installs some source. target points to where to install it on the system, *args are passed to the underlying package source to work on a specific branch.
 * gen_spec - creates a spec file specific to this build system

Cabal
is an implementation of a. It works on top of The Haskell Cabal, which is the build tool used for building Haskell packages. It uses the program  provided in Fedora for generating the spec file.

Possible BuildSystem Types
Currently we only support. Plugging in other build systems should be easily feasible, possible options include Autotools, Ant, Python's setuptools, Ruby Gems, CMake, and many other build systems. We may want to look into handling multiple build targets for cross compiling.

PackageSource
is a Python class of type. s are registered with a, which can load any directory from the filesystem as the properly subclassed   in Fedora Devshell. is a single source, be it a tarball or revision control that will go into an RPM Package.

It handles the management of patches between the original upstream source and the downstream sourcetree, generating tarballs and patches, Each implementation manages the storage and generation of these details.

It exposes these methods.
 * setup_sourceball(ver='') - sets up a source tarball if needed out of the revision control. ver is the upstream version
 * setup_sourceball_w_patches(ver='') - not sure of the behavior of this one yet, it's a placeholder
 * src_dir(*args) - a contextmanager that will run a block of code inside a specfic branch or checkout
 * src(*args) - a context manager that will run a block of code with a certain branch or checkout set as the active one.
 * set_current_src - a hook to be implemented by the subclass, notifies the subclass when the active source has changed and runs any arbitrary code.
 * set_cur_to(*args) - where *args is processed. This parses *args, and sets the source to some source directory accordingly. Must be overridden in a subclass
 * branch(*args) - returns the relative path to where a named branch is stored, based on *args.
 * branch_dir(*args) - returns the absolute patch of

It exposes these properties
 * pkg_src - a relative path where internal files to devshell are stored inside the package source
 * pkg_src_dir - the absolute pathname to pkg_src
 * branches - a relative path to where branches are stored inside their package source
 * branches_dir - the absolute pathname to branches
 * sourceball - a relative pathname to the current source tarball equivalent to this package source
 * sourceball_loc - the absolute pathname to sourceball
 * source - a relative pathname to the currently active sourcetree
 * source_dir - the absolute pathname to source
 * buildsystem - the type of buildsystem that this sourcetree uses
 * builder - a reference to a  that can build this package

SourceBall
A  is an implementation of. It maintains both an actively edited work tree, and a copy of the original source tree, along with the original sourcetarball for comparison. It can only generate a single stack of patches, on top its one branch, 'orig'.


 * *args - can either be 'head' or 'orig'.

RevisionControl
Subclasses  for all implementations that use. This is a very boring class.

Darcs
The only implementation of. It wraps around the Darcs Distributed Version Control System. It maintains multiple branches based on parameters passed directly to. It does not yet do patch management.


 * *args - can be one of several possibilities.
 * beings with 'new'  - creates a new branch by that name
 * 'head' - the top level active source tree
 * 'branch'  - switches to a named branch
 * anything else - passes these parameters to darcs and creates an unnamed patch after the hash of the arguments

Examples new foo branch bar - forks bar as foo new foo --tag 0.8 - forks tag 0.8 as foo --tag 0.8 - switches to tag 0.8 --to-match 'hash 4593894085394890' - switches to some patch identified by some hash

Possible PackageSource Types
Currently,  and   are the only supported types. does branching fundamentally different from Git, Bazaar, and Mercurial. Work will need to go into having the paradigms sit together nicely. Note that we aren't creating an all in one solution, but just using the important bits of each VCS to generate patches to go into RPMs. Ultimately, RPMs must be packaged with Tarballs, the use of  is only to ease and aid the process.

Fetcher
Provides an upstream source for multiple packages. Exposes:


 * latest_version(pkg) - returns the latest version of some package by name
 * url(pkg, ver) - given a package and version, return a URL where source can be found, namely source control and/or a tarball
 * latest_url(pkg) - shortcut function, given a package, return a URL to the latest release

Hackage
An implementation of  for downloading packages from HackageDB - The Haskell Package Repository

Possible Fetcher Types
There are many sources for packages, including Sourceforge, Launchpad, Fedora Hosted, CPan, Python Cheese Shop, and many many others. Implementing s is easy, so lets see some more.

Port
A  is analogous to a BSD or Gentoo Port. It combines the above components into a single Module that has an intellegent overview of the whole situation. It can be used to download, setup, configure, install, and package any type of package, so long there is a port to handle it.

It currently exposes:

NOTE: This will very likely turn into a utility function later on.
 * close_later(directory) - given some  object, mark it for closing later.  s may come across some   object it does not own, that it will pass on to the outside world. This marks it for deletion once the system is done running. This ensures that all changes to the   are persisted.

HaskellPort
This will download any package that is hosted on HackageDB or at darcs.haskell.org, test build it, install it, and generate spec files for it. It combines,  ,  , and.

Possible Port Types
Theoretically, there could be one port for every package that is being managed. Many do have in fact their own idiosyncronacies that do need to be managed. Despite that, many components in packages are standardized out of other common systems. One goal of ports is to be able to use as much generic code as possible. For example, a KDE port might use a KDEFetcher, CMakeBuilder, SVNSource, and. We may want to look into other ways to declaratively define s, and even include them as a repository somewhere.

Build
A wrapper for. It subclasses, and takes an rpmdev-tree (such as ~/rpmbuild, or /usr/src/redhat) as the directory. It can then perform common  operations on  s.

It exposes:
 * setup_source(package) - for some, import the spec file, tarballs, and patches into the rpmdev-tree using symlinks to save space and time
 * build_quick_rpm(package, profile=None) - given some package, and some optional profile, build a quick rpm using the current system as the buildroot. NB: This package should never be shared publically, it's only good for testing
 * build_source_rpm(package, profile=None) - given some package and some optional profile, build an SRPM
 * rpmbuild(param, package, profile=None) - runs  with some parameter, on some package with some optional profile, worker function for
 * fetch_rpms(target_dir) - fetches all the RPMs from the rpmdev-tree to some target directory, by moving them. should be run after every invocation of rpmbuild
 * fetch_build(package) - given some package, fetches the expanded, possibly built source tree from the rpmdev-tree, and deposits it in the package. this is for doing error debugging, test, and verification

Mock
is a simple subclass of a Module, it is merely a worker module with no state of its own. It uses a  for all the stateful bits. It does one thing simplemindedly, it builds a source RPM in mock given some profile and rpmdev-tree for doing temporary work. The initializer takes a  and

It exposes:
 * build_rpm(package) - given some package, use the  and   to build it cleanly in

Possible Mock Types
is a bit specific purpose. We may want to define a better clean room type builder. There are many technologies for distributing and cleanly building packages, including, Koji, and many other frameworks. We may want to create a generic RPMBuilder, that  is an implementation of. For example, via a single switch, we could send a package to  on a local system, do a scratch build in Koji, or inject it into a VM that can be sent across a grid using Condor.

Profile
is a profile to be used in. It manages seperate repos of packages built in mock before to be used as a base, version information, vis-a-vis which version of Fedora to use, and other important information about the build profile. It uses its own directory store mock configurations and results from mock.

It exposes the methods: And properties:
 * configure_from_system(branch) - given some branch (the same as in Fedora CVS), make a new profile based on the local system mock configurations (in ) in the directory.
 * dist - eg .fc7 or .olpc2 used in rpmmacro %dist
 * distvar - eg: fedora or rhel
 * distval - eg: 10 or 3 (string form)
 * koji_target - eg: dist-f11
 * dist_defines - a list of parameters for an rpm aware function to redefine certain macros to match the profile
 * mock_cfg - the name of the mock config/profile to be used to compile packages for this profile
 * mock_cfg_dir - directory where profile wide mock settings are kept
 * result_dir - where to store results from mock

Coding Guidelines
The following basic rules must be applied. Exceptions can be granted for any reason, provided there's a good reason.


 * Basic coding standards will follow PEP 8, except where noted below.
 * Code should be as big as your head. If you have a small head or a large monitor, lower your font resolution ;).
 * Code should be terse, and haiku like. It should explain a single concept or action clearly, without being overrun by scaffolding.
 * Scaffolding should be broken down into utility functions, all exposed in the API. They should be documented clearly.
 * Where logical, use context managers. This will isolate the alogrithm from the build up and break down.
 * High nesting of if/then statements is not allowed
 * High nesting of context managers and for loops are allowed, although extreme nesting is discouraged.
 * Read only attributes from  should be exposed as properties as such:

@property def spec_file(self): '''returns the name of the spec file as it should be accordingto fedora guidelines, good for double checking''' return self.pkg_name + '.spec'

@property def pkg_name(self): canonical name of the package in a source repository return self.cfg['pkg_name']

Submission Guidelines
All patches are welcome. The best way to submit patches initially is via email or mailing lists, or sending a pull request from another git repo. In order to get write access to the public repo, there is a one drink minimum. To demonstrate understanding of the Coding Guidelines, the submitter must have at least one other patch approved before gaining write access. This will ensure that the coder has had ample time to learn the simple guidelines.