New rpkg2 ========= .. attention:: This document is a summary of recent discussion of new rpkg2. This document does not aim to be a design document of new rpkg2. Content of each section has been keeping updated and also would be changed in the future. Background ---------- ``rpkg`` is a generic Python package to build package tools that builds package with dist-git and Koji. It is used to develop a few of package tools for various software communities, e.g. ``fedpkg`` for Fedora, ``centpkg`` for CentOS, ``rdopkg`` for RDO, and ``rhpkg`` for Red Hat internal package build. Some of them are built on top of rpkg by extending core class to add specific functionalities, and others, e.g. rdopkg, just call ``fedpkg`` to build packages in Koji. ``rpkg`` provides a set of core features, aka. commands, that allows packagers to manage package's sources files, test changes by building package locally, and do either scratch build or normal build in Koji. Meanwhile, downstream package tools built on top of ``rpkg`` could add their own commands and extend the command line interface so that packagers are able to use the new features. rpkg2 will be a rewritten version of rpkg totally with a proposed new architecture design. Major goals are * to reduce duplicate code when creating downstream package tools * to make it more reusable for creating CLI for downstream package tool * to make it easier to reuse rpkg by other applications to access package information * to make it easier to extend, customize and add new features according to specific community's requirement. For all discussion about this topic so far, please refer to issue `Propose new mechanism to refactor cli codebase`_. .. _Propose new mechanism to refactor cli codebase: https://pagure.io/rpkg/issue/49 Thoughts about new architecture ------------------------------- In new architecture, original rpkg will be modularized and each of them is dedicated for handling specific tasks. rpkg2 will be * a framework of CLI to allow developers to create CLI application easily without copy and paste. * a set of components (or they could be called libraries, modularize modules, whatever the name) that can be used to get various information from package and its git repository and interact with backend package infrastructure like dist-git and lookaside. They may include * **PackageRepo**: interface to get underlying git metadata, e.g. tags, branches * **Package**: interface to get package information (the metadata), e.g. module name, NVR, disttag, distval, distvar, epoch, giturl. * **Distgit**: interface to operate package repository, e.g. clone, commit, push * **Koji**: interface to interact with Koji to build packages, e.g. scratch-build, build, mock-config * **Lookaside**: interface to upload and download sources files * **LocalBuild**: interface to build package locally * **TestToolkit**: provide basic helper classes and methods to help developer to write test cases for downstream package tool easier. all these could be reuse by any other applications. * a framework to allow developers to write new commands for their package tool easily. A new command can be written, distributed and installed outside rpkg as long as it follows some rules defined by rpkg2 so that it is able to be discovered and loaded dynamically in the runtime. A little more details here * rpkg2 provides a set of core commands dedicated to package build and operations with Lookaside, which are good for sharing with downstream package tools. * downstream package tool developers can write their own commands, and those commands can be totally new, a customized based on core commands, or disabled. Currently, this would benefit fedpkg and rhpkg at least. Development ----------- Some development information of rpkg2 would be **Name**: rpkg2 **Repository**: pagure.io. It would be nice to have a place like github's organization, for example https://pagure.io/packagetool/rpkg2. There would be fedpkg2 https://pagure.io/packagetool/fedpkg2 **RPM**: python2-rpkg2 **PyPI**: rpkg2 Each source code file should declare its license rpkg2 should use ``koji`` as much as possible to avoid writing duplicate code of ``koji``. rpkg2 should be compatible with Python 3 by default. It would be nice to have a complete Jenkins pipeline from patch to RPM and PyPI, so that everyone who is interested in rpkg2 can try it easily. Use py.test instead of nose, since nose has been in maintenance for the past several years. Tests should run under both Python 2 and 3. Python 2.6 is the minimum version to support. Python 3.6 is maximum supported version, since it is in Fedora rawhide as of writing this. Do not use `GitPython`_, and use git command directly instead. Major reasons * GitPython does not support some core git feature, AFAIK which is worktree so far. IIRC, someone requested to support worktree for packaging, and it is really a useful feature for packager. * In rpkg, GitPython is usually used to get config items from repository, which can be migrated to a subprocess call against git command simply. * Other popular Python package for git is `pygit2`_, whose documentation tells "It is likely to work with Python 2.6 and 3.1, but these versions are not officially supported.", but supporting Python 2.6 is still required for rpkg2. However, GitPython can still be used in tests. .. _GitPython: http://gitpython.readthedocs.io/en/stable/ .. _pygit2: http://www.pygit2.org/ What you can do with rpkg2 -------------------------- Some use cases that could be done with rpkg2. Get package version from package repository ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An application could import rpkg2 and call a method to get the version. Before calling, the application has to ensure repository's path has been passed properly. .. code:: python from rpkg2.packagerepo import PackageRepo version = PackageRepo('path/to/repo').ver Write a new command document-build ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ New command ``document-build`` would be used for building product documentation in Koji, and it should be added to downstream package tool fedpkg. But, it is not a good idea to make it available along with fedpkg's installation, since this command is only useful for a small range of users and installing many dependent packages would be a noise for other packagers. So, as a developer, I can do following * start a separate repository as a new project * write some code to implement command ``document-build`` * hacking, testing, hacking testing ... * distribute this new project in either Python package or RPM * install the package Now, command ``document-build`` should be available and can be used with fedpkg:: fedpkg document-build -h Known Issues ------------ Here lists some known issues that rpkg2 should fix them. * `#164`_ Reuse bash completion .. _#164: https://pagure.io/rpkg/issue/164 To be determined ---------------- * Support el6 and epel7? Major consideration is some of dependent packages are too old in repository, especially in el6.