From Fedora Project Wiki

No edit summary
No edit summary
Line 1: Line 1:
== Fedora Package Process ==
=Introduction=
This cheat sheet will demonstrate how to create a RPM package using eclipse Fedora Packager.
=== Create Account ===
If you are a new package maintainer:
* Create a new account on [https://admin.fedoraproject.org/accounts Fedora Account System (FAS)]
** Click on 'New account' and fill in the blanks.
**After you create your account, please be sure to sign the CLA (if you click on the "My Account" link in the top right, you should see CLA: CLA Done).
**Also you need to upload a public RSA SSH key. You need to use the matching private key to access Fedora machines via SSH.
*** Here is information on [http://www.openbsd.org/cgi-bin/man.cgi?query=ssh-keygen creating SSH key].
* Create an account in Red Hat [https://bugzilla.redhat.com/createaccount.cgi bugzilla].
* Install the client tools
** To build Packages for the Fedora Collection or [http://fedoraproject.org/wiki/EPEL EPEL], you need [http://fedoraproject.org/wiki/Using_the_Koji_build_system Koji].
** You'll also need to [https://admin.fedoraproject.org/accounts/user/gencert generate a client side certificate at the Fedora Account System] and save the file in ~/.fedora.cert, where fedpkg will look for it by default.
** You can now use "koji" to try to build your RPM packages on platforms (e.g., PPC) or distributions you don't have. Note that you can test out builds ("scratch" builds) even when your package hasn't been approved and you don't have a sponsor. [http://fedoraproject.org/wiki/Join_the_package_collection_maintainers#Install_the_Client_Tools_.28Koji.29 Here] is a simple way to do a scratch build using koji on the command line.


=== Make a Package ===
There are many ways to generate help content in Eclipse. One particular method involves generating your help content from the wiki which allows you to crowdsource your documentation. By having your documentation on the wiki, you lower the barrier of entry for people to contribute documentation. The purpose of this wiki entry is to guide you through an example of how you can crowdsource your documentation using Mylyn WikiText.
* Make sure it is a new package. Check this [https://admin.fedoraproject.org/pkgdb/acls/list/ list] of existing packages.
* Create an RPM package:
** Select "File->New->Other..." from the main menu. Choose "RPM->RPM Project", click "Next".
** Input the "Project name". Click "Finish".
** Right click on SPECS, select "New->Other...", choose "RPM->Specfile Based on a Template", click "Next".
** Fill out the template or accept the default values and customize it based your package specifications.
*** You can also use RPM-stubby plug-in to create a spec-file stub.
** To send your SRPM/Spec-file for review, select "File->Export" from the main munu, choose "RPM->Source/Binary RPM", click "Next".
** Based on your requirements, select one or both options, click "Finish".
* Make sure it is a suitable package
** Check [http://fedoraproject.org/wiki/Packaging:Guidelines Packaging Guidelines] and [http://fedoraproject.org/wiki/Packaging:NamingGuidelines Packaging Naming Guidelines] to see if it meets the requirements
** Read some other package submissions to learn about packaging and gain familiarity with the process and requirements. One way of doing this is to join the package-review@lists.fedoraproject.org mailing list.


(provide space to enter URL of hosted SRPM/.spec or scp to fedorapeople.org) and create review-request bug (using mylyn)
=What is WikiText=


=== Submit For Review ===
Mylyn WikiText provides an extensible framework and tools for parsing, editing and presenting lightweight markup. On top of that, it has a wiki text editor for Eclipse and Ant tasks for converting lightweight markup to HTML and other formats. In this specific example, we will be using WikiText's ability to convert Mediawiki content into Eclipse help content.
* Join the mailing lists (--'''Are all these info for mailing list necessary here, or shall we just put a link to them: http://fedoraproject.org/wiki/Join_the_package_collection_maintainers#Join_the_important_Mailing_Lists?''')
** When a new package maintainer joins the Fedora Project, we request that he/she introduces themselves on the devel@lists.fedoraproject.org. To sign up for the list, visit the devel list signup page . The primary purpose of this is to begin the process of building trust by learning about the package maintainer and increase the chances of your review request being processed sooner.
***The purpose of all this is to break anonymity and foster real-world community within the project. You are under no obligation to reveal personal secrets. The objective is to establish a level of trust with yourself and the other members of the project.
***Subject: Self Introduction
***Body: Add any information you believe is applicable including past experience, a link to the review request you have filed and a brief description of yourself. You can also post your GPG key information if you want to.
**You must join the fedora devel-announce@lists.fedoraproject.org mailing list. It is a low traffic announcements only list, where important development information is posted.
**You can join the fedora devel@lists.fedoraproject.org mailing list, where discussions about the development of Fedora are held. This is a high traffic mailing list.
**You can also consider joining the package-announce@lists.fedoraproject.org mailing list -- The commits mailing list gets notifications on all commits in any package in the Fedora repository. This is a very high traffic mailing list. The Fedora package database sends commit mails for packages you (co-)maintain.
**Another mailing list you might consider (at least to view the archives) is packaging@lists.fedoraproject.org. This is the mailing list of the Fedora Packaging Committee, who determine the official packaging guidelines for Fedora projects.
* Upload your package
** Upload your SRPM and SPEC files onto the Internet somewhere. This can be anywhere accessible by a URL.
** If you need hosting space, please make a note of it in your ticket submission and someone will take care of you.
** If you have already got a Fedora Account then you can use your storage at <user-name>.fedorapeople.org for this.
* Create your review request
** Fill out this form at [https://bugzilla.redhat.com/bugzilla/enter_bug.cgi?product=Fedora&format=fedora-review bugzilla].
** Before submitting your request, be sure there’s not a previous request for the same package.
** Make sure that you put the name of the package (excluding version and release numbers) in the 'Review Summary' field, along with a very brief summary of what the package is.
** Put a description of your package (usually, this can be the same thing as what you put in the spec %description) in the 'Review Description' field.
** Include the URLs to your SRPM and SPEC files.
** If this is your first package, explain it and say that you need a sponsor.
* Watch the bugzilla report for feedback
** You should get notifications of changes by email. Fix any blockers that the reviewer(s) point out.


=== Ready to Ship ===
[[Image:What-is-wikitext.png]]
Follow these steps after your package is approved by reviewers.
* Obtain member sponsorship (to check in and build your package)
** you must separately obtain member sponsorship in order to check in and build your package. Sponsorship is not automatic and may require that you further participate in other ways in order to demonstrate your understanding of the packaging guidelines.
** Key to becoming sponsored is to convince an existing sponsor-level member that you understand and follow the project's guidelines and processes.
** See [http://fedoraproject.org/wiki/How_to_get_sponsored_into_the_packager_group how to get sponsored into the packager group] for more information.
*  Add package to Source Code Management (SCM) system and Set Owner
When the package is approved by the reviewer, request a git module and branches with the [http://fedoraproject.org/wiki/Package_SCM_admin_requests SCM admin requests].
* Check out the empty module from SCM
** Once you have the git module, checkout your module from git.
*** Select "import->Git->Projects from Fedora Git" and click on Next.
*** Input the name of the package in the "Package name".
** It is probably a good idea to make a "git" toplevel directory, then check-out your files inside of that. (--'''Do we need this line?''')


=== Update your SCM ===
=A Simple Example=
* Import and commit your SRPM into master branch
 
** Right click on your rpm file in SRPMS folder on your local RPM project in eclipse, select "Copy".
The best way to learn is by doing. As an example, we will take this wiki entry and create some Eclipse help content from it. To accomplish this, we will use Ant so you can easily integrate this into a build if you wish.
** Right click on your cloned project from Fedora Git, select "Paste". (--'''We can consider making it simpler by just one click on RPM project''')
 
** Right click on pasted file, select "Fedora Packager->Upload this File->Add to Existing Sources".
==Setting up the Classpath==
** Do the same steps for your .spec file.
 
* Build your package
The first order of business is to setup your Ant classpath so you can use the WikiText Ant tasks.
** Select your spec file, right click, select "Fedora Packager->Prepare for local build".
 
--'''what are the other steps here? -"Build for Local Architecture", then "Local Build using Mock" then "push the build to Koji"?'''
You can [http://www.eclipse.org/downloads/download.php?file=/tools/mylyn/update/weekly/mylyn-wikitext-standalone-latest.zip download] the WikiText SDK which contains the jars you'll need. In our case, we only need the org.eclipse.mylyn.wikitext.core_1.3.0.I20100116-0000-e3x.jar and org.eclipse.mylyn.wikitext.mediawiki.core_1.3.0.I20100116-0000-e3x.jar since we are only working with Mediawiki. If we were working with something like confluence, we would have to grab the respective JAR and put it on our classpath.
--'''Do we actually need to put all of them here or just refer to "Fedora Packager User Guide" in Eclipse help?'''
 
* Submit your package as update in Bodhi
In the end, your Ant file will look something like the snippet below.
**  If this package will be built for any version of Fedora that is already released please submit it for inclusion in the 'fedora-updates' repository for those versions of Fedora.  
 
** For doing this, right click on your project, select "Fedora Packager->Create New Bodhi Update".
<pre style="width: 60em;">
* Close the bugzilla account
<path id="wikitext.tasks.classpath">
** If the package built successfully, you should close it as NEXTRELEASE.
<fileset dir="lib">
* Add the package to the comp files
<include name="org.eclipse.mylyn.wikitext.*core*.jar"/>
** If appropriate for the package, make it available in "comps" files so that it can be selected during installation and included in yum's package group operations. See [http://fedoraproject.org/wiki/PackageMaintainers/CompsXml PackageMaintainers/CompsXml] for more info.
</fileset>
* Enable Upstream Release Monitoring
</path>
** Fedora has infrastructure available for monitoring new upstream releases of the software you are packaging. Refer to [http://fedoraproject.org/wiki/Upstream_Release_Monitoring Upstream Release Monitoring] for more details.
 
<taskdef classpathref="wikitext.tasks.classpath"
resource="org/eclipse/mylyn/internal/wikitext/mediawiki/core/tasks/tasks.properties"/>
<taskdef classpathref="wikitext.tasks.classpath"  
resource="org/eclipse/mylyn/wikitext/core/util/anttask/tasks.properties"/>
</pre>
 
==Converting to Eclipse Help==
 
After we have setup the classpath for our task, we need to go and fetch the wiki content and convert it to Eclipse help. This is accomplished via the '''mediawiki-to-eclipse-help Ant''' task.
 
In the end, your Ant file will look something like the snippet below.
 
<pre style="width: 60em;">
 
<mediawiki-to-eclipse-help
    wikiBaseUrl="http://wiki.eclipse.org"
validate="true"
failonvalidationerror="true"
prependImagePrefix="images"
formatoutput="true"
defaultAbsoluteLinkTarget="doc_external"
    dest="${basedir}"
    title="Crowdsourcing Documentation"
    generateUnifiedToc="false">
    <path name="DocumentationGuidelines/Example"
                title="Crowdsourcing Documentation"
                generateToc="true"/>
    ...
</pre>
 
When the Ant task is executed... the wiki content will be transformed into HTML and a respective Eclipse help TOC XML will be generated.
 
[[Image:Crowdsourcing-toc.png]]
 
When you self-host and launch Eclipse Help in the self-hosted instance, you should see this content there as shown below.
 
[[Image:Crowdsourcing-help-content.png]]
 
==Source Code==
 
The full source code for this example is available on [http://github.com/caniszczyk/mylyn-wikitext-examples GitHub].
 
=Other Approaches=
 
There are two main approaches when it comes to crowdsourcing documentation: '''wiki-based''' and '''file-based'''.
 
In this example, we focus on the wiki-based approach as it's the most collaborative approach. Anyone can contribute to documentation easily and the changes are instantly available online for people to see. The downside to this approach is that contributions are a bit less controlled which could impact quality depending on how a project structures its documentation process and depending on the wiki you choose, version history may be a bit harder to manage.
 
In the file-based approach, the origin of the content is only in the SCM and is transformed during build-time. The plus side to this is it's managed by your existing version control system. The two major downsides are that changes aren't immediately available to your users and that it's much harder to contribute to the documentation since you need to have commit access. There are merits to both approaches and it really depends on your needs. Thankfully, examples for both approaches can be found in the [[#Eclipse.org Reference Projects]] section.
 
=Tips and Tricks=
 
You can use <pageAppendum> to add some extra information to your documentation.  
 
It's common to add a blurb about how to update the documentation so users can be informed that the documentation they are reading has been generating from the wiki.
 
=Eclipse.org Reference Projects=
 
If you're looking for other examples on how to crowdsource your documentation at Eclipse, the best place is to look and see what other Eclipse.org projects are doing.
 
==Mylyn==
 
The Mylyn project uses two wiki pages on Eclipsepedia for its documentation: [[Mylyn/User Guide]] and [[Mylyn/FAQ]].
 
Each of these have their own Help table of contents, which are arranged into a common document via a top-level table of contents with links.
 
You can browse the source code [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mylyn/org.eclipse.mylyn.help.ui/?root=Mylyn_Project here]. Pay attention to [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mylyn/org.eclipse.mylyn.help.ui/build-helper.xml?root=Mylyn_Project&view=markup build-helper.xml], which drives the generation tasks, and [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mylyn/org.eclipse.mylyn.help.ui/toc.xml?root=Mylyn_Project&view=markup toc.xml], which links the generated table of contents into a single document.
 
==EGit==
 
The EGit project uses a single wiki page on Eclipsepedia for its [http://wiki.eclipse.org/EGit/User_Guide documentation].
 
Check out the '''org.eclipse.egit.doc''' plug-in.
 
==Xtext==
 
The Xtext project uses a Textile and Wikitext for its documentation. The code is available in the [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext.doc/?root=Modeling_Project org.eclipse.xtext.doc] plug-in. The documentation is sourced from various Textile files stored in source control and then transformed at build-time. This is different from the previous two examples which used the wiki as the source of documentation.
 
Pay attention to the [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext.doc/customBuild.xml?root=Modeling_Project&view=markup customBuild.xml] and the [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext.doc/doc/?root=Modeling_Project doc] folder.
 
==OSEE==
 
For modularity reasons, the OSEE project has multiple help projects which generate Eclipse online help from different parts of the [[OSEE#Documentation|OSEE wiki]]. The central help plugin is [https://dev.eclipse.org/svnroot/technology/org.eclipse.osee/trunk/plugins/org.eclipse.osee.framework.help.ui/ org.eclipse.osee.framework.help.ui], which contains a [https://dev.eclipse.org/svnroot/technology/org.eclipse.osee/trunk/plugins/org.eclipse.osee.framework.help.ui/build-helper.xml build-helper.xml] file that calls <tt>mediawiki-to-eclipse-help</tt> and specifies the different help chapters:
 
<source lang="xml">
<mediawiki-to-eclipse-help wikiBaseUrl="${osee.help.doc.url.base}"
validate="true" failonvalidationerror="true" prependImagePrefix="images"
formatoutput="true" defaultAbsoluteLinkTarget="osee_external" dest="${basedir}"
navigationimages="true" title="OSEE User's Guide"
generateUnifiedToc="true">
<path name="OSEE/Users_Guide/Getting_Started" title="Introduction" />
<path name="OSEE/Users_Guide/Concepts" title="Concepts" />
<path name="OSEE/Users_Guide/Features" title="Features" />
<path name="OSEE/Users_Guide/Tips" title="Tips" />
<path name="OSEE/Users_Guide/New" title="New" />
<stylesheet url="book.css" />
<pageAppendum>
= Updating This Document =
 
This document is maintained in a collaborative wiki. If you wish to
update or modify this document please visit {url}
</pageAppendum>
</mediawiki-to-eclipse-help>
</source>
 
Classpath/taskdef setup is performed in [https://dev.eclipse.org/svnroot/technology/org.eclipse.osee/trunk/plugins/org.eclipse.osee.framework.help.ui/scripts/help-build-common.xml help-build-common.xml]. Another example of an OSEE help plugin is [https://dev.eclipse.org/svnroot/technology/org.eclipse.osee/trunk/plugins/org.eclipse.osee.ats.help.ui/ org.eclipse.osee.ats.help.ui], which uses the libraries and taskdefs from org.eclipse.osee.framework.help.ui.
 
== PsychoPath XPath 2.0 Processor ==
 
The [[PsychoPathXPathProcessor/UserManual | PsychoPath Processor documentation]] is maintained on the eclipse wiki.  The documentation is then generated into DocBook format using Mylyn WikiText.  The documentation is then cleaned up as some of the conversions do not necessarily fit our specific needs.  The Docbook Project's XSL stylesheets are then used to generate the help into a variety of formats including eclipse help.
 
* See [[Authoring_Eclipse_Help_Using_DocBook]] for more information on using Docbook to generate the final output produced by WikiText.
 
== XSL Tools ==
 
Like the Psychopath Processor, the webtools [[XSLT_Project | XSL Tools documentation]] is also available on the wiki and generated back into DocBook format using Mylyn WikiText.
 
* See [[Authoring_Eclipse_Help_Using_DocBook]] for more information on using Docbook to generate the final output produced by WikiText.

Revision as of 18:09, 27 May 2011

Introduction

There are many ways to generate help content in Eclipse. One particular method involves generating your help content from the wiki which allows you to crowdsource your documentation. By having your documentation on the wiki, you lower the barrier of entry for people to contribute documentation. The purpose of this wiki entry is to guide you through an example of how you can crowdsource your documentation using Mylyn WikiText.

What is WikiText

Mylyn WikiText provides an extensible framework and tools for parsing, editing and presenting lightweight markup. On top of that, it has a wiki text editor for Eclipse and Ant tasks for converting lightweight markup to HTML and other formats. In this specific example, we will be using WikiText's ability to convert Mediawiki content into Eclipse help content.

File:What-is-wikitext.png

A Simple Example

The best way to learn is by doing. As an example, we will take this wiki entry and create some Eclipse help content from it. To accomplish this, we will use Ant so you can easily integrate this into a build if you wish.

Setting up the Classpath

The first order of business is to setup your Ant classpath so you can use the WikiText Ant tasks.

You can download the WikiText SDK which contains the jars you'll need. In our case, we only need the org.eclipse.mylyn.wikitext.core_1.3.0.I20100116-0000-e3x.jar and org.eclipse.mylyn.wikitext.mediawiki.core_1.3.0.I20100116-0000-e3x.jar since we are only working with Mediawiki. If we were working with something like confluence, we would have to grab the respective JAR and put it on our classpath.

In the end, your Ant file will look something like the snippet below.

<path id="wikitext.tasks.classpath">
	<fileset dir="lib">
		<include name="org.eclipse.mylyn.wikitext.*core*.jar"/>
	</fileset>
</path>

<taskdef classpathref="wikitext.tasks.classpath" 
 resource="org/eclipse/mylyn/internal/wikitext/mediawiki/core/tasks/tasks.properties"/>
<taskdef classpathref="wikitext.tasks.classpath" 
 resource="org/eclipse/mylyn/wikitext/core/util/anttask/tasks.properties"/>

Converting to Eclipse Help

After we have setup the classpath for our task, we need to go and fetch the wiki content and convert it to Eclipse help. This is accomplished via the mediawiki-to-eclipse-help Ant task.

In the end, your Ant file will look something like the snippet below.


<mediawiki-to-eclipse-help
    	wikiBaseUrl="http://wiki.eclipse.org"
		validate="true"
		failonvalidationerror="true"
		prependImagePrefix="images"
		formatoutput="true"
		defaultAbsoluteLinkTarget="doc_external"
    	dest="${basedir}"
    	title="Crowdsourcing Documentation"
    	generateUnifiedToc="false">
    	<path name="DocumentationGuidelines/Example" 
                title="Crowdsourcing Documentation" 
                generateToc="true"/>
    	...

When the Ant task is executed... the wiki content will be transformed into HTML and a respective Eclipse help TOC XML will be generated.

File:Crowdsourcing-toc.png

When you self-host and launch Eclipse Help in the self-hosted instance, you should see this content there as shown below.

File:Crowdsourcing-help-content.png

Source Code

The full source code for this example is available on GitHub.

Other Approaches

There are two main approaches when it comes to crowdsourcing documentation: wiki-based and file-based.

In this example, we focus on the wiki-based approach as it's the most collaborative approach. Anyone can contribute to documentation easily and the changes are instantly available online for people to see. The downside to this approach is that contributions are a bit less controlled which could impact quality depending on how a project structures its documentation process and depending on the wiki you choose, version history may be a bit harder to manage.

In the file-based approach, the origin of the content is only in the SCM and is transformed during build-time. The plus side to this is it's managed by your existing version control system. The two major downsides are that changes aren't immediately available to your users and that it's much harder to contribute to the documentation since you need to have commit access. There are merits to both approaches and it really depends on your needs. Thankfully, examples for both approaches can be found in the #Eclipse.org Reference Projects section.

Tips and Tricks

You can use <pageAppendum> to add some extra information to your documentation.

It's common to add a blurb about how to update the documentation so users can be informed that the documentation they are reading has been generating from the wiki.

Eclipse.org Reference Projects

If you're looking for other examples on how to crowdsource your documentation at Eclipse, the best place is to look and see what other Eclipse.org projects are doing.

Mylyn

The Mylyn project uses two wiki pages on Eclipsepedia for its documentation: Mylyn/User Guide and Mylyn/FAQ.

Each of these have their own Help table of contents, which are arranged into a common document via a top-level table of contents with links.

You can browse the source code here. Pay attention to build-helper.xml, which drives the generation tasks, and toc.xml, which links the generated table of contents into a single document.

EGit

The EGit project uses a single wiki page on Eclipsepedia for its documentation.

Check out the org.eclipse.egit.doc plug-in.

Xtext

The Xtext project uses a Textile and Wikitext for its documentation. The code is available in the org.eclipse.xtext.doc plug-in. The documentation is sourced from various Textile files stored in source control and then transformed at build-time. This is different from the previous two examples which used the wiki as the source of documentation.

Pay attention to the customBuild.xml and the doc folder.

OSEE

For modularity reasons, the OSEE project has multiple help projects which generate Eclipse online help from different parts of the OSEE wiki. The central help plugin is org.eclipse.osee.framework.help.ui, which contains a build-helper.xml file that calls mediawiki-to-eclipse-help and specifies the different help chapters:

<source lang="xml"> <mediawiki-to-eclipse-help wikiBaseUrl="${osee.help.doc.url.base}" validate="true" failonvalidationerror="true" prependImagePrefix="images" formatoutput="true" defaultAbsoluteLinkTarget="osee_external" dest="${basedir}" navigationimages="true" title="OSEE User's Guide" generateUnifiedToc="true"> <path name="OSEE/Users_Guide/Getting_Started" title="Introduction" /> <path name="OSEE/Users_Guide/Concepts" title="Concepts" /> <path name="OSEE/Users_Guide/Features" title="Features" /> <path name="OSEE/Users_Guide/Tips" title="Tips" /> <path name="OSEE/Users_Guide/New" title="New" /> <stylesheet url="book.css" /> <pageAppendum>

Updating This Document

This document is maintained in a collaborative wiki. If you wish to update or modify this document please visit {url} </pageAppendum> </mediawiki-to-eclipse-help> </source>

Classpath/taskdef setup is performed in help-build-common.xml. Another example of an OSEE help plugin is org.eclipse.osee.ats.help.ui, which uses the libraries and taskdefs from org.eclipse.osee.framework.help.ui.

PsychoPath XPath 2.0 Processor

The PsychoPath Processor documentation is maintained on the eclipse wiki. The documentation is then generated into DocBook format using Mylyn WikiText. The documentation is then cleaned up as some of the conversions do not necessarily fit our specific needs. The Docbook Project's XSL stylesheets are then used to generate the help into a variety of formats including eclipse help.

XSL Tools

Like the Psychopath Processor, the webtools XSL Tools documentation is also available on the wiki and generated back into DocBook format using Mylyn WikiText.