From Fedora Project Wiki
No edit summary
(review doc)
 
(19 intermediate revisions by 10 users not shown)
Line 1: Line 1:
= Building anaconda =
= Building anaconda =


Building and packaging anaconda is not hard once you have all the sources checked out.  These two tasks will be described separately, even though they are pretty similar.  Building is just compiling the handful of programs and libraries.  Packaging is building, followed by putting everything together into an RPM.  These will be described using the official Fedora build infrastructure.
Building and packaging anaconda is not overly difficult, but it does involve a lot of steps.  Building anaconda refers to building the source in place.  Packaging refers to creating either a local package or an official build in the Fedora build system.  These will be described using the official Fedora build infrastructure.


The first step is having both the anaconda source and the Fedora package source checked out.  Checking out the anaconda source is described in detail on the main [[Anaconda| ]]  page.  Checking out the package source is described in detail on the [[Infrastructure/CVS| ]]  page.  For now, let's assume the anaconda source is in src/ and the package source is in pkg/.
The first step is having both the anaconda source and the Fedora package source checked out.  Checking out the anaconda source is described in detail on the main [[Anaconda| ]]  page.  Checking out the package source is described in detail on the  
[[Infrastructure/CVS| ]]  page.  For now, let's assume the anaconda source is in src/ and the package source is in pkg/.


== Building ==
The rest of this page is laid out based on what task you are trying to accomplish.


Why might you just want to build anaconda instead of building a package?  Of course there's the obvious reason that you are working on something in the loader and need to see if your changes still compile.  You may also want to build a new loader for testing out in a new initrd.  Also, sometimes it's good to do a test build real quick to make sure sources still compile and translations still work before submitting it to the build system.
== Installing Build Dependencies ==


First you need all the requirements installed.  If you are running the same version of Fedora that you want to build anaconda for (say, you're using F8 and want to build the F8 anaconda sources), this is pretty straightforwardThe easiest way to do this is to build a SRPM file and then automatically install all the build requirements from that using the yum-builddep tool included in the yum-utils package.
Before you can do any local builds, you need to have the build dependencies installed.  We have a Makefile target to help with that, but if you have never built anaconda before, the steps needed to generate the real Makefile won't work.  So you can just grab the line out of the Makefile.am and do it yourself:
 
<pre>
$ cd src/
$ sudo yum install libtool $(grep ^BuildRequires: anaconda.spec.in | awk '{print $2}')
</pre>
 
This may take a little while but will end up pulling in all the programs required to build anaconda.  This includes compilers and basic libraries if you don't already have them installed.
 
== Configure Transifex ==
 
You will also need to configure Transifex, for translations.
 
<pre>
$ sudo yum install transifex-client
</pre>
 
Create a Transifex account at https://fedora.transifex.com/
(This system is not linked to FAS, it's hosted by another company; so it requires another account at this time.  In the near future, we will be migrating to use [https://translate.zanata.org/zanata/ Zanata], which should work with your FAS account.)
 
 
Configure 'tx' on your system, by running
 
<pre>
tx init /tmp
</pre>
 
Accept default host, fill in your username and password generated in #2.
 
This command both configures tx in general and sets up a project. The "/tmp" keeps it from overwriting the already-configured project in the anaconda tree, while still writing the general configuration.
 
And then (back in the anaconda tree, after you run ./autogen.sh && ./configure):
 
<pre>
make po-pull
</pre>
 
which may take a little while.
 
== Building the Source ==
 
Why might you just want to compile anaconda instead of building a package?  Sometimes it's good to do a test build real quick to make sure sources still compile and translations still work before submitting it to the build system.
 
Once you've got the build dependencies installed, building anaconda is a simple matter of:
 
<pre>
clumens@exeter:~/src$ ./autogen.sh
clumens@exeter:~/src$ ./configure
clumens@exeter:~/src$ make
</pre>
 
== Building a Test Package and Tree Compose ==
 
We no longer automatically generate nightly trees for rawhideThis makes it a little hard to test updates to tree composition scripts.  For this, you need to create a test package of anaconda and then use that test package in a run of pungi to compose a new tree.  In order to do those tasks, you need anaconda's build dependencies installed (see above), pungi installed, enough disk space to store a tree, and space somewhere to put a small package repository.
 
Again, this is easiest if you are building a tree for the same architecture as you will be testing.  First, you need to build a new anaconda binary package:


<pre>
<pre>
clumens@exeter:~$ cd src/
clumens@exeter:~$ cd src/
clumens@exeter:~/src$ make src
clumens@exeter:~/src$ ./autogen.sh
...
clumens@exeter:~$ ./configure --disable-static  --enable-introspection --enable-gtk-doc
...
...
Wrote: /home/clumens/rpm/SRPMS/anaconda-11.4.0.59-1.src.rpm
clumens@exeter:~/src$ yum-builddep /home/clumens/rpm/SRPMS/anaconda-11.4.0.59-1.src.rpm
</pre>
</pre>


This may take a little while, but will end up pulling in all the programs required to build anaconda.  This includes compilers and basic libraries if you don't already have them installed.  Of course, you will need make to build the SRPM. It's possible to build an SRPM without the makefile target, but it's so much less convenient.
Then edit configure.ac and increment the version number in the following line:
 
<pre>
AC_INIT([anaconda], [15.5], [anaconda-devel-list@redhat.com])
</pre>


Then, building anaconda is a simple matter of:
Commit it temporarily:


<pre>
<pre>
clumens@exeter:~/src$ make
clumens@exeter:~/src$ git commit -m 'Temporary build.' configure.ac
...
</pre>
</pre>


If you aren't running the same version of Fedora that you want to target a build of anaconda for (say, you're running F8 and want to build the Rawhide anaconda sources), it's still possible.  You'll first need to install the mock package and install all the build dependencies of the anaconda package in the right mock chroot.
We have makefile targets to do this last bit automatically, but you don't want to use them for test builds.  The real targets also add a new %changelog entry to the spec file and tag the build with a GPG signed tag.  You don't really want to accidentally push a new tag to the remote repo.
 
Then you create a new tarball, build an RPM from it, install it to your local system (you need the updated versions of the scripts on the local system for pungi to find), and create a package repo containing the updated package:


<pre>
<pre>
clumens@exeter:~/src$ make src
clumens@exeter:~/src$ make scratch
...
clumens@exeter:~/src$ rpmbuild -tb anaconda-15.5.tar.bz2
...
Wrote: /home/clumens/rpm/RPMS/x86_64/anaconda-15.5-1.fc15.x86_64.rpm
Wrote: /home/clumens/rpm/RPMS/x86_64/anaconda-debuginfo-15.5-1.fc15.x86_64.rpm
...
...
Wrote: /home/clumens/rpm/SRPMS/anaconda-11.4.0.59-1.src.rpm
clumens@exeter:~/src$ sudo rpm -Uvh --force /home/clumens/rpm/RPMS/x86_64/anaconda-15.5-1.fc15.x86_64.rpm
clumens@exeter:~/src$ mock -r fedora-8-i386 /home/clumens/rpm/SRPMS/anaconda-11.4.0.59-1.src.rpm
...
...
clumens@exeter:~/src$ mkdir ~/repo
clumens@exeter:~/src$ cp /home/clumens/rpm/RPMS/x86_64/anaconda-15.5-1.fc15.x86_64.rpm ~/repo
clumens@exeter:~/src$ cd
clumens@exeter:~$ createrepo -p -d repo
1/1 - anaconda-15.5-1.fc15.x86_64.rpm                                         
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
clumens@exeter:~$
</pre>
At this point, you are all set to run pungi and create a whole new Fedora tree to test with.  In order to run pungi, you need the fedora-kickstarts package installed.  pungi takes a kickstart file as input and generates a tree using the various repo commands and the %packages section.  I copy one of the basic kickstart files from that package as the basis.  I remove a bunch of the stuff out of the %packages section since I don't care, and then I add in a new repo line so pungi finds my updated anaconda in preference to whatever we're shipping in the distribution right now (that's also why you've got to update the version number in configure.ac above).
Add the following to your pungi kickstart file:
<pre>
repo --name=anaconda --baseurl=file:///home/clumens/repo
</pre>
</pre>


The first time you run this, it will take a long while as it downloads and installs all the package build dependencies. The slower your repos, the longer it will take.  Afterwards, you will have a mock chroot populated with all the libraries and programs needed to compile anaconda.  You can then chroot into it and do a build:
And then it's time to run pungi.  This will take a good long while.


<pre>
<pre>
clumens@exeter:~$ sudo su -
clumens@exeter:~$ sudo setenforce permissive
root@exeter:~# mount -o bind /home/clumens/src /var/lib/mock/fedora-8-i386/root/root
clumens@exeter:~$ sudo pungi --destdir=/home/clumens/install --nosource --nodebuginfo --nosplitmedia -GCBI -c fedora-install-fedora.ks
root@exeter:~# chroot /var/lib/mock/fedora-8-i386/root/
root:~# make
...
...
</pre>
</pre>


== Packaging ==
== Building an Official Package ==


Doing an official build of anaconda for Fedora is not hard either.  We have some makefile targets to make things much easier.  The first step is to build a tarball from anaconda source.  Before doing that, it's good to make sure you've got the latest stuff.
Doing an official build of anaconda for Fedora is much simpler because of our Makefile targets and the fact that you don't need to do any tree composition.  The first step is to build a tarball from anaconda source.  Before doing that, it's good to make sure you've got the latest stuff.


<pre>
<pre>
clumens@exeter:~$ cd src/
dcantrel@mitre ~$ cd src/
clumens@exeter:~/src$ git fetch && git rebase origin
dcantrel@mitre src (master)$ git fetch && git rebase origin
clumens@exeter:~/src$ make bumpver
dcantrel@mitre src (master)$ make bumpver
</pre>
 
This last command updates the anaconda.spec.in file with a new %changelog entry, increments the version number in configure.ac, and regenerates po/anaconda.pot for the translators.  The %changelog entry is made automatically from the git logs, so you should examine what was written to anaconda.spec.in and remove unnecessary messages.  We remove 'Sending translation for' messages as well as merge commit messages.  When you are finished with anaconda.spec.in, check in the files as 'New version' and the new version number to signify we are making a new version:
 
<pre>
dcantrel@mitre anaconda (master)$ git add anaconda.spec.in configure.ac po/anaconda.pot
dcantrel@mitre anaconda (master)$ git commit -m "New version. ($version-number)"
</pre>
 
Now we have the version updated and the spec file template updated.  We need to tag the git repository and generate a new release archive.  Here's how I do that:
 
<pre>
dcantrel@mitre src (master)$ git clean -d -x -f
dcantrel@mitre src (master)$ ./autogen.sh && ./configure
dcantrel@mitre src (master)$ make release
</pre>
 
The git tag procedure will attempt to sign the tag using the GnuPG key for the default email address in your local git configuration.  If you do not have GnuPG set up, now is a good time.
 
Now it's time to push the new version changes and tags and move the files over to the package GIT directory:
 
<pre>
dcantrel@mitre src (master)$ git push && git push --tags
dcantrel@mitre src (master)$ mv anaconda.spec *.bz2 ~/pkg/devel/
dcantrel@mitre src (master)$ cd ~/pkg/devel/
dcantrel@mitre pkg$ fedpkg upload *.bz2
</pre>
</pre>


This last command updates the anaconda.spec file to increase the version number and write out a %changelog entry.  Since this entry is made automatically from the git logs, it's a good idea to check it by hand and make sure there's nothing strange in there. In particular, it's good to get rid of merge commit messages.  No one cares about those.  Then it's time to make use of that package CVS checkout from earlier.
Now update the sources file like you are told to and remove the old entry.  I also remove old entries from the .gitignore file.  Then continue like so:


<pre>
<pre>
clumens@exeter:~/src$ make archive
dcantrel@mitre pkg$ fedpkg clog
clumens@exeter:~/src$ git push
dcantrel@mitre pkg$ fedpkg commit -F clog
clumens@exeter:~/src$ git push --tags
dcantrel@mitre pkg$ fedpkg clean
clumens@exeter:~/src$ cp anaconda.spec *.bz2 ~/pkg/devel/
dcantrel@mitre pkg$ fedpkg push
clumens@exeter:~/src$ cd ~/pkg/devel/
dcantrel@mitre pkg$ fedpkg build
clumens@exeter:~/pkg/devel$ make upload FILES=*.bz2
</pre>
</pre>


Now update the sources file like you are told to and remove the old entry.  Then continue like so:
Then you just wait for the package to work its way through the build system.  If all goes well, there's nothing left to do.  However, if the build fails, you can check the provided link and view the log files to determine the cause of the problem.  Then it's back to the beginning of the whole packaging process to try again.
 
== Building for other Releases with mock ==
 
If the release you want to build doesn't match what you are running on your development system you will need to use mock. eg. building for rawhide from a f14 system.


* <code> yum install mock </code>
* Add your user to the mock group (never run mock as root, weird things will happen)
** <code> useradd -a -G mock </code>
* Edit the <code>/etc/mock/site-defaults.cfg</code> file to bind your working directories inside the mock root. This keeps mock from wiping those directories on --init
<pre>
<pre>
clumens@exeter:~/pkg/devel$ cvs ci
config_opts['plugin_conf']['bind_mount_enable'] = True
clumens@exeter:~/pkg/devel$ make tag
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/home/clumens/anaconda', '/root/src' ))
clumens@exeter:~/pkg/devel$ make build
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/local/repo', '/root/repo' ))
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/local/pungi', '/root/pungi' ))
</pre>
</pre>
* Setup the initial mock environment, replace the fedora-13-i386 with the mock file edited above
** <code>mock -r fedora-rawhide-i386 --init</code>
** <code>mock -r fedora-rawhide-i386 --no-clean --install pungi</code>


Then you just wait for the package to work its way through the build system. If all goes well, there's nothing left to do.  However if the build fails, you can check the provided link and view the log files to determine the cause of the problem.  Then it's back to the beginning of the whole packaging process to try again.
Now you have a mock chroot environment setup that will use the proxy cache for packages. You can enter it with <code>mock -r fedora-rawhide-i386 --shell</code> and build anaconda as described above.


[[Category:Anaconda]]
[[Category:Anaconda]]

Latest revision as of 00:05, 8 August 2018

Building anaconda

Building and packaging anaconda is not overly difficult, but it does involve a lot of steps. Building anaconda refers to building the source in place. Packaging refers to creating either a local package or an official build in the Fedora build system. These will be described using the official Fedora build infrastructure.

The first step is having both the anaconda source and the Fedora package source checked out. Checking out the anaconda source is described in detail on the main page. Checking out the package source is described in detail on the page. For now, let's assume the anaconda source is in src/ and the package source is in pkg/.

The rest of this page is laid out based on what task you are trying to accomplish.

Installing Build Dependencies

Before you can do any local builds, you need to have the build dependencies installed. We have a Makefile target to help with that, but if you have never built anaconda before, the steps needed to generate the real Makefile won't work. So you can just grab the line out of the Makefile.am and do it yourself:

$ cd src/
$ sudo yum install libtool $(grep ^BuildRequires: anaconda.spec.in | awk '{print $2}')

This may take a little while but will end up pulling in all the programs required to build anaconda. This includes compilers and basic libraries if you don't already have them installed.

Configure Transifex

You will also need to configure Transifex, for translations.

$ sudo yum install transifex-client

Create a Transifex account at https://fedora.transifex.com/ (This system is not linked to FAS, it's hosted by another company; so it requires another account at this time. In the near future, we will be migrating to use Zanata, which should work with your FAS account.)


Configure 'tx' on your system, by running

tx init /tmp

Accept default host, fill in your username and password generated in #2.

This command both configures tx in general and sets up a project. The "/tmp" keeps it from overwriting the already-configured project in the anaconda tree, while still writing the general configuration.

And then (back in the anaconda tree, after you run ./autogen.sh && ./configure):

make po-pull

which may take a little while.

Building the Source

Why might you just want to compile anaconda instead of building a package? Sometimes it's good to do a test build real quick to make sure sources still compile and translations still work before submitting it to the build system.

Once you've got the build dependencies installed, building anaconda is a simple matter of:

clumens@exeter:~/src$ ./autogen.sh
clumens@exeter:~/src$ ./configure
clumens@exeter:~/src$ make

Building a Test Package and Tree Compose

We no longer automatically generate nightly trees for rawhide. This makes it a little hard to test updates to tree composition scripts. For this, you need to create a test package of anaconda and then use that test package in a run of pungi to compose a new tree. In order to do those tasks, you need anaconda's build dependencies installed (see above), pungi installed, enough disk space to store a tree, and space somewhere to put a small package repository.

Again, this is easiest if you are building a tree for the same architecture as you will be testing. First, you need to build a new anaconda binary package:

clumens@exeter:~$ cd src/
clumens@exeter:~/src$ ./autogen.sh
...
clumens@exeter:~$ ./configure --disable-static  --enable-introspection --enable-gtk-doc
...

Then edit configure.ac and increment the version number in the following line:

AC_INIT([anaconda], [15.5], [anaconda-devel-list@redhat.com])

Commit it temporarily:

clumens@exeter:~/src$ git commit -m 'Temporary build.' configure.ac
...

We have makefile targets to do this last bit automatically, but you don't want to use them for test builds. The real targets also add a new %changelog entry to the spec file and tag the build with a GPG signed tag. You don't really want to accidentally push a new tag to the remote repo.

Then you create a new tarball, build an RPM from it, install it to your local system (you need the updated versions of the scripts on the local system for pungi to find), and create a package repo containing the updated package:

clumens@exeter:~/src$ make scratch
...
clumens@exeter:~/src$ rpmbuild -tb anaconda-15.5.tar.bz2
...
Wrote: /home/clumens/rpm/RPMS/x86_64/anaconda-15.5-1.fc15.x86_64.rpm
Wrote: /home/clumens/rpm/RPMS/x86_64/anaconda-debuginfo-15.5-1.fc15.x86_64.rpm
...
clumens@exeter:~/src$ sudo rpm -Uvh --force /home/clumens/rpm/RPMS/x86_64/anaconda-15.5-1.fc15.x86_64.rpm
...
clumens@exeter:~/src$ mkdir ~/repo
clumens@exeter:~/src$ cp /home/clumens/rpm/RPMS/x86_64/anaconda-15.5-1.fc15.x86_64.rpm ~/repo
clumens@exeter:~/src$ cd
clumens@exeter:~$ createrepo -p -d repo
1/1 - anaconda-15.5-1.fc15.x86_64.rpm                                           
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
clumens@exeter:~$

At this point, you are all set to run pungi and create a whole new Fedora tree to test with. In order to run pungi, you need the fedora-kickstarts package installed. pungi takes a kickstart file as input and generates a tree using the various repo commands and the %packages section. I copy one of the basic kickstart files from that package as the basis. I remove a bunch of the stuff out of the %packages section since I don't care, and then I add in a new repo line so pungi finds my updated anaconda in preference to whatever we're shipping in the distribution right now (that's also why you've got to update the version number in configure.ac above).

Add the following to your pungi kickstart file:

repo --name=anaconda --baseurl=file:///home/clumens/repo

And then it's time to run pungi. This will take a good long while.

clumens@exeter:~$ sudo setenforce permissive
clumens@exeter:~$ sudo pungi --destdir=/home/clumens/install --nosource --nodebuginfo --nosplitmedia -GCBI -c fedora-install-fedora.ks
...

Building an Official Package

Doing an official build of anaconda for Fedora is much simpler because of our Makefile targets and the fact that you don't need to do any tree composition. The first step is to build a tarball from anaconda source. Before doing that, it's good to make sure you've got the latest stuff.

dcantrel@mitre ~$ cd src/
dcantrel@mitre src (master)$ git fetch && git rebase origin
dcantrel@mitre src (master)$ make bumpver

This last command updates the anaconda.spec.in file with a new %changelog entry, increments the version number in configure.ac, and regenerates po/anaconda.pot for the translators. The %changelog entry is made automatically from the git logs, so you should examine what was written to anaconda.spec.in and remove unnecessary messages. We remove 'Sending translation for' messages as well as merge commit messages. When you are finished with anaconda.spec.in, check in the files as 'New version' and the new version number to signify we are making a new version:

dcantrel@mitre anaconda (master)$ git add anaconda.spec.in configure.ac po/anaconda.pot
dcantrel@mitre anaconda (master)$ git commit -m "New version. ($version-number)"

Now we have the version updated and the spec file template updated. We need to tag the git repository and generate a new release archive. Here's how I do that:

dcantrel@mitre src (master)$ git clean -d -x -f
dcantrel@mitre src (master)$ ./autogen.sh && ./configure
dcantrel@mitre src (master)$ make release

The git tag procedure will attempt to sign the tag using the GnuPG key for the default email address in your local git configuration. If you do not have GnuPG set up, now is a good time.

Now it's time to push the new version changes and tags and move the files over to the package GIT directory:

dcantrel@mitre src (master)$ git push && git push --tags
dcantrel@mitre src (master)$ mv anaconda.spec *.bz2 ~/pkg/devel/
dcantrel@mitre src (master)$ cd ~/pkg/devel/
dcantrel@mitre pkg$ fedpkg upload *.bz2

Now update the sources file like you are told to and remove the old entry. I also remove old entries from the .gitignore file. Then continue like so:

dcantrel@mitre pkg$ fedpkg clog
dcantrel@mitre pkg$ fedpkg commit -F clog
dcantrel@mitre pkg$ fedpkg clean
dcantrel@mitre pkg$ fedpkg push
dcantrel@mitre pkg$ fedpkg build

Then you just wait for the package to work its way through the build system. If all goes well, there's nothing left to do. However, if the build fails, you can check the provided link and view the log files to determine the cause of the problem. Then it's back to the beginning of the whole packaging process to try again.

Building for other Releases with mock

If the release you want to build doesn't match what you are running on your development system you will need to use mock. eg. building for rawhide from a f14 system.

  • yum install mock
  • Add your user to the mock group (never run mock as root, weird things will happen)
    • useradd -a -G mock
  • Edit the /etc/mock/site-defaults.cfg file to bind your working directories inside the mock root. This keeps mock from wiping those directories on --init
config_opts['plugin_conf']['bind_mount_enable'] = True
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/home/clumens/anaconda', '/root/src' ))
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/local/repo', '/root/repo' ))
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/local/pungi', '/root/pungi' ))
  • Setup the initial mock environment, replace the fedora-13-i386 with the mock file edited above
    • mock -r fedora-rawhide-i386 --init
    • mock -r fedora-rawhide-i386 --no-clean --install pungi

Now you have a mock chroot environment setup that will use the proxy cache for packages. You can enter it with mock -r fedora-rawhide-i386 --shell and build anaconda as described above.