BinutilsRawhideSync

Rawhide synchronization for the GNU Binutils

This page documents the process by which the rawhide binutils package is synced with the upstream GNU Binutils project. The aim of this process is to make syncing simple and easy so that it can be performed on a regular basis. Due to the fact that the binutils is an important component in the toolchain used to build many of the packages in rawhide however, it is important to extensively test a new binutils before putting it into the buildroot. This does make the syncing process more time consuming and complicated.

This process is based upon a similar process for the GNU C Library (GlibcRawhideSync), but tweaked to fit the binutils' way of handling patches.

This process assumes that you already have a clone of the Fedora rawhide binutils.

The decision for when to perform a sync is a complicated one. In general if there is a newly made official upstream release then that should be used. Otherwise if there is a recent upstream branch snapshot available then that should be used. Otherwise if there has not been a sync in the last two weeks then it is probably time for this procedure to be run.

Follow these steps:

1. Check to see if there is a recent snapshot available:

 https://snapshots.sourceware.org/binutils/trunk/

If there is, then download it and proceed to step 2. Otherwise proceed to step 5.

2. Go to your clone of the fedora binutils package and make sure that it is clean and up to date:

cd <rawhide-binutils>
git switch rawhide
git pull
git clean -fdx
git status 

3. Delete the current source tarball and then copy in the source tarball downloaded in step 1:

rm -f *.tar.xz
cp ~/Downloads/binutils-*.tar.xz . 

4. View the binutils.spec file and check that the source variable is set to snapshot and that the Version field is set to the version number of the tarball. If necessary change them to the correct values:

grep -e "Version:" -e "define source" binutils.spec

Then set the commit_id variable to the commit id of the new tarball:

sed --in-place -e 's/define commit_id.*/define commit_id %(echo "COMMIT_ID")/' binutils.spec 

Replace COMMIT_ID in the above command with the commit id of the tarball. So for example if the tarball is called binutils-2.43.50-1f4aee70ed1.tar.xz then COMMIT_ID is 1f4aee70ed1.

Also increment the Release: field by one and add a changelog entry for the update:

rpmdev-bumpspec -c "- Rebase to commit COMMIT_ID" binutils.spec

Replace COMMIT_ID in the above command with the commit id of the tarball.

Goto step 10.

5. (This step is reached from step 1 if a suitable upstream source snapshot is unavailable).

Make sure that your clone of the upstream GNU Binutils is up to date, on the right branch and clean. If there any problems or uncommitted changes, resolve them before continuing.

cd <upstream-binutils>
git switch HEAD
git pull
git clean -fdx
git status

Delete any old release tarballs and then create a source tarball from the upstream sources. Change the version number in the tarball's name to a commit id.

rm -f *.xz
export COMMIT_ID=`git rev-parse HEAD`
./src-release.sh -x -r `git log -1 --format=%cd --date=format:%F $COMMIT_ID` binutils
mv binutils-*.tar.xz binutils-$COMMIT_ID.tar.xz 

6. Go to your clone of the fedora binutils package and make sure that it is clean and up to date:

pushd <rawhide-binutils>
git switch rawhide
git pull
git clean -fdx
git status 

7. Delete the current source tarball and then copy in the source tarball created in step 5:

rm -f *.tar.xz
cp <upstream-binutils>/binutils*.tar.xz . 

8. [Optional] Tidy up the upstream repository.

pushd <upstream-binutils>
git clean -fdx
popd

9. View the binutils.spec file and check that the source variable is set to tarball and that the Version field is set to the version number of the tarball. If necessary change them to the correct values:

grep -e "Version:" -e "define source" binutils.spec

Then set the commit_id variable to the commit id of the new tarball:

sed --in-place -e 's/define commit_id.*/define commit_id %(echo "COMMI_ID")/' binutils.spec 

Replace COMMIT_ID in the above command with the commit id of the tarball.

Also increment the Release: field by one and add a changelog entry for the update:

rpmdev-bumpspec -c "- Rebase to commit $COMMIT_ID" binutils.spec

10. Create a source tree and discover if any local patches no longer apply:

fedpkg prep 

If a patch does not apply, decide if it is no longer necessary, or if it needs to be adjusted. Take whatever actions are necessary to resolve this issue and then repeat step 5 until a clean run is obtained.

If a patch is not needed, remove its entry from the binutils.spec file.

Also schedule the patch to be deleted from the repository:

git rm <unneeded-patch>

Optionally add a mention of this to the changelog entry created in step 5.

- Retire: <unneeded-patch>

Optionally renumber the remaining patches referenced in the binutils.spec file so that they are in a monotonically increasing order.

11. Run a local build to ensure that there are no surprises.

 fedpkg local 

If there are problems, fix them. Note, local builds run in parallel, so it is not always easy to discover the location of a build failure. If the failure is due to a bug in the upstream sources, then it may be better to abort this sync and wait for the bug to be fixed. Or to fix the bug upstream first and then start the sync process again. If the failure is due to the local patches that are applied to the sources then they may need to be adjusted. If a failure is due to a testsuite problem then either the test might need to be tweaked, or just skipped if there is no easy way to correct it.

12. Then run a scratch build.

rm -f *.rpm
fedpkg srpm
fedpkg scratch-build --srpm *.rpm 

At the same time also run a scratch build for the RISC-V target:

koji -p riscv build --scratch rawhide *.rpm

If the build fails, investigate and fix the problems. The most likely cause at this point is new failures in one of the testsuites. Note - the build.log file created for the build for each architecture will include a uuencoded compressed tarball of the logs from the testsuite runs. Hence the logs can be downloaded and examined locally by:

[download build.log file]
uudecode build.log
tar xvf binutils-*.tar.xz

13. [Optional] This next step is optional, but recommended. It assumes that you have command line access to the COPR system and you have the mass-prebuild package installed. Warning - this step will take a long time to complete, as in 3-4 days...

Set up a config script for the mass pre-build tool.

archs: x86_64 aarch64 ppc64le s390x
chroot: fedora-rawhide
name: binutils-$COMMIT_ID
packages:
  binutils:
    src_type: file
    src: <path-to-source-rpm-created-in-step-8>
data: <path-to-results-directory>
verbose: 1

Then use it to run a test of the new binutils

mpb --config <config-file>

If there are problems, try to find out what went wrong.

14. Finally check that the fedora repository has not received any other commits, add the new sources, and commit the changes.

git pull
fedpkg new-sources binutils-*.tar.xz binutils-2.19.50.0.1-output-format.sed
fedpkg commit -m "Rebase to commit $COMMIT_ID"
fedpkg push
fedpkg build

15. Sit back and relax. Have a cup of tea. Then wait for the bug reports to flood in.