MinGW/CrossCompilerFramework

From FedoraProject

< MinGW(Difference between revisions)
Jump to: navigation, search
(Porting guide)
Line 21: Line 21:
  
 
Filelists:
 
Filelists:
  %{_cross-...}    Used for files which are common among all possible targets
+
  %{_cross_...}    Used for files which are common among all possible targets
  %{_mingw-...}    Used for files which are used on both Win32 and Win64
+
  %{_mingw_...}    Used for files which are used on both Win32 and Win64
  %{_mingw32-...}  Used for files which are only used on Win32
+
  %{_mingw32_...}  Used for files which are only used on Win32
  %{_mingw64-...}  Used for files which are only used on Win64
+
  %{_mingw64_...}  Used for files which are only used on Win64
  
 
= Development and testing repository =
 
= Development and testing repository =
Line 38: Line 38:
 
To provide a clean upgrade path and as little breakage as possible the introduction of this framework can be split in several phases.
 
To provide a clean upgrade path and as little breakage as possible the introduction of this framework can be split in several phases.
  
== Phase 1: Get the mingw-w64 toolchain in Fedora ==
+
=== Phase 1: Get the mingw-w64 toolchain in Fedora ===
  
 
This requires some new packages to be approved using package reviews.
 
This requires some new packages to be approved using package reviews.
Line 45: Line 45:
 
The keep the amount of breakage as low as possible all these 5 packages have to be imported at once.
 
The keep the amount of breakage as low as possible all these 5 packages have to be imported at once.
  
== Phase 2: Bootstrap the mingw-w64 toolchain ==
+
=== Phase 2: Bootstrap the mingw-w64 toolchain ===
  
 
Once the 5 packages mentioned in phase 1 are approved they can be imported. In order to bootstrap the mingw-w64 toolchain this has to be done in a specific order.
 
Once the 5 packages mentioned in phase 1 are approved they can be imported. In order to bootstrap the mingw-w64 toolchain this has to be done in a specific order.
Line 55: Line 55:
 
  6. Rebuild the cross-gcc package with the %bootstrap flag set to 0 and wait for it to enter the buildroot
 
  6. Rebuild the cross-gcc package with the %bootstrap flag set to 0 and wait for it to enter the buildroot
  
== Phase 3: Rebuild all mingw32 packages ==
+
=== Phase 3: Rebuild all mingw32 packages ===
  
 
The mingw-w64 toolchain is now operational and can build binaries for both Win32 and Win64.
 
The mingw-w64 toolchain is now operational and can build binaries for both Win32 and Win64.
Line 65: Line 65:
 
Packages which have the triplet 'i686-pc-mingw32' hardcoded in their .spec files need to updated as they aren't conforming to the packaging guidelines
 
Packages which have the triplet 'i686-pc-mingw32' hardcoded in their .spec files need to updated as they aren't conforming to the packaging guidelines
  
== Phase 4: Drop the old mingw32 toolchain packages ==
+
=== Phase 4: Drop the old mingw32 toolchain packages ===
  
 
Now that the mingw-w64 based toolchain is imported the original mingw32 based toolchain has been obsoleted.
 
Now that the mingw-w64 based toolchain is imported the original mingw32 based toolchain has been obsoleted.
 
The packages mingw32-filesystem, mingw32-binutils, mingw32-gcc, mingw32-w32api and mingw32-runtime can be dropped from the repository without breaking anything
 
The packages mingw32-filesystem, mingw32-binutils, mingw32-gcc, mingw32-w32api and mingw32-runtime can be dropped from the repository without breaking anything
  
== Phase 5: Rename and port all mingw32-packages to the new framework ==
+
=== Phase 5: Rename and port all mingw32-packages to the new framework ===
  
 
This phase is also the most time-consuming one. Right now all mingw32 packages use the prefix 'mingw32-'.
 
This phase is also the most time-consuming one. Right now all mingw32 packages use the prefix 'mingw32-'.
Line 76: Line 76:
 
In additional several macros will have to be replaced. See the [[#Porting guide|porting guide]]
 
In additional several macros will have to be replaced. See the [[#Porting guide|porting guide]]
  
== Phase 6: Drop deprecated compatibility macros (post F15) ==
+
=== Phase 6: Drop deprecated compatibility macros (post F15) ===
  
 
Once all packages are ported to the new framework the mingw32 compatibility macros can be dropped from the cross-filesystem package. This can happen after F15 has been released and development for F16 has started
 
Once all packages are ported to the new framework the mingw32 compatibility macros can be dropped from the cross-filesystem package. This can happen after F15 has been released and development for F16 has started
Line 85: Line 85:
 
This has been done to indicate that the new mingw-w64 toolchain is used instead of the old mingw.org one.
 
This has been done to indicate that the new mingw-w64 toolchain is used instead of the old mingw.org one.
 
All paths have been updated to use this new name as well. So now all files are kept in /usr/i686-w64-mingw32 instead of /usr/i686-pc-mingw32 for Win32.
 
All paths have been updated to use this new name as well. So now all files are kept in /usr/i686-w64-mingw32 instead of /usr/i686-pc-mingw32 for Win32.
After the initial rebuild (as mentioned in phase 4) all files will be moved to this new location.
+
After the initial rebuild (as mentioned in phase 3) all files will be moved to this new location.
  
 
If you want your package to be built for multiple targets instead of just Win32 you need to rename the package to use the 'cross-' or 'mingw-' prefix (depending on the package) and apply some changes in the .spec file.
 
If you want your package to be built for multiple targets instead of just Win32 you need to rename the package to use the 'cross-' or 'mingw-' prefix (depending on the package) and apply some changes in the .spec file.
 
If you don't want to add support for other targets you can leave the .spec file as is and ignore the rest of this porting guide.
 
If you don't want to add support for other targets you can leave the .spec file as is and ignore the rest of this porting guide.
  
== Generic macros ==
+
=== Generic macros ===
  
 
Most Fedora MinGW packages contain this set of instructions at the top of the .spec file:
 
Most Fedora MinGW packages contain this set of instructions at the top of the .spec file:
Line 110: Line 110:
 
Packages containing macros to generate debuginfo subpackages should also update the call to %{?_mingw32_debug_package} to %{?_cross_debug_package}
 
Packages containing macros to generate debuginfo subpackages should also update the call to %{?_mingw32_debug_package} to %{?_cross_debug_package}
  
== Build targets ==
+
=== Build targets ===
  
 
To indicate which targets you want to have the package build you need to add one of these (or all) lines to your .spec file.
 
To indicate which targets you want to have the package build you need to add one of these (or all) lines to your .spec file.
Line 118: Line 118:
 
  %global _cross_build_win64 1
 
  %global _cross_build_win64 1
  
== Description and Summary ==
+
=== Description and Summary ===
  
 
Most packages mention in their description and/or summary that the package in question is targeted for Win32. Though not strictly required, it's recommend to update
 
Most packages mention in their description and/or summary that the package in question is targeted for Win32. Though not strictly required, it's recommend to update
 
the description and summary tags to indicate the targets which this package supports
 
the description and summary tags to indicate the targets which this package supports
  
== BuildRequires ==
+
=== BuildRequires ===
  
 
As a new 'cross-' prefix has been introduced for the base toolchain packages the BuildRequires have to be updated:
 
As a new 'cross-' prefix has been introduced for the base toolchain packages the BuildRequires have to be updated:
Line 138: Line 138:
 
For other dependencies it depends on whether that specific package has been ported to the new cross compiler framework
 
For other dependencies it depends on whether that specific package has been ported to the new cross compiler framework
  
== Dropped packages ==
+
=== Running the ./configure script ===
 +
 
 +
The original Fedora Mingw toolchain contained a macro called %{_mingw32_configure} which could be used to call the ./configure script with the proper arguments set.
 +
For cross compilation to multiple targets a new macro has been introduced called %{_cross_configure}.
 +
The macro creates a separate folder for each target (out of source compilation) and executes the configure script for every target (set with the flags mentioned earlier).
 +
 
 +
Almost all packages support out of source compilation or require slight patching. The only known exceptions to date are zlib and openssl.
 +
Packages which don't support out of source compilation may require a different approach like performing everything in the %install phase.
 +
If you happen to stumble across a package which requires a different approach feel free to contact us on the Fedora MinGW mailing list
 +
 
 +
If you want to supply arguments to the ./configure call, you have to use a slightly different approach than before.
 +
Say you've got something like the call below:
 +
%{_mingw32_configure} --disable-xlib --enable-win32
 +
This has to be changed to something like this:
 +
%{_cross_configure "--disable-xlib" "--enable-win32"}
 +
The quotes mentioned above are mandatory! If you forget them then rpmbuild will fail or the arguments might not get passed to the ./configure calls.
 +
Also note that the '}' character needs to be placed after all the arguments
 +
 
 +
Some packages need to be built multiple times for each target. Examples of this are packages which have to be built once for a static version and once for a shared version.
 +
Such packages can add a custom suffix to the build directory used.
 +
Say you've got something like below:
 +
mkdir build_shared
 +
pushd build_shared
 +
%{_mingw32_configure} --enable-shared
 +
popd
 +
mkdir build_static
 +
pushd build_static
 +
%{_mingw32_configure} --enable-static
 +
popd
 +
This can be rewritten to something like this:
 +
%{_cross_configure -s shared "--enable-shared"}
 +
%{_cross_configure -s static "--enable-static"}
 +
 
 +
=== Building and installing the package ===
 +
 
 +
Most packages used the command 'make %{?_smp_mflags}' to build the package.
 +
In the cross compiler framework you have to use '%{_cross_make %{?_smp_mflags}' to build the package for all configured targets
 +
As with the %{_cross_configure} macro you can also use the '-s' argument to indicate a custom suffix to the build directory used
 +
 
 +
To install the package the command 'make install DESTDIR=$RPM_BUILD_ROOT' was used in almost all cases.
 +
This can be rewritten to '%{_cross_make_install DESTDIR=$RPM_BUILD_ROOT}' to install the package for all configured targets
 +
The '-s' argument can also be used here
 +
 
 +
=== Custom %install instructions ===
 +
 
 +
Some packages require some custom instructions before the files are ready to be packaged.
 +
Such code can remain as is. However, you may need to duplicate these instructions multiple times (for all configured targets).
 +
 
 +
=== File lists ===
 +
 
 +
All the original filelist macros from the original Fedora MinGW toolchain (like %{_mingw32_bindir}) are still available.
 +
However, when adding support for multiple targets these macros aren't sufficient.
 +
To create filelists which support multiple targets you can use a different set of macros as can be seen at the [[#Namespaces]] chapter.
 +
 
 +
The following macros exists for use in filelists:
 +
%{_cross_bindir}
 +
%{_cross_libdir}
 +
%{_cross_includedir}
 +
%{_cross_datadir}
 +
%{_cross_sysconfdir}
 +
%{_cross_docdir}
 +
 +
%{_mingw_bindir}
 +
%{_mingw_libdir}
 +
%{_mingw_includedir}
 +
%{_mingw_datadir}
 +
%{_mingw_sysconfdir}
 +
%{_mingw_docdir}
 +
 +
%{_mingw32_sysroot}
 +
%{_mingw32_prefix}
 +
%{_mingw32_exec_prefix}
 +
%{_mingw32_bindir}
 +
%{_mingw32_libdir}
 +
%{_mingw32_includedir}
 +
%{_mingw32_datadir}
 +
%{_mingw32_sysconfdir}
 +
%{_mingw32_docdir}
 +
%{_mingw32_sbindir}
 +
%{_mingw32_libexecdir}
 +
%{_mingw32_infodir}
 +
%{_mingw32_mandir}
 +
%{_mingw32_sharedstatedir}
 +
%{_mingw32_localstatedir}
 +
 +
%{_mingw64_sysroot}
 +
%{_mingw64_prefix}
 +
%{_mingw64_exec_prefix}
 +
%{_mingw64_bindir}
 +
%{_mingw64_libdir}
 +
%{_mingw64_includedir}
 +
%{_mingw64_datadir}
 +
%{_mingw64_sysconfdir}
 +
%{_mingw64_docdir}
 +
%{_mingw64_sbindir}
 +
%{_mingw64_libexecdir}
 +
%{_mingw64_infodir}
 +
%{_mingw64_mandir}
 +
%{_mingw64_sharedstatedir}
 +
%{_mingw64_localstatedir}
 +
 
 +
When using the %{_cross_...} or %{_mingw_...} macros please note that the expect an argument.
 +
For example, these set of files:
 +
%{_mingw32_libdir}/pkgconfig/libsoup-2.4.pc
 +
%{_mingw64_libdir}/pkgconfig/libsoup-2.4.pc
 +
can be merged in one by using the following macro:
 +
%{_cross_libdir /pkgconfig/libsoup-2.4.pc}
 +
 
 +
=== Dropped packages ===
  
 
Some packages aren't needed anymore to compile software these days so they won't be ported to the new cross compiler framework and will be dropped from the repository eventually:
 
Some packages aren't needed anymore to compile software these days so they won't be ported to the new cross compiler framework and will be dropped from the repository eventually:
 
  mingw32-dlfcn
 
  mingw32-dlfcn

Revision as of 14:28, 11 October 2010

Contents

Introduction

One of the new features of F15 will be the introduction of a Cross Compiler Framework. This framework will obsolete the Fedora MinGW toolchain which can be found in current versions of Fedora.

Initially, this framework will have support for building binaries for Win32 and Win64 targets. The mingw-w64 toolchain will be used for this. However support for other targets (like Mac OS X) can be added quite easily.

The goal of this framework is to provide a generic method to cross-compile binaries for multiple targets with as little packaging overhead as possible.

Namespaces

The framework will use several namespaces. Each namespace has it own characteristics. These will be explained here Package names:

cross-            Used for packages which can be built for multiple targets
mingw-            Used for packages which can be built for both Win32 and Win64 but not for other targets
mingw32-          Used for packages which can only be built for Win32
mingw64-          Used for packages which can only be built for Win64
darwinx-          Used for packages which can only be built for Mac OS X (future work)

Filelists:

%{_cross_...}     Used for files which are common among all possible targets
%{_mingw_...}     Used for files which are used on both Win32 and Win64
%{_mingw32_...}   Used for files which are only used on Win32
%{_mingw64_...}   Used for files which are only used on Win64

Development and testing repository

All development of this toolchain is currently being done in a SVN repository at http://svn.openftd.org/svn/fedora_cross (SVNWeb: http://svn.openftd.org/viewvc/Fedora%20Cross%20Compilers%20Environment/) It has anonymous read-only access. For write access, please contact epienbro on FreeNode #fedora-mingw.

A yum testing repository containing (x86_64) binaries for all packages will be published soon. Note that this testing repository will obsolete any mingw32 packages you might have installed!

Roadmap

To provide a clean upgrade path and as little breakage as possible the introduction of this framework can be split in several phases.

Phase 1: Get the mingw-w64 toolchain in Fedora

This requires some new packages to be approved using package reviews. The packages which have to be approved are cross-filesystem, cross-binutils, cross-gcc, mingw-crt and mingw-headers

The keep the amount of breakage as low as possible all these 5 packages have to be imported at once.

Phase 2: Bootstrap the mingw-w64 toolchain

Once the 5 packages mentioned in phase 1 are approved they can be imported. In order to bootstrap the mingw-w64 toolchain this has to be done in a specific order.

1. Import the cross-filesystem package, build it and wait for it to enter the buildroot
2. Import the cross-binutils package, build it and wait for it to enter the buildroot
3. Import the mingw-headers package, build it and wait for it to enter the buildroot
4. Import the cross-gcc package, build it (with the %bootstrap flag set to 1) and wait for it to enter the buildroot
5. Import the mingw-crt package, build it and wait for it to enter the buildroot
6. Rebuild the cross-gcc package with the %bootstrap flag set to 0 and wait for it to enter the buildroot

Phase 3: Rebuild all mingw32 packages

The mingw-w64 toolchain is now operational and can build binaries for both Win32 and Win64. As the name of the triplet has been changed (in the cross-filesystem package) from i686-pc-mingw32 to i686-w64-mingw64 as mingw32 packages have to be rebuild. All mingw32 packages should have been using macros like %{_mingw32_bindir} as mentioned in the MinGW Packaging Guidelines. As all these macros have been moved to the cross-filesystem package (and some deprecated macros are kept for backwards compatibility) a plain rebuild should be sufficient to get mingw32 packages built against the new toolchain.

Packages which have the triplet 'i686-pc-mingw32' hardcoded in their .spec files need to updated as they aren't conforming to the packaging guidelines

Phase 4: Drop the old mingw32 toolchain packages

Now that the mingw-w64 based toolchain is imported the original mingw32 based toolchain has been obsoleted. The packages mingw32-filesystem, mingw32-binutils, mingw32-gcc, mingw32-w32api and mingw32-runtime can be dropped from the repository without breaking anything

Phase 5: Rename and port all mingw32-packages to the new framework

This phase is also the most time-consuming one. Right now all mingw32 packages use the prefix 'mingw32-'. All packages (which want to be built for both win32 and win64) have to be renamed to use the prefix 'cross-' instead of 'mingw32-'. In additional several macros will have to be replaced. See the porting guide

Phase 6: Drop deprecated compatibility macros (post F15)

Once all packages are ported to the new framework the mingw32 compatibility macros can be dropped from the cross-filesystem package. This can happen after F15 has been released and development for F16 has started

Porting guide

First thing to keep in mind is that the name of the triplet for Win32 has been changed. Originally this was i686-pc-mingw32 while with the new cross compiler framework this has changed to i686-w64-mingw32. This has been done to indicate that the new mingw-w64 toolchain is used instead of the old mingw.org one. All paths have been updated to use this new name as well. So now all files are kept in /usr/i686-w64-mingw32 instead of /usr/i686-pc-mingw32 for Win32. After the initial rebuild (as mentioned in phase 3) all files will be moved to this new location.

If you want your package to be built for multiple targets instead of just Win32 you need to rename the package to use the 'cross-' or 'mingw-' prefix (depending on the package) and apply some changes in the .spec file. If you don't want to add support for other targets you can leave the .spec file as is and ignore the rest of this porting guide.

Generic macros

Most Fedora MinGW packages contain this set of instructions at the top of the .spec file:

%global __strip %{_mingw32_strip}
%global __objdump %{_mingw32_objdump}
%global _use_internal_dependency_generator 0
%global __find_requires %{_mingw32_findrequires}
%global __find_provides %{_mingw32_findprovides}
%define __debug_install_post %{_mingw32_debug_install_post}

These have to be adjusted so that the generic cross compilation macros are used:

%global __strip %{_cross_strip}
%global __objdump %{_cross_objdump}
%global _use_internal_dependency_generator 0
%global __find_requires %{_cross_findrequires}
%global __find_provides %{_cross_findprovides}
%define __debug_install_post %{_cross_debug_install_post}

Packages containing macros to generate debuginfo subpackages should also update the call to %{?_mingw32_debug_package} to %{?_cross_debug_package}

Build targets

To indicate which targets you want to have the package build you need to add one of these (or all) lines to your .spec file. To build for Win32:

%global _cross_build_win32 1

To build for Win64:

%global _cross_build_win64 1

Description and Summary

Most packages mention in their description and/or summary that the package in question is targeted for Win32. Though not strictly required, it's recommend to update the description and summary tags to indicate the targets which this package supports

BuildRequires

As a new 'cross-' prefix has been introduced for the base toolchain packages the BuildRequires have to be updated:

Original                             New
------------------------             ----------------------
mingw32-filesystem >= xx             cross-filesystem >= 25
mingw32-binutils                     cross-binutils
mingw32-cpp                          cross-cpp
mingw32-gcc                          cross-gcc
mingw32-gcc-c++                      cross-gcc-c++
mingw32-runtime                      mingw-runtime
mingw32-w32api                       mingw-headers

For other dependencies it depends on whether that specific package has been ported to the new cross compiler framework

Running the ./configure script

The original Fedora Mingw toolchain contained a macro called %{_mingw32_configure} which could be used to call the ./configure script with the proper arguments set. For cross compilation to multiple targets a new macro has been introduced called %{_cross_configure}. The macro creates a separate folder for each target (out of source compilation) and executes the configure script for every target (set with the flags mentioned earlier).

Almost all packages support out of source compilation or require slight patching. The only known exceptions to date are zlib and openssl. Packages which don't support out of source compilation may require a different approach like performing everything in the %install phase. If you happen to stumble across a package which requires a different approach feel free to contact us on the Fedora MinGW mailing list

If you want to supply arguments to the ./configure call, you have to use a slightly different approach than before. Say you've got something like the call below:

%{_mingw32_configure} --disable-xlib --enable-win32

This has to be changed to something like this:

%{_cross_configure "--disable-xlib" "--enable-win32"}

The quotes mentioned above are mandatory! If you forget them then rpmbuild will fail or the arguments might not get passed to the ./configure calls. Also note that the '}' character needs to be placed after all the arguments

Some packages need to be built multiple times for each target. Examples of this are packages which have to be built once for a static version and once for a shared version. Such packages can add a custom suffix to the build directory used. Say you've got something like below:

mkdir build_shared
pushd build_shared
%{_mingw32_configure} --enable-shared
popd
mkdir build_static
pushd build_static
%{_mingw32_configure} --enable-static
popd

This can be rewritten to something like this:

%{_cross_configure -s shared "--enable-shared"}
%{_cross_configure -s static "--enable-static"}

Building and installing the package

Most packages used the command 'make %{?_smp_mflags}' to build the package. In the cross compiler framework you have to use '%{_cross_make %{?_smp_mflags}' to build the package for all configured targets As with the %{_cross_configure} macro you can also use the '-s' argument to indicate a custom suffix to the build directory used

To install the package the command 'make install DESTDIR=$RPM_BUILD_ROOT' was used in almost all cases. This can be rewritten to '%{_cross_make_install DESTDIR=$RPM_BUILD_ROOT}' to install the package for all configured targets The '-s' argument can also be used here

Custom %install instructions

Some packages require some custom instructions before the files are ready to be packaged. Such code can remain as is. However, you may need to duplicate these instructions multiple times (for all configured targets).

File lists

All the original filelist macros from the original Fedora MinGW toolchain (like %{_mingw32_bindir}) are still available. However, when adding support for multiple targets these macros aren't sufficient. To create filelists which support multiple targets you can use a different set of macros as can be seen at the #Namespaces chapter.

The following macros exists for use in filelists:

%{_cross_bindir}
%{_cross_libdir}
%{_cross_includedir}
%{_cross_datadir}
%{_cross_sysconfdir}
%{_cross_docdir}

%{_mingw_bindir}
%{_mingw_libdir}
%{_mingw_includedir}
%{_mingw_datadir}
%{_mingw_sysconfdir}
%{_mingw_docdir}

%{_mingw32_sysroot}
%{_mingw32_prefix}
%{_mingw32_exec_prefix}
%{_mingw32_bindir}
%{_mingw32_libdir}
%{_mingw32_includedir}
%{_mingw32_datadir}
%{_mingw32_sysconfdir}
%{_mingw32_docdir}
%{_mingw32_sbindir}
%{_mingw32_libexecdir}
%{_mingw32_infodir}
%{_mingw32_mandir}
%{_mingw32_sharedstatedir}
%{_mingw32_localstatedir}

%{_mingw64_sysroot}
%{_mingw64_prefix}
%{_mingw64_exec_prefix}
%{_mingw64_bindir}
%{_mingw64_libdir}
%{_mingw64_includedir}
%{_mingw64_datadir}
%{_mingw64_sysconfdir}
%{_mingw64_docdir}
%{_mingw64_sbindir}
%{_mingw64_libexecdir}
%{_mingw64_infodir}
%{_mingw64_mandir}
%{_mingw64_sharedstatedir}
%{_mingw64_localstatedir}

When using the %{_cross_...} or %{_mingw_...} macros please note that the expect an argument. For example, these set of files:

%{_mingw32_libdir}/pkgconfig/libsoup-2.4.pc
%{_mingw64_libdir}/pkgconfig/libsoup-2.4.pc

can be merged in one by using the following macro:

%{_cross_libdir /pkgconfig/libsoup-2.4.pc}

Dropped packages

Some packages aren't needed anymore to compile software these days so they won't be ported to the new cross compiler framework and will be dropped from the repository eventually:

mingw32-dlfcn