Windows loaders packaging draft: Difference between revisions

Latest revision as of 15:28, 19 October 2011

Exception for windows programs that link to unpackaged libraries

Short version

For the Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries section

If a package provides Windows executables (with corresponding source code), but those executables link to multiple Windows DLLs that we do not ship and thus cannot be rebuilt from source, you may be allowed to ship the pre-compiled Windows binary from upstream. See Precompiled Windows Programs for details.

Precompiled Windows programs

Longer version for its own page.

Software installers aimed at upstream developers may want to ship bits of code that can be used to install the upstream developers code on Windows or other operating systems. Sometimes these bits of code can be compiled from source using a cross-compiler like MinGW. Other times, they cannot be shipped that way due to the limitations of Windows and MinGW. This page will explain how to tell the difference. In all cases, the upstream sources must ship with the source code for the binary; you just are allowed to ship from upstream's pre-compilation of that source in specific cases.

The Windows dynamic library loader is a bit different from the Linux loader. It is capable of loading multiple versions of a library into a single process. One ramification of this is that some software may use one version of a library in its core but a library (or plugin) that it loads will use a different version. When this occurs, the application code has to be careful not to send data from its version of the library where the other version of the library may try to operate on it as if it was created by itself otherwise invalid access to memory or silent bugs may occur. This is especially bad with the Windows C library (known as the CRT -- C RunTime). If there are multiple versions of the CRT, simple things like stdout and stdin will fail to work since the FILE * handles for one of the versions will be attempting to write to the files opened by the other version and Windows will disallow that.

The latter has an especially problematic interaction with MinGW. MinGW does not provide a CRT since every current version of Windows ships with one. This means that when we cross-compile with MinGW, the application we create must be dynamically linked to the CRT. If we also link dynamically to other libraries then the application and the other libraries may ask the Windows loader to load two separate versions of the CRT and we'll run into the problems outlined above.

What this means for us is when the software we're packaging is intended to run on Windows end user's systems and the software links against multiple libraries then recompiling it locally will lead to code which may have errors when it finally gets to the Windows user's system simply because the code uses common functions from the C library. Shipping the upstream files which are built using a Windows native compiler and have statically linked to a CRT allows this software to run on Windows more reliably.

Links

Microsoft post on using the CRT http://msdn.microsoft.com/en-us/library/ms235460%28v=VS.100%29.aspx
A developer's thoughts on working on this issue with Apache on Windows http://www.syndicateofideas.com/posts/fighting-the-msvcrt-dll-hell

@@ Line 1: / Line 1: @@
-== Exception for precompiled windows loaders ==
+== Exception for windows programs that link to unpackaged libraries ==
 === Short version ===
 {{admon/note||For the [[Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries]] section}}
-* Windows wrapper executables for scripts may be allowed.  See [[Precompiled Windows Script Wrappers]] for details
+* If a package provides Windows executables (with corresponding source code), but those executables link to multiple Windows DLLs that we do not ship and thus cannot be rebuilt from source, you may be allowed to ship the pre-compiled Windows binary from upstream. See [[Precompiled Windows Programs]] for details.
-=== Precompiled Windows Script Wrappers ===
+=== Precompiled Windows programs ===
 {{admon/note||Longer version for its own page.}}
-Windows does not support the shebang line like Unix systems do.  For this reason, Windows has an idiom of creating wrapper executables for scripts.  These wrappers check what name they were invoked under and then attempt to launch an interpreter with the script in question.  For instance, a python loader may be called foo-app.exe.  When run on Windows, it will try to run a python interpreter from the Windows PATH on the file foo-app.py in the same directory as the wrapper.
+Software installers aimed at upstream developers may want to ship bits of code that can be used to install the upstream developers code on Windows or other operating systems.  Sometimes these bits of code can be compiled from source using a cross-compiler like MinGW.  Other times, they cannot be shipped that way due to the limitations of Windows and MinGW.  This page will explain how to tell the difference.  In all cases, the upstream sources must ship with the source code for the binary; you just are allowed to ship from upstream's pre-compilation of that source in specific cases.
-In some cases we may want to package software in Fedora which contains generic wrappers.  The purpose here would be so that developers can create a script on Linux but package it up for use on Windows.  The software package in Fedora would allow the developer to copy the generic wrapper into their software to run their script when installed on Windows.  It therefore makes sense for us to ship those compiled wrappers in our package.
+The Windows dynamic library loader is a bit different from the Linux loader.  It is capable of loading multiple versions of a library into a single process.  One ramification of this is that some software may use one version of a library in its core but a library (or plugin) that it loads will use a different version.  When this occurs, the application code has to be careful not to send data from its version of the library where the other version of the library may try to operate on it as if it was created by itself otherwise invalid access to memory or silent bugs may occur.  This is especially bad with the Windows C library (known as the <b>CRT</b> -- C RunTime).  If there are multiple versions of the <code>CRT</code>, simple things like stdout and stdin will fail to work since the <code>FILE *</code> handles for one of the versions will be attempting to write to the files opened by the other version and Windows will disallow that.
-{{admon/question||Does the following make sense?  I don't know Windows, just piecing together the issue from an IRC discussion.  Not sure precisely why loading the CRT library at the bottom would cause conflicts when, at the same time, mingw says when you cross-compile a program, that program should work on any system, no matter what version of MSVCRT is installed.}}
+The latter has an especially problematic interaction with MinGW.  MinGW does not provide a <code>CRT</code> since every current version of Windows ships with one.  This means that when we cross-compile with MinGW, the application we create must be dynamically linked to the <code>CRT</code>.  If we also link dynamically to other libraries then the application and the other libraries may ask the Windows loader to load two separate versions of the <code>CRT</code> and we'll run into the problems outlined above.
-Now, as to why those wrappers need to be pre-compiled, as opposed to our cross-compiling them with something like mingw.  These wrappers are going to need to link against a C library for Windows (known as a CRT -- C-RunTime).  Mingw does not contain a CRT.dll so if we were to compile the wrapper script using mingw, the CRT would need to be linked dynamically.  Oftentimes, the script interpreters installed on Windows are statically linked to a specific CRT library.  If the wrapper is dynamic, Windows will load the most recent CRT.dll that it has and then run it.  When the wrapper tries to load the statically linked interpreter, the symbols from the CRT library that's embedded in the interpreter will conflict with the symbols provided by the dynamic CRT.dll.  The wrapper and interpreter must both be statically linked to avoid this conflict.  Since we cannot cross-compile in this manner with mingw, we need an exception to ship the upstream supplied pre-compiled binaries.
+What this means for us is when the software we're packaging is intended to run on Windows end user's systems and the software links against multiple libraries then recompiling it locally will lead to code which may have errors when it finally gets to the Windows user's system simply because the code uses common functions from the C library.  Shipping the upstream files which are built using a Windows native compiler and have statically linked to a <code>CRT</code> allows this software to run on Windows more reliably.
+==== Links ====
+* Microsoft post on using the CRT http://msdn.microsoft.com/en-us/library/ms235460%28v=VS.100%29.aspx
+* A developer's thoughts on working on this issue with Apache on Windows http://www.syndicateofideas.com/posts/fighting-the-msvcrt-dll-hell
 [[Category:Packaging guidelines drafts]]

Search