From Fedora Project Wiki

(Notes to ext/ files and example followup)
m (internal link cleaning)
 
(7 intermediate revisions by 2 users not shown)
Line 3: Line 3:
== Current status ==
== Current status ==
Current packaging guidelines on Fedora about Ruby Gems are mainly written on
Current packaging guidelines on Fedora about Ruby Gems are mainly written on
https://fedoraproject.org/wiki/Packaging/Ruby#Ruby_Gems
[[Packaging/Ruby#Ruby_Gems]]


The guideline says:
The guideline says:
Line 40: Line 40:
==== Solution to B ====
==== Solution to B ====
Usually ruby extension uses ruby script file named <code>extconf.rb</code> which has <code>require 'mkmf'</code>. <code>mkmf.rb</code> module accepts
Usually ruby extension uses ruby script file named <code>extconf.rb</code> which has <code>require 'mkmf'</code>. <code>mkmf.rb</code> module accepts
<code>CONFIGURE_ARGS</code> environment to specify <code>CFLAGS</code> so using this is
<code>CONFIGURE_ARGS</code> environment to specify <code>CFLAGS</code> so using this is preferred.
preferred.
* even for non-Gem Ruby packages, many packages containing C shared libraries
** have <code>extconf.rb</code> in the source archives
** or have <code>Rakefile</code> which create <code>extconf.rb</code>
which usually contain <code>require mkmf</code>. In such case
<pre>
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
</pre>
can be used, too. So using this method is preferred even for non-Gem packages.
* A package which uses <code>extconf.rb</code> directly can take <code>--with-cflags=</code> argument directly, like <code>ruby extconf.rb --with-cflags="%{optflags}"</code>, however in the case that <code>Rakefile</code> creates <code>extconf.rb</code>, usually <code>rake build</code> does not accept <code>--with-cflags=</code> argument.


== Proposition ==
== Proposition ==
When Ruby Gem contains extension libraries written in C,
* Add the following sentences at the end of '''Ruby packages with binary content/shared libraries'''
* First <code>%{_builddir}/%{buildsubdir}</code> '''must''' be created at <code>%prep</code> stage.
 
* The following line '''should''' be used to pass <code>CFLAGS</code> to <code>Makefile</code> to be created correctly.
For packages which create C shared libraries using <code>extconf.rb</code>
<pre>
<pre>
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
</pre>
</pre>
'''should''' be used to pass <code>CFLAGS</code> to <code>Makefile</code> correctly.
This also applies to Ruby Gems.
* And add below.
'''Ruby Gem with extension libraries written in C'''
When Ruby Gem contains extension libraries written in C,
* First <code>%{_builddir}/%{buildsubdir}</code> '''must''' be created at <code>%prep</code> stage.
* Then the Ruby Gem '''must''' be installed under <code>%{_builddir}/%{buildsubdir}</code> at <code>%build</code> stage to get C libraries compiled under the directory.
* Then the Ruby Gem '''must''' be installed under <code>%{_builddir}/%{buildsubdir}</code> at <code>%build</code> stage to get C libraries compiled under the directory.
* When using <code>gem</code> command to install Gem file, using <code>-V</code> option is recommend to check if <code>CFLAGS</code> is correctly honored.
* When using <code>gem</code> command to install Gem file, using <code>-V</code> option is recommend to check if <code>CFLAGS</code> is correctly honored.
Line 56: Line 74:
* Installed C codes (usually under %{geminstdir}/etc) '''may''' be removed even if <code>gem contents %{gemname}</code> reports that installed C codes should be found there.
* Installed C codes (usually under %{geminstdir}/etc) '''may''' be removed even if <code>gem contents %{gemname}</code> reports that installed C codes should be found there.


== Revised proposition ==
From the comments I received, it seems that the following proposition is clearer.
(The part <code>CONFIGURE_ARGS=...</code> stays unchanged)
'''Ruby Gem with extension libraries written in C'''
When Ruby Gem contains extension libraries written in C,
* First, <code>%prep</code> stage '''must''' contain <code>%setup -q -c -T</code> to create the directory where C libraries are compiled.
* Then at %build stage the Ruby Gem '''must''' be installed under the directory created at %prep stage to get C libraries compiled under there.
* When <code>gem install</code> is used to install Gem file, using <code>-V</code> option is '''recommend''' to check if <code>CFLAGS</code> is correctly honored.
* Finally at <code>%install</code> stage the whole tree under the directory created at %prep stage '''should''' be '''copied''' (not moved) to under <code>%{buildroot}%{gemdir}</code>.
** When all tree under the directory created at %prep stage is moved to under <code>%{buildroot}</code>, <code>find_debuginfo.sh</code> will complain that the corresponding source files are missing.
* Installed C codes (usually under <code>%{geminstdir}/etc</code>) '''may''' be removed even if <code>gem contents %{gemname}</code> reports that installed C codes should be found there.


=== Note ===
=== Note ===
Line 97: Line 129:


== Issues ==
== Issues ==
=== Installed C codes ===
=== [SOLVED] Installed C codes ===
Currently all C codes (in Gem file) are also installed with <code>gem install</code> (usually under <code>%{geminstdir}/ext</code>). In my recognition these files are used only for creating C libraries and are not needed anymore. So actually some packagers simply remove these files.
Currently all C codes (in Gem file) are also installed with <code>gem install</code> (usually under <code>%{geminstdir}/ext</code>). In my recognition these files are used only for creating C libraries and are not needed anymore. So actually some packagers simply remove these files.
* Note that when also non-Gem type of source archive is provided, when using non-Gem version such C codes are usually not installed.
* Note that when also non-Gem type of source archive is provided, when using non-Gem version such C codes are usually not installed.


However Gem has its own metadata and Gem actually reports that these files should have been installed.
However Gem has its own metadata and Gem actually reports that these files should have been installed.
=== Example ===
* Example
* Currently in rubygem-pam rpm no files are under <code>%{geminstdir}/ext</code> (on i386 this is currently <code>/usr/lib/ruby/gems/1.8/gems/pam-1.5.3/ext</code>).
** Currently in rubygem-pam rpm no files are under <code>%{geminstdir}/ext</code> (on i386 this is currently <code>/usr/lib/ruby/gems/1.8/gems/pam-1.5.3/ext</code>).
* However <code>gem contents pam</code> actually returns some files under this directory.
** However <code>gem contents pam</code> actually returns some files under this directory.


How should we treat these files?
How should we treat these files?
Line 111: Line 143:
* seperate these files into subpackages (e.g. -source subpackage)
* seperate these files into subpackages (e.g. -source subpackage)
* or anything else
* or anything else
=== [SOLVED] To pass CFLAGS correctly ===
I noticed that even for non-Gem Ruby packages, many packages containing C shared libraries
* have <code>extconf.rb</code> in the source archives
* or have <code>Rakefile</code> which create <code>extconf.rb</code>
which usually contain <code>require mkmf</code>. In such case
<pre>
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
</pre>
can be used.
* A package which uses <code>extconf.rb</code> directly can take <code>--with-cflags=</code> argument directly, like <code>ruby extconf.rb --with-cflags="%{optflags}"</code>, however in the case that <code>Rakefile</code> creates <code>extconf.rb</code>, usually <code>rake build</code> does not accept <code>--with-cflags=</code> argument.
So, maybe the part of how to pass <code>CFLAGS</code> correctly in the proposal written above can be replaced by more general guideline like following.
==== Alternative proposal (for both Gem and non-Gem) ====
* For packages which create C shared libraries using <code>extconf.rb</code>
<pre>
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
</pre>
'''should''' be used to pass <code>CFLAGS</code> to <code>Makefile</code> correctly.
This also applies to Ruby Gems.
[[Category:Archived packaging guideline drafts]]

Latest revision as of 07:49, 18 September 2016

Proposal for package Ruby Gem including extension library written in C

Current status

Current packaging guidelines on Fedora about Ruby Gems are mainly written on Packaging/Ruby#Ruby_Gems

The guideline says:

The install should be performed with the command
$ gem install --local --install-dir %{buildroot}%{gemdir} --force %{SOURCE0}

Problem

This causes some problems especially when Ruby Gem contains extension libraries written in C because of the following reason.

A-1

First of all the above procedure will cause the error like following.

+ /usr/lib/rpm/check-buildroot
Binary file /home/tasaka1/rpmbuild/INSTROOT/rubygem-zoom-0.4.1-4.fc9p-root-tasaka1/usr/lib/debug/usr/lib/ruby/site_ruby/1.8/i386-linux/zoom.so.debug matches
Found '/home/tasaka1/rpmbuild/INSTROOT/rubygem-zoom-0.4.1-4.fc9p-root-tasaka1' in installed files; aborting
error: Bad exit status from /home/tasaka1/rpmbuild/INSTROOT/rpm-tmp.84251 (%install)

This is because with above command:

  • gem first unpacks files in the gem file under %{buildroot}%{gemdir}
  • Then gem (actually gcc) compiles C source files installed under there.

So the rebuilt C library may contain %{buildroot} string.

A-2

So to avoid this error some maintainers on Fedora of rubygems including extension written in C simply strip compiled C libraries before %install stage ends.

  • Of course this won't create debuginfo file correctly.
  • Also, when Gem is directly installed under %{buildroot} and %prep is blank, find_debuginfo.sh won't be called anyway because %{buildsubdir} is not defined (this is because debugXXX.list can't be written when %{_builddir}/%{buildsubdir} is missing).

Solution to A

The problem written in A can be resolved when Ruby Gems are installed under %{_builddir} first.

B

Currently there is no mention about how to pass Fedora specific compilation flags correctly.

Solution to B

Usually ruby extension uses ruby script file named extconf.rb which has require 'mkmf'. mkmf.rb module accepts CONFIGURE_ARGS environment to specify CFLAGS so using this is preferred.

  • even for non-Gem Ruby packages, many packages containing C shared libraries
    • have extconf.rb in the source archives
    • or have Rakefile which create extconf.rb

which usually contain require mkmf. In such case

export CONFIGURE_ARGS="--with-cflags='%{optflags}'"

can be used, too. So using this method is preferred even for non-Gem packages.

  • A package which uses extconf.rb directly can take --with-cflags= argument directly, like ruby extconf.rb --with-cflags="%{optflags}", however in the case that Rakefile creates extconf.rb, usually rake build does not accept --with-cflags= argument.

Proposition

  • Add the following sentences at the end of Ruby packages with binary content/shared libraries

For packages which create C shared libraries using extconf.rb

export CONFIGURE_ARGS="--with-cflags='%{optflags}'"

should be used to pass CFLAGS to Makefile correctly. This also applies to Ruby Gems.


  • And add below.

Ruby Gem with extension libraries written in C

When Ruby Gem contains extension libraries written in C,

  • First %{_builddir}/%{buildsubdir} must be created at %prep stage.
  • Then the Ruby Gem must be installed under %{_builddir}/%{buildsubdir} at %build stage to get C libraries compiled under the directory.
  • When using gem command to install Gem file, using -V option is recommend to check if CFLAGS is correctly honored.
  • Finally at %install stage the whole tree under %{_builddir}/%{buildsubdir}%{gemdir} should be copied (not moved) to under %{buildroot}%{gemdir}.
    • When all tree under %{_builddir}/%{buildsubdir} is moved to under %{buildroot}, find_debuginfo.sh will complain that the corresponding source files are missing.
  • Installed C codes (usually under %{geminstdir}/etc) may be removed even if gem contents %{gemname} reports that installed C codes should be found there.

Revised proposition

From the comments I received, it seems that the following proposition is clearer.

(The part CONFIGURE_ARGS=... stays unchanged)

Ruby Gem with extension libraries written in C

When Ruby Gem contains extension libraries written in C,

  • First, %prep stage must contain %setup -q -c -T to create the directory where C libraries are compiled.
  • Then at %build stage the Ruby Gem must be installed under the directory created at %prep stage to get C libraries compiled under there.
  • When gem install is used to install Gem file, using -V option is recommend to check if CFLAGS is correctly honored.
  • Finally at %install stage the whole tree under the directory created at %prep stage should be copied (not moved) to under %{buildroot}%{gemdir}.
    • When all tree under the directory created at %prep stage is moved to under %{buildroot}, find_debuginfo.sh will complain that the corresponding source files are missing.
  • Installed C codes (usually under %{geminstdir}/etc) may be removed even if gem contents %{gemname} reports that installed C codes should be found there.

Note

The current guideline

If the Gem contains binary content (e.g., for a database driver), it must be marked 
as architecture specific, and all architecture specific content must be moved 
from the %{gemdir} to the [#ruby_sitearch %{ruby_sitearch} directory] during %install

must still apply.

Example

%prep
%setup -q -T -c

%build
mkdir -p ./%{gemdir}
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
gem install --local --install-dir ./%{gemdir} -V --force %{SOURCE0}

%install
rm -rf %{buildroot}
mkdir -p %{buildroot}%{gemdir}
cp -a ./%{gemdir}/* %{buildroot}%{gemdir}

mkdir -p %{buildroot}%{ruby_sitearch}
mv %{buildroot}%{geminstdir}/lib/*.so %{buildroot}%{ruby_sitearch}
rm -rf %{buildroot}%{geminstdir}/ext

%clean
rm -rf %{buildroot}

%files
%defattr(-,root,root,-)
%{ruby_sitearch}/*.so
%{geminstdir}/
%{gemdir}/cache/%{gemname}-%{version}.gem
%{gemdir}/specifications/%{gemname}-%{version}.gemspec

Issues

[SOLVED] Installed C codes

Currently all C codes (in Gem file) are also installed with gem install (usually under %{geminstdir}/ext). In my recognition these files are used only for creating C libraries and are not needed anymore. So actually some packagers simply remove these files.

  • Note that when also non-Gem type of source archive is provided, when using non-Gem version such C codes are usually not installed.

However Gem has its own metadata and Gem actually reports that these files should have been installed.

  • Example
    • Currently in rubygem-pam rpm no files are under %{geminstdir}/ext (on i386 this is currently /usr/lib/ruby/gems/1.8/gems/pam-1.5.3/ext).
    • However gem contents pam actually returns some files under this directory.

How should we treat these files?

  • simply don't remove these files
  • ignore gem report in this case and remove these anyway
  • seperate these files into subpackages (e.g. -source subpackage)
  • or anything else

[SOLVED] To pass CFLAGS correctly

I noticed that even for non-Gem Ruby packages, many packages containing C shared libraries

  • have extconf.rb in the source archives
  • or have Rakefile which create extconf.rb

which usually contain require mkmf. In such case

export CONFIGURE_ARGS="--with-cflags='%{optflags}'"

can be used.

  • A package which uses extconf.rb directly can take --with-cflags= argument directly, like ruby extconf.rb --with-cflags="%{optflags}", however in the case that Rakefile creates extconf.rb, usually rake build does not accept --with-cflags= argument.

So, maybe the part of how to pass CFLAGS correctly in the proposal written above can be replaced by more general guideline like following.

Alternative proposal (for both Gem and non-Gem)

  • For packages which create C shared libraries using extconf.rb
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"

should be used to pass CFLAGS to Makefile correctly. This also applies to Ruby Gems.