[ Content | View menu ]

Gemspec: Loading Dependent Gems Based On The User’s System

Mark Mzyk | May 21, 2012

It’s a scenario that shouldn’t be hard. When installing a gem, have that gem load dependent gems based on what state the system is in.

Yet RubyGems provide no mechanism for doing this. You won’t find mention of it on Rubygems.org.

When you create a gem your gemspec is executed at creation time. When the user installs the gem, nothing that could react to the user’s system is run.

With one exception: extensions.

If your gem has an extension component (most commonly C), then it has to be compiled on the user’s system.

This process can be used to make your gem install dependencies or execute other ruby code at install time. I discovered this when looking for a way to install dependent gems on a user’s system based on the ruby version they were running.

The wikibook on RubyGems mentions this process (“How to install different version of gems depending on which version of ruby the installee is using”), but doesn’t do a great job going into detail on how it works or exactly what is needed.

This is a hack, in the sense that we’re using a system to do something other than what it was designed for. Due to this, if you use this process, it will appear to the user that your gem is installing an extension, when in fact it isn’t.

To go along with the explanation that follows, example code can be found on github: gem_dependency_example. You can install the gem gem_dep_example to see the process in action. This is the gem built from the example github repo. The gem will install itself along with a dependent gem based on the ruby version it detects on your system. It will install the gem gem_dep_shine if you have Ruby 1.9 or greater and it will install the gem gem_dep_polish if you have Ruby 1.8.7 (or older).

Let’s step through how this is accomplished. Looking at the code in the github repo, you’ll see in the gem_dep_example folder that it is a very simple gem. There is the standard gemspec and lib folder. There is also the ext, or extension, folder, which you normally don’t see unless a gem is installing an extension.

Look at the gemspec first. Nothing in the gemspec should seem surprising, except for a line near the bottom.

s.extensions = ["ext/mkrf_conf.rb"]

This line tells RubyGems that it has an extension to install and that the extension can be found in the file mkrf_conf.rb in the ext folder. This is a RubyGem convention, where it looks for this file in this location to know that an extension should be installed. In this case, the name of the file also tells RubyGems it is installing a Ruby extension and not a C extension.

Go the the ext folder and open mkrf_conf.rb, the only file there. You’ll notice some boiler plate code you’ll have to include, where you load the RubyGem’s dependency installer. If it doesn’t find this it will fail with an error.

After loading the installer, write your code to do what you want. This will run on the user’s system, so you can inspect the system and take conditional action based on what you find. In this case the code picks which gem to install based on the user’s Ruby version.

begin
  if RUBY_VERSION < "1.9"
    installer.install "gem_dep_polish", ">=0"
  else 
    installer.install "gem_dep_shine", ">=0"
  end

  rescue
    #Exit with a non-zero value to let rubygems know something went wrong
    exit(1)
end  

The extension file needs to end with these lines:

f = File.open(File.join(File.dirname(__FILE__), "Rakefile"), "w")
f.write("task :default\n")
f.close

The reason for this goes back to this being a hack. RubyGems thinks it is installing a native extension. To finish installing the extension, it will look for the make or rake file that it assumes was generated and run that to complete the installation. In this case, the relevant code has already run, but RubyGems will error out unless it finds a rake file to run. To make RubyGems happy, we create an empty rake file with an empty default task for RubyGems to run so it will exit normally.

Since it isn’t entirely clear to the user installing the gem what is happening, I recommend you avoid using this trick unless you have no other choice. I wish RubyGems included a more intuitive way of achieving this, but it doesn’t, so this will do for now.

A final note: you shouldn’t use this if you’re looking to install different gems based on the user’s platform (Windows, JRuby, Ruby, etc). In this case the accepted best practice is to create different gems for each platform and leave it to the user to install the appropriate one.

Filed in: Programming.

6 Comments

  1. Comment by Patrick Neve:

    Unfortunately I’ve been requested to do just what is not recommended at the end of the post. Am getting a rake failed: error:


    Building native extensions. This could take a while…
    Exception `Gem::InstallError’ at C:/Ruby187/lib/ruby/site_ruby/1.8/rubygems/ext/builder.rb:51 – rake failed:

    C:/Ruby187/bin/ruby.exe mkrf_conf.rb
    i386-mingw32

    mkrf_conf.rb:

    require ‘rubygems/dependency_installer.rb’
    installer = Gem::DependencyInstaller.new
    begin
    if !!((RUBY_PLATFORM =~ /(win|w)(32|64)$/) || (RUBY_PLATFORM=~ /mswin|mingw/))
    puts “#{RUBY_PLATFORM}”
    installer.install “watirloo”, “>=0″
    installer.install “win32-process”, “>=0″
    installer.install “win32ole”, “>=0″
    elsif RUBY_PLATFORM =~ /darwin/i
    puts “#{RUBY_PLATFORM}”
    installer.install “safariwatir”, “>=0″
    #installer.install “appscript”, “>=0″
    end
    rescue
    exit(1)
    end
    f = File.open(File.join(File.dirname(__FILE__), “Rakefile”), “w”)
    f.write(“task :default\n”)
    f.close

    awetestlib.gemspec file:

    require “lib/version”

    Gem::Specification.new do |s|

    s.name = %q{awetestlib}
    s.version = Awetestlib::VERSION
    s.date = Awetestlib::VERSION_DATE
    s.platform = Gem::Platform::RUBY

    s.required_rubygems_version = Gem::Requirement.new(“>= 0″) if s.respond_to? :required_rubygems_version=
    s.authors = [“Anthony Woo”, “Patrick Neve”]
    s.email = %q{patrick@3qilabs.com}
    s.summary = %q{Awetest DSL for automated testing of browser-based applications.}
    s.homepage = %q{http://3qilabs.com}
    s.description = %q{Awetest DSL for automated testing of browser-based applications.}

    s.add_dependency(‘watir-webdriver’)
    s.add_dependency(‘watir’)
    s.add_dependency(‘activesupport’)
    s.add_dependency(‘andand’)

    s.require_paths = [“lib”,”ext”]
    s.files = `git ls-files`.split(“\n”)
    s.executables = `git ls-files — bin/*`.split(“\n”).map{ |f| File.basename(f) }

    s.extensions = [“ext\\mkrf_conf.rb”]
    end

    Anything obvious (other than we really shouldn’t do this?)
    Thanks!!

    June 13, 2012 @ 19:01
  2. Comment by Mark Mzyk:

    Patrick,

    Nothing obvious jumps out at me from the code you pasted. I would suggest a couple of things to try and figure out what is going on:

    1) Run rubygems in as verbose a mode as you can possibly get. I haven’t done a lot of looking, but I know it has verbose options for various commands it offers, and based off the source of rubygems builder.rb I can see it looks for a verbose flag. This might tell you more information.

    2) I think your code might be falling into the rescue block for some reason. Try catching and printing out the error there before issuing the exit(1).

    3) Don’t use this, even though I told you about it (which you obviously know). It looks like you’re trying to install gems based on if the user is running windows or not. I’m sure you read in my blog that the accepted practice is just to cut two gems. I would recommend that way first.

    If is also possible that rubygems has been updated since I wrote my code and this is somehow no longer allowed. I don’t think this is the case, but it is always a possibility.

    Hope that helps. If you figure out the error, I’d be curious to know what it was.

    – Mark

    June 13, 2012 @ 20:40
  3. Comment by Benjamin:

    Why can’t you just put ruby code in your gemspec like in metric_fu https://github.com/metricfu/metric_fu/blob/master/metric_fu.gemspec ?

    January 2, 2013 @ 01:01
  4. Comment by Mark Mzyk:

    @Benjamin

    You can put Ruby in your gemspec and that is absolute the right way to go, most of the time. The problem is that the Ruby in the gemspec is executed at gem creation time, not gem install time, so there is no way to branch execution based on the user’s system. That’s what my post was detailing how to do. Most of the time you don’t want to do this, or you achieve the same result by having the user download the appropriate gem, so the user does the branching before execution time.

    Of course, it’s possible something with gems has changed since I wrote this post, as I haven’t checked recently. If that is the case, please feel free to set me straight.

    – Mark

    January 2, 2013 @ 10:36
  5. Comment by Benjamin:

    Hmm.. I was wondering if that was happening. So, I probably should publish a 1.8 version and a 1.9 version that I build from 1.8 and 1.9 respectively… `gem.name = “metric_fu#{(RUBY_VERSION.to_f*10.0).to_i}”` conversely, if people install the gem via a bundler Gemfile as `gem ‘metric_fu’, :git => ‘git://github.com/metricfu/metric_fu.git’` they should always get the correct platform, no?

    January 3, 2013 @ 17:27
  6. Comment by Mark Mzyk:

    @Benjamin

    Yes on the publishing different versions for each platform. My understanding is that is the accepted best practice, assuming you need a different version for 1.8 vs. 1.9 (I haven’t looked at your code).

    As for bundler installing the correct gem, I have no idea. It’s not something I’ve looked into and I haven’t had to deal with that issue. If you figure out what the behavior is, leave a comment letting me know.

    Thanks.

    January 4, 2013 @ 11:25