Just a PSA: the number of gems you have installed on your system will impact your rails boot time. Why is this?
When you require a file, rubygems must find the file. If the file isn’t on the load path, it must find it in your installed gems. To do this, it loads all gemspecs and searches for the file. This means all gemspecs on your system are evaluated, so the more gems you have, the longer it takes. You can read more in the rubygems comments.
The second problem is when bundler comes in to the picture. When rubygems loads the gemspec files, it turns them in to Gem::Specification objects and caches them. Sometimes bundler must modify your GEM_HOME and GEM_PATH environment variables. These environment variables tell rubygems where to locate gems on your file system. Unfortunately, if these variables are modified after the specification cache is built, that means the cache must be cleared. Until recently, bundler always cleared the rubygems cache. Since rubygems can load gemspecs before bundler gets activated (or may even while bundler is being activated!), this means it’s not unusual for all gemspecs on your system to be evaluated twice. Earlier today, I submitted a patch that will only clear the cache if the environment variables are modified. Hopefully this will mean fewer gemspec evals in the next version of bundler.
To help reduce boot time, just remember:
- Use gemsets (if you’re on RVM)
- Clean unused gems often
To clean up your gems, just use the command gem cleanup.
Hope that helps!
The most extreme version of this approach would be to leave your system gems empty except for Bundler, and use `bundle install –path vendor/bundle` in each project — that way only gems for the project will be loaded etc.
Cleanup will still leave some gems installed. I have this aliased to gem_purge in my console.
gem list | cut -d” ” -f1 | xargs gem uninstall -aIx
if you’re using bundler, can’t you just do a bundle install –standalone and only require the gems you use ?
@deepfryed no, that won’t really help. Just using the `bundle` bin stub will evaluate every gemspec on your system.
Good point! Thanks.
… creating clean gemsets for my apps!
Or if you’re on OSX:
gem list | cut -f1 -d ” ” | xargs gem uninstall -aIx
bundle clean –force (with bundler 1.1)
Would it make sense for RubyGems or Bundler to cache the mapping of require -> gem path somewhere on disk so it wouldn’t have to scan for them every time?
Really guys, why aren’t ruby, rvm, rubygems, and bundler on speaking terms with each other to straighten this out?
Why is rubygems unaware of Gemfile? Shouldn’t that be part of its domain?
Why do we still need “bundle exec” in front of everything? There was a proposal from RVM to manage that in the presence of a Gemfile, but it was rejected. (I think rbenv has a plugin to do that).
I know I’m just ranting, but this whole gems thing shouldn’t be so clumsy…
Personally I only install Bundler and a few other gems which provide command line utilities I use. The rest is handled with Bundler on a per-project basis.
I recently blogged about my setup here: http://jimeh.me/blog/2011/11/01/my-ruby-development-environment/
Just to play devils advocate:
Folks who work on many different projects often spend more time waiting for “bundle install” than they do waiting for rails to boot.
I blogged about this issue many years ago, and it’s still really difficult to solve. The problem is that stat(2) (and indeed almost every viable discovery method for filesystem paths) actually touches the disk, rather than just looking in the filesystem cache.
This can be avoided / solved a number of ways:
* Run your app in a VM, as then even the stat(2) calls get cached
* Run your app off of a ramdisk (because ramfs is much faster at stat(2) than even Sata-III and a big on-disk-cache)
* Mount your app on an in-process stat-cached filesystem backed fusefs driver (no really, that can work fast, and relieve the need to remember to copy files back over to disk from a ramdisk).
There are some things we can do in RubyGems and Ruby to avoid the mad stat calls, but the best solution without changing the existing semantics will rely on Eric Hodels $LOAD_PATH object proposal in order to be cleanly implemented. Ideally, in order to mainain a reasonable time-space efficiency, we need a lightweight heap datastructure in the ruby stdlib that can represent the filesystem tree that RubyGems has installed, so we can walk that heap instead of smashing the filesystem all the time. This isn’t flawless, but as stated (once you research it) it can be solved quite nicely with the $LOAD_PATH object. The issue with implementing it without, is that the load path is managed by more than just RubyGems (sadly), and this has real practical order limitations.
It’s also worth noting though, that load path objects will be a semantic change that will break some popular libraries out there, so as with so many of these age old problems, we’re in for a fun ride if we want to actually fix the root cause.
As for gemsets, once again, please people, realize that if you’re going to install the same gems again and again and again, you may not actually be saving yourself time.
Ramdisk for OSX: https://gist.github.com/2764761
Gah, third comment..
It’s also worth noting (potentially for future Rubygems patches), that RubyGems doesn’t need whole gemspecs to do it’s general work (for booting apps and such). In fact, on JIT’d rubies, loading specs is quite a lot more expensive than doing some potential alternatives.
Consider for example, a simple data file containing: [name, version, dependencies, load_paths]
Make this datafile in simple data structures (e.g. tuples) and keep parsing cheap. Use a weakref cache and you’ll be reasonably balanced, other than a small amount of GC fragmentation in the early slots (which is a problem now anyway – just dump ObjectSpace and see how much crap gets retained from gemspec loads and the like even after a bunch of GC runs).
Now crunch those indexes into a single file and maybe (if ruby-core adds mmap to stdlib) mmap it, suddenly all but the stat problem is largely solved.
Food for thought.