Tuesday, May 22, 2012

Backgrounded 2.0 Beta

Backgrounded 2.0 beta is now available with some really exciting changes. I've taken this release as an opportunity to critically evaluate the perfect API for invoking background jobs. The new API is not backwards compatible with previous versions, but it is SEXY as HELL!

class User
  # instance method to do some stuff
  def do_stuff
  end

  # class method to do something else
  def self.do_something_else
  end
end

# execute class method in background
User.backgrounded.do_something_else

user = User.new
# execute instance method in background
user.backgrounded.do_stuff

Oh snap...Did you see what happened there? I just ran class and instance methods in the background with no code changes whatsoever! This new API allows for any method on any ruby object to be executed in the background. But that's not all! The new API also simplified passing custom options into the background handlers (which is awesome for resque users).

# execute instance method on a custom resque queue
user.backgrounded(:queue => 'custom').do_stuff

It's glorious...

Extension Gems

One of Backgrounded's core missions is to be a simple and lightweight wrapper around any background processing framework. With this release, I've decided to extract logic for each library into separate gems. This helps simplify the core testsuite and makes it more extensible in the long run to release updates to specific libraries without bumping the core library version.

The backgrounded-resque gem is now available on github.

Bundler config

gem 'backgrounded-resque'

If any other developers are interested in building a backgrounded wrapper, spin up a gem and drop me a line!

after_commit_backgrounded Callback

Using ActiveRecord callbacks to perform background work is an extremely common pattern. Background jobs should only be invoked after the data has been committed to the database to ensure that the background job will have access to the recent changes and backgrounded now makes it even easier to do the "right thing".

The after_commit_backgrounded callback is just like your standard Rails after_commit hook with a few minor upgrades. It executes the referenced method in the background (duh), and it allows for an optional :backgrounded option to customize the options for the Backgrounded::Handler.

class User < ActiveRecord::Base
  # execute :do_something in the background
  after_commit_backgrounded :do_something

  # execute :do_something_else in the background
  # passing custom options to the backgrounded handler
  after_commit_backgrounded :do_something_else, 
    :backgrounded => {:priority => :high}
end

Grab the latest beta version via rubygems and send any feedback my way!

gem install backgrounded --pre

Thursday, May 3, 2012

Bundler Nirvana With Custom Groups

Managing gem dependencies in Rails applications has been an ongoing struggle over the years. Anyone out there still remember vendoring and freezing gems/plugins into their application? Ah yes, the good old days. Thank goodness for bundler. For all of it's warts, it's an exceptional piece of software and has made my life infinitely better.

The basics of bundler are all well and good, but my goal here is to take your bundler-fu to the next level.

The Problem

As applications grow, managing your Gemfile dependencies is an ongoing issue. The number of dependencies grows...and grows...and grows, until you are no longer able to grok what your application needs to get up and running with a simple visual scan.

Enter Custom Bundler Groups!

Inspired by the great article by Iain Hecker, I began exploring more advanced usage of bundler groups. The rails-console-tweaks gem was a step in the right direction, but there's plenty more work to be done!

After launching a few Rails applications from the ground up, I found a number of reusable themes that could be expressed quite well using bundler groups.

At a minimum, a basic Rails application should have a bundle group for each of these areas:

  • :app - gems for front end appservers (ex: rack middleware)
  • :worker - gems for resque backend processing jobs (ex: resque-ensure-connected)
  • :console - gems for firing up a rails console (ex: hirb, awesome_print)
  • :development - gems for day to day development tasks (ex: email_preview, mailcatcher, etc)
  • :test - gems for running testcases (ex: rspec, webmock, etc)
  • :debug - gems for firing up the debugger (ex: pry)
  • :darwin/:linux/etc - gems specific to your particular OS (ex: growl) (might be improved at some point see https://github.com/carlhuda/bundler/issues/663)
  • :ci - gems for running continuous integration by automated build processes (ex: jslint)
  • :ct - gems for continuous testing workflow (ex: guard, autotest, etc)

An example annotated skeleton Gemfile is available in this Gist.

Adding these specialized bundler groups gives your app an instant boost in:

  • Clarity - Each of these self describing groups will help avoid paralysis when you get 100+ gems in your app
  • Speed - I've personally seen a 500ms difference in application startup time for even a simple usecase. Every little bit helps!
  • Safety - Isolating non-prodution gems from your production environment.

Putting the pieces together...

Once you have grouped your gem dependencies, the next step is to ensure that the proper groups are loaded in the proper places.

Rails Application Config

The Rails application.rb config file is used to declare which custom groups are loaded based on the current Rails environment (production, development or test). In our configuration, we want to ensure that the debug gems are loaded into our development and test environments.

Rack Appserver Config

The appserver specific gems are autoloaded using the Rack config.ru configuration file by configuring the standard RAILS_GROUPS environment variable. It's a simple and elegant solution.

Resque Background Job Config

Autoloading gems into your background workers can also easily be done by passing the RAILS_GROUPS config to your resque workers.

$ RAILS_GROUPS=worker QUEUES=* RAILS_ENV=production rake resque:work

Capistrano Deployment Config

It's important to make sure that these development and test gems are excluded from your production environment. It's quite easy to tweak the Rails default capistrano deployment task to exclude additional custom groups

Versioning Best Practices

When building your Gemfile, It's important to lock down all production gems with a STRICT version requirement (ex: "1.0.0"). This prevents a careless developer from running "bundle update" and pulling in gems that are not fully tested. On the other hand, it is highly recommended to use a LOOSE version requirement for development and test gems to easily upgrade to the bleeding edge version (ex: "~> 1.0.0").

A full list of all files referenced in this post can be found in this Gist. Comments and suggestions are always welcome!

Saturday, March 24, 2012

Rails console tweaks gem

Every developer toolbox has a certain set of tweaks for use in the Rails console. There is no shortage of blog posts for customizing your Rails console, however, there is a considerable gap in tooling to simplify the experience. The Rails console tweaks gem is a solution to help you bootstrap a new app and get the best default console experience right out of the box.

Just drop this gem into your Rails application Gemfile and watch the magic happen the next time you open your Rails console!

# Gemfile
gem 'rails-console-tweaks'

SQL Logging

By default, all SQL statements executed in the console will be logged to standard out. This is invaluable when debugging applications in order to inspect what SQL is being executed during an action.

Bundler Console Group

The Rails console tweaks gem autoloads any gems defined in the console bundler group which makes it easy to plug in new fancy console tools as they are released. Creating a separate bundler group has the added benefit that these debugging tools will not be exposed to your standard production environment. Using custom bundler groups is slowly becoming a best practice for grouping related gems for several reasons such as: speed, safety, clarity.

# Gemfile
gem 'rails-console-tweaks'

# any gems in this group are only available
# within the rails console
group :console do
  gem 'wirb'
  gem 'hirb'
  gem 'awesome_print'
end

HIRB/WIRB Integration (optional)

If you use the HIRB or WIRB ruby gems in your application, the Rails console tweaks gem will auto-enable the gems to customize your console output.

The Rails console tweaks gem is 100% opensource and any comments or suggestions are welcome!