I'm going to come right out and say it...
Every ruby background job solution sucks.
I've used a number of the popular libraries on various projects over the past year, and have yet to find one that "feels right". They took an incredibly complex problem and made and even more complex solution. It really shouldn't be this hard. All I want to do is take a unit of work from my model and kick it off into "magical background job land". I don't want to create separate "Worker" or "Job" classes which only exist to call back into my model object. If your background jobs are doing more work than that, you're doing something wrong!
This is the API I want:
user = User.new user.do_stuff_backgrounded
This is the contract I envision for enabling background processing:
class User backgrounded :do_stuff def do_stuff #long running process here end end
I've released a Backgrounded on github at http://github.com/wireframe/backgrounded as a simple and concise API for invoking background jobs. It also provides a thin wrapper around the underlying job handler framework to support pluggable implementations. DelayedJob is supported out of the box, but it's trivial to drop in support for bj, workling, or any other implementation.
Backgrounded also works great in unit tests as well by providing a "synchronous" handler so that your tests don't need to wait for the background work to complete. No need for Thread.sleep!
I'm excited about this API for a number of reasons. The meta programming declaration doubles as documentation and makes it very clear what the developer is intending to do. It also allows for backgrounded internals to declaratively create the backgrounded methods ahead of time and remove the need for ugly "method_missing" handling. Yay for clean stacktraces and accurate listing of methods (including IRB method autocompletion)!
DelayedJob supports a slightly similar *alternative* API (ex: User.send_later(:do_something)), but their primary usecase revolves around creating a secondary Job class to perform your work. I've thought about forking the delayed job project to support my syntax, but for now I'll leave it as a separate project. Let me know if there's any interest in having this merged into the core delayed_job project.
Awesome!
ReplyDeleteI think I will be using this plugin very soon, I never liked the code required to start background processes that other solutions provide. This is excellent!
DelayedJob does support that kind of usage with the handle_asynchronously method.
ReplyDeleteIf the solutions all suck, why are you just slapping new code on top of them?
ReplyDeleteMaybe state that the APIs for the solutions suck, instead of the solutions themselves.
As I stated pretty clearly in the post, the reasons this library exists are because:
ReplyDelete* the API should be clean/concise. the alternative delayed_job API is okay, but not great.
* testability. none of the libraries I've worked with have been particularly good about working in unit tests.
* portability. It's trivial to implent your own worker queue, and delayed_job is included by default just to help users get up and running. It would be trivial for me to extract that into a separate gem and leave the core as platform agnostic.
Cool stuff Ryan!
ReplyDeleteKeep on working on this but please don't couple it too much with some specific job handler (for specific framework). It would be cool if I could use it not only in Rails but also in Merb, Sinatra and so on.
Does it work well with JRuby and it's Ruby threads based on Java threads?
ReplyDeleteI must say, as significantly as I enjoyed reading what you had to say, I couldnt help but lose interest after a while. Its as if you had a fantastic grasp to the subject matter, but you forgot to include your readers. Perhaps you should think about this from a lot more than 1 angle. Or maybe you shouldnt generalise so much. Its better if you think about what others may have to say instead of just going for a gut reaction to the topic. Think about adjusting your personal thought process and giving others who may read this the benefit of the doubt.
ReplyDelete