When I first began implementing background jobs I found myself moving my code into an appropriate background class whether it was in
lib somewhere. I found it frustrating that I needed to shift code around based only on the decision to run it in the background. It seemed to be the conventional wisdom to do this, or at least that's what I thought.
It's not uncommon to find a reason to move some of your application into the background.
We build up systems that do a lot of work and at a certain point find that the work to be done takes too long. This often means that we reach for a new background job class, move our code to it, stick it in a place like
app/jobs and then we're done with it.
Our code now lives in the background jobs and we move along to the next feature. So we separate
But this ends up feeling like I have two applications. I have my user facing application where most of my work happens and the one where things must happen in the background. But this is mostly a false dichotomy. That's not really how it works nor how any of us think about the system.
While it is important that background processing happen when necessary, I'd rather make those decisions as I determine them. I'd rather those descisions not require I rearrange my code like that.
Make decisions about background processing without rearranging your code
Here's an example of being able to make those decisions as you determine them. Developers using Rails may be familiar with this easy to change aspect of ActionMailer:
Notifier.welcome(some_user).deliver_now # or with a minor change to the code... Notifier.welcome(some_user).deliver_later
This feature is built-in and ready to use. Few things are easier to change than altering now
We can do the same in our code too.
To be able to handle this same approach, we'd need to create objects which can be initialized with data from your background data store.
A specialized class to write our object data to the background data store can do the job well, and it's pretty easy to do.
Here's a quick example.
Let's say you need to run some process.
class SomeProcess def initialize(some_id) @object = User.find(some_id) end def call # perform some long-running action (use your imagination) end end process = SomeProcess.new('ee6f1d66-b4e5-11e6-80f5-76304dec7eb7') process.call
But you've implemented a long running process that runs as soon as some user requests it.
If you follow the example from ActionMailer, maybe could just change
I want to have the ability to make decisions as easy as this when I need to run the
Process code. When you keep all related code together, it’s easier to understand and make changes.
To make this work, we'll need that class to have a
call_later method. We might have other methods we want to be able to run later too. Implementing
method_missing can make this work...
class SomeProcess def method_missing(method_name, *args, &block) if method_name.to_s =~ /_later\z/ # run it in the background else super # do the normal thing ruby does end end end
The above implementation of
method_missing will catch any methods ending in
_later, but I'd rather not do that.
method_missing hides the implementation of our hook into the world of background jobs. It's hard to know that it is there and will probably difficult to find later when you want to understand how it works.
Instead, I'm going to write some code so that I can run my code in the backgound by using
later(:call) instead of
call. It might not be as elegant as appending
_later to a method name, but the implementation easier is to get going and will put the code in a place where you can more easily find it.
Saving for later
So here's where we'll start:
class SomeProcess def later(trigger_method) # ...now what? end end
We've made a
later method that will accept a single argument of some method that we want to run.
That's easy enough but now we need to actually save this to the background. Usually this involves referring to some background job class and saving it.
We need to create a class that will save our object information to the background data store, and then later initialize the object and run our
Writing the data to your background store will be handled by whatever library you use for managing background jobs. This example will use Que but the differences with yours won't matter much.
Our backgroud class needs to initialize the
SomeProcess object and tell it to run the
There's a trick to doing this. Our backgroud class needs to know what attributes are required to initialize the object, and which one to use as the method we're calling.
First, let's make a minor change to our initializer to store the argument we're given:
class SomeProcess def initialize(some_id) @some_id = some_id # <-- keeping track of the argument @object = User.find(some_id) end attr_reader :some_id end
This change allows us to reference the argument we're given so that we can use it when we enqueue our background job. Rails allows us to pass an object into an ActiveJob instance (which we're not using here) and make it's best guess about how to serialize and deserialize the data to initialize our objects. Given our simple example here, we don't really need that feature (but we could implement the same if we like).
We really only need the class loaded by the background process to be a thin mediator between the data in the job store and the class which defines our business process. So I just make the class as small and isolated as possible.
class SomeProcess class Later < Que::Job def run(*args) options = args.pop # get the hash passed to enqueue ::SomeProcess.new(args).send(options['trigger_method']) end end def later(which_method) Later.enqueue(some_id, 'trigger_method' => which_method) end end
There's no need to make a new thing inside of
app/jobs (or anywhere else) since we never directly access this
If my main purpose in the code is to initialize
Process and use the
call method. The only decision I need to make is to either run it immediately, or run it later.
Once I began organizing my code with small and focused background classes, I was able to push aside the concern of when it would run until I needed to make that decision. Reading my code left my head clearer when I kept it all together.
With this code, I can make that decision as I determine the need for it.
Here's the final code for the class:
class SomeProcess class Later < Que::Job def run(*args) options = args.pop # get the hash passed to enqueue ::SomeProcess.new(args).send(options['trigger_method']) end end def initialize(some_id) @some_id = some_id @object = User.find(some_id) end attr_reader :some_id def later(which_method) Later.enqueue(some_id, 'trigger_method' => which_method) end def call # perform some long-running action end end
There are some other features we could add to this, but this small class and
later method get us where we want to be.
Now our decision to run this immediately or at a later time is as simple as changing this:
Are the objects in your system able to change like this? How do you handle decisions to move actions into the background?