Ruby Forwardable deep dive

The Forwardable library is one of my favorite tools from Ruby's standard library both for simplifying my own code and learning how to make simple libraries.

I find that the best way to understand how code works is to first understand why it exists and how you use it. In a previous article I wrote about the value of using Forwardable. It takes code like this:

def street
      address.street
    end

    def city
      address.city
    end

    def state
      address.state
    end

And makes it as short as this:

delegate [:street, :city, :state] => :address

Shrinking our code without losing behavior is a great feature which Forwardable provides. So how does it work?

Modules and their context

Forwardable is a module, which can be used to add behavior to an object. Most of the of modules I see tend to be used like this:

class Person
      include SuperSpecial
    end

But Forwardable is different and is designed to be used with the extend method.

require 'forwardable'
    class Person
      extend Forwardable
    end

Using extend includes the module into the singleton_class of the current object. There's a bit more to it than that, but here's a simple model to keep in mind: use include in your class to add instance methods; use extend in your class to add class methods.

Now that we have that out of the way, to use Forwardable, use extend.

Defining forwarding rules

My most often used feature of Forwardable is the one you saw above: delegate. It accepts a hash where the keys can be symbol or string method names, or an array of symbols and strings. The values provided are accessors for the object to which you'll be forwarding the method names.

class Person
      extend Forwardable

      delegate [:message_to_forward, :another_method_name] => :object_to_receive_message,
                :single_method => :other_object
    end

Other shortcuts

Forwardable provides a few methods, and most commonly you'll see their shortened versions: delegate, def_delegator, and def_delegators. These are actually alias methods of the originals.

alias delegate instance_delegate
    alias def_delegators def_instance_delegators
    alias def_delegator def_instance_delegator

The delegate method we reviewed above is a bit of a shortcut for similar behavior that other methods provide in Forwardable.

The def_delegators method accepts multiple arguments but it's sometimes hard for me to remember that one argument in particular is important. The first argument is the reference to the related object, the next arguments are used to create methods to forward.

class SpecialCollection
      extend Forwardable

      def_delegators :@collection, :clear, :first, :push, :shift, :size
      # The above is equivalent to:
      delegate [:clear, :first, :push, :shift, :size] => :@collection
    end

As you can see, with delegate there's a visual separation between the accessor and the list of methods.

There's more of a difference between delegate and def_delegators too.

def instance_delegate(hash) # aliased as delegate
      hash.each{ |methods, accessor|
        methods = [methods] unless methods.respond_to?(:each)
        methods.each{ |method|
          def_instance_delegator(accessor, method)
        }
      }
    end

Here the code loops through the hash argument changing the keys into arrays of methods if they aren't already arrays, and then calls the def_instance_delegator method for each item in the array. Here's what def_instance_delegators looks like. Note that this is the plural version:

def def_instance_delegators(accessor, *methods) # aliased as def_delegators
      methods.delete("__send__")
      methods.delete("__id__")
      for method in methods
        def_instance_delegator(accessor, method)
      end
    end

This method speficially restricts the use of __send__ and __id__ in forwarded messages. These methods are particularly important in communicating with the forwarding object and determining its identity. If you only used delegate and (for some strange reason) you specify either of __send__ or __id__ then those methods will pass right through. That might do exactly what you want or it might introduce some buggy behavior. This is mostly easy to avoid since you'll likely specify all the methods you need.

The different behavior is important to know, however, if you want to do a blanket forward for all methods from another class of objects:

class SpecialCollection
      extend Forwardable

      def_delegators :@collection, *Array.instance_methods
      # The above is equivalent to:
      delegate [*Array.instance_methods] => :@collection
    end

If you do that, you'll likely see warnings from Ruby like this:

warning: redefining `__send__' may cause serious problems

Don't say Ruby didn't warn you!

But def_delegators is a plural version of def_delegator which provides more options than the two we've been reviewing.

class SpecialCollection
      extend Forwardable

      def_delegator :@collection, :clear, :remove
      def_delegator :@collection, :first
    end

The method def_delegator accepts only three arguments. The first is the accessor for the related object (which will receive the forwarded message) and the second is the name of the message to be sent to the related object. The third argument is the name of the method to be created on the current class and is optional; if you don't specify it then the second argument will be used.

Here's what the above def_delegator configurations would look like if you wrote out the feature yourself:

class SpecialCollection
      extend Forwardable

      # def_delegator :@collection, :clear, :remove
      def remove
        @collection.clear
      end

      # def_delegator :@collection, :first
      def first
        @collection.first
      end
    end

You can see how the optional third argument is used as the name of the method on your class (e.g. remove instead of clear).

How the methods are created

We looked at how Forwardable adds class methods to your class. Let's look at the most important one:

def def_instance_delegator(accessor, method, ali = method)
      line_no = __LINE__; str = %{
        def #{ali}(*args, &block)
          begin
            #{accessor}.__send__(:#{method}, *args, &block)
          rescue Exception
            $@.delete_if{|s| Forwardable::FILE_REGEXP =~ s} unless Forwardable::debug
            ::Kernel::raise
          end
        end
      }
      # If it's not a class or module, it's an instance
      begin
        module_eval(str, __FILE__, line_no)
      rescue
        instance_eval(str, __FILE__, line_no)
      end
    end

It looks like a lot, and it is, but let's strip it down to it's simplest form rather than review everything at once. Here's a simpler version:

def def_instance_delegator(accessor, method, ali = method)
      str = %{
        def #{ali}(*args, &block)
          #{accessor}.__send__(:#{method}, *args, &block)
        end
      }
      module_eval(str, __FILE__, __LINE__)
    end

Remembering, of course, that def_instance_delegator is aliased as def_delegator we can see that a string is created which represents what the method definition will be and saved to the str variable. Then then that variable is passed into module_eval.

It's good to know that module_eval is the same as class_eval because I know I often see class_eval but rarely see the other. Regardless, class_eval is merely an alias for module_eval.

The string for the generated method is used by module_eval to create the actual instance method. It evaluates the string and turns it into Ruby code.

Taking this command def_delegator :@collection, :clear, :remove here's what string will be generated:

%{
      def remove(*args, &block)
        @collection.__send__(:clear, *args, &block)
      end
    }

Now it's a bit clearer what's going to be created.

If you're not familiar with __send__, know that it's also aliased as send. If you need to use the send method to match your domain language, you can use it and rely on __send__ for the original behavior. Here, the Forwardable code is cautiously avoiding any clashes with your domain language just in case you do use "send" as a behavior for some object in your system.

Maybe you're scratching your head about what either of those methods are at all. What the heck is send anyway!?

The simplest way to describe it is to show it. This @collection.__send__(:clear, *args, &block) is equivalent to:

@collection.clear(*args, &block)

All Ruby objects accept messages via the __send__ method. It just so happens that you can use the dot notation to send messages too. For any method in your object, you could pass it's name as a string or symbol to __send__ and it would work the same.

It's important to note that using __send__ or send will run private methods as well. If the clear method on @collection is marked as private, the use of __send__ will circumvent that.

The methods defined by Forwardable will accept any arguments as specified by *args. And each method may optionally accept a block as referred to in &block.

It's likely that the acceptance of any arguments and block will not affect your use of the forwarding method, but it's good to know. If you send more arguments than the receiving method accepts, your forwarding method will happily pass them along and your receiving method will raise an ArgumentError.

Managing errors

Forwardable maintains a regular expression that it uses to strip out references to itself in error messages. 

FILE_REGEXP = %r"#{Regexp.quote(__FILE__)}"

This creates a regular expression where the current file path as specified by __FILE__ is escaped for characters which might interfere with a regular expression.

That seems a bit useless by itself, but remembering the original implementation of def_instance_delegator we'll see how it's used:

str = %{
      def #{ali}(*args, &block)
        begin
          #{accessor}.__send__(:#{method}, *args, &block)
        rescue Exception
          $@.delete_if{|s| Forwardable::FILE_REGEXP =~ s} unless Forwardable::debug
          ::Kernel::raise
        end
      end
    }

This code recues any exceptions from the forwarded message and removes references to the Forwardable file.

The $@ or "dollar-at" global variable in Ruby refers to the backtrace for the last exception raised. A backtrace is an array of filenames plus their relevant line numbers and other reference information. Forwardable defines these forwarding methods to remove any lines which mention Forwardable itself. When your receive an error, you'll want the error to point to your code, and not the code from the library which generated it.

Looking at this implementation we can also see a reference to Forwardable::debug which when set to a truthy value will not remove the Forwardable lines from the backtrace. Just use Forwardable.debug = true if you run into trouble and want to see the full errors. I've never needed that myself, but at least it's there.

The next thing to do, of course, is to re-raise the cleaned up backtrace. Again Forwardable will be careful to avoid any overrides you may have defined for a method named raise and explicitly uses ::Kernel::raise.

The double colon preceding Kernel tells the Ruby interpreter to search from the top-level namespace for Kernel. That means that if, for some crazy reason, you've defined a Kernel underneath some other module name (such as MyApp::Kernel) then Forwardable will use the standard behavior for raise as defined in Ruby's Kernel and not yours. That makes for predictable behavior.

Applying the generated methods

After creating the strings for the forwarding methods, Forwardable will attempt to use module_eval to define the methods. 

# If it's not a class or module, it's an instance
    begin
      module_eval(str, __FILE__, line_no)
    rescue
      instance_eval(str, __FILE__, line_no)
    end

If the use of module_eval raises an error, then it will fallback to instance_eval.

I've yet to find a place where I've needed this instance eval feature, but it's good to know about. What this means is that not only can you extend a class or module with Forwardable, but you can extend an individual object with it too.

object = Object.new
    object.extend(Forwardable)
    object.def_delegator ...

This code works, depending of course on what you put in your def_delegator call.

Forwarding at the class or module level

All these shortcuts for defining methods are great, but they only work for instances of objects.

Fortunately forwardable.rb also provides SingleForwardable, specifically designed for use with modules (classes are modules too).

class Person
      extend Forwardable
      extend SingleForwardable
    end

In the above sample you can see that Person is extended with both Forwardable and SingleForwardable. This means that this class can use shortcuts for forwarding methods for both instances and the class itself.

The reason this library defines those longform methods like def_instance_delegator instead of just def_delegator is for a scenario like this. If you wanted to use def_delegator and those methods were not aliased, you'd need to choose only one part of this library.

class Person
      extend Forwardable
      extend SingleForwardable

      single_delegate [:store_exception] => :ExceptionTracker
      instance_delegate [:street, :city, :state] => :address
    end

As you can probably guess from the above code, the names of each library's methods matter.

alias delegate single_delegate
    alias def_delegators def_single_delegators
    alias def_delegator def_single_delegator

If you use both Forwardable and SingleForwardable, you'll want to avoid the shortened versions like delegate and be more specific by using instance_delegate for Forwardable, or single_delegate for SingleForwardable.

If you liked this article, please join my mailing list at http://clean-ruby.com or pick up the book today!