Blog
Simplifying JS function arguments for recursion with this
by Jim Gay
In a previous post, you read about how to ensure that a deeply nested object structure exists. We wrote a simple function ensure_struct that takes a string, splits it up, and turns it into a nested object.
In the original function it accepted 2 arguments: the string and an optional scope.
We used the optional scope as a way to aid in recursion. The function grabbed the first part of our string and made sure there was an object with that name, then it took the rest of the string and passed that object in as the scope. Down it went through the object structure until there were no more parts of the string left.
The string flamethrower.fuel.fire became the object {flamethrower:{fuel:{fire:{}}}}.
JS is powerful and leaning on the use of the this keyword instead of passing an optional argument makes the function even simpler. Let’s look at the final function:
var ensure_struct = function(attributes) {
if (attributes == '')
return this;
var parts = attributes.split('.');
var first_part = parts.shift();
if (typeof this[first_part] === 'undefined')
this[first_part] = {};
ensure_struct.call(this[first_part], parts.join('.'));
return this;
}
This is mostly the same as last time, but what’s different is that we’re not checking for a value for scope, the optional argument. Last time we had var scope = scope || this; to check if there was a scope given, or just set it to this.
But we can ignore that entirely because the value of this can be set by using either of 2 core functions in JS.
Instead of calling the function and passing the scope, we can set the value of this by using either call or apply.
It’s here where that magic happens:
ensure_struct.call(this[first_part], parts.join('.'));
That line takes the function and uses the call function on it. By using call we can pass an object that we want to be the value of this when the function is run. It’s the same as just running the function ensure_struct(parts.join('.'); but telling it to use a specific value for this (which is the last object created in our function).
While we’re still recursively applying the function to our objects and strings, we no longer need to track an additional variable.
We can do the same thing with apply but the additional arguments we send must be in an array:
ensure_struct.apply(this[first_part], [parts.join('.')]);
Now we can either call our function and set a nested object on our window (what the value of this would be originally) or we can tell the function to apply a specific value to this.
var ensure_struct = function(attributes) {
if (attributes == '')
return this;
var parts = attributes.split('.');
var first_part = parts.shift();
if (typeof this[first_part] === 'undefined')
this[first_part] = {};
ensure_struct.call(this[first_part], parts.join('.'));
return this;
}
var some_object = {};
ensure_struct.call(some_object, 'deeply.nested.structure');
Ruby delegate.rb secrets
by Jim Gay
You’ve seen SimpleDelegator in action and used it a bit yourself. The delegate library is more than just a fancy method_missing wrapper.
Easy Wrappers
First and foremost, SimpleDelegator is a fancy method_missing wrapper. I know I said the library was more than that, just bear with me.
Here’s some sample code:
jim = Person.new # some object
class Displayer < SimpleDelegator
def name_with_location
"#{__getobj__.name} of #{__getobj__.city}"
end
end
displayer = Displayer.new(jim)
puts displayer.name_with_location #=> "Jim of Some City"
That Displayer class initializes with an object and automatically sets it as @delegate_sd_obj. You’ll also get both a __getobj__ and a __setobj__ method to handle the assignment of the @delegate_sd_obj.
You may want to alias those methods so they won’t be so ugly when you use them: alias_method :object, :__getobj__.
Method Missing
Here’s an expanded view of how it handles method_missing:
target = self.__getobj__ # Get the target object
if target.respond_to?(the_missing_method)
target.__send__(the_missing_method, *arguments, &block)
else
super
end
The actual code is a bit more compact than that, but it’s that simple. SimpleDelegator is so simple, in fact, that you can create your own implementation just like this:
class MyWrapper
def initialize(target)
@target = target
end
attr_reader :target
def method_missing(method_name, *args, &block)
target.respond_to?(method_name) ? target.__send__(method_name, *args, &block) : super
end
end
That’s not everything, but if all you need is simple use of method_missing, this is how it works.
SimpleDelegator Methods
SimpleDelegator adds some convenient ways to see what methods are available. For example, if we have our jim object wrapped by displayer, what can we do with it? Well if we call displayer.methods we’ll get back a unique collection of both the object’s and wrapper’s methods.
Here’s what it does:
def methods(all=true)
__getobj__.methods(all) | super
end
It defines the methods method and uses the union method “|” from Array to make a unique collection. The object’s methods are combined with those of the wrapper.
['a','b'] | ['c','b'] #=> ['a','b','c']
The same behavior is implemented for public_methods and protected_methods but not private_methods. Private methods are private, so you probably shouldn’t be accessing those from the outside anyway.
Why does it do this? Don’t we want to know that the main object and the SimpleDelegator object have methods of the same name?
Not really.
From the outside all we care to know is what messages we can send to an object. If both your main object and your wrapper have methods of the same name, the wrapper will intercept the message and handle it. What you choose to do inside your wrapper is up to you, but all these methods lists need to provide is that the wrapper can receive any of those messages.
Handling clone and dup
SimpleDelegator will also prepare clones and dups for your target object.
def initialize_clone(obj) # :nodoc:
self.__setobj__(obj.__getobj__.clone)
end
def initialize_dup(obj) # :nodoc:
self.__setobj__(obj.__getobj__.dup)
end
Read Jon Leighton’s post about initialize_clone, initialize_dup and initialize_copy in Ruby for more details about when those methods are called.
Making your own SimpleDelegator
SimpleDelegator actually inherits almost all of this from Delegator. In fact, the only changes that SimpleDelegator makes is 2 convenience methods.
class SimpleDelegator < Delegator
def __getobj__
@delegate_sd_obj
end
def __setobj__(obj)
raise ArgumentError, "cannot delegate to self" if self.equal?(obj)
@delegate_sd_obj = obj
end
end
Subtracting all the comments around what those methods mean, that’s the entirety of the class definition as it is in the standard library. If you prefer to use your own and call it SuperFantasticDelegator, you only need to make these same getter and setter methods and you’ve got all that you need to replace SimpleDelegator.
Keep in mind, however, that the __setobj__ method has some protection in there against setting the target object to the wrapper itself. You’ll need to do that too unless you want to get stuck in an endless method_missing loop.
Using DelegateClass
The delegate library also provides a method called DelegateClass which returns a new class.
Here’s how you might use it:
class Tempfile < DelegateClass(File)
def initialize(basename, tmpdir=Dir::tmpdir)
@tmpfile = File.open(tmpname, File::RDWR|File::CREAT|File::EXCL, 0600)
super(@tmpfile)
end
# more methods here...
end
This creates a Tempfile class that has all the methods defined on File but it automatically sets up the message forwarding with method_missing.
Inside the DelegateClass method it creates a new class with klass = Class.new(Delegator).
Then it gathers a collection of methods to define on this new class.
methods = superclass.instance_methods
methods -= ::Delegator.public_api
methods -= [:to_s,:inspect,:=~,:!~,:===]
It gets the instance_methods from the superclass and subtracts and methods already in the Delegator.public_api (which is just the public_instance_methods). Then it removes some special string and comparison methods (probably because you’ll want to control these yourself and not have any surprises).
Next it opens up the klass that it created and defines all the leftover methods.
klass.module_eval do
def __getobj__ # :nodoc:
@delegate_dc_obj
end
def __setobj__(obj) # :nodoc:
raise ArgumentError, "cannot delegate to self" if self.equal?(obj)
@delegate_dc_obj = obj
end
methods.each do |method|
define_method(method, Delegator.delegating_block(method))
end
end
The code is sure to define the __getobj__ and __setobj__ methods so that it will behave like SimpleDelegator. Remember, it’s copying methods from Delegator which doesn’t define __getobj__ or __setobj__.
What’s interesting here is that it’s using Delegator.delegating_block(method) to create each of the methods. That delegating_block returns a lambda that is used as the block for the method definition. As it defines each of those methods in the methods collection, it creates a forwarding call to the target object. Here’s the equivalent of what each of those methods will do:
target = self.__getobj__
target.__send__(method_name, *arguments, &block)
For every method that it gathers to define on this new DelegateClass it forwards the message to the target object as defined by __getobj__. Pay close attention to that. Remember that I pointed out how you can make your own SimpleDelegator and create your own getter and setter methods? Well DelegateClass creates methods that expect __getobj__ specifically. So if you want to use DelegateClass but don’t want to use that method explicitly, you’ll need to rely on alias_method to name it something else. All of your automatically defined methods rely on __getobj__.
Lastly, before returning the klass, the public_instance_methods and protected_instance_methods are defined. There’s some interesting things going on in those method definitions, but I’ll keep the explanation simple for now.
This Tempfile class that we created is actually exactly how the standard library’s Tempfile is defined.
If you’re not familiar with it, you can use it like this:
require 'tempfile'
file = Tempfile.new('foo')
# then do whatever you need with a tempfile
If you dive into that library you’ll see:
class Tempfile < DelegateClass(File)
The tempfile library relies on the delegate library, but not in the way that you might find in the wild. Often I see developers using only the SimpleDelegator class, but as you can see there’s a handful of other ways to make use of delegate to handle message forwarding for you.
Ensure nested objects/hashes exist
by Jim Gay
Recently I was working on someone else’s code and found quite a lot of repetition of similar code in their JS files. Here’s an example:
if(typeof(flamethrower) === 'undefined'){ flamethrower = {} };
if(typeof(flamethrower.fuel) === 'undefined'){ flamethrower.fuel = {} };
if(typeof(flamethrower.fuel.fire) === 'undefined'){ flamethrower.fuel.fire = {} };
This sort of pattern kept coming up in file after file.
There are 2 things to learn about the project from this code. The first is that there is obviously a loading problem.
The number of files that followed this pattern was a serious indicator that someone either wasn’t investigating a load order problem, or that the developer was being overly protective of the code in a particular file to avoid errors. So the first step when encountering this is to investigate dependencies and load order.
But sometimes you may actually need to ensure that some nested structure exists. Your load order may need to be necessarily arbitrary or unpredictable.
To DRY up this code we can write a recursive function and get something as simple as:
app.ensure_struct('flamethrower.fuel.fire')
This makes our code much easier to process and understand. In the first example you see so much code that you are required to slow down and read carefully to ensure you haven’t missed anything. A simple typographical error would be very easy to miss and could introduce an unexpected bug.
How can recursion help us here? Let’s take a look at the final function first:
var app = {}
app.ensure_struct = function(atts,scope){
var scope = scope || this;
if (atts == "") {
return scope;
}
var parts = atts.split('.'),
first_attribute = parts.shift();
if (typeof (scope[first_attribute]) === 'undefined') {
scope[first_attribute] = {};
}
app.ensure_struct(parts.join('.'), scope[first_attribute]);
return scope;
}
We’ve defined this on an app object but that can be whatever you want. First, what we care about is some string representing the structure that we need. In our example we used “flamethrower.fuel.fire” so that’s the first argument that we need to handle.
app.ensure_struct = function(atts){
var parts = atts.split('.');
}
This gives us an array of parts to turn into objects. We’ll also need a starter object to add these items and we can begin to check if they exist or set them.
app.ensure_struct = function(atts){
var parts = atts.split('.');
var starter_object = {};
if(typeof(starter_object[parts[0]]) === 'undefined'){
starter_object[parts[0]] = {};
}
}
The problem here is that we have begun by getting the first of the attributes from our array. We still need to address the rest of our attributes. One way to do this would be to loop over the array of parts and add those named objects to the last object we had in hand:
var last_object = starter_object;
var index = 0;
while (index < parts.length) {
if (typeof (last_object[parts[index]]) === 'undefined') {
last_object[parts[index]] = {};
}
last_object = starter_object[parts[index]];
index++;
}
You could instead use a for loop to shrink it a bit:
for(var i = 0, o = starter_object, l = parts.length; i < l; i++){
attribute = parts[i]
if(typeof(o[attribute]) === 'undefined'){
o[attribute] = {};
}
o = o[attribute];
}
As an alternative, we can use recursion to tackle our needs. This helps simplify the code a bit:
var parts = atts.split('.'),
first_attribute = parts.shift();
if (typeof (scope[first_attribute]) === 'undefined') {
scope[first_attribute] = {};
}
app.ensure_struct(parts.join('.'), scope[first_attribute]);
I’m happier with that. Inside our ensure_struct function we reuse what we’ve already written by joining our leftovers and sending a new to the fuction. Of course, to get this to work we need to change the arguments that the function accepts and check that we haven’t sent a blank string in for the attributes. When we hit our last part, the leftover_attributes is an empty array and joining it will return a blank string.
app.ensure_struct = function(atts,scope){
var scope = scope || this;
if (atts == "") {
return scope;
}
var parts = atts.split('.'),
first_attribute = parts.shift();
if (typeof (scope[first_attribute]) === 'undefined') {
scope[first_attribute] = {};
}
app.ensure_struct(parts.join('.'), scope[first_attribute]);
return scope;
}
Now, we can set our scope from the outside if we like.
app.ensure_struct('values.one.two', other_object);
And yes, this comes from an actual project named flamethrower.
Searching through your bundled gems
by Jim Gay
You’ve probably been working on a project where someone added a feature from a gem which caused an error, but you had no idea where to find the offending code.
For me, that happened yesterday. Someone had removed a database table without diving in to see what code actually required the table. Unfortunately, this is a rescue project which barely has tests.
Here’s a quick summary of what you can do:
ag thethingyouneed `bundle show --paths`
I began writing some acceptance tests to cover our asses and found that when I attempted to delete a user, the test blew up. Deleting a user was completely unrelated to what I actually cared about but it coincidentally happened for my test.
The failure was because I was missing a receipts table. The fact that this table was needed was news to me, so I asked other team members to find out what it was.
“Oh, yeah. Have you rebased onto develop? It was removed.”
WTF?! Why? Why would anyone remove a table and not make sure to investigate that no leftover code required it? Unfortunately lazy developers make mistakes.
So I began digging. I knew the requirement for the offending table existed somewhere in our project, but there was no direct reference in our application code.
Bundler must have a way to search through it’s gems, right? Wrong.
I found a few pull requests related to this need and realized that it was much simpler than relying on bundler to do the job for me.
Bundler can provide you with all the load paths that it creates with
bundle show --paths
And nix tools like grep or ack can take a collection of paths to search through. I really like using the_silver_searcher however, so that’s my go-to tool. It’s super fast.
All I needed to do was pass that collection of paths to ag and I had what I needed.
ag receipts `bundle show --paths`
Then I was on my way and found that we were using a gem that provided an acts_as_messagable method that expected a receipts table to exist was used in one of our models and was no longer needed.
There are 3 lessons here.
First, you can compose things with unix tools and I don’t need Bundler to do the entire job for me.
Second, you really should be writing tests. It’s the presence and execution of a test that found this bug for me.
Third, always make sure you properly clean up your project. Consider the impact of your laziness on the rest of the team. An ounce of prevention is worth a pound of cure.
Chubby models are still fat with Concerns. DCI focuses on how things work together
by Jim Gay
You’ve seen complicated codebases. You’ve dug through other people’s code without a clue to what’s going on or how it all fits together. Perhaps it was your own code or perhaps someone else’s but you know how difficult it can be to figure out how this or that aspect of a system implements some required feature.
Often, code organization is a key aspect of aiding communication. Today on the 37signals blog, David Heinemeier Hansson wrote about how ActiveSupport::Concern can really help clean up your code. Rails 4 will even have new concerns directories in the load path by default, making it even easier to use this feature.
He points out that Basecamp has almost 40 concerns that organize features required for multiple models. Giving an example, he says “This concern can then be mixed into all the models that are taggable and you’ll have a single place to update the logic and reason about it.”
What’s great about this is that single place. To understand something thats “Taggable” or “Searchable” you only need to find that concern and you have all that you need. Making changes is just as easy; it’s all right there.
Addressing some of the objections to these fat models, he says:
It’s true that this will lead to a proliferation of methods on some objects, but that has never bothered me. I care about how I interact with my code base through the source. That concerns happen to mix it all together into a big model under the hood is irrelevant to the understanding of the domain model.
I agree with the first part of this. Most of the time an object that has many unrelated methods isn’t much of a concern. But when we need to drop into a debugger and look at what the object is and can do, then we are met with far more than we need. Inspecting the runtime object becomes difficult because of the overwhelming features and responsibilities that it has.
Almost 100% of the time, however, we interact with code through the source. This is spot on. We read and write code. But the fact that concerns happen to mix it all together under the hood is not irrelevant to understanding a domain model. It’s important how many of these mixins are used and how complicated an individual class can become.
But even more important is how it works. Not how things are mixed in, but how your program works. What is going on?
Why do you care that a model is Taggable? What purpose does it serve and what other actor in the system will have a need for it?
Using concerns can be helpful, but if it’s used by many models it may effectively become a fragile base class problem where changes to it have unintended ripple effects through those models that inherit its features.
ActiveSupport::Concerns and Roles in DCI are not the same. Roles in DCI are ONLY defined within a context. Your context represents a use case for your system and is larger than a single procedure. A use case covers success paths, failure paths, and alternate paths to the execution.
I had one reader of Clean Ruby, Drew Ulmer, write to me and say:
As far as putting roles directly into contexts, I can’t tell you how much I love this idea. It’s fantastic to have all of the logic and model behavior for a specific activity in one place and not have to jump around files. It keeps the models focused around how they are defined and not around what we use them for or how they behave; it keeps the behavior focused and easy to reason about; and it defines a very clear API that must be implemented for a context.
I passed this along to my mailing list and after reading it, I got this response from Anthony Burton saying:
To be honest, reading this I kept thinking “that’s just the way things were supposed to be”. Nevertheless, I always get so frustrated trying to dig through a project (new or old, right?), and trying to trace the flow of execution and how things fit together. This made me think of a project I worked on a few years back (which thankfully had a substantial logging capability) where I wrote a little perl script that would parse logs files and show how things were linked together. It was a very popular utility with the other devs and I was always curious, 1) why no one had written something like it before me; and 2) why the code wasn’t just organized differently.
This is a valuable lesson for communicating what your software is supposed to do. You not only need to read the code, but you need to understand how different pieces work together. DCI is a great way to organize code and achieve better communication.
Here’s a dumb example of what your context might look like:
class SwappingDetails
def initialize(source, recipient)
@source, @recipient = source.extend(Source), recipient.extend(Recipent)
end
def swap
# trigger the appropriate methods
end
def other_trigger
# trigger alternate behavior
end
module Source
# some related methods here
end
module Recipent
# some related methods here
end
end
Those modules represent the methods required for an object playing the role. With concerns, you must understand more about the classes of objects ahead of time whereas within a DCI context you’d focus only on roles and their interaction. When using concerns, you understand less about what will trigger the action, what other objects are necessary, and what roles those other objects play.
Concerns can be useful, but just like anything else they can be done poorly. Thinking in terms of interacting roles is not the same as abstracting related behaviors into a module.
Regardless of what you do in your application, communicating it’s purpose is key to managing change. The better others can understand how something works, the easier it will be to make necessary changes.
Permalink… Comments: 1
$15,000 in Income From an eBook, How I Did It
by Jim Gay
Over less than 6 months I’ve made more than $15k in revenue from Clean Ruby. It’s an unfinished book and I’ve been working to polish it up and get new content out to my customers but I wanted to take a step back and review where I am.
Grand Plan Gone Wrong
I already had a product idea, I just needed help getting it going. Or so I thought.
I asked Eric Davis, who wrote Refactoring Redmine what he thought of Amy Hoy’s 30x500 course. For him, it was an absolutely positive experience and he let me know that when you take it once, you can keep taking it.
I was cautious about the cost of the course, but with that reaction I decided it would be a good investment to finally make my CMS hosting service, Solar System a reality.
When the course began, I was super eager. I already had a leg up because I had most of my product put together. I just needed to figure out how to market it.
And the first thing I learned was that my product was dead, would remain dead, and was a waste of further time and effort. I fundamentally had begun in the wrong place and needed to first find what people wanted.
For me, it was like building a shop in the middle of nowhere. I chose a place that seemed like it would be great for a trainload of customers to pass through. After the course however, I knew how to look for existing tracks first and then listen for the train approaching.
How I Chose My Product
I started with this Recurring Revenue Roundup and wish I had built that product first, given some of the problems I had accepting some people’s money in my initial launch.
But I was really distracted from this course by a new idea called DCI. If you’re not a programmer, don’t worry. It’s a new and challenging way of thinking about programming. That’s all you really need to know.
I was trying to build my other product, but was instead constantly, and eagerly, distracted by DCI. I read about it. I wrote notes. I tried out simple ideas in code. I started applying what I learned to my daily work. “Damn it!” I thought. I want to build a product but I can’t focus on that.
This DCI idea is new. There were practically no places it was being discussed. I thought that would preclude me from using it to make a product. Who cares about it? I have no idea, I can’t find them.
I write on my blog from time to time. Not often enough. I planned a blog post because the idea is important and powerful. Someone should be blogging about it.
So I wrote my first post and decided just before I published it that I would add an email sign-up form. http://www.saturnflyer.com/blog/jim/2011/09/28/4-simple-steps-extending-ruby-objects-the-tip-of-the-iceberg-with-dci/
I rarely get comments on my blog. I still don’t get many, but within a few short hours I had 0 comments, but 12 people on my mailing list!
WTF!?
Apparently people will give your their email sooner than they will make a comment. I didn’t quite do customer research the way she teaches, but I went fishing. Little did I know, but I wrote what Amy calls an E-Bomb (for Education Bomb).
Finding the Ears to Listen
The thing about programmers (aside from the fact that some are strangely offended by that term and prefer “developers”) is that they really enjoy technical stuff.
But most people don’t do technical stuff because it’s there and it’s technical. They read, write, and study because it gets them to some greater goal.
That is what Clean Ruby does.
It’s not a book about DCI. It’s a book that will help you better manage your development effort to reflect your business. It’s a book about thinking.
I noticed that despite great developers being great, they often overlook things. There is a lot to consider when developing an application and seemingly small things can fall through the cracks.
I took great inspiration form both Lean Architecture and Clean Code. Both of these books are about thinking too. These aren’t books about patterns or code wankery, they teach you to slow down and do things properly.
Encouragement and Support
I’m thankful to have the support of many people. I’ve been fortunate to have guidance and understanding on programming concepts by Jim Coplien and Trygve Reenskaug. They have been both critical and supportive exactly where I need it. And I’ve been lucky enough to get a deeper understanding of things like delegation by asking David Ungar and Henry Lieberman directly.
I’d asked Alistair Cockburn for his opinion about my book. In an email I mentioned that “I’m often the developer who spends the most time thinking about method, class, or variable names. And I often take the time to go back and clarify when I’ve met a misunderstanding somewhere.”
He replied with this: “I got in trouble at a small co in 1983 cuz I wrote that way in Radio shack TRS-80 assembler. One of the obscure coders said I was the worst programmer he’d ever seen. Wonder what he’s doing with his life these days.”
All this and more is leading me to create a book that will be a long-lasting, helpful product. There are plenty of people working on their own products in the 30x500 group who help keep me focused: Noah Gibbs, Brennan Dunn, Alex Korban and more.
Revenue
With all my research, thinking, experimenting, questioning, and daily work going on I’ve been selling books.
I was very nervous to launch the book. I’ve never written a book before and I was planning to release it before it was done. I actually chose to do that because I’d gotten pressure from anxious people waiting to become customers.
My pre-launch sales have been fantastic; much better than I expected.
The initial spike of sales was a total surprise to me. I began collecting comments from Twitter about my book and was happy to see my book called “long-awaited”.
I was building a product that people wanted. I wasn’t wasting time figuring out how to market something that I thought possibly could be useful like my CMS hosting. My book is actively helping developers take a second look at their applications and think harder about the real business value of what they do.
After a tip from my friend Roy Tomeij (who is also writing a helpful book) I gathered comments and relevant info in a story so I wouldn’t lose that feedback and encouragement.
After less than 6 months, this is how it looks.
But my sales have dipped as my personal life has been turned upside down. I’ve gone through the process of buying a new home, having a tree fall on it hours after we signed the deed, had a daughter born a day after that (my 4th child), found mold in the basement and had it removed, found plumbing problems, electrical problems, and finally more mold.
Despite all that, I feel great. I’m now selling a product that doesn’t require my time for every dollar. Hourly consulting is great, but it requires hours. Building a product completely removes me from needing to provide hours for money. This is a life-changing experience.
I am eager to get back to my research and finish the book and my customers are eager to get it.
The book has helped getting my name out for my consulting work as well. A broader audience now sees that I know how to help them build software in large systems and how I can train them to do the same. What’s great about writing this book is that the small companies and individuals who can’t afford that kind of consulting are able to learn from the book for a fraction of what they charge for an hour’s worth of work.
Buy Clean Ruby and learn to write Ruby applications that better reflect your business. And let me know what you think.
Permalink… Comments: 10