Note: This is not about Python’s language feature called decorators (with the @ symbol), but about the design patterns called “decorator” and “delegate.”
In this series we’re going to learn two very useful design patterns: the delegation pattern and the decorator pattern. Each pattern is useful in that they help us enforce the single responsibility principle of object oriented programming. Getting these patterns to work as expected in Python requires a relatively deep dive into Python syntax as well as the always scary technique of metaprogramming. Let’s jump in!
Motivation
I had the idea for this article while working on LuluTest, my open source browser automation framework. One of my classes had several methods that shared identical lines either before or after the method fired, but the methods themselves did not have identical signatures or responsibilities, so there was no opportunity to abstract away the reused code using traditional inheritance or composition.
There is actually a pretty easy way to do this in Ruby, and I thought the same would be true for Python. This turned out not the case though and it took some work to figure out how to copy Ruby’s delegate feature in Python. Check out this file to see how I implemented this workaround. To save readers the trouble of figuring it out on their own, and for the added bonus of learning more about OOP, I decided to write this article.
What is Delegation?
According to Wikipedia, “the delegation pattern is an object-oriented design pattern that allows object composition to achieve the same code reuse as inheritance.” If you read that sentence four or five times you might get it but let me try to make it simpler.
Let’s say we have a Dog class that is a subclass (and thus inherits the functionality of) an Animal class. If Animal has a method called get_number_of_legs, any instantiation of the Dog class can call the get_number_of_legs method. In Python, an implementation might look like this:
class Animal:
def __init__(self, name, num_of_legs):
self.name = name
self.num_of_legs = num_of_legs
def get_number_of_legs(self):
print(f"I have {self.num_of_legs} legs")
class Dog(Animal):
def __init__(self, name, num_of_legs):
super().__init__(name, num_of_legs)
dog = Dog('Fido', 4)
dog.get_number_of_legs()
# Outputs "I have 4 legs"It would be technically incorrect to say that Dog delegates get_number_of_legs to Animal because the Dog class actually has that method since it inherits the Animal class. This is what the Wikipedia definition is talking about when it refers to “code reuse.” This is what delegation will duplicate when we use composition.
Kitchens are Compositions
Let’s do a pseudocode example to understand delegation before we dig into a Python implementation. For this example, let’s think of a class called Kitchen. In real life, kitchens–as rooms–do not have any functionality, but we think of a kitchen’s functionality as a composition of the appliances it has. If my kitchen has a microwave, I can heat things up in my kitchen; likewise, with a dishwasher, I can wash dishes in my kitchen.
Now let’s think about this from an object oriented design position. I want to abstract away the appliance implementations and just think of heating up food and washing dishes as kitchen functions. In other words, I want to write my code like this:
# Pseudocode
kitchen = new Kitchen();
kitchen.heat_up_food();
# Food is being microwaved
kitchen.wash_dishes();
# Dishwasher startingIn order to do this, in my Kitchen class definition, I probably have Microwave and Dishwasher classes as attributes of the Kitchen class, allowing the Kitchen class access to their methods. This is essentially what composition is. Now, let’s start really implementing this in Python so we can understand the rest of that Wikipedia definition.
Pythonic Kitchen
Ok, let’s talk about how this kitchen would look in Python code. I was thinking something like this:
class Microwave:
def __init__(self):
pass
def heat_up_food(self):
print("Food is being microwaved")
class Dishwasher:
def __init__(self):
pass
def wash_dishes(self):
print("Dishwasher starting")
class Kitchen:
def __init__(self):
self.microwave = Microwave()
self.dishwasher = Dishwasher()Just like we discussed, the Microwave and Dishwasher classes are attributes of the Kitchen class. Now our kitchen has access to their methods, right? Well, let’s see it in the console:
>>> from kitchen import Kitchen
>>> kitchen = Kitchen()
>>> kitchen.heat_up_food()
Traceback (most recent call last):
File ".\kitchen.py", line 21, in <module>
kitchen.heat_up_food()
AttributeError: 'Kitchen' object has no attribute 'heat_up_food'
Why doesn’t that work? Well, if we want to use the Kitchen‘s Microwave, we have to refer to it as kitchen.microwave.heat_up_food(). This is not the kind of code reuse the delegation definition is talking about. So how do we get kitchen.heat_up_food() to give us what we want?
Implementing the Delegation
The most obvious way would be to use inheritance. If we implement the Kitchen class as inheriting Microwave and Dishwasher, we can use kitchen.heat_up_food() with no syntactical problems.
However, this solution would be very poor design. Inheritance is typically saved for classes that have “is a” relationships. Earlier we created a Dog class that inherited from an Animal class because a Dog is a Animal. But a Kitchen is not a Microwave or Dishwasher, so inheritance here would not make sense.
This is why we have to delegate. One simple way of delegation would be to just create wrapper methods in the Kitchen class like so:
# ... Microwave and Dishwasher truncated
class Kitchen:
def __init__(self):
self.microwave = Microwave()
self.dishwasher = Dishwasher()
def heat_up_food(self):
self.microwave.heat_up_food()
def wash_dishes(self):
self.dishwasher.wash_dishes()Now, the same thing we did in the Python console earlier works as we want:
>>> from kitchen import Kitchen
>>> kitchen = Kitchen()
>>> kitchen.heat_up_food()
Food is being microwaved
>>> kitchen.wash_dishes()
Dishwasher startingNow, when we call heat_up_food on the Kitchen class, we are actually delegating it to the Microwave class. We are getting closer to that “code reuse” for this composite class, but we’re not quite there yet.
The reason we’re “not there” yet is because we haven’t really reused the Microwave and Dishwasher methods, we just wrapped them in identically named functions in the Kitchen class. And really, this is fine if your composite class is only going to have one or two methods to delegate. But what if we want to extend the Microwave and Dishwasher classes with more methods? Now, every time we write a method in those classes, we have to go write another wrapper in the Kitchen class to delegate those functions. This is not code reuse, it’s duplication.
So what are our options? Well, I’m afraid we might have to start using metaprogramming to get what we want.
__getattr__, getattr, and dir
We are going to use three builtin methods that don’t get a lot of use outside of general Python wizarding. Let’s talk about __getattr__, getattr, and dir.
__getattr__ is a function that all Python classes have. This method is called by default anytime an unknown or non-existent attribute is called on a class. For example, earlier when we called kitchen.heat_up_food() before we delegated the method, it was the builtin __getattr__ method which raised that AttributeError.
getattr is a builtin Python function that takes as input a class instance and attribute, and returns the designated class’s designated attribute. We can even execute class functions by appending a () at the end. Let’s go back to the Dog class we created earlier for an example:
>>> from animal import Dog
>>> dog = Dog('fido', 4)
>>> getattr(dog, 'name')
'fido'
>>> getattr(dog, 'get_number_of_legs')()
I have 4 legsFinally, dir is a function that returns an array of strings for all the method names in a class instantiation. Check it out on Dishwasher:
>>> dishwasher = Dishwasher()
>>> dir(dishwasher)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'wash_dishes']We will combine these three methods in the Kitchen class to delegate all Dishwasher and Microwave functions.
Delegating the Kitchen
So let’s think about what we know. The dir method gives us a list available methods inside of a class (the ones starting with and underscore are Python built-ins that we can ignore), and the __getattr__ method is called when a non-existent attribute is called on a class.
What we should do, then, is overwrite the Kitchen‘s __getattr__ method and check and see if the non-existent attribute exists in either its Dishwasher or Microwave attributes. If it is, we can use the getattr method to call the requested method on the appropriate class. Here’s how we’ll do it:
# Microwave and Dishwasher are the same as earlier
class Kitchen:
def __init__(self):
self.microwave = Microwave()
self.dishwasher = Dishwasher()
self.microwave_methods = [f for f in dir(Microwave) if not f.startswith('_')]
self.dishwasher_methods = [f for f in dir(Dishwasher) if not f.startswith('_')]
def __getattr__(self, func):
def method(*args):
if func in self.microwave_methods:
return getattr(self.microwave, func)(*args)
elif func in self.dishwasher_methods:
return getattr(self.dishwasher, func)(*args)
else:
raise AttributeError
return method
What’s going on here? Well, in the Kitchen‘s constructor, we not only set the microwave and dishwasher attributes, we also set an array of available methods from within those classes (we ignore the methods that start with an underscore because those are Ptyhon built-in methods and we don’t want to hijack those).
Then, we overwrite Kitchen‘s __getattr__ method by defining another method within it. Since this method will be called anytime an attribute that doesn’t exist in Kitchen is called, this is the ideal place to do our delegation. We catch the called method in the func argument, then check if it is within the list of available methods in Microwave or Dishwasher. If it is, we call that function with (*args), which represents any arguments the non-existent function might have been called with (though in our current case, this is irrelevant).
This has probably been confusing so let’s see it in action and talk about what is actually going on. In your Python console:
>>> from kitchen import *
>>> kitchen = Kitchen()
>>> kitchen.heat_up_food()
Food is being microwaved
>>> kitchen.wash_dishes()
Dishwasher is startingOk so here’s what happened. We created an instance of Kitchen called kitchen which is obvious. When we called heat_up_food, kitchen‘s __getattr__ method was called, because heat_up_food is not a function or attribute of Kitchen. Inside this method, we checked to see if the called method existed in microwave_methods, which it did. Because it did, we used getattr to call kitchen‘s heat_up_food on its microwave instance and returned that method. That’s why it looks like kitchen.heat_up_food() did exactly what we wanted.
Note: The line raising the AttributeError is absolutely necessary. Without it, any otherwise non-existent method/attribute calls will error silently. In other words, something like kitchen.hey_look_at_me() will simply return nothing.
Why is This Better?
Functionally, this looks the same as what we had before. However, unlike the previous implementation of our delegator pattern, we can add methods to Microwave and Dishwasher with reckless abandon and know that they will be available in the Kitchen class without any other work from us. To illustrate, let’s extend the Microwave class:
class Microwave:
def __init__(self):
pass
def heat_up_food(self):
print("Food is being microwaved")
def timed_heat(self, minutes):
print(f"Microwaving the food for {minutes} minutes")
Now, we don’t have to do anything else to our Kitchen class to allow it to use this new timed_heat class. In the console:
>>> from kitchen import *
>>> kitchen = Kitchen()
>>> kitchen.timed_heat(4)
Microwaving the food for 4 minutesAnd that is the true code reuse achievement we get from inheritance without actually using inheritance. With some magic, we have pooled the functionality of a kitchen’s surrogate parts to abstract those parts away.
One note about this particular implementation is that it assumes the classes composing Kitchen do not have any attributes. If the Microwave class had an attribute called color for example, calling Kitchen.color would return an error. We will talk more about this when we get into Decorators in the next installment of this series.
Delegation Wrap-Up
This technique is not without its downsides. For example, since delegated methods aren’t explicitly defined inside composite classes, your IDE might have a hard time offering code completion for delegated methods. In fact, it might even highlight calls to delegated methods as potential errors since, when analyzed statically, it seems like those methods don’t exist.
Additionally, the use of metaprogramming (which is what we did when we overwrote the __getattr__ method) can slow scripts down. What you lose in execution speed, you might gain in development velocity however; all software design decisions are trade-offs.
This pattern is a powerful tool in object oriented programming and for making code readable and extendable. It is also paramount to the next pattern we cover in this series: the Decorator Pattern.