Delegate and Decorate in Python: Part 3 – Reusable Decorators

Note: This is not about Python’s language feature called decorators (with the @ symbol), but about the design patterns called “decorator” and “delegate.”

In the final installment of this series, we will take the universal concepts of delegation and decoration and put them into a base Decorator class. Doing so will allow us to abstract the universal functionality these patterns to be inherited by more specific decorator classes, maximizing code reuse. Let’s dive in.

The Plan

In this article, we’re going to think about the decorator classes we made in the last article and see if we can identify some universal functionality. We’ll put this universal functionality into a base class called Decorator so that more concrete implementations of the pattern can focus solely on their responsibilities.

Set Up

Let’s first quickly write a simple class to use with our decorators. I’m going to just make a simple Animal and Dog class like we made in part 1:

class Animal:
  def __init__(self, name, num_of_legs):
    self.name = name
    self.num_of_legs = num_of_legs
  
  def get_number_of_legs(self):
    print(f"I have {self.num_of_legs} legs")

class Dog(Animal):
  def __init__(self, name, num_of_legs):
    super().__init__(name, num_of_legs)

  def bark(self):
    print("Woof woof")

Common Functionality

In each of the examples in previous articles, our decorator classes had at least three attributes: an instance of the decorated model, a list of its methods, and a list of its attributes. This seems like as good a place as any to start with our Decorator class, let’s try:

class Decorator:
  def __init__(self, model):
    self.model = model
    self.model_methods = [f for f in dir(type(self.model)) if not f.startswith('_')]
    self.model_attributes = [a for a in self.model.__dict__.keys()]
  
  def __getattr__(self, func):
    if func in self.model_methods:
      def method(*args):
        return getattr(self.model, func)(*args)
      return method
    elif func in self.model_attributes:
      return getattr(self.model, func)
    else:
      raise AttributeError

Let’s test this out by decorating an instance of the Dog class. The decorated instance won’t actually have any added functionality, we’re just making sure it still behaves like a dog:

>>> dog = Dog('Fido', 4)
>>> dog = Decorator(dog)
>>> dog.bark()
Woof woof
>>> dog.get_number_of_legs()
I have 4 legs

Great, looks like everything is fine. We will not be using the base Decorator class to decorate models after this, this was just a smoke test.

An Actual Decorator

Let’s write a non abstract decorator now, we’ll call it the LeashedDogDecorator, who will have the added functionality of tugging on its leash. In order to do this, the class will have to inherit the Decorator class, gaining the delegation features written in that class and simply adding its own decorating functionality. Check it out:

class LeashedDogDecorator(Decorator):
  def __init__(self, dog):
    super().__init__(dog)
  
  def tug_on_leash(self):
    print("Let's GOOOOO!!!")

Let’s make sure that our decorated Dog instances have both the base and decorated functionality:

>>> dog = Dog('Fido', 4)
>>> dog = LeashedDogDecorator(dog)
>>> dog.tug_on_leash()
Let's GOOOOO!!!
>>> dog.bark()
Woof woof

Excellent. Everything seems to be working as we like.

Enforcing Model Types

The Decorator class exists so we can write short and concise decorator classes and reuse the tedium of delegation required to make decoration work. The Decorator can decorate anything, which is how it should be. What about the LeashedDogDecorator though? There is nothing stopping decoration of a Report class instance with this, meaning we could have Report objects floating around that can tug_on_leash. Is there anything we can do to prevent this?

One obvious solution is of course, to consciously not decorate models with decorators that don’t make sense. This is easy enough in a system that has models as different from each other as Dogs and Reports, but in many systems, class differences can be subtle.

Python offers a solution to this, called type hints. Let’s take a quick look at how this works.

The goal is making sure that the only models passed to the LeashedDogDecorator are instances of the Dog class. Using python’s type checking, we would do something liek this:

class LeashedDogDecorator(Decorator):
  def __init__(self, dog: Dog):
    super().__init__(dog)
  
  def tug_on_leash(self):
    print("Let's GOOOOO!!!")

Notice in the constructor arguments we put dog: Dog. This little addition has a couple of benefits. First of all, using type hints like this will cause most IDEs to give you better completion hints, which is cool. Additionally, and again in most IDEs, if we try to pass something that isn’t a Dog to this decorator, we will see a little red squiggly under the line trying to this.

Note: As I’m writing the code to go with this article, I am using Visual Studio Code, because it is lightweight. With the standard Python linter, the IDE doesn’t seem to be helping me out at all with type hints. I had to install this plugin to get hints in code completion: https://marketplace.visualstudio.com/items?itemName=njqdev.vscode-python-typehint. I usually use JetBrains’ PyCharm in larger scale projects and I can confirm that using type hints in PyCharm will give users both code-completion hints as well as warnings when type mismatches occur.

Python will not, however, throw an error at run time if you pass incompatible class instances to type-hinted arguments. Whether you consider this a good or bad thing, I leave up to you. The point of type hints in cases like this are to make our IDEs help us more and to write self-documenting code.

Overriding or Wrapping Methods

What other functionality could we put into our Decorator class? Well, the last time I used decorators was because I had a class in which some methods required a few variables to be checked before running. Likewise, this class had a few methods that needed to set a few status variables after running. The code was identical in each of the methods, so I had no code reuse.

In this case, inheritance didn’t make sense. So I wrote a decorator for the class the listed all the functions needing methods to fire before, and those needing methods to fire after. After that, it was delegation as usual. Let’s talk about introducing that functionality into our Decorator class.

A word on nomenclature

This is a Python article written by someone heavily influenced by working professionally with Ruby. In Ruby, we have method callbacks for classes, known as before, after, and around. In our classes we can designate a method to run after another method has run. Same with before methods are run. The around callback actually wraps a method with both before and after processing.

Since this is how these concepts are named in Ruby, it’s what I will call them in this Pythonic implementation. You are free to call them something else if you like, just make sure they make sense.

before and after hook

Let’s see think about how we would design a decorator to run a method before its base model method. Keep in mind that something like this is an edge case, so a user shouldn’t be forced to implement a before method in the decorator if they don’t want.

First, we should think about how we want the user to tell us there’s a method to be called before or after another method. I decided to go with a list of dictionaries containing a key that is either 'before' or 'after' with the value of the method in question, and a key called do that is the method to be called before or after.

This is getting pretty complicated, as most forays into metaprogramming are, so I’ll just show you the code and then we’ll talk about it. Here’s the Decorator class with support for before and after method callbacks:

class Decorator:
  def __init__(self, model, callback_methods=[]):
    self.model = model
    self.model_methods = [f for f in dir(type(self.model)) if not f.startswith('_')]
    self.model_attributes = [a for a in self.model.__dict__.keys()]
    self.callback_methods = callback_methods
    self.list_of_callback_methods = []
    self.__divide_callback_functions()
  
  def __getattr__(self, func):
    if func in self.model_methods and func not in self.list_of_callback_methods:
      def method(*args):
        return getattr(self.model, func)(*args)
      return method
    elif func in self.model_attributes:
      return getattr(self.model, func)
    elif func in self.list_of_callback_methods:
      def method(*args):
        return self.__callback_switchboard(func, *args)
      return method
    else:
      raise AttributeError
  
  def __divide_callback_functions(self):
    for method in self.callback_methods:
      if 'before' in method.keys(): self.list_of_callback_methods.append(method['before'])
      if 'after' in method.keys(): self.list_of_callback_methods.append(method['after'])

  def __callback_switchboard(self, func, *args):
    for method in self.callback_methods:
      if 'before' in method.keys() and func == method['before']:
        getattr(self, method['do'])()
        getattr(self.model, func)(*args)
      if 'after' in method.keys() and func == method['after']:
        getattr(self.model, func)(*args)
        getattr(self, method['do'])()

Yikes, what’s going on here? Well let’s start with the callback_methods parameter. This is an optional parameter in the Decorator constructor that takes a list of dictionary objects. Before we get into the LeashedDogDecorator which will implement a before hook, let me just show you what one of these dictionaries will look like to better understand the code we’re looking at now. One item in this array will look like this: {'before': 'bark', 'do': 'something'} where 'something' is the name of the method in the decorator to run before the bark method.

Also in the constructor is a call to __divide_callback_functions, if you’ve never seen this leading double underscore naming convention before, this is the equivalent of Java’s private class method functionality, but in Python. The method parses through the callback methods and puts them into our list_of_callback_methods.

Moving onto the __getattr__ overwritten method, we modified the first if function to only fire for methods not included in the list_of_callback_methods populated earlier. When the __getattr__ method does receive a func that exists in the list_of_callback_methods, we call the private method __callback_switchboard. The switchboard parses through the callback_methods, checking to see if the passed func has a before callback or after. Once it has made this distinction, it will either run the callback function, then the model function, or vice versa depending on if it is a before or after callback.

Using the before hook

Now we’re going to update the LeashedDogDecorator to make use of the new callback abilities. We’re going to make a method called growl and we’re going to make it fire before the Dog‘s bark method. Here’s what the decorator looks like now:

class LeashedDogDecorator(Decorator):
  def __init__(self, dog: Dog):
    callbacks = [
      {'before': 'bark', 'do': 'growl'},
    ]
    super().__init__(dog, callback_methods=callbacks)
  
  def tug_on_leash(self):
    print("Let's GOOOOO!!!")
  
  def growl(self):
    print("Grrr")
  

Now, let’s fire up the console and see it in action:

>>> from decorator import *
>>> dog = Dog('Fido', 4)
>>> dog = LeashedDogDecorator(dog)
>>> dog.bark()
Grrr
Woof woof

Isn’t that cool??

One Small Problem

There’s a problem with the current implementation, can you guess what it is? Let’s see what happens when we try to set bark to be called after tug_on_leash.

class LeashedDogDecorator(Decorator):
  def __init__(self, dog: Dog):
    callbacks = [
      {'before': 'bark', 'do': 'growl'},
      {'after': 'tug_on_leash', 'do': 'bark'}
    ]
    super().__init__(dog, callback_methods=callbacks)
  
  def tug_on_leash(self):
    print("Let's GOOOOO!!!")
  
  def growl(self):
    print("Grrr")
  

But if we run dog.tug_on_leash() in the console, the bark method does not fire afterwards like we expect. Why is this? Well, it’s because the Decorator base class is assuming that the only methods that will have before or after hooks are ones found in the decorated model, and the callbacks will only come from the decorator. In other words, because tug_on_leash isn’t part of base class and because bark isn’t part of the decorator class, this implementation doesn’t work.

Whether or not you think this is a problem depends on how you might feel about design. You might think only decorating methods should be callback functions. You might think the entire base/decorator ecosystem should be treated as one. Or perhaps you’re somewhere in the middle and believe that all functions can be callbacks but only that decorating functions can be callbacks for model functions and model functions for decorating functions (so decorating functions cannot be callbacks for other decorating functions).

In order to make that happen, we would have to do a lot more work and use the built in Python method __getattribute__, which unlike its abbreviated cousin __getattr__ fires every time a method or attribute is called on a class.

For the purposes of finishing up this article, I leave that implementation as an exercise to the reader (and maybe to myself if I ever have the time *shrug*).

Conclusion

This concludes our exploration of the delegation and decorator patterns and their implementation in Python. Some things we didn’t cover include really digging into when these patterns are useful vs. when other’s might be better. We also did not cover efficient strategies for testing classes implementing these patterns.

Design patterns are an important tool in every developers toolbox, and as such, will be covered more and in different ways in future articles. For now, I hope you’ve enjoyed what we’ve learned so far. Please feel free to reach out regarding any questions you might have. ewhiting@erikscode.space

A Word on Metaprogramming

What we’ve been doing–overriding built in methods like __getattr__ and checking the string names of function/methods–is called “metaprogramming.” Solutions in which metaprogramming is involved will always be complicated and tricky.

It’s actually pretty uncommon to need to solve a challenge with metaprogramming. In fact, usually these solutions are to help the developer or development team responsible for building or managing the solution rather than contributing to a system’s features. Creative metaprogramming solutions can increase development velocity (the speed in which new features go from user-stories to code live in production) but obviously can hinder it pretty significantly as well. Always be careful when implementing metaprogrammign solutions as you will lose a lot of the support you get from IDEs and debuggers.

Good luck!