Decorators

1. Overview

In general a decorator function:

• Takes a function as an argument
• Returns a closure
• The closure usually accepts any combination of parameters
• Runs some code in the inner function (closure)
• The closure function calls the original function using the arguments passed to the closure
• Returns whatever is returned by that function call

2. Description

2.1 Decorator function

def counter(fn):
    count = 0
    
    def inner(*args, **kwargs):
        nonlocal count
        count += 1
        print('Function {0} was called {1} times'.format(fn.__name__, count))
        return fn(*args, **kwargs)
    return inner
    

def add(a, b=0):
    """
    returns the sum of a and b
    """
    return a + b
    
add = counter(add)
result = add(1, 2)

Using *args, **kwargs means we can call any function fn with any combination of positional and keyword-only arguments. We essentially modified our add function by wrapping it inside another function that added some functionality to it. We also say that we decorated our function add with the function counter. And we call counter a decorator function.

2.2 Decorators and the @ Symbol

In general, if func is a decorator function, we decorate another function my_func using

my_func = func(my_func)

This is so common that Python provides a convenient way of writing that with 2.1 code

@counter
def add(a, b=0):
    """
    returns the sum of a and b
    """
    return a + b

2.3 Retain metadata of the original function

The functools module has a wrap function that we can use to fix the metadata of our inner function in our decorator.

from functools import wraps

In fact, the wraps function is itself a decorator but it needs to know what was our original function

def counter(fn):
    count = 0
    
    @wraps(fn)
    def inner(*args, **kwargs):
        nonlocal count
        count += 1
        print("{0} was called {1} times".format(fn.__name__, count))

    return inner
  
@counter
def add(a: int, b: int=10) -> int:
    """
    returns sum of two integers
    """
    return a + b
    
help(add)
# Help on function add in module __main__:
#
# add(a:int, b:int=10) -> int
#     returns sum of two integers

And introspection using the inspect module works as expected

inspect.signature(add)
# import sys; print('Python %s on %s' % (sys.version, sys.platform))
# sys.path.extend(['/Users/demyank88/workspace/practice/python/practice'])

inspect.signature(add)
# <Signature (*args, **kwargs)>

2.4 Decorator parameter

Let's modify a timer runs the function multiple times and calculates the average run time:

def calc_fib_recurse(n):
    return 1 if n < 3 else calc_fib_recurse(n-1) + calc_fib_recurse(n-2)

def fib(n):
    return calc_fib_recurse(n)

# that value of 10 which is repetition has been hardcoded.
def timed(fn):
    from time import perf_counter

    def inner(*args, **kwargs):
        total_elapsed = 0
        for i in range(10):
            start = perf_counter()
            result = fn(*args, **kwargs)
            end = perf_counter()
            total_elapsed += (perf_counter() - start)
        avg_elapsed = total_elapsed / 10
        print('Avg Run time: {0:.6f}s'.format(avg_elapsed))
        return result
    
    return inner
    
    
fib = timed(fib)
fib(30)

Now to decorate our Fibonacci function we have to use the long syntax (The @ syntax will not work):

def timed(fn, num_reps):
    from time import perf_counter
    
    def inner(*args, **kwargs):
        total_elapsed = 0
        for i in range(num_reps):
            start = perf_counter()
            result = fn(*args, **kwargs)
            end = perf_counter()
            total_elapsed += (perf_counter() - start)
        avg_elapsed = total_elapsed / num_reps
        print('Avg Run time: {0:.6f}s ({1} reps)'.format(avg_elapsed,
                                                        num_reps))
        return result
    
    return inner
    
    
def fib(n):
    return calc_fib_recurse(n)

fib = timed(fib, 5)
fib(28)

The problem is that we cannot use the @ decorator syntax because when using that syntax Python passes a single argument to the decorator: the function we are decorating - nothing else.

Of course, we could just use what we did above, but the decorator syntax is kind of neat, so it would be nice to retain the ability to use it. We just need to change our thinking a little bit to do this:

First, when we see the following syntax:

@dec 
def my_func(): 
	pass

we see that dec must be a function that takes a single argument, the function being decorated.

You'll note that dec is just a function, but we do not call dec when we decorate my_func, we simply use the label dec.

Then Python does:

my_func = dec(my_func)

Let's try a concrete example:

def dec(fn):
    print ("running dec")
    
    def inner(*args, **kwargs):
        print("running inner")
        return fn(*args, **kwargs)
              
    return inner
    
    
@dec
def my_func():
    print('running my_func')

As we can see, when we decorated my_func, the dec function was called at that time.

Because Python did this:

my_func = dec(my_func)
my_func()

# running inner
# running my_func

so dec was called

And when we now call my_func, we see that the inner function is called, followed by the original my_func

2.5 Decorator Factories

The outer function is not itself a decorator instead it returns a decorator when called. And any arguments passed to outer can be referenced (as free variables) inside our decorator. We call this outer function a decorator factory function. (It is a function that creates a new decorator each time it is called)

def dec_factory():
    print('running dec_factory')
    def dec(fn):
        print('running dec')
        def inner(*args, **kwargs):
            print('running inner')
            return fn(*args, **kwargs)
        return inner
    return dec
    
# Approach 1
@dec_factory()
def my_func(a, b):
    print(a, b)
    
    
my_func(10, 20)
# running inner
# 10 20


# Approach 2
dec = dec_factory()
@dec
def my_func():
    print('running my_func')

my_func()
#running inner
#running my_func


# Approach 3
dec = dec_factory()

def my_func():
    print('running my_func')

my_func = dec(my_func)
my_func()
#running inner
#running my_func

3. Examples

3.1 Introspecting 1

def counter(fn):
    count = 0

    def inner(*args, **kwargs):
        nonlocal count
        count += 1
        print('Function {0} was called {1} times'.format(fn.__name__, count))
        return fn(*args, **kwargs)

    return inner

@counter
def mult(a: float, b: float = 1, c: float = 1) -> float:
    """
    returns the product of a, b, and c
    """
    return a * b * c
    
# Remember we could equally have written like below
mult = counter(mult)

mult.__name__ 
# inner, but not mult

help(mult)
# Help on function inner in module __main__:
# inner(*args, **kwargs)

mult's name changed when we decorated it. They are not the same function after all. We have also lost our docstring and even the original function signature. Even using the inspect module's signature does not yield better results.

3.2 Decorator Factory

All we need to do is create an outer function around our timed decorator, and pass the num_reps argument to that outer function instead:

# approach 1
def calc_fib_recurse(n):
    return 1 if n < 3 else calc_fib_recurse(n-1) + calc_fib_recurse(n-2)
    
def timed_factory(num_reps=1):
    def timed(fn):
        from time import perf_counter

        def inner(*args, **kwargs):
            total_elapsed = 0
            for i in range(num_reps):
                start = perf_counter()
                result = fn(*args, **kwargs)
                end = perf_counter()
                total_elapsed += (perf_counter() - start)
            avg_elapsed = total_elapsed / num_reps
            print('Avg Run time: {0:.6f}s ({1} reps)'.format(avg_elapsed,
                                                            num_reps))
            return result
        return inner
    return timed  

@timed_factory(5)
def fib(n):
    return calc_fib_recurse(n)
    
    
fib(30)
# Avg Run time: 0.249934s (5 reps)

# approach 2
# using wraps in functools
from functools import wraps

def timed(num_reps=1):
    def decorator(fn):
        from time import perf_counter

        @wraps(fn)
        def inner(*args, **kwargs):
            total_elapsed = 0
            for i in range(num_reps):
                start = perf_counter()
                result = fn(*args, **kwargs)
                end = perf_counter()
                total_elapsed += (perf_counter() - start)
            avg_elapsed = total_elapsed / num_reps
            print('Avg Run time: {0:.6f}s ({1} reps)'.format(avg_elapsed,
                                                            num_reps))
            return result
        return inner
    return decorator  
    
@timed(5)
def fib(n):
    return calc_fib_recurse(n)

fib(30)
# Avg Run time: 0.253744s (5 reps)

Just to put the finishing touch on this, we probably don't want to have our outer function named the way it is (timed_factory). Instead we probably just want to call it timed. 

4. References

https://mathbyteacademy.com

https://en.wikipedia.org/wiki/Decorator_pattern