Using IPython for timing and profiling

Before you read

I found out the code here doesn’t work outside IPython, I didn’t realize that get_ipython() only works when running inside IPython. But the examples here work in isolated scripts if you run the script using ipython: ipython benchmarh.py.

Introduction

I like using IPython to measure the execution time of Python functions. You can also time functions using the timeit module, but the IPython utilities are just more convenient, and I install IPython in almost all my virtual environments anyway.

Opening an ipython shell just to use the %%timeit magic is not very convenient. This is how you can call that magic from inside a normal Python script. Just make sure to run the scrip using ipython <script> instead of python3 <script>.

Timing

from IPython import get_ipython
from typing import List

ip = get_ipython()

timeit = ip.magics_manager.magics["cell"]["timeit"]

def f():
    x = 1
    b = x + 10
    return b * 2


# Measure execution time and just print results to stdout
# Let IPython figure out how many times it should run the code
timeit("f()")

# -r: repeat code 4 times
# -n: for each `repeat`, execute the code, 5 times
timeit("-n 5 -r 4", "f()")

# -o: return the TimeitResult object
res = timeit("-o -n 2 -r 4", "f()")

What I like about using IPython for timing functions is that it does the right thing by default. Measuring execution time can be tricky, and just using timeit and getting a single value is not correct. The code must run multiple times. Among other things, you need to check the standard deviation between all the runs and look at the difference between the fastest and slowest run. The IPython timing boilerplate already provides all of that¹.

Profiling

IPython has the %prun magic to run the code using the Python code profiler.

from IPython import get_ipython

ip = get_ipython()

prof_run = ip.magics_manager.magics["cell"]["prun"]

def f():
    x = 1
    b = x + 10
    return b * 2

# -s cumulative: sort by cumulative time
# -s time: sort by internal time
# -T prof.txt: Save results to TXT file
# -D save profile stats to file
# -q: Do not show results in stdout, we already have them in the files
prof_run("-s cumulative -s time -T prof.txt -D prof.stats -q", "f()")

# to save the results, run the code as a variable assignment
prof_run("-s cumulative -s time -T prof.txt -D prof.stats -q", "result = f()")
print(result)

Other utilities

Sometimes you want to compare different implementations and their relative performance improvements. The following function can be used/adapted to sort multiple exection times and show how much faster is each one, relative to the slowest one.

def compare_execution_times(execution_times: List[float]):
    """
    Sorts a list of floating point numbers representing execution times in
    descending order, then calculates how much slower each number is relative
    to the fastest one. Returns a list of tuples, where each tuple contains the
    execution time and its corresponding speed factor relative to the fastest
    execution time.
    """
    sorted_times = sorted(execution_times, reverse=True)
    slowest = sorted_times[0]
    result = []
    for time in sorted_times:
        factor = slowest / time
        result.append((time, factor))
    return result

You can use snakeviz to visalize the dumped .stats file:

snakeviz prof.stats

It will start a server and give you a URL. You can open the URL in your browser to have an interactive look at the results.

Another alternative I’ve used is copy-pasting the code that IPython uses internally to a separate module. This is for when I don’t want/can’t install ipython ↩︎