Using IPython for timing and profiling
Table of contents
Before you read
I found out the code here doesn’t work outside IPython, I didn’t realize that
get_ipython() only works when running inside IPython. But the examples here work in isolated scripts if you run the script using ipython:
I like using IPython to measure the execution time of Python functions. You can also time functions using the timeit module, but the IPython utilities are just more convenient, and I install IPython in almost all my virtual environments anyway.
ipython shell just to use the
%%timeit magic is not very convenient. This is how you can call that magic from inside a normal Python script. Just make sure to run the scrip using
ipython <script> instead of
from IPython import get_ipython from typing import List ip = get_ipython() timeit = ip.magics_manager.magics["cell"]["timeit"] def f(): x = 1 b = x + 10 return b * 2 # Measure execution time and just print results to stdout # Let IPython figure out how many times it should run the code timeit("f()") # -r: repeat code 4 times # -n: for each `repeat`, execute the code, 5 times timeit("-n 5 -r 4", "f()") # -o: return the TimeitResult object res = timeit("-o -n 2 -r 4", "f()")
What I like about using IPython for timing functions is that it does the right thing by default. Measuring execution time can be tricky, and just using
timeit and getting a single value is not correct. The code must run multiple times. Among other things, you need to check the standard deviation between all the runs and look at the difference between the fastest and slowest run. The IPython timing boilerplate already provides all of that1.
IPython has the
%prun magic to run the code using the Python code profiler.
from IPython import get_ipython ip = get_ipython() prof_run = ip.magics_manager.magics["cell"]["prun"] def f(): x = 1 b = x + 10 return b * 2 # -s cumulative: sort by cumulative time # -s time: sort by internal time # -T prof.txt: Save results to TXT file # -D save profile stats to file # -q: Do not show results in stdout, we already have them in the files prof_run("-s cumulative -s time -T prof.txt -D prof.stats -q", "f()") # to save the results, run the code as a variable assignment prof_run("-s cumulative -s time -T prof.txt -D prof.stats -q", "result = f()") print(result)
Sometimes you want to compare different implementations and their relative performance improvements. The following function can be used/adapted to sort multiple exection times and show how much faster is each one, relative to the slowest one.
def compare_execution_times(execution_times: List[float]): """ Sorts a list of floating point numbers representing execution times in descending order, then calculates how much slower each number is relative to the fastest one. Returns a list of tuples, where each tuple contains the execution time and its corresponding speed factor relative to the fastest execution time. """ sorted_times = sorted(execution_times, reverse=True) slowest = sorted_times result =  for time in sorted_times: factor = slowest / time result.append((time, factor)) return result
You can use snakeviz to visalize the dumped
It will start a server and give you a URL. You can open the URL in your browser to have an interactive look at the results.
Another alternative I’ve used is copy-pasting the code that IPython uses internally to a separate module. This is for when I don’t want/can’t install