Tuesday, January 5, 2010

Let's timeit!

Every now and then you might want to time snippets just to make sure that you choose the more efficient solution. In those cases you can use the timeit module to measure execution time for snippets.

Introduction
It's very easy to setup and measure the execution time for a snippet with timeit. The module contains a class, Timer, which is used to perform the measurement. The class has one constructor and three methods:
  • Timer([stmt='pass'[, setup='pass'[, timer=<timer function>]]]) - stmt is the statement to be timed and setup is called once before executing the main statement. A timer function can be specified and default is time.time() for all platforms but windows which is set to time.clock() instead (according to my timeit.py)
  • timeit([number=1000000]) - Executes the main statement passed to the constructor number of times and returns the result in seconds as a float.
  • repeat([repeat=3[, number=1000000]]) - Convenience function that calls timeit(number) repeat times. Returns a list with the results.
  • print_exc([file=None]) - Helper to print a traceback from the timed snippet.
Starting with Python 2.6 the timeit module also defines two convenience functions, timeit.timeit() and timeit.repeat(). They are basically wrappers around the Timer class.

Example
Suppose that I would like to create a list containing 100 'c':s like this ['c', 'c', ...]. There are at least two ways of doing this:
lst = ['c'] * 100
# or
lst = ['c' for i in xrange(100)]
Which one should I choose? Well, let's execute both statements with timeit and measure the execution time.
>>> import timeit
>>> t = timeit.Timer(stmt="lst = ['c'] * 100")
>>> print t.timeit()
1.10580182076
>>> t = timeit.Timer(stmt="lst = ['c' for x in xrange(100)]")
>>> print t.timeit()
7.66900897026
Ok, I think I'll stick with the first snippet :)

The result returned is the total execution time in seconds. In this particular case when we are executing the snippet 1000000 times the result is also the execution time in microseconds for one single pass (1000000*exe_time/1000000 == exe_time).

Normally, the timeit module doesn't have access to things that you have defined in your module. If you would like to measure a function that you have defined in your module you can specify the import statement in the setup parameter:
>>> def create_lst(size):
...    return ['c'] * size
...
>>> t = timeit.Timer(stmt="create_lst(100)", setup="from __main__ import create_lst")
>>> print t.timeit()
1.21339488029
This will introduce a little overhead since the create_lst() function is called in the measurement loop instead of just executing an inlined snippet.

Note: Timer.timeit() will by default disable garbage collection during timing. To enable GC you can pass 'gc.enable()' as a setup statement.

I find the timeit module as a simple and convenient way to measure execution time for small snippets.

5 comments:

  1. Nice writeup! For quick tests, I really like using the module as an script, by running it with 'python -m timeit [statements]'. You can see the parameters it accepts using 'python -m timeit --help'.

    ReplyDelete
  2. I totally agree with you. It's much easier to use the command line version when you just have a one-liner statement and the output from the tool is nice :)

    Thanks for pointing this out to the readers.

    ReplyDelete
  3. %time and $timeit with ipython.

    eg.

    >>> %timeit a=1

    cu!

    ReplyDelete
  4. I didn't know about ipython, finally tab-completion in the interactive python prompt :)

    ReplyDelete
  5. I really need to spend more time with the IPython docs, I've been using it for years and never noticed the timing capabilities until now.

    Cheers, illume, very very handy.

    ReplyDelete