Arch Linux

Google

Member

From: Mountain View, California

Registered: 2010-05-31

Posts: 484

Website

How to accurately test code in Python?

I am trying to compare some custom C modules with the standard Python modules. For instance, I wanted to test if a random function in C could out-perform Pythons randint method.

I have tried using the timeit module, but I am curious if I am skewing my own results in any way.

import timeit
import ctypes
import random

my_test_lib = ctypes.cdll.LoadLibrary('/home/name/workspace/Testing/test.so')


crand_timer = timeit.Timer("crand()", "from __main__ import crand")
prand_timer = timeit.Timer("prand()", "from __main__ import prand")

def crand():
    for i in range(10):
        my_test_lib.get_random(1, 10)
        
def prand():
    for i in range(10):
        random.randint(1,10)
    

if __name__ == '__main__':
    print crand_timer.timeit()
    print prand_timer.timeit()

Just for kicks, here is the C code:

#include <stdlib.h>
#include <time.h>

#define TRUE  1
#define FALSE 0

static int randomized = FALSE;

void randomize(void)
{
  srand(time(NULL));
  randomized = TRUE;
}

/* returns a random number between base and max, inclusive */
int get_random(int base, int max)
{
  if (randomized == FALSE){
    randomize();
  }
  return rand() % (max - base + 1) + base;
}

I compiled the C code with O3. The results seem to indicated the C code is about 2.3 times faster than the Python randint method.

Am I skewing the results in any way? I have a large amount of C code to test. I will be integrating a bunch of low level C modules into a Python program I am going to build. I want to learn how to test efficiently before I start.

edit:

If anyone needs/wants/interested in knowing-- I plan to build a tile based game with multi-threaded procedural dungeon generator. I want to build the generator in C and make it as fast as possible.

Last edited by Google (2011-10-27 07:21:54)

trontonic

Member

Registered: 2008-07-21

Posts: 80

Re: How to accurately test code in Python?

Hi,

If you increase the samples (say loop 5000 times instead of 10), the answers will be better, but accurate benchmarking is really hard.
Have you seen The Computer Language Benchmarks Game. I find it interesting.
When it comes to compilation flags, -O2 vs -O3 is seldom the problem. It's hard to choose the best algorithm and it's hard to make it scale as intented.
Do you know of Big O notation? That could be helpful.

Good luck with the game and dungeon generator.

Google

Member

From: Mountain View, California

Registered: 2010-05-31

Posts: 484

Website

Re: How to accurately test code in Python?

I didn't know about the benchmark site-- thanks!

I know a bit about Big-Oh and complexity. I try to keep it in mind-- essentially try not to re-peat myself when not needed and try to pass through my data as few times as possible.

Thanks