Torture testing Python and really cool results!

vsk · 2008-07-23 19:26:50

As many of you probably know, the beta version of Python (Py3k) has been out for a while.
I've decided to write a few tests and put it to the limit! (Well, sort of.)

I'm no expert, but the results speak for themselves;

q ~/Desktop  $  ./test_py.sh 
Starting Python tests!

---
Py2.x - No Optimization
Start Time: 1216840813.97
    Benchmark 1: 6.35747790337
    Benchmark 2: 6.20724105835
    Benchmark 3: 6.18551802635
    Benchmark 4: 6.18940997124
    Benchmark 5: 6.18906211853
Average Run: 6.22574181557

---
Py2.x - With Psyco
Start Time: 1216840845.31
    Benchmark 1: 2.65315485001
    Benchmark 2: 2.73273706436
    Benchmark 3: 2.72992992401
    Benchmark 4: 2.73208212852
    Benchmark 5: 2.71949505806
Average Run: 2.71347980499

---
Py3k - No Optimization
Start Time: 1216840858.93
    Benchmark 1: 4.4160349369
    Benchmark 2: 4.40270519257
    Benchmark 3: 4.41088199615
    Benchmark 4: 4.38550496101
    Benchmark 5: 4.44915294647
Average Run: 4.41285600662
---

Finished tests!

The new beta Py3k performs consistently faster than the current stable version! I'm very excited about this!
Strangely enough, my system monitor showed that once Py3k took over, the memory usage dropped almost by half!
Of course, many parts of the language are far from complete (gui, network, profile related modules especially).
But it'll catch on..

I'm sure you guys wanted to see what code I used to test, and again, if it turns out that it's invalid - sorry! I'm not an expert.
(Edit: For the record: I didn't have any other running processes. This was done on an i686 Intel Pentium4 with 1 GB of ram.)

#!/bin/bash
# A shell script to easily run the tests.

cd ~/Desktop

echo "Starting Python tests!"

python test_2.x.py
python test_2.x_psyco.py
/usr/local/bin/python3.0 test_3.x.py3k

echo "---"
echo
echo "Finished tests!"

import time

# Only used in ONE of the tests! Please look at the results!
###########
# import psyco
# psyco.full()
###########

print("\n---\n[Python Version] [Type of test.]")

def test():
    var = 0
    for x in range(0,10000000):
        var += (x+1)
    while var > 1:
        var -= var**3

def main():
    start = time.time()
    print("Start Time: %s" % start)
    times = []
    for y in range(1,6):
        test()
        next = time.time()
        next -= start
        start += next
        print("\tBenchmark %s: %s" % (y, next))
        times.append(next)
    avg = 0
    for z in times:
        avg += z
    avg /= len(times)
    print("Average Run: %s" % avg)

main()

I can't wait to see what the developer behind psyco can achieve with Py3k!!!
Afaik, he used a specialized JIT compilation technique using the py/C API that replaced python's native bytecode parsing method 'eval_frame' with his own magic.
It'll be interesting to see if the same technique can still be used!

Last edited by vsk (2008-07-23 19:32:54)

freakcode · 2008-07-23 20:15:43

Good timing, considering that Py3 doesn't have a JIT.

But this kind of test is too artificial, not real use cases for most of things Python is used. Also, this script benefits greatly from the Psyco JIT, as it's simply calling the same function in a loop and doing nothing I/O intensive nor using memory.

vsk · 2008-07-23 21:00:29

I should add;

* File I/O (with and later without a buffer)
* Multi-threading
* Random stuff like; os.system Usage, var = os.walk('/'), zipfile Usage
* importing modules
* add more functions
* create classes within the test() scope (tee hee)

to test() to make it well-rounded, yes?
I'll give it a go!

skottish · 2008-07-23 21:11:51

If you intend to distribute your programs, be forewarned that psycho doesn't work on anything other than i386 machines. The psycho developer recommends PyPy instead for all architectures.

vsk · 2008-07-23 21:39:30

For you extremely brave souls out there, I have created a compilation of what must be the absolute worst, pointless, memory exhausting, CPU hogging, and badly styled Python functions in the known universe. It violates every sacred Pythonic law, maxed out my memory, swap, and CPU until I killed it, and I couldn't even get ONE cycle through.

Before you run this (yes, it is Py3k compatible) - please note that I warned you!! This is risky! Do this only if you have absolutely NOTHING better to do;

import time

print("\n---\nPy3k - No Optimization")

# The following Python code may cause brain damage, and stunt the
# development of logical programming skills. Be warned.
def test():
    
    class DUMMY (str):
        def __init__(self):
            pass
        def __pow__(self):
            pass
        def __div__(self):
            pass
    
    try:
        import threading
    except ImportError:
        raise IOError
    
    import os
    import sys
    import math
    import platform
    import subprocess
    import distutils
    import email
    import pickle
    
    obj = open("uyegrutyergtbueyrgtsuyregtbsueyrgtbkuerytnkuscyergnutksdysregtbufisyergtbfkusgytrbo",'a')
    for q in range(0,10000000):
        obj.write("%suyegrutyergtbueyrgtsuyregtbsueyrgtbkuerytnkuscyergnutksdysregtbufisyergtbfkusgytrbo\n" % q)
    obj.close()
    del obj
    
    var = 0
    for x in range(0,10000000):
        var += (x+1)
    while var > 1:
        var -= var**2
    
    obj = open("uyegrutyergtbueyrgtsuyregtbsueyrgtbkuerytnkuscyergnutksdysregtbufisyergtbfkusgytrbo", 'r')
    junk = obj.readlines()
    obj.close()
    
    del var
    
    def var(fp='/'):
        mywalk = os.walk(fp)
        while True:
            try:
                crap = mywalk.__next__()
            except AttributeError:
                crap = mywalk.next()
            except StopIteration:
                break
            os.system("echo %s >> uyegrutyergtbueyrgtsuyregtbsueyrgtbkuerytnkuscyergnutksdysregtbufisyergtbfkusgytrbo" % crap)
            
            arch_is_the_best = DUMMY()
    
    t1 = threading.Thread(target=var)
    t2 = threading.Thread(target=var)
    t1.start()
    t2.start()

def main():
    start = time.time()
    print("Start Time: %s" % start)
    times = []
    for y in range(1,6):
        test()
        next = time.time()
        next -= start
        start += next
        print("\tBenchmark %s: %s" % (y, next))
        times.append(next)
        os.system("rm uyegrutyergtbueyrgtsuyregtbsueyrgtbkuerytnkuscyergnutksdysregtbufisyergtbfkusgytrbo")
    avg = 0
    for z in times:
        avg += z
    avg /= len(times)
    print("Average Run: %s" % avg)

main()

vsk · 2008-07-23 21:41:58

skottish wrote:

If you intend to distribute your programs, be forewarned that psycho doesn't work on anything other than i386 machines. The psycho developer recommends PyPy instead for all architectures.

Thank you for pointing this out.

For others who may use psyco, please note that you can safely do this;

#!/usr/bin/env python

try:
    import psyco
    psyco.full()
except ImportError:
    pass

This may get your code running much faster - or not.

Last edited by vsk (2008-07-23 21:48:15)

andre.ramaciotti · 2008-07-23 21:43:22

vsk wrote:

I should add;
* File I/O (with and later without a buffer)
* Multi-threading
* Random stuff like; os.system Usage, var = os.walk('/'), zipfile Usage
* importing modules
* add more functions
* create classes within the test() scope (tee hee)
to test() to make it well-rounded, yes?
I'll give it a go!

Tests with strings would be nice too.

vsk · 2008-07-23 21:50:33

Tests with strings would be nice too.

What sort of test(s)?
Translations to uppercase, lowercase and the sort?

Arch Linux

#1 2008-07-23 19:26:50

Torture testing Python and really cool results!

#2 2008-07-23 20:15:43

Re: Torture testing Python and really cool results!

#3 2008-07-23 21:00:29

Re: Torture testing Python and really cool results!

#4 2008-07-23 21:11:51

Re: Torture testing Python and really cool results!

#5 2008-07-23 21:39:30

Re: Torture testing Python and really cool results!

#6 2008-07-23 21:41:58

Re: Torture testing Python and really cool results!

#7 2008-07-23 21:43:22

Re: Torture testing Python and really cool results!

#8 2008-07-23 21:50:33

Re: Torture testing Python and really cool results!

Board footer