Doug's Weblog

In the official CPython implementation, the Python interpreter mixes the state of Python’s stack with the state of the underlying C code’s stack. Under normal operation, this design choice has little impact, but if one looks more closely there are some major drawbacks.

One such drawback is that Python’s stack depth is now limited by the maximum stack depth supported by the underlying CPython process. This might not seem so bad until you consider that one function call in Python may cause many C functions to be pushed onto the stack.

A solution to this problem is to separate Python’s stack from C’s stack. This is exactly what Stackless Python does. Now, Python’s stack size is only limited by the amount of memory in the machine.

I’ve implemented the Sieve of Eratosthenes example from last time using the tasklets and channels that Stackless provides. I haven’t really written any Stackless code until now, but I feel this is a decent translation.

import stackless

def ints(n, ch):
    while 1:
        ch.send(n)
        n = n + 1

def filter_mul(n, i, o):
    while 1:
        num = i.receive()
        if (num % n):
            o.send(num)

def sieve(i, o):
    while 1:
        head = i.receive()
        o.send(head)
        n = stackless.channel()
        stackless.tasklet(filter_mul)(head, i, n)
        i = n

def printer(i):
    while 1:
        print i.receive()

def gather(i, o, n):
    l = []
    for x in range(int(n)):
        l.append(i.receive())
    o.send(l)

def main():
    intch = stackless.channel()
    intt = stackless.tasklet(ints)(2, intch)
    sieveo = stackless.channel()
    sievet = stackless.tasklet(sieve)(intch, sieveo)
    #printert = stackless.tasklet(printer)(sieveo)
    gathero = stackless.channel()
    gathert = stackless.tasklet(gather)(sieveo, gathero, 1e4)
    print len(gathero.receive())

    stackless.run()

if __name__ == "__main__":
    main()

Running this results in decent performance:

--( 9:22 : $)-- ~/local/stackless/bin/python --version
Python 2.6.2
--( 9:24 : $)-- time ~/local/stackless/bin/python sless.py
10000

real    0m47.369s
user    0m46.123s
sys     0m0.076s

I tried running the same example to find the millionth prime, but after hours of execution time I killed the process. While the memory consumption was reasonable, the execution time suffered from, what I believe to be, much overhead from object creation and switching between tasklets. Still, it’s encouraging that this version didn’t fail due to stack depth problems.

Stackless Python is an interesting experiment, and I wish the features implemented here were part of the official Python release. If you haven’t heard of Stackless before, I encourage you to read about what it offers to Python and why this can be useful beyond the toy example I have here.

Threads are evil.

There, I said it, and I stand by it too. If you find this sentiment a strange one, I suggest you start with a little reading.

read more »