Python is an amazing language and (by now, given well-crafted code) is also performing really well in terms of speed. If we, however, need some serious speed boost, Python allows us to use good old C in our programs and scripts. Put very simply, if you can do something in C you will be able to do it in Python.

Traditionally, we would probably write a C extension for CPython. While this approach still is widely used and has its benefits, since 2006 ctypes is available in Pythonland.

Python’s ctype library provides C compatible data types and allows us to call functions in DLLs (Dynamic Link Library; Windows) or shared libraries with ease. Compared to C exentions, ctypes often is easier to implement and (often) does require less C programming since we can easily use existing C functions. That being said, a ctype wrapper is usally still much slower than pure C / C extensions.

Before looking at two simple examples we should think about x86 calling conventions for a second. Without going into any detail, we should know that there are (at least) cdecl (WINAPIV) and stdcall (WINAPI) available. While the first one (the default option for C/C++) does not support variable lists of arguments, the second one does. Also, stdcall is the de facto standard for Microsoft Win32.

Example 1: Creating a Little GUI Window (Windows)

While there are many very good libraries for GUI programming (PyQt, wxPython, etc.), it’s fun to do it fully manually.

In Windows, the go to API for this kind of task is the almighty User32.dll. Looking at MSDN reveals that there is a MessageBox function within the API that seems very promising. The function takes four arguments: hWnd, lpText, lpCaption, and uType. Since ownership (hWnd) is not of our concern right now, the first argument is irrelevant - the others are rather self-explanatory. Let’s go!

import ctypes

label = ctypes.c_char_p('A Message Box')
message = ctypes.c_char_p('...with a message!')

ctypes.windll.user32.MessageBoxA(0, message, label, 0x00000000) # we need to do an stdcall, hence windll.

User32.dll MessageBox Example

As we can see, the API returns a return value with which we can work now. In case of this simple ‘OK dialogue’, pressing OK will just result in 1. With the more complex dialogues, there are more, finely nuanced, return values.

Example 2: Using a C Function in Python

Let’s move on to a more interesting Linux (Unix) example. In the beginning I said we would use this to speed up a Python script - that’s what we’re going to do.

Using the same concept as above, we will replace a Python function with a faster alternative of said function written in C. If there are no premade C functions available, we have to write our own shared library containing the new (replacement) function first. Afterwards we can call this new function from within Python.

Does this really work and increase speed? In order to test this, I implemented the same (stupid) functionality in both pure Python and in C. The function counts down from 1000000 to 0 and prints the current number. In other words: It’s simply a loop and some standard output.

I measured the runtime of both variants in order to figure out wheter the C variant runs faster. Based on eight runs (using time), the C implementation clearly won. In terms of just user mode, it ran three times faster. Based on sys time, it ran 20% faster. Such an increase in execution (CPU) time could, given specific circumstances, really make a difference. Also, we can expect even larger gains if it comes to computationally more complex problems.

forlib.c

#include <stdio.h>

void test_loop(void);

void test_loop()
{
    int i;
    int remaining;
    for (i=0; i<1000000; i++) {
        remaining = 1000000 - i;
        printf("%d\n", remaining);
    }
}

//gcc -shared -Wl,-soname,forlib -o forlib.so -fPIC forlib.c

test-ctype.py

import ctypes

forlib = ctypes.CDLL('/root/ctypes/forlib.so') # we need to do an cdecl call, hence CDLL.
forlib.test_loop()

test-python.py

for i in range(1000000):
    remaining = 1000000 - i
    print(remaining)