Testing the performance of Python with and without GIL

Mon, 25 Nov 2024 03:51:53 +0000

Since Python 3.13, the famous Python GIL is optional. This is a long requested functionality but, is the performance gain of threaded code real?

Let's test the performance of a threaded application with and without the GIL.

What is GIL?

GIL stands for Global Interpreter Lock, and is a mechanism only present in the standard Python implementation (CPython) that ensures the coherence and safety of Python internal and runtime objects. The GIL makes sure that only a single thread is executed at the same time, losing the benefits of a multicore environment.

The new versions make this security mechanism entirely optional, opening whole word of possible optimizations in multicore environments.

NOTE: this functionality is not yet available in the official releases of Python 3.14, but it will in the future.

Installing Python 3.14

Use these commands to install Python 3.14 in Ubuntu:

add-apt-repository ppa:deadsnakes/ppa
apt install python3.14-full python3.14-nogil

After the installation, we can test the installation with:

python3.14 --version

Great, now we have a working Python 3.14 installation. Now, we can create a virtual environment with:

python3.14 -m venv ~/venv314
source ~/venv314/bin/activate

Now we are ready to execute our tests!

Testing the same code with and without GIL

We have coded this application:

import time
import threading
import math

def test_function(num_thread):
    thread_start_time = time.time()
    print(F"Executing thread #{num_thread}...")
    math.factorial(250000)
    thread_execution_time = time.time() - thread_start_time
    print(F"...thread #{num_thread} finalized in {thread_execution_time} seconds")

print("Initializing thread test...")
start_time = time.time()

# Create multiple threads
threads = []
for num_thread in range(5):
    thread = threading.Thread(target=test_function, args=(num_thread,))
    thread.start()
    threads.append(thread)

# Wait for all threads to finish
for thread in threads:
    thread.join()

execution_time = time.time() - start_time
print(f"Finalized! Execution time: {execution_time} seconds")

Is a really simple application that spawns some threads and performs a CPU intensive task in each thread (a factorial). Now, we can execute the test code in each version, with and without GIL:

With GIL enabled:

PYTHON_GIL=1 python3.14-nogil test.py

Initializing thread test...
Executing thread 0...
Executing thread 1...
Executing thread 2...
...thread 0 finalized in 1.6720194816589355 seconds
...thread 1 finalized in 1.6916308403015137 seconds
Executing thread 3...
...thread 2 finalized in 1.8445136547088623 seconds
Executing thread 4...
...thread 3 finalized in 0.9945917129516602 seconds
...thread 4 finalized in 1.0793228149414062 seconds
Finalized! Execution time: 4.599931955337524 seconds

And without GIL:

PYTHON_GIL=0 python3.14-nogil test.py

Initializing thread test...  
Executing thread 0...
Executing thread 1...
Executing thread 2...
Executing thread 3...
Executing thread 4...
...thread 0 finalized in 0.8833539485931396 seconds
...thread 2 finalized in 0.8942356109619141 seconds
...thread 3 finalized in 0.934593677520752 seconds
...thread 4 finalized in 1.2945129871368408 seconds
...thread 1 finalized in 1.2981431484222412 seconds
Finalized! Execution time: 1.3003475666046143 seconds

Wow! The performance gain is notable!

A brilliant future

The possibility of running Python code without GIL with the default Python interpreter opens a lot of optimization opportunities to the Python ecosystem. We can expect an improved Django, a faster FastAPI, more optimized ASGI or WSGI servers...

Let's wait for this future together!

About the list

Among the Python and Docker posts, I will also write about other related topics, like:

Software architecture
Programming environments
Linux operating system
Etc.

If you found some interesting technology, programming language or whatever, please, let me know! I'm always open to learning something new!

About the author

I'm Andrés, a full-stack software developer based in Palma, on a personal journey to improve my coding skills. I'm also a self-published fantasy writer with four published novels to my name. Feel free to ask me anything!