Coder Legion - Recent posts tagged multiprocessing

Telegram Chatbot’s Evolution: Decoupling Parallel Python’s Shared State for Clarity

Tue, 03 Jun 2025 06:59:38 +0000

After building the intro animation last week,,) I finally took the time to make a video based on the article on jQuery as promised. In the future, articles that are accompanied by a video will all have a play button emoji ▶️ added to the title. While video is still made irregularly, we are going to meet here more regularly. This week, let’s revisit our chatbot project discussed not too long ago. It mostly works fine, but there’s one thing I want to improve on — removing the hardcoded global queue.

Illustration from copilot for the topic

The Global Headache: When a Simple Assignment Becomes a Core Problem

Spreading workload to run in parallel is a hard problem, especially when it comes to keeping them synchronized. Previously in our chatbot, we split the bot and webhook to run in 2 different processes, managed by the ProcessPoolExecutor. The whole architecture mimics my experimental bot named BigMeow ..) The project is definitely flawed, as it is a culmination of my learning on AsyncIO without a formal introduction to the topic.

Not too long ago I was going through a job interview. As part of the process, I was asked to work on a take-home assignment. It was unimaginative as ever, though I took the opportunity to implement an experimental background batch processing module. The whole design, again, is similar to the chatbot, so it works as expected.

Photo by Pascal van de Vendel on Unsplash

When I finally started to write tests to validate my work, that was when my nightmare began.

Though I hacked through the tests, there were multiple minor issues left unresolved after I submitted my work. The hardcoded queue in the settings module severely limits the flexibility and hacks like patches needed to be applied in tests. Despite rejecting the offer, the frustration lingered on my mind for quite a while. How can I do better to avoid spending endless hours fixing tests? Most of my code is rather testable, but there must be something I am missing. During the weekend, I finally took some time going through the code to prepare a new article, and a revelation came and it clicked.

What if I stop hard-coding the queue and pass it into each process explicitly, like the exit event?

With some help from my LLM chatbot, I managed to remove the hardcoded queue. Then I proceeded to pass it into the child processes explicitly, for both the background processing module, and the FastAPI web application. We would talk more about it, including test preparation, in the coming week. For now, let’s shift the focus back to our chatbot.

Exposing the Entanglement: Diving into the Original Architecture

Photo by Josh Redd on Unsplash

Let’s briefly review the current setup, as I try to recall why it was written this way. By understanding the original design decision, we can then start working out a solution, addressing the inflexibility that came with the hard-coded problem.

The reason for having a settings module was basically I needed a place to store some configuration globals. Over time, the list of configurable options piles up. Seeing how convenient it is, I eventually set up a queue to allow the web application process to pass incoming messages from the webhook endpoint to the process hosting the Telegram bot. At the time, it was just a simple queue I created from multiprocessing.Queue. Nothing fancy, just a boring process-safe queue that can be safely shared by multiple processes.

On the other hand, in the main process, we created a multiprocessing.Manager, and created an Event object with it. The created Event object is also process-safe, i.e. can be shared by multiple processes. That event object is then passed explicitly to each child process, to be listened to for a cue to exit.

The multiprocessing.Manager here acts almost like a coordinator, making sure processes can communicate effectively with each other whenever needed. Unfortunately, this means messages would be passing through an additional layer before hitting the target. Still the overhead is worth it, when it comes to queues. There is a risk of non-managed queue getting stuck in a shut down, especially when it is not empty. The Queue object created with the multiprocessing.Manager would handle it gracefully.

Delving deeper into multiprocessing.Manager isn’t the point of the article, and I am clearly not capable of doing it as of now. Just understand it is beneficial in orchestrating synchronization between processes, and we are going to utilize it in our Telegram bot.

From Global to Explicit: Refactoring with Centralized State

Knowing the limitation is one thing; next we are going to fix it. As we tackle the hard-coding problem, we will see how it becomes more flexible, while not breaking the inter-process communication.

Photo by U. Storsberg on Unsplash

Since we created a multiprocessing.Manager already to create an Event object, let’s extend it to also create the queue. Move the telegram_queue to our main module

import signal
import multiprocessing
from functools import partial

def shutdown_handler(
    _signum, _frame, exit_event: threading.Event
) -> None:
    exit_event.set()

manager = multiprocessing.Manager()
exit_event = manager.Event()
telegram_queue = manager.Queue() # MOVED telegram_queue here

for s in (signal.SIGHUP, signal.SIGTERM, signal.SIGINT):
    signal.signal(
        s, partial(shutdown_handler, exit_event=exit_event)
    )

with ProcessPoolExecutor() as executor:
    executor.submit(process_run, tg_run, exit_event)
    executor.submit(process_run, web_run, exit_event)

They can be passed into the child processes as-is, but I prefer to group them into an object (for reference, BigMeow now has 11 synchronization primitives, namely queues, locks, and events). Let’s create a simple dataclass for this:

from dataclasses import dataclass
from threading import Event
from queue import Queue

@dataclass
class SyncStore:
    exit_event: Event
    telegram_queue: Queue

Notice multiprocessing.Manager creates proxies of the synchronization primitives. They often have the same programming interface as existing objects, hence the type annotation specifies threading.Event and queue.Queue. Next we instantiate an object by

import multiprocessing

manager = multiprocessing.Manager()
sync_store = SyncStore(
    manager.Event(),
    manager.Queue()
)

After that, we adapt the changes in our main module, remember to also pass the new shiny sync_store into our child processes in the executor.submit call

for s in (signal.SIGHUP, signal.SIGTERM, signal.SIGINT):
    signal.signal(
        s, partial(shutdown_handler, exit_event=sync_store.exit_event)
    )

with ProcessPoolExecutor() as executor:
    executor.submit(process_run, tg_run, sync_store)
    executor.submit(process_run, web_run, sync_store)

There is more work for the FastAPI web application module, so we are doing it last, For now, we first modify the function signature of run() in our Telegram bot, and update the exit_event.wait() call accordingly

async def run(sync_store: SyncStore) -> None:
    async with application:
        ...

        await asyncio.to_thread(sync_store.exit_event.wait)

        ...

Previously, the coroutine function message_consume implicitly read from settings.telegram_queue. As it is now made available to run(), we can proceed to pass it as an argument. Also there’s no reason why the bot’s application object to be global, let’s move everything into the function:

# remove this line
# application = ApplicationBuilder().token("Your telegram token").build()

async def run(sync_store: SyncStore) -> None:
    application = ApplicationBuilder().token("Your telegram token").build()
    async with application:
        await setup(application) # pass the application object explicitly
        # Starting the bot
        await application.start()
        # Receive messages (pass the object explicitly)
        asyncio.create_task(loop_queue(message_consume, application, sync_store.telegram_queue))
        # Wait for exit signal
        await asyncio.to_thread(sync_store.exit_event.wait)
        # Stopping the bot
        await application.stop()

Both loop_queue and message_consume would need to adapt to the change

from typing import Any
from telegram.ext import Application
from queue import Queue

# Take additional args to be passed to the coroutine function
async def loop_queue(coro_func: Callable[[], Awaitable[None]], *args: Any) -> None:
    while True:
        await coro_func(*args)

async def message_consume(application: Application, queue: Queue) -> None:
    with suppress(queue.Empty):
        await application.update_queue.put(
            Update.de_json(
                # no more consuming from settings.telegram_queue
                await asyncio.to_thread(queue.get, timeout=10),
                application.bot,
            )
        )

Finally, applying the changes to the FastAPI web application module. Like the Telegram bot module, we first revise the run() function signature

async def run(sync_store: SyncStore) -> None:
    ...

Then, we will have to register our store to the app.state ..) Also update the uvicorn config accordingly as app is modified in our function.

async def run(sync_store: SyncStore) -> None:
    app.state.sync_store = sync_store

    server = uvicorn.Server(
        uvicorn.Config(
            app, # we made changes to global app, so specify explicitly
            host="0.0.0.0",
            port=80,
            log_level="info",
        )
    )

    ...

Usually this is done in the lifespan, where we yield an object, that can be later accessed through request.state in the handler functions, as shown below:

from fastapi import FastAPI, Request

@asynccontextmanager
async def lifespan(app: FastAPI):
    yield {'foo', 'bar'} # yield the object here

app = FastAPI(lifespan=lifespan)

@app.get('/')
async def index(request: Request):
    print(request.state.foo) # 'bar'

Unfortunately, in our case app is instantiated outside of run() and it has no knowledge of sync_store at the time. Considering now we just store it at app.state directly, we could still define a lifespan handler, to check for the presence of the store, at startup:

@asynccontextmanager
async def lifespan(app: FastAPI):
    # do a check when the application starts up to ensure sync_store is available
    if not (hasattr("sync_store", app.state) and isinstance(app.state.sync_store, SyncStore)):
        raise RuntimeError("SyncStore object is not found")

    yield

app = FastAPI(lifespan=lifespan)

Lastly, update the webhook, to use the queue from sync_store, via request.app.state.sync_store.telegram_queue

@app.post("/webhook/telegram", include_in_schema=False)
async def telegram_webhook(
    request: Request,
    x_telegram_bot_api_secret_token: Annotated[
        str, Header(pattern=settings.TELEGRAM_SECRET_TOKEN, strict=True)
    ],
) -> None:
    asyncio.create_task(
        asyncio.to_thread(
            # change how the queue is accessed
            request.app.state.sync_store.telegram_queue.put,
            await request.json()
        )
    )

Although the refactoring looks rather significant, the core logic underneath is unaffected. Apart from changing how the queue is accessed in both the Telegram and the FastAPI web application, there’s no change on how it is done. Data is still being produced by the web application via .put(), and consumed by the .get() in the bot.

Beyond the Fix: Testability, Scalability, and What’s Next

So back to the original objective of this refactoring — remove the hard-coded telegram_queue. Fortunately, we have done so successfully, though the process was somewhat involved. The biggest advantage of passing them explicitly; is how it improves testability. No more patching and replacing globals are needed, just pass the test queue to the function, or properly populate the app.state while creating FastAPI’s TestClient fixture.

Testing would be covered next week.

We also explored grouping all the synchronization primitives into one object. When the number of child processes increases, the number of such objects would accumulate inevitably. Even if we cherry-pick, ensuring only the relevant ones go into each child process, some process would end up taking too many arguments. Thus, creating an object containing them can be a good workaround to the problem.

Other than the rather involved gymnastics of moving things around, there’s no change to the underlying workflow. Still it brings consistency compared to the original implementation. Now all synchronization primitives are all defined in the same place, all managed by the multiprocessing.Manager, and are introduced to each child process explicitly as arguments. Additionally, the usage of multiprocessing.Manager makes it easy to scale the application as the manager can run as a server.

Photo by Sharon Co Images on Unsplash

This is it, the journey started from a very unassuming take-home assignment and in the end became a valuable lesson. On a high level it isn’t quite significant, as it doesn’t bring much optimization (if not slower due to the change to the queue managed by multiprocessing.Manager). Even so, by applying what I learned to my chatbot, it immediately became relevant as the bot is still running out there serving its purpose. Turning the experience into this article; is also how I reciprocate to the community.

Hmm, that was the third time my LLM Editor suggested I should use a semicolon ..)

Anyway, that’s all I have for this week, thanks for reading, and I shall write again, next week.

Just so you know, while an AI assistant helped me refine this article’s clarity and structure, the voice, all the technical content, and every line of code presented here are entirely mine. This piece reflects my personal experiences and insights. If you’re interested in job opportunities or collaborating on projects, please feel free to reach out via LinkedIn or connect with me right here on Medium.

Concurrency vs. Parallelism: Achieving Scalability with ProcessPoolExecutor

Tue, 08 Apr 2025 14:57:43 +0000

In our previous exploration, we dove deep into AsyncIO by writing a task scheduler. We discovered how IO-bound operations were scheduled to the event loop concurrently while building our task scheduler project. Building on what we’ve done, we will revisit the project and scale it to run CPU-bound operations.

More chefs, more dishes, illustration generated by Microsoft Copilot

How to write an AsyncIO Telegram bot in Python
Understanding Awaitables: Coroutines, Tasks, and Futures in Python
AsyncIO Task Management: A Hands-On Scheduler Project
Concurrency vs. Parallelism: Achieving Scalability with ProcessPoolExecutor

AsyncIO: A Foundation for Concurrency

Photo by John Barkiple on Unsplash

It is hard to avoid talking about concurrency in discussions about asynchronous programming. However, when it gets mentioned, often times it gets mixed up with parallelism. When Golang came out, people raved about how goroutines are efficient, and that was the first time I hear people discussing concurrency after my undergrad days. Rob Pike eventually delivered a talk aiming to distinguish the two concepts and it became quite popular among my peers.

While building our project last week, I mentioned event loops functions like a timetable where we slot in awaitables for execution. They would be scheduled to run one at a time, switching to another whenever one is done, or is busy doing IO-operations. For example, when we send multiple dex tasks, it would attempt to (order is not necessarily exact, just to illustrate the idea)

Start an HTTP request to the PokeAPI
Wait for the response for (1)
Start another HTTP request to the PokeAPI
Wait for the response for (3)
Continue the task started in (1), and compute the result
Start another HTTP request to the PokeAPI
Wait for the response for (6)
Continue the task started in (3), and compute the result
Continue the task started in (6), and compute the result

We can see the event loop improves the efficiency by keeping the CPU busy, while IO is happening in the background. This is an example of concurrency, in Pike’s word: dealing with a lot of things at once. In a way, it is like planning our time to minimize time waste on waiting in the kitchen. For example, we can proceed to cut ingredients for other dishes, while waiting for our pot of soup to boil. This allows us to concurrently prepare or cook multiple dishes at once.

So we now know we can achieve concurrency with AsyncIO for IO-bound operations. How about CPU-bound operations?

We are fortunate to live in an era where computing devices sports multicore CPU by default. Theoretically this should improve efficiency, but spreading work to multiple cores is a challenge. Continuing our cooking analogy, having an extra core or CPU is like having another kitchen and chef. Both chefs can now work together and get more dishes prepared at once. They can both choose to schedule their time wisely, to work concurrently as well.

Go’s goroutines can be scheduled to both run concurrently, or to be distributed to multiple CPU cores to run at once. In Python, we can achieve concurrency with AsyncIO. However, for parallelism, we would have to use tools like ProcessPoolExecutor in addition to AsyncIO. Let’s get started by continuing our example project last week by making fib run in parallel.

Harnessing Parallelism: ProcessPoolExecutor in Action

Photo by Becca Tapert on Unsplash

We usually opt to spawn a new thread, when we want to distribute work across the available cores. With threads, data can be shared among the main process and the spawn threads. However, it also means a race condition can occur when multiple threads are writing to the same variable. Hence, the global interpreter lock (GIL) was introduced, to limit the behaviour. While it protects the shared data, it makes threads inefficient as it depends on the lock.

Ensuring data is properly shared across processes and threads is therefore one of the biggest challenge in parallelism. There are ongoing efforts to deal with the limitations, or even removing GIL entirely in recent releases. But for now, most of the time we would just start new processes to distribute works to be done at once. In our case, let’s set up ProcessPoolExecutor in our repl function:

from concurrent.futures import ProcessPoolExecutor

async def repl() -> None:
    with (
        ProcessPoolExecutor() as executor,
        suppress(asyncio.CancelledError),
    ):
        ...

Previously, in our evaluator, we would spawn a new thread whenever we work on fib, through the use of asyncio.to_thread. We also know it is just an abstraction of loop.run_in_executor. Now that we are using our own process executor, let’s revert to use run_in_executor, with our own ProcessPoolExecutor

async def evaluate(…):
    match expression:
        ...

        case Expression(command=Command.FIB):
            return asyncio.get_running_loop().run_in_executor(
                executor, fib, expression.args[0]
            )

Now, whenever we issue a fib command, a new process is spawn instead of a thread to execute the (potentially) super slow CPU-bound operation. Starting a new process is considered more expensive, as we are pending resource allocation from the operating system. Sharing data and synchronization between processes also now require more effort, and we will discuss further about this later.

The loop.run_in_executor is essentially an abstraction of executor.submit call, where it takes function and the corresponding parameters. Therefore, in general, we would write it as

with ProcessPoolExecutor() as executor:
    future = executor.submit(fib, 100)

The returned future object, has a nearly identical design compared to the asyncio counterpart, sans the asynchronous magic.

While we are at it, let’s add a new command to send multiple fib operations, where the format is fib-multi [ …]. For instance, when we do fib-multi 100 200 300, it would schedule fib 100, fib 200 and fib 300 to run in parallel. For that, we add a new case in our evaluator

 async def evaluate(...):
    match expression:
        ...

        case Expression(command=Command.FIB_MULTI):
            return asyncio.gather(
                *[
                    asyncio.get_running_loop().run_in_executor(
                        executor, fib2, nth
                    )
                    for nth in expression.args
                ]
            )

When we use asyncio.gather, it takes all the awaitables passed as parameters and return a Future object. The result of the awaitables will be returned as a list when .result() is called. In our case, since we are setting the executor explicitly to a ProcessPoolExecutor, the futures are then scheduled to run in parallel.

And for that, we need to revise our job and dash to properly display the result, namely to handle cases where the result returned is a list.

def job(
    task: tuple[
        str,
        Event | None,
        asyncio.Task[str] | asyncio.Future[str] | asyncio.Future[list[str]],
    ],
) -> str:
    try:
        match task[-1]:
            case asyncio.Future():
                return (
                    "; ".join(task[-1].result())
                    if isinstance(task[-1].result(), list)
                    else task[-1].result()
                ) # type: ignore

            case _:
                return "Still waiting"

    except asyncio.InvalidStateError:
        return "Still waiting"

def dash(
    tasks: Sequence[
        tuple[
            str,
            Event | None,
            asyncio.Task[str] | asyncio.Future[str] | asyncio.Future[list[str]],
        ]
    ],
) -> str:
    result = [
        "\t".join(("id", f"{'command':16s}", "done?", "result")),
        "\t".join(("==", f"{'=' * 16}", "=====", "======")),
    ]

    for num, (line, _, task) in enumerate(tasks, 1):
        task_done, task_result = False, None

        if task_done := task.done():
            task_result = (
                "; ".join(task.result())
                if isinstance(task.result(), list)
                else task.result()
            )

        result.append(
            "\t".join(
                (
                    str(num),
                    f"{line:16s}",
                    str(task_done),
                    task_done and str(task_result)[:16] or "",
                )
            )
        )

    return "\n".join(result)

We can now issue an actual fib-multi command, and check our system monitor to see the operating system does indeed allocate multiple processes to do the job.

A screen-capture from my system monitor

On the other hand, dex is still unaffected, and would continue to work concurrently despite the changes introduced to fib. This exercise show how it is possible to introduce parallelism into a concurrent program.

Managing Processes: Cancellation and Communication

Photo by Mark Boss on Unsplash

A big drawback of starting a new process is that communication takes a lot of effort, and we lose some control over it. For instance, if we create a task in asyncio, we can cancel the task whenever needed with .cancel(). However, despite the ProcessPoolExecutor provides a .cancel() method as well, it does not always work for scheduled futures that are in progress.

One way to workaround the problem is through event objects. In our example, we will need 2 types of events, one for task cancellation, and another for application exit. Let’s get fib to subscribe to both of them.

def fib(nth, exit_event=None, cancel_event=None):
    ...

    for i in range(1, nth + 1):
        if (exit_event and exit_event.is_set()) or (
            cancel_event and cancel_event.is_set()
        ):
            return "Cancelled"

        ...

The key is to stop the loop, whenever either of the events is set. In our case, would be either user is exiting the program with quit, or cancelling the job with kill . Now we move on to the evaluator to ensure both these events are sent to fib.

from multiprocessing.managers import SyncManager

async def evaluate(
    …
    executor: ProcessPoolExecutor,
    exit_event: Event,
    manager: SyncManager,
    expression: Expression,
):
    match expression:
        ...

        case Expression(command=Command.FIB):
            cancel_event = manager.Event()
            return cancel_event, asyncio.get_running_loop().run_in_executor(
                executor, fib, expression.args[0], exit_event, cancel_event
        )

        case Expression(command=Command.FIB_MULTI):
            cancel_event = manager.Event()
            return cancel_event, asyncio.gather(
                *[
                    asyncio.get_running_loop().run_in_executor(
                        executor, fib, nth, exit_event, cancel_event
                    )
                    for nth in expression.args
                ]
            )

The application exit_event is going to be applicable for all fib operations, so we are requiring it in the parameters. On the other hand, a cancel_event is only relevant for a particular job, hence it is created just for it. Then we set up the global exit_event in repl().

import multiprocessing

async def repl() -> None:
    with (
        ProcessPoolExecutor(max_workers=5) as executor,
        suppress(asyncio.CancelledError),
    ):
        manager = multiprocessing.Manager()
        exit_event = manager.Event()
        …

We create Event() object through the multiprocessing.Manager() object because it is generally more reliable, especially we are passing the object between processes.

The ShutdownHandler would require a little bit of revision so it is aware of the exit_event.

@dataclass
class ShutdownHandler:
    exit_event: Event
    signal: int | None = None

    async def __call__ (self) -> None:
        logger.info("Shutting down tasks. signal:%s", self.signal or "")

        self.exit_event.set()

        ...

Now we can prompt fib, which is our CPU-bound task to return when the application exits. The changes in the handler, also require init() function to take exit_event as parameter

def init(exit_event: Event) -> None:
    loop = asyncio.get_running_loop()

    for sig in (signal.SIGTERM, signal.SIGHUP, signal.SIGINT):
        handler = ShutdownHandler(exit_event, sig)
        loop.add_signal_handler(sig, lambda: asyncio.create_task(handler()))

    loop.set_exception_handler(ExceptionHandler(ShutdownHandler(exit_event)))

Now the application exit part is done, let’s move on to task cancellation. In the evaluator, we see the CPU-bound jobs also return the cancel_event, so we need to properly store them in our list of tasks in repl()

async def repl() -> None:
    ...

                match result := await thread_last(
                    line,
                    (tokenize, t),
                    (parse, g),
                    (
                        evaluate,
                        client,
                        executor,
                        exit_event,
                        manager,
                        tasks,
                        ShutdownHandler(exit_event),
                    ),
                ):
                    case (_, asyncio.Future()):
                        tasks.append((line, *result))
                        print(f"Task {len(tasks)} is submitted")

                    case asyncio.Task():
                        tasks.append((line, None, result))
                        print(f"Task {len(tasks)} is submitted")

                    case _:
                        print(result)

And the corresponding kill function is now

def kill(task):
    match task[-1]:
        case asyncio.Task():
            task[-1].cancel()

        case asyncio.Future():
            assert task[1]

            task[1].set()

    return f'Task "{task[0]}" is killed'

Note that asyncio.Task is a subclass of asyncio.Future, so the case order matters.

The Power of the Hybrid Approach

Photo by Elena Mozhvilo on Unsplash

In our example, we can use ProcessPoolExecutor to introduce parallelism to our application. This allows us to scale CPU-bound operations so that they can be distributed to other cores of the CPU, allowing them to be executed simultaneously. On the other hand, IO-bound operations are still scheduled to run concurrently.

Remember the cooking analogy we mentioned earlier? Doing things concurrently is very similar to cooking. It is possible to prepare multiple dishes at a time with clever time management. While we wait for our chicken in the oven, we can go stir-fry whatever that is in the pot.

On the other hand, doing it in parallel means adding a new chef (and maybe even another kitchen). Adding a new chef, would definitely increase throughput, allowing more dishes to be done at once. On the other hand, either or both of them can still be working concurrently, increasing efficiency. In our case, it would be starting separate processes, and each manages their own event loop to take on tasks.

I hope I am doing a good enough job in this tour of AsyncIO and asynchronous programming. I certainly learned a lot throughout these few weeks, revisiting the documentation, and through building the example project. The updated project can still be accessed on GitHub ,,) with week1 and week2 code separated, allowing comparison. Thank you again for your time, and I shall write again, next week.

This article was enhanced by the editorial assistance of a large language model, which provided valuable feedback on clarity, flow, and adherence to writing principles. While the core ideas and code remain my own, the LLM’s suggestions significantly contributed to the overall quality and readability of the piece. I am always open to collaboration on interesting projects and exploring job opportunities. Feel free to connect with me here on Medium or via my LinkedIn profile to discuss further.

Multithreading and Multiprocessing Guide in Python

Fri, 07 Jun 2024 22:41:00 +0000

Have you ever wondered why your Python application lags when handling large volumes of data or numerous user requests simultaneously? Or how to run multiple operations in Python without having them interfere with each other? The solution is concurrent execution that helps to execute parallel tasks in Python and scale with increasing demand. In modern computing, concurrent execution ensures that an application performs multiple operations simultaneously. It is crucial to enhance the efficiency and performance of the application. We can achieve this through the multiprocessing and multithreading concepts of Python. The multithreading technique is bound with I/O-related tasks facilitated by the Python threading module, whereas the multiprocessing technique is related to parallelism, which allows multiple tasks to run in parallel on distinct processes.
Therefore, in this article, we will learn in detail about the fundamentals of concurrency and how you can do it through multithreading and multiprocessing techniques in Python. So, let's get started.

Understanding Concurrency

1. Definition Of Concurrency And Its Significance In Programming

Concurrency means that an application is making progress on more than one task seemingly simultaneously. However, since the CPU can only execute one program at a time, it cannot execute more than one task at the same time; instead, it executes one task for some specific time then switches to another task and executes it for some time and so on. These tasks are called threads, and the switching between them is unnoticeable by the user, which shows the concurrent execution.

2. Difference Between Concurrency And Parallelism

3. Importance Of Concurrency In Software Performance And Responsiveness

In modern computing, concurrency is the fundamental concept of Python that is crucial to know for developing an application because it not only improves the efficiency of the application but makes it more responsive. The importance of concurrent execution lies in its ability to handle multiple tasks simultaneously. There is no need to wait to get a response from the server or send a response to the web request and perform the other task. We can now achieve this through concurrent execution, which can significantly enhance the responsiveness and speed of the application.

Note: Remember that task, thread, and process are relatable but different terms in the context of concurrency. As task is the unit of work, thread is the smallest unit of execution within a process, and process is the instance of a program running in a computer having its own memory and resources.

Multithreading in Python

1. Introduction To Multithreading

Multithreading is a technique for achieving concurrency within a single process. It allows you to run different parts of your process simultaneously. These different parts, known as threads, are the single unit of execution, within the process. They share the same resources and run in parallel but independently of each other. We call them multithreading or multiple threading when multiple threads work together to run within a single process.

Note: Thread is a sequence of instructions in a program that can be executed independent of the remaining program.

2. Global Interpreter Lock (GIL) And Its Impact On Multithreading

Now, moving toward the execution of the thread, let's understand the Global interpreter lock. GIL is a mechanism of applying a global lock on the Python interpreter. This lock ensures the execution of a single thread at a time. However, this mechanism lacks the execution of multiple threads. What happened is that all the threads try to acquire the memory at once because of single thread execution restriction, which causes the data override problem in memory. Even if your system processor has multiple cores, it still allows only single-thread execution one at a time. The reason lies in the thread working mode acquiring the GIL and, when needed for any I/O operation, releasing the lock so other threads can work and so on. Hence, this way GIL impacts the working of multithreading by limiting the thread operations, restricting the parallel execution, and ensuring that only one single thread will run in the interpreter at a time.

Note: Remember, GIL is only supported in CPython interpreter.

3. Illustration Of Creating And Managing Threads Using The 'threading' Module

In Python, we can perform multithreading using the built-in threading module. This module is directly imported into the program and does not require further installation. Additionally, threads are represented as objects in Python. These thread objects can execute functions that work with data, be stored in iterables (data structures), and be passed as parameters to function. Now, let's take an example and understand the threading module.

import time #importing time module
import threading  #importing threading module

def calc_square(Numbers): #create first thread task
    print("Calculate Square Numbers")
    for n in Numbers:
        time.sleep(0.2) #To pause execution for checking the total time taken
        print("Square:", n*n)


def calc_cube(Numbers): #create second thread task
    print("Calculate Cube of Numbers")
    for n in Numbers:
        time.sleep(0.2) #To pause the execution for checking total time taken
        print("Cube:", n*n*n)

arr = [2,4,7] #array of numbers
t = time.time()

#creating the threads
t1 = threading.Thread(target=calc_square, args = (arr,)) #thread 1
t2 = threading.Thread(target=calc_cube, args = (arr,)) #thread 2

#starting the threads
t1.start()
t2.start()

#waiting for the threads to complete
t1.join()
t2.join()

print("Thread has completed its execution in :", time.time() - t)

Tip: After creating a thread, it does not start executing until you call its start() method. Additionally, you need to wait for a thread to finish its task before the main program can continue or terminate. This is done using the join() method.

Output

Note: Keep in mind that Python's GIL allows only one thread to execute at a time, even in a multithreaded architecture. It means threads are not truly concurrent when executing CPU-bound tasks.

Synchronization and Communication Between Threads

1. Synchronization And Communication Techniques For Coordinating Threads

If multiple threads execute concurrently, there is a possibility of race or deadlock conditions occurring. Managing shared resources and coordination between threads within the process is essential to avoid these conditions. They tried to access and modify the shared data, causing the data override or inconsistency problem, and sometimes threads lead to starvation and live lock conditions. Therefore, the Python threading module has provided many techniques that follow strict synchronization between coordinating threads.

2. Thread Synchronization Primitives Like Locks, Semaphores, And Barriers

To overcome these concerns, Python provides various tools for thread synchronization to ensure that threads execute in a controlled manner. We can achieve thread synchronization using the threading module. Using the "threading" module, we can achieve thread synchronization, which allows multiple threads to work together without interfering with each other, especially when they need to share data or resources. The most common primitives are:

1. Lock

It is the most common mechanism for thread synchronization provided by Python. By using a lock mechanism, only one thread will enter a critical block of code at a time. It provides prevention against data corruption and inconsistencies.

import threading #import thread library
import time #import time library for measuring total time taken

# Create a lock object
lock = threading.Lock()

# Function that simulates a task using a lock
def task_with_lock(thread_ID):
    lock.acquire()  # Acquire the lock before accessing the shared resource
    try:
        print(f"Thread {thread_ID} has entered the critical section")
        time.sleep(1)  # Simulate work
    finally:
        lock.release()  # Release the lock when done
        print(f"Thread {thread_ID} is leaving the critical section")

# Create multiple threads to demonstrate lock usage
ThreadsList = [threading.Thread(target=task_with_lock, args=(i,)) for i in range(3)]
#starting the threads
for Thread in ThreadsList:
    Thread.start()
#waiting for the threads to complete
for Thread in ThreadsList:
    Thread.join()

Output

2. Semaphores

Semaphore is the advanced mechanism for thread synchronization. This mechanism limits the number of threads that can access a particular resource or all the resources. Because, sometimes, we want the following number of threads to access the resources e.g. only four members are allowed to access the database at a time. Therefore, we just specify how many threads we want to run concurrently.

import threading #import thread library
import time #import time library for measuring total time taken

# Create a semaphore with 3 threads 
Semaphore = threading.Semaphore(3)

# Function to access a limited resource
def access_resource(thread_ID):
    with Semaphore:
        # Accessing the shared resource
        print(f"Thread {thread_ID} is accessing the resource")
        time.sleep(1)  # Simulate resource usage
        print(f"Thread {thread_ID} is releasing the resource")

# Create multiple threads to demonstrate semaphore usage
ThreadsList = [threading.Thread(target= access_resource, args=(i,)) for i in range(3)]
#starting the threads
for Thread in ThreadsList:
    Thread.start()
#waiting for the threads to complete
for Thread in ThreadsList:
    Thread.join()

Output

Note: Remember, args=(i,) is a tuple that contains the arguments to pass to the target function. We can pass as many arguments as required to the targeted function to run by using ', '.

3. Barriers

One of the primitives of thread synchronization is a barrier that ensures all the threads wait for each other to reach a certain point of execution before continuing. This means that these threads completed their specific tasks before synchronizing their execution.

import threading #import thread library

# Create a barrier for two threads
barrier = threading.Barrier(2)

# Function that reaches a synchronization point
def synchronize_point(Thread_ID):
    print(f"Thread {Thread_ID} is working...")
    barrier.wait()  # Wait for all threads to reach this point
    print(f"Thread {Thread_ID} passed the barrier")

# Create multiple threads to demonstrate semaphore usage
ThreadsList = [threading.Thread(target= synchronize_point, args=(i,)) for i in range(2)]
#starting the threads
for Thread in ThreadsList:
    Thread.start()
#waiting for the threads to complete
for Thread in ThreadsList:
    Thread.join()

Output

Multiprocessing in Python

1. Introduction To Multiprocessing

Multiprocessing is another technique for achieving true parallelism by utilizing multiple processes. It enables the execution of multiple processes by allowing them to process concurrently on multi-core systems. Additionally, the working of each process is independent of the other, which can significantly reduce the time required compared to performing the task in a single process. Python provides us with a built-in module named "multiprocessing" which creates a separate process, each having its interpreter and memory space.

Tip: Process is an instance of a program that consists of at least one thread called the main thread.

2. Difference Between Multithreading And Multiprocessing

3. Illustration Of Creating And Managing Processes Using The 'Multiprocessing' Module

Now, let's understand how the multiprocessing module works in Python with a simple example where we want to calculate the square of each number in the array in one function and the cube of each number in the second function. Then, we will use multiple processing to execute them separately as an individual process.

import time #importing time module
import multiprocessing  #importing multiprocessing module

def calc_square(Numbers): #create first process task
    print("Process 1 started Woring")
    print("Calculate Square Numbers")
    for n in Numbers:
        time.sleep(0.2) #To pause execution for checking the total ime taken
        print("Square:", n*n)


def calc_cube(Numbers): #create second process task
    print("Process 2 started Woring")
    print("Calculate Cube of Numbers")
    for n in Numbers:
        time.sleep(0.2) #To pause execution for checking total time taken
        print("Cube:", n*n*n)

if __name__ == '__main__':
    arr = [2,4,7] #array of numbers
    t = time.time()
    
    #creating the Processes
    Process1 = multiprocessing.Process(target=calc_square, args = (arr,)) #process 1
    Process2 = multiprocessing.Process(target=calc_cube, args = (arr,)) #process 2
    
    #starting the processes
    Process1.start()
    Process2.start()
    
    #waiting for the processes to complete
    Process1.join()
    Process2.join()
    
    print("Both Processes has completed its execution in :", time.time() - t)

Output

Caution: Remember to implement the main function under if __name__ == "__main__" or otherwise the multiprocessing module will complain. It ensures that Python finishes analyzing the program before the sub-process is created.

Wrapping Up

In conclusion, we discussed how Python programs become efficient if they work concurrently. The main point of concurrent execution is to overcome the deadlock condition by allowing the application to perform multiple tasks concurrently. It improves the performance of the application by utilizing the maximum CPU resources. Not only that, but it also enhances the responsiveness of the application. We also learned about the two mechanisms of achieving concurrent execution, i.e. multithreading and multiprocessing. Each has its own use case and can be helpful in different cases. So, when working with tasks that require I/O operations multithreading is efficient; otherwise, multiprocessing is ideal for CPU-bound tasks. As a developer, don't just ignore these mechanisms, as they significantly affect your application performance. I hope this guide is helpful for you. Thank you for reading this guide. Happy Coding!

Additional Reference

Concurrency in Python: Detailed Documentation
Multiprocessing module: A process based parallelism Python Documentation
Threading module: A thread based parallelism Python Documentation