<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
<channel>
<title>Coder Legion - Recent posts tagged bullmq</title>
<link>https://coderlegion.com/tag/bullmq</link>
<description>Powered by Question2Answer</description>
<item>
<title>Why I Used BullMQ + Redis for Background Jobs in My NestJS Application — and When I Wouldn't</title>
<link>https://coderlegion.com/29737/why-i-used-bullmq-redis-for-background-jobs-in-my-nestjs-application-and-when-i-wouldnt</link>
<description>&lt;p&gt;While working on &lt;strong&gt;Waitless&lt;/strong&gt;, a real-time queue management platform, I came across a backend problem that looked simple at first:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;A customer reaches the front of the queue, and the system needs to send notifications to other customers.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The first instinct could be to simply send the email inside the HTTP request.&lt;/p&gt;
&lt;p&gt;But once I looked at the problem as a backend architecture problem rather than just an implementation task, I realized that sending the email and handling the user's request are two different workloads.&lt;/p&gt;
&lt;p&gt;That led me to &lt;strong&gt;BullMQ + Redis&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The interesting part wasn't learning the API. The interesting part was understanding &lt;strong&gt;why this architecture made sense, what Redis was actually doing, and when I would choose something else instead.&lt;/strong&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;the-problem-in-waitless&quot;&gt;The problem in Waitless&lt;/h3&gt;
&lt;p&gt;Waitless allows customers to join a digital queue and track their position without creating an account.&lt;/p&gt;
&lt;p&gt;When one customer is called, another customer may need to receive a notification such as:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;“It's your turn.”&lt;/li&gt;
&lt;li&gt;“You're almost next.”&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I already had the queue state stored in PostgreSQL. The question was:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Should the API send the email immediately, or should it create background work?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;A synchronous approach could look like this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;HTTP request
     ↓
NestJS
     ↓
PostgreSQL
     ↓
Send email
     ↓
HTTP response
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The problem is that sending an email depends on an external service.&lt;/p&gt;
&lt;p&gt;If that service is slow or temporarily unavailable, my API request is affected.&lt;/p&gt;
&lt;p&gt;I don't necessarily need the email to be completed before responding to the user.&lt;/p&gt;
&lt;p&gt;What I actually need is:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;“Remember that this email has to be sent, and process it separately.”&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That is exactly the kind of problem a job queue is designed to solve.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;introducing-bullmq&quot;&gt;Introducing BullMQ&lt;/h2&gt;
&lt;p&gt;BullMQ provides a job queue abstraction for Node.js applications.&lt;/p&gt;
&lt;p&gt;Instead of doing the work immediately, the application creates a job:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Application
     ↓
   Queue
     ↓
    Job
     ↓
   Worker
     ↓
 Actual work
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The application that creates the job is the &lt;strong&gt;producer&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The worker that processes the job is the &lt;strong&gt;consumer/worker&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;This gives me a clean separation between:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Creating work
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;and:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Executing work
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That separation becomes particularly useful when the work is slow, retryable, delayed, or needs to be processed independently from the HTTP request.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;where-redis-comes-in&quot;&gt;Where Redis comes in&lt;/h2&gt;
&lt;p&gt;One thing I initially had to clarify for myself was the relationship between BullMQ and Redis.&lt;/p&gt;
&lt;p&gt;They are not the same thing.&lt;/p&gt;
&lt;p&gt;A simplified mental model is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;NestJS
   ↓
BullMQ
   ↓
Redis
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;BullMQ is the queue system.&lt;/p&gt;
&lt;p&gt;Redis is the backend that BullMQ uses to store and coordinate queue-related state.&lt;/p&gt;
&lt;p&gt;In my NestJS application, I configure the connection like this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;BullModule.forRoot({
  connection: {
    host: 'localhost',
    port: 6379,
  },
});
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Port &lt;code&gt;6379&lt;/code&gt; is Redis's default port, so this configuration tells BullMQ which Redis instance to connect to.&lt;/p&gt;
&lt;p&gt;The important distinction is:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;BullMQ is not “running on port 6379.”&lt;br&gt;
BullMQ is connecting to the Redis server running there.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id=&quot;creating-a-notification-job&quot;&gt;Creating a notification job&lt;/h2&gt;
&lt;p&gt;In Waitless, I inject the notification queue into my service:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;@InjectQueue('notifications')
private notifQueue: Queue
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now I can add jobs to that queue.&lt;/p&gt;
&lt;p&gt;For example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;if (next.email) {
  await this.notifQueue.add('your-turn-now', {
    email: next.email,
    name: next.name,
    queueName: queue.name,
  });
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This does not mean:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;“Send the email now.”&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;It means:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;“Create a job called &lt;code&gt;your-turn-now&lt;/code&gt; and put it into the notifications queue.”&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The payload contains the information the worker will need later.&lt;/p&gt;
&lt;p&gt;Conceptually:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;NestJS
   ↓
notifQueue.add(...)
   ↓
BullMQ
   ↓
Redis
   ↓
notifications queue
   ↓
your-turn-now job
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The actual email is the responsibility of the worker.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;delayed-jobs&quot;&gt;Delayed jobs&lt;/h2&gt;
&lt;p&gt;One of the features I needed was delayed notification.&lt;/p&gt;
&lt;p&gt;If another customer is second in line, I want to notify them shortly before their turn.&lt;/p&gt;
&lt;p&gt;My implementation looks like this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;const secondInLine =
  await this.entryRepo.findSecondWaiting(queueId, next.id);

if (secondInLine?.email) {
  await this.notifQueue.add(
    'your-turn-soon',
    {
      email: secondInLine.email,
      name: secondInLine.name,
      position: 2,
      queueName: queue.name,
      trackingUrl: `http://localhost:3000/track/${secondInLine.token}`,
    },
    {
      delay: 30_000,
    },
  );
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The important part is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;{ delay: 30_000 }
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The job doesn't need to be processed immediately.&lt;/p&gt;
&lt;p&gt;The architecture becomes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Customer becomes second in line
            ↓
      Create BullMQ job
            ↓
       Delayed job
            ↓
          30 sec
            ↓
        Worker picks it up
            ↓
        Send notification
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is much more reliable than simply doing:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;setTimeout(() =&amp;gt; {
  sendEmail();
}, 30_000);
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A timer belongs to the application process itself. A queue gives the work an explicit lifecycle outside the original request.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;why-not-just-use-postgresql-for-the-queue-&quot;&gt;Why not just use PostgreSQL for the queue?&lt;/h2&gt;
&lt;p&gt;This was one of the more interesting questions I explored.&lt;/p&gt;
&lt;p&gt;At first glance, PostgreSQL looks like a perfectly reasonable choice.&lt;/p&gt;
&lt;p&gt;I could create a table such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;jobs
-------------------
id
type
payload
status
created_at
processed_at
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;and continuously insert and delete jobs.&lt;/p&gt;
&lt;p&gt;The problem is that the queue workload has different characteristics from my core business data.&lt;/p&gt;
&lt;p&gt;My PostgreSQL database is responsible for things such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;businesses
queues
customers
queue entries
relationships
transactions
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Those are my core relational business data.&lt;/p&gt;
&lt;p&gt;The queue workload is different:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;job created
job becomes active
job succeeds
job fails
job retries
job is removed
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is a &lt;strong&gt;high-churn operational workload&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;With PostgreSQL's MVCC model, repeatedly inserting and deleting rows creates dead tuples that later have to be cleaned up by VACUUM.&lt;/p&gt;
&lt;p&gt;That doesn't mean PostgreSQL cannot implement queues. It absolutely can.&lt;/p&gt;
&lt;p&gt;The real architectural question is:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Do I want the queue workload competing with my transactional business workload inside the same database?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;For a small application, the answer might be yes.&lt;/p&gt;
&lt;p&gt;For a higher-throughput system, separating those workloads can become valuable.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;why-redis-made-sense-for-my-queue&quot;&gt;Why Redis made sense for my queue&lt;/h2&gt;
&lt;p&gt;Redis is designed around very fast operations on in-memory data structures.&lt;/p&gt;
&lt;p&gt;That makes it a natural fit for workloads such as:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;caching&lt;/li&gt;
&lt;li&gt;counters&lt;/li&gt;
&lt;li&gt;temporary state&lt;/li&gt;
&lt;li&gt;queues&lt;/li&gt;
&lt;li&gt;rate limiting&lt;/li&gt;
&lt;li&gt;pub/sub&lt;/li&gt;
&lt;li&gt;fast lookups&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For my notification queue, the important characteristics were:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Asynchronous
High frequency
Short-lived job state
Low latency
Retries
Delayed execution
Worker concurrency
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;So instead of:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;PostgreSQL
   ├── business data
   └── queue workload
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I could separate them:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;PostgreSQL
     ↓
Business state


Redis
     ↓
BullMQ
     ↓
Background jobs
     ↓
Workers
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That separation was more important to me than simply saying:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;“Redis is faster.”&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id=&quot;what-i-learned-about-workload-driven-architecture&quot;&gt;What I learned about workload-driven architecture&lt;/h2&gt;
&lt;p&gt;This changed the way I think about databases.&lt;/p&gt;
&lt;p&gt;The question shouldn't simply be:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Which database is better?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;A better question is:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;What workload am I dealing with?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;For example:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;&lt;th&gt;Requirement                    &lt;/th&gt;&lt;th&gt;Possible choice&lt;/th&gt;&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;&lt;td&gt;Relational transactional data  &lt;/td&gt;&lt;td&gt;PostgreSQL&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Very fast temporary state      &lt;/td&gt;&lt;td&gt;Redis&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Background jobs with retries   &lt;/td&gt;&lt;td&gt;BullMQ + Redis&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Document-oriented event data   &lt;/td&gt;&lt;td&gt;MongoDB&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Large event streams and replay &lt;/td&gt;&lt;td&gt;Kafka&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Small simple background jobs   &lt;/td&gt;&lt;td&gt;PostgreSQL or a scheduler&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Managed cloud queue            &lt;/td&gt;&lt;td&gt;SQS or similar service&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;These aren't absolute rules.&lt;/p&gt;
&lt;p&gt;They are starting points for architectural reasoning.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;bullmq-gives-me-more-than-a-queue&quot;&gt;BullMQ gives me more than a queue&lt;/h2&gt;
&lt;p&gt;Another thing I learned is that the real value of a job system is not just putting something into a list.&lt;/p&gt;
&lt;p&gt;BullMQ provides concepts such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Jobs
Workers
Retries
Backoff
Delayed jobs
Concurrency
Rate limiting
Failed jobs
Job lifecycle
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That matters because real systems fail.&lt;/p&gt;
&lt;p&gt;For example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Worker
   ↓
Send email
   ↓
Email provider unavailable
   ↓
Job fails
   ↓
Retry
   ↓
Worker tries again
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The system can therefore treat background work as something with a lifecycle instead of a simple function call.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;but-there-is-an-important-failure-model&quot;&gt;But there is an important failure model&lt;/h2&gt;
&lt;p&gt;A queue doesn't automatically mean:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;“The job runs exactly once.”&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;A worker can crash.&lt;/p&gt;
&lt;p&gt;A network call can time out.&lt;/p&gt;
&lt;p&gt;A job can be retried.&lt;/p&gt;
&lt;p&gt;The same operation may therefore be executed more than once.&lt;/p&gt;
&lt;p&gt;That means job handlers should often be designed with &lt;strong&gt;idempotency&lt;/strong&gt; in mind.&lt;/p&gt;
&lt;p&gt;For example, a payment operation is much more dangerous to retry blindly than sending a harmless notification.&lt;/p&gt;
&lt;p&gt;This was an important lesson for me:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Using a queue changes the failure model of the application.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;You have to think about retries, duplicates, timeouts, and partial failures.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;could-i-use-bullmq-without-redis-&quot;&gt;Could I use BullMQ without Redis?&lt;/h2&gt;
&lt;p&gt;Interestingly, yes.&lt;/p&gt;
&lt;p&gt;Modern BullMQ also provides a PostgreSQL backend.&lt;/p&gt;
&lt;p&gt;That means the conceptual architecture can also be:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;NestJS
   ↓
BullMQ
   ↓
PostgreSQL
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This can make sense when I want to keep the infrastructure simpler and the queue workload is moderate.&lt;/p&gt;
&lt;p&gt;But there is a trade-off.&lt;/p&gt;
&lt;p&gt;Using PostgreSQL for both workloads means:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;PostgreSQL
 ├── transactional business data
 └── queue workload
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;So the queue and business operations share database resources.&lt;/p&gt;
&lt;p&gt;That can be perfectly reasonable for a smaller application.&lt;/p&gt;
&lt;p&gt;For a system with large queue throughput, isolating the queue workload may become more attractive.&lt;/p&gt;
&lt;p&gt;This is why there isn't one universally correct answer.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;when-i-would-not-use-bullmq-redis&quot;&gt;When I would NOT use BullMQ + Redis&lt;/h2&gt;
&lt;p&gt;This was probably the most useful part of the exercise.&lt;/p&gt;
&lt;p&gt;I don't want to add Redis and BullMQ just because they are popular technologies.&lt;/p&gt;
&lt;p&gt;For something like:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;const result = calculateTax(data);
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;there is no reason to create a queue.&lt;/p&gt;
&lt;p&gt;If a task takes a few milliseconds and the user needs the result immediately, synchronous processing is simpler.&lt;/p&gt;
&lt;p&gt;I also might avoid BullMQ for a very small scheduled task:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Every day at 08:00
Generate one report
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A simple scheduler could be enough.&lt;/p&gt;
&lt;p&gt;And if the application already uses PostgreSQL and has only a small number of background jobs, a PostgreSQL-backed solution may be simpler than adding another infrastructure component.&lt;/p&gt;
&lt;p&gt;The lesson is:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;&lt;strong&gt;Don't introduce distributed infrastructure without a workload that justifies it.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id=&quot;the-decision-i-now-make-before-choosing-a-queue&quot;&gt;The decision I now make before choosing a queue&lt;/h2&gt;
&lt;p&gt;Instead of thinking:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;“Should I use BullMQ?”&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;I ask:&lt;/p&gt;
&lt;h4 id=&quot;1-does-this-work-need-to-happen-asynchronously-&quot;&gt;1. Does this work need to happen asynchronously?&lt;/h4&gt;
&lt;p&gt;If yes, continue.&lt;/p&gt;
&lt;h4 id=&quot;2-does-it-need-retries-or-failure-recovery-&quot;&gt;2. Does it need retries or failure recovery?&lt;/h4&gt;
&lt;p&gt;If yes, a real job system becomes more useful.&lt;/p&gt;
&lt;h4 id=&quot;3-does-it-need-delayed-execution-&quot;&gt;3. Does it need delayed execution?&lt;/h4&gt;
&lt;p&gt;For example:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Do this after 30 seconds.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h4 id=&quot;4-does-it-need-multiple-workers-or-concurrency-&quot;&gt;4. Does it need multiple workers or concurrency?&lt;/h4&gt;
&lt;p&gt;For example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Queue
 ├── Worker 1
 ├── Worker 2
 └── Worker 3
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;5-does-the-workload-justify-another-infrastructure-component-&quot;&gt;5. Does the workload justify another infrastructure component?&lt;/h4&gt;
&lt;p&gt;This is the trade-off question.&lt;/p&gt;
&lt;p&gt;Only after answering those questions would I choose the technology.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;my-current-mental-model&quot;&gt;My current mental model&lt;/h2&gt;
&lt;p&gt;The biggest thing I learned from implementing this feature is that architecture is less about memorizing technologies and more about matching &lt;strong&gt;workloads to the right tools&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;For Waitless, my current architecture can be thought of as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;                    Waitless
                       │
          ┌────────────┼────────────┐
          ↓            ↓            ↓
     PostgreSQL      Redis       MongoDB
          │            │            │
   business state    jobs      event history
                    BullMQ
                       │
                    Workers
                       │
                   Email/API
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;PostgreSQL answers:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;&lt;strong&gt;What is the current business state?&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Redis + BullMQ answers:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;&lt;strong&gt;What work needs to be processed asynchronously?&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;MongoDB answers:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;&lt;strong&gt;What happened in the event history, and what can I derive from those events?&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The important thing is not that I used three technologies.&lt;/p&gt;
&lt;p&gt;The important thing is that each technology was chosen for a different workload.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;final-takeaway&quot;&gt;Final takeaway&lt;/h2&gt;
&lt;p&gt;The best lesson I took from this wasn't actually “how to use BullMQ.”&lt;/p&gt;
&lt;p&gt;It was this:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;&lt;strong&gt;Before choosing a technology, understand the workload.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Ask:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;What problem am I solving?
What are the access patterns?
What guarantees do I need?
How does the system fail?
What are the trade-offs?
What simpler alternative could work?
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Once I started thinking this way, Redis, BullMQ, PostgreSQL, MongoDB and other technologies stopped looking like isolated tools.&lt;/p&gt;
&lt;p&gt;They became different tools for different engineering problems.&lt;/p&gt;
&lt;p&gt;And that is the mindset I want to continue developing as I work toward designing larger backend systems.&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/29737/why-i-used-bullmq-redis-for-background-jobs-in-my-nestjs-application-and-when-i-wouldnt</guid>
<pubDate>Sun, 04 Oct 2026 15:45:42 +0000</pubDate>
</item>
</channel>
</rss>