Coder Legion - Recent posts tagged cloud

Cloudflare vs QUIC.cloud: Which CDN Best Improves Core Web Vitals?

Mon, 26 Oct 2026 17:17:51 +0000

Website speed affects how visitors experience a website, how quickly they can access its content, and whether they can interact with its pages without delays. For website owners, improving performance is also an important part of search engine optimization. Google recommends good Core Web Vitals as part of delivering a strong page experience.

Two content delivery network services worth comparing are Cloudflare and QUIC.cloud. Both aim to improve website performance by delivering content through distributed infrastructure, but they approach optimization differently.

Cloudflare offers a broad CDN and security platform for websites across different technologies. QUIC.cloud focuses on WordPress performance and integrates with the LiteSpeed Cache plugin to provide caching and optimization features designed for that platform.

The choice between them depends on the website's technology, caching requirements, traffic patterns, and existing hosting environment. Neither service automatically guarantees better Core Web Vitals scores.

Understanding Core Web Vitals Before Choosing a CDN
Core Web Vitals are Google's metrics for evaluating three important aspects of the real-world experience of using a website: loading performance, responsiveness, and visual stability.

Before comparing Cloudflare and QUIC.cloud, website owners need to understand which problems these metrics measure.

Largest Contentful Paint (LCP)
Largest Contentful Paint measures how long it takes for the largest visible content element in the viewport to render. That element might be a featured image, a large heading, or a prominent image near the top of an article.

Google's recommended threshold for a good LCP score is 2.5 seconds or less.

A CDN can help improve LCP by reducing the time needed to deliver cached resources, including images, stylesheets, and other assets. Depending on the caching configuration, it can also reduce the time required to retrieve page content from the origin server.

However, a CDN cannot automatically correct an oversized hero image, a slow database query, or excessive server-side processing. These problems may continue to delay the largest visible element even when static assets are delivered quickly.

Interaction to Next Paint (INP)
Interaction to Next Paint measures how responsive a page is to user interactions, including clicks, taps, and keyboard input.

Google recommends an INP score of 200 milliseconds or less for a good user experience.

A CDN can contribute indirectly to better responsiveness by delivering JavaScript files and other resources efficiently. Faster delivery may allow necessary scripts to become available sooner.

Nevertheless, INP is heavily influenced by the work performed by the browser. Long-running JavaScript tasks, inefficient event handlers, and excessive main-thread processing can delay visual responses to user input.

Changing CDN providers alone will not resolve those problems. Developers may need to reduce JavaScript execution time, remove unnecessary scripts, or improve how interactive components operate.

Cumulative Layout Shift (CLS)
Cumulative Layout Shift measures unexpected movement of visible page content during a visitor's session.

Google recommends a CLS score of 0.1 or less.

Unexpected movement can occur when images load without reserved dimensions, advertisements appear without allocated space, web fonts change text dimensions, or dynamically inserted elements push existing content down the page.

A CDN may help deliver images and fonts more efficiently, but faster delivery does not guarantee a lower CLS score. The page must still reserve appropriate space for its content and load resources in a way that avoids unexpected movement.

These distinctions matter because a CDN is only one part of a website's performance system. The best choice is the provider that addresses the site's actual delivery and caching problems without introducing new complications.

Cloudflare: A Flexible CDN for Different Website Platforms
Cloudflare provides content delivery, caching, network security, and performance optimization services. It can support websites built on WordPress, other content management systems, and custom web applications.

Its main advantage in this comparison is flexibility. Website owners can use its CDN without committing to the WordPress and LiteSpeed ecosystem.

Global content delivery and caching
Cloudflare caches eligible website resources across its distributed network. When a visitor requests a cached resource, Cloudflare can serve it from an edge location instead of retrieving it from the origin server.

This can reduce latency, decrease origin server requests, and improve the delivery of images, stylesheets, JavaScript files, and other cacheable resources.

Cloudflare's standard caching configuration does not automatically cache every HTML page. Dynamic content generally requires additional configuration through features such as Cache Rules or Automatic Platform Optimization for WordPress.

This distinction matters when evaluating performance. A website may deliver images quickly through Cloudflare while continuing to generate every page request at its origin server. In that situation, the CDN may improve asset delivery without substantially reducing the delay before the HTML document arrives.

Website owners should therefore examine which resources are being cached rather than assuming that adding Cloudflare automatically places their entire website on the edge network.

Automatic Platform Optimization for WordPress
Cloudflare's Automatic Platform Optimization, commonly called APO, extends its performance capabilities for WordPress websites.

APO can cache eligible HTML responses at Cloudflare's edge, reducing repeated requests to the origin server. This can improve time to first byte, or TTFB, which measures how long a browser waits before receiving the first byte of a response.

For a WordPress website with a slow origin server, effective HTML caching can make a meaningful difference to page delivery.

Cloudflare also provides cache management mechanisms that can update cached content when a WordPress site changes, provided the appropriate integration is configured.

However, APO is an additional consideration when comparing costs. Cloudflare's standard CDN and its WordPress-specific optimization offering are not identical. Website owners should verify which features their selected plan includes before making a purchasing decision.

Image optimization
Images frequently account for a substantial portion of a webpage's transferred data. Large images can delay LCP, especially when the largest visible image is also the page's main content element.

Cloudflare offers image optimization features, including Polish, which can compress eligible images and reduce their transfer size. Its image services also support transformations such as resizing and format conversion.

These capabilities can help websites deliver more appropriately sized images to visitors.

There is an important qualification: Polish is not included in Cloudflare's Free plan according to its published plan comparison. Website owners should check the current feature availability and pricing before treating it as part of a free optimization setup.

Cloudflare also supports HTTP/3, which uses the QUIC transport protocol. This can improve connection performance under certain network conditions, including networks experiencing packet loss.

HTTP/3 is useful infrastructure, but enabling it does not guarantee a specific improvement in LCP, INP, or CLS. Actual results depend on the website, the visitor's connection, and the resources required to render the page.

Who benefits most from Cloudflare?
Cloudflare is a strong candidate for website owners who need a CDN that works across different platforms, configurable caching, network security, and additional performance services.

It is particularly relevant when the website uses a technology stack that does not depend on LiteSpeed.

It can also suit WordPress websites, especially when the owner wants to combine CDN delivery with HTML caching through APO.

The main requirement is correct configuration. Cache rules, cookie handling, dynamic content, and interactions with existing caching plugins all affect the outcome.

QUIC.cloud: WordPress Performance Through LiteSpeed Integration
QUIC.cloud is a CDN and optimization service developed by the team behind LiteSpeed Technologies. Its central advantage is its close integration with the LiteSpeed Cache plugin for WordPress.

Unlike a general-purpose CDN that primarily delivers eligible cached resources, QUIC.cloud combines CDN delivery with services designed to optimize WordPress pages.

This makes it particularly relevant for website owners who already use LiteSpeed-compatible hosting and the LiteSpeed Cache plugin.

Static and dynamic content caching
QUIC.cloud can cache static resources and dynamically generated WordPress pages through its integration with LiteSpeed Cache.

Dynamic caching is important because WordPress often generates pages using PHP, database queries, themes, and plugins. If a page can be safely cached, serving a stored response can reduce the need to repeat that work for every visitor.

The result may be a faster response from the website and fewer requests reaching the origin server.

QUIC.cloud's documentation explains that LiteSpeed Cache controls dynamic caching and that the CDN requires the LiteSpeed Cache for WordPress plugin.

This is a significant difference when choosing between the two services. QUIC.cloud is not simply a CDN that can be connected to any website regardless of its technology. Its core WordPress caching workflow depends on the required LiteSpeed integration.

For sites that already use this setup, the integration can reduce the need to assemble separate tools for different optimization tasks.

Page optimization beyond CDN delivery
QUIC.cloud also provides online services for optimizing WordPress page resources.

These include Critical CSS, Unique CSS, and Viewport Images.

Critical CSS identifies the styles needed to render above-the-fold content. Delivering those styles efficiently can help the browser display the visible portion of a page sooner.

Unique CSS reduces unnecessary CSS by generating a streamlined set of styles for an individual page.

Viewport Images identifies images that appear in the initial visible area, allowing LiteSpeed Cache to exclude those images from lazy loading where appropriate.

That last feature matters because lazy loading the main image visible when a page opens can delay LCP. Lazy loading is useful for images farther down the page, but it should not be applied indiscriminately to important above-the-fold content.

QUIC.cloud also offers image optimization services that can compress images and generate WebP or AVIF versions when configured.

These features address problems that a CDN alone may not resolve. They can reduce the amount of CSS and image data a browser must process before presenting the main content.

However, optimization settings require testing. Combining CSS transformations, asynchronous loading, and other performance techniques without checking the rendered page can introduce visual problems or interfere with scripts.

Security and network delivery
QUIC.cloud offers CDN-level security features, with capabilities depending on the selected plan. Its Standard plan includes additional security controls compared with its Free plan.

Its HTTP/3 support also contributes to its network delivery capabilities.

Website owners should distinguish between the availability of a protocol and the quality of the entire website. A modern transport protocol can improve resource delivery, but it cannot eliminate slow database queries, inefficient JavaScript, or excessive third-party scripts.

Who benefits most from QUIC.cloud?
QUIC.cloud is particularly suitable for WordPress websites running LiteSpeed-compatible infrastructure and using LiteSpeed Cache.

Publishers, bloggers, and content-heavy websites can benefit from the combination of page caching, image optimization, and CSS optimization when these features address their existing performance problems.

Its strongest advantage is the integration of several WordPress optimization functions into one workflow.

For a website that does not use the required LiteSpeed Cache plugin, however, QUIC.cloud's core CDN offering is not a straightforward replacement for a general-purpose CDN.

Cloudflare vs QUIC.cloud: The Main Differences
Here is the comparison organized into a clear, easy-to-read Markdown table:

Feature

Cloudflare

QUIC.cloud

Primary focus

General-purpose CDN, security, and performance

WordPress CDN and optimization

Platform compatibility

Supports a wide range of website technologies

WordPress websites

CDN requires

No specific server requirement

LiteSpeed Cache for WordPress

Static content caching

Yes

Dynamic HTML caching

Available through features such as APO and appropriate cache configuration

Integrated with LiteSpeed Cache for WordPress

Image optimization

Available through products such as Polish and Cloudflare Images

Available through QUIC.cloud image optimization services

CSS optimization

Available through various configuration options and services

Provides Critical CSS and Unique CSS services through its WordPress integration

HTTP/3

Supported

Free CDN option

Yes

Advanced features

Availability depends on the selected product and plan

Availability depends on the selected plan and service

Best starting point

Websites requiring a flexible CDN and security platform

WordPress websites using LiteSpeed Cache

The table highlights an important distinction. Cloudflare offers broader platform compatibility, while QUIC.cloud provides a more specialized optimization workflow for WordPress.

Neither approach is universally better. The relevant question is which service can address the specific causes of poor performance on the website.

Which CDN Is Better for Each Core Web Vital?
Best for improving LCP
Both services can improve LCP, but QUIC.cloud has a practical advantage for WordPress sites using LiteSpeed Cache when page optimization is the main objective.

QUIC.cloud combines CDN caching with image optimization and tools that help deliver critical CSS. These features can reduce delays associated with retrieving page content and rendering the main visible elements.

Cloudflare can also improve LCP through edge caching, image optimization, and WordPress HTML caching with APO.

The deciding factor is the bottleneck.

If a website spends too long generating HTML, effective page caching may provide the greatest benefit. If the main image is too large, image compression and appropriate dimensions may matter more. If the server responds quickly but the browser waits for CSS, stylesheet optimization may be the better target.

Neither provider can guarantee a particular LCP score without addressing the underlying cause.

Best for improving INP
Neither CDN has an automatic advantage for INP.

A website with poor INP may have excessive JavaScript execution, inefficient interactive components, or scripts from advertising and analytics services that compete for browser processing time.

Cloudflare and QUIC.cloud can help deliver resources, but network delivery is only one part of responsiveness.

Website owners should inspect long browser tasks, reduce unnecessary JavaScript, and review third-party scripts. They should then test whether their CDN configuration contributes to an improvement in the actual user experience.

Choosing a provider solely because it advertises faster delivery is not a reliable way to fix poor INP.

Best for improving CLS
Neither provider automatically solves CLS problems.

A page may receive every resource quickly and still have a poor CLS score because images lack reserved dimensions or advertisements introduce unexpected content movement.

QUIC.cloud's Viewport Images service can help manage the loading of above-the-fold images, while Cloudflare can improve resource delivery. Neither feature replaces correct page layout and resource dimensions.

To improve CLS, developers should reserve space for images and embedded content, avoid inserting content above existing elements without allocating space, and ensure that font loading does not cause unexpected movement.

The CDN should support these measures, not substitute for them.

Pricing: Which Service Offers Better Value?
Price matters, especially for independent publishers and small businesses operating within limited budgets.

Cloudflare offers a Free plan that includes CDN services. Its published pricing page also lists paid plans, including Pro at $20 per month when billed annually or $25 per month when billed monthly. Product availability varies by plan.

QUIC.cloud offers a Free CDN plan and a Standard plan with additional network coverage and security features.

Its published documentation lists six CDN nodes in selected North American and European locations for the Free plan, compared with a global network of more than 80 points of presence for the Standard plan. The service's pricing documentation states that Standard plan bandwidth costs vary by region, with published rates ranging from $0.02 to $0.08 per GB.

These figures should be checked against the providers' current pricing pages before purchase because plan features and charges can change.

The comparison also needs to account for the complete optimization setup, not just the CDN subscription.

A WordPress site using QUIC.cloud may already have LiteSpeed Cache available within its existing hosting arrangement. A Cloudflare setup may require additional spending for particular image optimization or WordPress caching features.

The most economical option depends on which features the website needs and which services its hosting provider already supplies.

Do not pay for features that duplicate existing functionality without first testing whether the additional service produces a measurable improvement.

How to Test Which CDN Improves Your Website
Selecting a CDN based on feature lists alone is insufficient. The website's actual performance provides a stronger basis for the decision.

Use the following process to compare Cloudflare and QUIC.cloud.

Step 1: Record your existing Core Web Vitals
Before changing providers or optimization settings, establish a baseline.

Use Google Search Console's Core Web Vitals report to review real-world performance data for groups of pages. Use PageSpeed Insights to examine individual URLs and identify potential performance problems.

Record LCP, INP, and CLS separately. Where possible, examine mobile and desktop performance independently.

Step 2: Identify the main performance bottleneck
Determine whether the primary problem involves server response time, large images, render-blocking CSS, JavaScript execution, or unexpected layout movement.

This step prevents unnecessary configuration changes.

For example, a site with a large featured image may need image optimization more urgently than a different caching configuration. A site with excessive JavaScript may require code-level improvements regardless of its CDN.

Step 3: Configure the selected service correctly
For Cloudflare, check caching rules, resource delivery, and whether WordPress HTML caching through APO is appropriate for the site.

For QUIC.cloud, confirm that LiteSpeed Cache is installed and configured correctly. Review its page caching, image optimization, and CSS optimization settings.

Avoid enabling several overlapping optimization features simultaneously without testing their interactions.

Step 4: Test representative pages
Test the homepage, a typical article, a category page, and any important landing pages.

For online stores, include product pages and other relevant customer-facing pages. Ensure that personalized content, shopping carts, and logged-in sessions continue to work correctly.

Run repeated tests under comparable conditions. A single laboratory test cannot establish how a website performs for every visitor.

Step 5: Compare the results
Review changes in LCP, INP, and CLS rather than relying on a single overall performance score.

Check server response times and caching behavior as well. A CDN that reduces the time needed to retrieve HTML may improve loading performance even if its effect on other metrics is limited.

For a reliable assessment, allow enough time for real-world performance data to accumulate. Google Search Console's field data and laboratory testing measure different aspects of performance, so their results should not be treated as interchangeable.

Step 6: Check for functional problems
Performance improvements are not useful if pages stop displaying correctly.

Test navigation, forms, search, account access, checkout processes, and content updates. Verify that cached pages do not expose private information or continue displaying outdated content after updates.

The final decision should account for performance, compatibility, maintenance, and total cost.

Final Verdict: Cloudflare or QUIC.cloud?
Cloudflare is the stronger starting point for website owners who need a flexible CDN that supports different platforms, provides network security, and offers configurable caching. Its WordPress-specific APO feature also makes it a credible option for websites that need HTML caching at the network edge.

QUIC.cloud is the stronger starting point for WordPress websites already using LiteSpeed Cache, particularly when the goal is to combine CDN delivery with image and CSS optimization in one integrated workflow.

For LCP, either service can help, depending on the main cause of slow loading. QUIC.cloud offers an integrated route for optimizing LiteSpeed-based WordPress sites, while Cloudflare provides flexible caching and image delivery options.

For INP, neither service replaces the need to optimize JavaScript and browser execution.

For CLS, correct layout implementation remains essential regardless of the CDN.

The practical recommendation is straightforward: choose Cloudflare for broad platform compatibility and flexible infrastructure, or choose QUIC.cloud when its LiteSpeed integration matches your WordPress environment and optimization needs.

Do not assume that changing CDN providers will automatically improve search rankings or produce good Core Web Vitals. Measure existing performance, identify the main bottleneck, configure the service, and compare the results.

A well-configured CDN combined with efficient page code, appropriately sized images, and reliable hosting is a more dependable approach than selecting a provider based on marketing claims alone.

From Big Data to the Cloud, Part 1 — Why One Machine Is Never Enough

Sun, 25 Oct 2026 05:46:00 +0000

A calm, sequential guide for anyone entering the field. The article is divided into smaller chapters to make it easier to understand.

This series grows out of the Cloud Computing labs I teach to fourth-year Computer Science students at the University of Hilla. The labs move quickly, because a lab has ninety minutes. The articles can afford to slow down. Here we take the same path the labs take, but we stop at every step and ask the question a careful student always asks: why this, and not something simpler?

The path itself is short to state. Data has grown past the point where one machine can store it, process it, or analyse it efficiently. The answer is not a bigger machine but many ordinary machines working together. And the framework that manages those machines is Apache Hadoop. Each of those three sentences hides a chain of reasoning, and the purpose of this part is to lay that chain out link by link — so that by the end, the cluster does not look like a fashionable choice but like the answer the problem forces on us.

The destination of this series is the cloud. But we will not begin there, because a definition of the cloud means little until you have felt the problem it was built to solve. So the series takes the long road on purpose: first the problem (this part), then the cluster that answered it and the burden that answer carried (Part 2), and only then the cloud — defined in Part 3, and explored service by service in the parts after it.

We will not touch a configuration file in this part. What we build here is the problem — precisely enough that when Part 2 opens Hadoop up and looks at HDFS, MapReduce, and YARN from the inside, each component reads as the answer to a question we have already asked.

Chapter 1: Where All This Data Comes From

Every Action Leaves a Record
Before we define anything, it is worth noticing how ordinary the sources of big data are. Every click, every purchase, every email, every chat message, and every social post creates data. None of these events is large on its own.

But the sources do not stop at people typing. Sensors and IoT devices report what they measure. Banks record every transaction. Hospitals keep patient records. Streaming services log what their users watch. Each of these produces records continuously, whether or not anything interesting is happening.

The individual record is small. What makes the situation new is the multiplication: small records, produced by enormous numbers of people and devices, without end. Data now reaches billions of records, and petabytes every day. The important words there are every day. The data does not arrive once. It keeps arriving.

Chapter 2: What Big Data Actually Means

A Definition About Tools, Not Size
The most useful definition of big data is not a number. It is this: big data is datasets so large and complex that traditional tools — spreadsheets, a single database server — cannot store, process, or analyse them efficiently.

It is worth being exact about why the definition is phrased this way. Students often ask how large data must be before it “counts” as big. The definition answers by pointing at the tools rather than at a size: data becomes big at the moment the ordinary tool stops coping with it. Notice also that it names three separate jobs — store, process, analyse — and a tool can fail at any one of them.

The Goal Is Insight, Not Storage
A second point is easy to miss: the goal is not only storing data. The goal is extracting insights from it — turning a mountain of records into a decision someone can act on.

The classic example is an online store. It already keeps every purchase as a matter of bookkeeping. The value appears only when it analyses customer habits: what people buy, when, and together with what. With that analysis, the store can make targeted offers and discounts, which increases revenue. The data was there all along; what matters is the ability to read it. Hold onto this, because a system that can store huge data but cannot analyze it has solved only half the problem.

Three Shapes of Data
Data does not only come in different sizes. It comes in different shapes, and the shape decides which tools can handle it.

Structured data is the familiar case: every row has the same columns. A student table with a name, a department, and an age is structured.
Semi-structured data carries its own labels — tags in XML and HTML, keys in JSON — but does not promise that every record has the same fields. The keys tell you what each value means, yet there is no rigid schema forcing every record into one shape.
Unstructured data has no predefined format at all. An email body, a social post, an image, a video: the meaning is in there, but no column tells you where.

And here is the uncomfortable fact: most of today’s data is unstructured, and that is exactly what traditional databases handle worst. They were built around the first row of that table. The world moved to the third.

Chapter 3: The Five V’s

A Vocabulary for “Why It Is Hard”
“Too large and too complex” is a good definition, but it is too blunt to design a system against. We need to name the separate ways data can be difficult, because each one places a different demand on the machine that handles it. The standard vocabulary for this is the five V’s.

One at a Time
01 — Volume: how much. The huge amount of data, from terabytes to petabytes. Volume is the V that breaks storage first.

02 — Velocity: how fast. How fast data is created and must be processed — think of stock trading or live streams. The second half of that sentence is the important one: these systems need answers quickly, not just storage quickly. Velocity is the V that breaks time.

03 — Variety: how many shapes. Many types and formats of data — the structured, semi-structured, and unstructured shapes from Chapter 2, often mixed together. Variety is the V that breaks fixed schemas.

04 — Veracity: how trustworthy. How accurate and trustworthy the data is. Real data is noisy, messy, and has missing values, so cleaning and checking quality stands between you and a conclusion you can defend.

05 — Value: what we get out of it. The useful insight we finally get out of the data. This is the V that justifies the other four — the point Chapter 2 already made, now given its own name.

Notice the split: the first three describe the data itself, while the last two describe whether we can trust it and whether it was worth the effort.

Chapter 4: Why Traditional Systems Fail
What “Traditional” Means Here
By a traditional system we mean one built for structured data on a single powerful machine. All the data lives on that machine; all the work runs on it. This design is good at what it was built for. The point of this chapter is that big data asks it to do something it was never built for — and it breaks in five places.

The Five Breaking Points

It was built for structured data on one machine. Its whole design assumes fixed rows and columns, and one computer doing all the work.
It cannot keep up with the volume and speed of modern data. This is Volume and Velocity together: too many bytes, arriving too fast, for one machine to absorb.
It has difficulty handling unstructured data — images, video, text. This is Variety. And since most of today’s data is unstructured, this is not an edge case; it is most of the data.
Scaling up means buying a bigger, very expensive server — and there is always a limit. When one server is not enough, the traditional move is to scale up: replace it with a bigger one. That works for a while, but each step costs more, and eventually there is no bigger machine to buy.
It is slow to extract insights from huge datasets. Put the first four together and the last failure follows on its own. This is the failure that matters most, because it is a failure of Value.

Three Challenges, Stated Plainly
All five breaking points reduce to three questions any big data system must answer.

Storage — where do we keep data that keeps growing?
Processing — how do we analyse it fast enough to be useful?
Scalability — how do we grow (or shrink) resources as data changes?

Read the third question twice, because it hides a detail that will matter later in this series. It does not only ask how to grow. It asks how to grow or shrink — how to have exactly as much capacity as the data needs, no more and no less. Keep that small word “shrink” in your pocket; we will come back for it.

The single powerful machine answers none of them well once the data is big enough. So the question becomes: what architecture answers all three?

Chapter 5: The Distributed Idea

Many Ordinary Machines, Working Together
The answer is to stop making one machine bigger. Instead of one big machine (monolithic), we use many ordinary machines working together, called a cluster. Each machine in the cluster is a node. Both the data and the processing are split across the nodes. A cluster can range from a few nodes to thousands — and cheap commodity hardware is enough: normal PCs and servers, not specialised machines.

This strategy is called scaling out: when you need more capacity, you do not replace the machine — you add more machines.

The Price of Distribution
Scaling out is not free. The moment we spread data across many machines, we inherit three problems the single machine never had:

Which data lives on which node? With one machine, a file is simply on its disk. With many, something must remember where every piece went.
How do nodes communicate and coordinate? Many machines working on the same job is only useful if the work is divided and the results are brought back together.
What happens when a machine crashes? On one server, a crash is a rare event. In large clusters, failures are normal.
That last point changes how the whole system must be designed. If failure is the normal state of affairs, the system cannot treat it as an emergency. It must expect machines to fail and keep working anyway, without losing data.

Stated as a General Rule
Scaling out trades one hard problem — a machine that cannot grow past its limit — for three new ones: tracking where data lives, coordinating the nodes, and surviving failure. The trade is worth it only if software solves those three problems, so that the people using the cluster do not have to. What the field needed was a framework that solves them once, for everyone.

Chapter 6: Enter Hadoop

What Hadoop Is
Apache Hadoop is an open-source framework for distributed storage and parallel processing of big data. It runs on clusters of commodity hardware — exactly the cluster Chapter 5 described. It was created around 2005, when the volume of web data, from sites like Facebook and YouTube, exploded. It is free to download from Apache, and commercial distributions also exist, such as Cloudera, Hortonworks, and MapR.

Its Core Components, at a Glance
The cleanest way to understand Hadoop’s design is to hold it up against the three challenges from Chapter 4. Each core component answers one of them.

We deliberately stay at this altitude for now. How HDFS splits and tracks blocks, why MapReduce runs the program where the data already is, and how YARN hands out CPU and memory are the subject of Part 2.

What Hadoop Buys

Scalable — add more nodes as data grows. (Notice: as data grows. The “shrink” half of the Scalability question is still waiting.)
Fault tolerant — keeps copies of data on different machines, so a hardware failure does not lose data. This is the answer to “what happens when a machine crashes?”
Cost-effective — uses cheap commodity machines.
Flexible — handles structured and unstructured data. This is the answer to Variety.
Parallel — many nodes work at the same time.

What Hadoop Costs
An honest account includes the other column.

It is designed for batch processing, not real time. For real-time work, Apache Spark is better.
It is complex to configure and maintain — which is exactly what the labs practise.
Replication uses extra storage space. Keeping copies on different machines is what makes Hadoop fault tolerant, but every copy takes up disk.

Where It Is Used
Healthcare: patient records, medical images, early disease prediction.
Banking: fraud detection, transaction analysis.
Social media: sentiment analysis of posts.
E-commerce: recommendations, customer behaviour, inventory management — the online store from Chapter 2, now at full scale.

What Comes in Part 2

We now have the problem in full, and the shape of the answer. We know why one machine is never enough, why a cluster is the way out, and which new problems the cluster creates. We have met Hadoop’s components by name. What we do not yet have is the inside view.

Part 2 opens the components one at a time. We follow a 400 MB file as HDFS splits it into 128 MB blocks, and see why, with three copies of each block, that file ends up stored as twelve blocks across the cluster. We meet the NameNode that keeps track of which block is on which machine, and the DataNodes that send it a heartbeat to say “I’m alive” — and we ask what happens when one of them stops. We walk through MapReduce by hand, on a word count and on an average-age example, from Map through Shuffle & Sort to Reduce. And we watch YARN take a submitted job, start an ApplicationMaster for it, and request containers on the nodes that hold the data.

The single principle running through all of it: Hadoop moves the work to the data, not the data to the work.

And Part 2 will end where the rest of this series begins. Hadoop solves storage, processing, and growth — but it leaves us holding a cluster: machines to buy, a framework that is complex to configure and maintain, and capacity that can grow but not easily shrink. That remaining burden is exactly what the cloud takes off our hands, and it is where Part 3 picks up the story.

References

Deshpande, T., Karanth, S., & Turkington, G. Hadoop: Data Processing and Modelling (Learning Path). Packt Publishing.
Oracle. What Is Big Data? https://www.oracle.com/big-data/what-is-big-data/
Big Data — Understanding Hadoop: Architecture, Features, Challenges & Applications. Medium. https://medium.com/@rp99452/big-data-understanding-hadoop-architecturefeatures-challenges-applications-c7ab7011e375
Mehta, C. B. (2022). A Comprehensive Review of Hadoop Ecosystem for Big Data Processing and Analytics. IJNRD, 7(10).

Best Managed Cloud Hosting Platforms for High-Traffic Tech Blogs

Fri, 23 Oct 2026 16:53:18 +0000

High-traffic technology blogs need hosting infrastructure that can handle sudden visitor spikes, deliver pages quickly, protect published content, and keep websites available when demand rises. A hosting plan that works for a new blog may become unreliable or expensive as its audience grows, its media library expands, and more readers arrive from search engines, newsletters, and social media.

Managed cloud hosting can reduce the amount of server administration required from a publisher. Depending on the provider and plan, the hosting company may handle server maintenance, security updates, automated backups, monitoring, performance optimization, and technical support. The exact services vary, so site owners must check what each package includes before paying.

The best platform depends on the website's traffic patterns, content management system, technical requirements, budget, and need for direct control over the infrastructure. WordPress publishers may benefit from a managed WordPress service, while developers running custom applications may need a cloud platform with more control over server configuration.

This guide examines established managed hosting and cloud infrastructure platforms that tech bloggers can evaluate as their websites grow. The focus is on hosting capabilities, scalability, security, maintenance, and the practical trade-offs involved in choosing a provider.

Kinsta: Managed WordPress Hosting for Growing Tech Publications
kinsta.com
Kinsta is a strong candidate for technology blogs built on WordPress. Its hosting service includes a global content delivery network, edge caching, automated backups, and a management dashboard for website administration.

For publishers who would rather concentrate on reporting, editing, search optimization, and audience growth than server administration, managed WordPress hosting can reduce the amount of infrastructure work they must handle themselves.

Key features
Global content delivery: Kinsta includes a CDN powered by Cloudflare, with image optimization and support for HTTP/3.

Edge caching: Cached content can be delivered closer to readers, reducing the need to process every request on the main server.

Automated backups: Daily backups help publishers recover website files and databases after certain failures or unwanted changes.

Website migration: Kinsta offers free WordPress migrations, subject to its service terms.

Multiple hosting locations: Its documentation lists 30 available data centers.

Security measures: Cloudflare integration and infrastructure-level protections help defend websites against certain attacks.

Why tech bloggers should consider it
A technology blog may publish software reviews, product comparisons, tutorials, industry news, and long-form guides. These pages often contain images, screenshots, code examples, and embedded media. A CDN and caching system can reduce the work required from the origin server when readers request frequently accessed pages.

Kinsta also provides plan options based on factors such as site count, storage, bandwidth, and estimated monthly visits. Publishers should compare these limits with their actual traffic and resource usage before choosing a package.

What to consider before buying
Kinsta is focused on WordPress hosting rather than providing an unrestricted general-purpose cloud server. That makes it a better fit for publishers who want a managed WordPress environment than for developers who need to configure a custom application stack.

Budget is another consideration. Storage, bandwidth, CDN usage, and other plan limits matter, and exceeding included allowances can lead to additional charges. Check the current pricing and overage policy before committing.

Best suited for: Established WordPress tech blogs that want managed infrastructure, integrated caching, automated backups, and less server administration.

Official website: Kinsta

Cloudways: Managed Cloud Hosting With More Infrastructure Choice
oakcassidy.com
Cloudways provides a managed hosting layer for applications including WordPress. It is worth considering for publishers who want cloud server resources without handling every aspect of server setup and maintenance themselves.

The platform offers server management tools, backup and restore functions, caching options, application deployment, and vertical scaling. Its available infrastructure and costs depend on the selected plan and cloud provider.

Key features
Managed server administration: The platform handles various server management tasks and provides a central interface for managing applications.

Backup and restoration: Automated and on-demand backups can be stored off-site, with one-click restoration options.

Caching: Supported caching technologies help reduce repeated processing for frequently requested pages.

Vertical scaling: Users can increase server resources as their application requirements grow, subject to the selected configuration.

Application support: The platform supports WordPress and other application types.

Security features: Available services include SSL, firewall protections, and security-related tools.

Why tech bloggers should consider it
Cloudways can suit a publication that has outgrown basic shared hosting but does not need a fully custom infrastructure managed by an in-house engineering team.

For example, a WordPress blog might receive steady traffic most of the month and experience a large increase after a major software announcement. More server capacity, effective caching, and a properly configured CDN can help the website handle increased demand.

Cloudways also offers a useful middle ground for publishers who manage multiple websites. Its management interface can make it easier to oversee applications and server resources without administering every component directly.

However, increasing server capacity is not a complete performance strategy. Poorly optimized database queries, heavy plugins, inefficient themes, and uncompressed images can still cause delays. Publishers should measure application performance before spending more on infrastructure.

What to consider before buying
Cloudways pricing varies according to the selected hosting configuration and service options. The lowest-priced server should not automatically be treated as suitable for a high-traffic publication.

Review the available memory, CPU resources, storage, bandwidth allowances, backup charges, and CDN costs. Also confirm which technical tasks are covered by standard support and which require additional services.

Best suited for: Growing WordPress publications, multi-site operators, and publishers who want managed cloud hosting with greater server configuration flexibility.

Official website: Cloudways

WP Engine: Managed WordPress Hosting for Editorial Teams
he.hostadvice.com
WP Engine focuses on WordPress hosting and development workflows. Its services are relevant to technology publishers who need managed maintenance, backup options, security tools, and separate environments for testing website changes.

WP Engine lists daily and on-demand backups, SSL, security patching, plugin risk scans, staging environments, and its EverCache performance technology among its hosting features. Available features depend on the plan.

Key features
Managed WordPress maintenance: Hosting plans include management of key components of the WordPress environment.

Backups: Daily and on-demand backup options support recovery from unwanted changes or technical problems.

Staging environments: Publishers can test theme changes, plugins, and other updates before deploying them to a live site.

Performance technology: EverCache is designed to improve website performance.

Security features: Available protections include security patching, plugin risk scans, SSL, and DDoS protection.

Developer workflows: Certain plans support development environments and automated deployments.

Why tech bloggers should consider it
A technology publication may have several contributors, editors, developers, and administrators working on the same website. Changes to a theme, caching configuration, or plugin can affect the live site, especially when the website uses multiple integrations.

Staging environments provide a place to test those changes before publishing them. This can help teams identify compatibility problems before they affect readers.

WP Engine is also relevant to publishers who want managed WordPress operations rather than responsibility for configuring and maintaining a general-purpose cloud server.

What to consider before buying
The main limitation is its WordPress focus. A blog built around a custom application or a different content management system may need another provider.

Publishers should also compare plan limits and support terms. A package intended for a smaller website may not offer the resources, features, or service commitments required by a publication receiving heavy traffic.

Best suited for: Editorial teams, established WordPress publications, and businesses that value managed maintenance and structured development workflows.

Official website: WP Engine

SiteGround Cloud Hosting: Dedicated Resources and Managed Operations
hostingcanada.org
SiteGround offers cloud hosting with dedicated resources, automated scaling, daily backups, on-demand backups, free SSL, CDN access, and WordPress management tools. Its cloud service is worth evaluating for publishers who want managed hosting with the option to configure their resource allocation.

Key features
Dedicated resources: Cloud plans allocate CPU and memory resources to the hosting environment.

Automated scaling: The service lists automated scaling among its cloud hosting features.

Backup options: Daily and on-demand backups support website recovery.

Caching: Dynamic caching and Memcached are included among its listed cloud features.

WordPress tools: Available functions include WordPress installation, automatic updates, migration tools, and staging.

Technical support: SiteGround lists 24/7 priority support for its cloud hosting service.

Why tech bloggers should consider it
A publication with a growing audience needs sufficient processing capacity, memory, and storage to support its workload. SiteGround's cloud hosting allows customers to select resource configurations and offers automated scaling as part of the service.

That can appeal to publishers who want more resources than a basic shared hosting plan provides without taking full responsibility for server administration.

Its staging tools are also useful when a website undergoes frequent changes. Editors and developers can test updates before applying them to the production site.

What to consider before buying
Cloud hosting can cost more than entry-level shared hosting, so the decision should be based on actual resource needs rather than traffic figures alone.

Before purchasing, compare the available CPU, memory, storage, and data transfer limits. Confirm how automated scaling works, whether it creates additional charges, and which resources are included in the selected package.

Best suited for: Growing technology publications that want managed cloud resources, automated scaling, and integrated WordPress tools.

Official website: SiteGround Cloud Hosting

DigitalOcean App Platform: Managed Infrastructure for Custom Tech Blogs
digitalocean.com
DigitalOcean offers App Platform, a managed platform-as-a-service for deploying applications from Git repositories or container images. It is particularly relevant to developers building technology publications with custom software, rather than publishers who only need conventional WordPress hosting.

DigitalOcean describes App Platform as a service that builds, deploys, and scales application components while managing the underlying infrastructure. It can connect to DigitalOcean Managed Databases for production data storage.

Key features
Managed application deployment: Applications can be deployed from supported repositories or container images.

Application scaling: App Platform can scale application components.

Managed databases: DigitalOcean offers database services with features such as backups, standby nodes, and scaling options, depending on the database engine and configuration.

Object storage: DigitalOcean Spaces provides object storage for files and other application data.

Development workflows: Deployment from repositories supports code-based workflows and application updates.

Why tech bloggers should consider it
Some technology blogs are more than conventional content management websites. They may include searchable software directories, interactive tools, custom APIs, developer documentation, user accounts, or other application features.

For these projects, a managed application platform can offer more flexibility than a hosting service designed exclusively for WordPress.

A developer can deploy the application separately from its database and use managed database features appropriate for production workloads. Object storage can also keep uploaded files separate from the application's temporary filesystem.

What to consider before buying
DigitalOcean App Platform requires more application-level technical knowledge than a typical managed WordPress service. The publisher or development team must choose an appropriate architecture, manage application code, and understand how the application connects to its database and file storage.

DigitalOcean's documentation also states that App Platform's local container filesystem is not persistent across deployments and container replacements. Production files should therefore be stored in an appropriate persistent service, such as Spaces or a managed database where applicable. Its development databases do not include built-in backups, so production deployments should use an appropriate managed database configuration.

Best suited for: Developer-led publications with custom applications, APIs, searchable databases, and infrastructure requirements beyond a standard WordPress installation.

Official website: DigitalOcean App Platform

How to Choose the Right Hosting Platform
The provider matters, but the configuration matters just as much. A costly hosting plan can still perform poorly if the website uses inefficient plugins, lacks caching, or has an overloaded database.

Use the following criteria when comparing hosting options.

Match the hosting model to your website
Choose managed WordPress hosting if your publication runs on WordPress and you want the provider to handle much of the underlying infrastructure work.

Consider managed cloud hosting if you need greater control over server resources or plan to run several applications.

Choose a managed application platform if the publication relies on custom code, APIs, or a separate application architecture.

Check how traffic limits work
Do not evaluate a plan using monthly visitor estimates alone. Check bandwidth, CPU capacity, memory, storage, CDN allowances, concurrent request handling, and any usage-based charges.

A website with heavy images and downloadable files can consume considerable bandwidth even if its visitor count is lower than that of a text-focused blog.

Confirm backup and recovery terms
Automated backups are useful, but publishers should check their frequency, retention period, storage location, and restoration process.

A backup stored only within the same infrastructure may not provide sufficient protection against every disaster. Consider keeping an independent copy of critical website files and databases, and test restoration procedures before an emergency occurs.

Review security responsibilities
Check whether the hosting provider includes SSL certificates, security patching, firewall protection, malware detection, DDoS protection, and access controls.

Managed hosting does not remove every security responsibility. Website owners still need to maintain secure administrator accounts, review plugin permissions, update application code where required, and limit access to trusted contributors.

Measure performance before upgrading
Use website monitoring and performance tests to identify the source of slow responses. Check server response time, database performance, cache effectiveness, image sizes, and third-party scripts.

If cached pages load quickly but the WordPress dashboard is slow, increasing CDN capacity may not solve the problem. If the server is overloaded during traffic spikes, additional resources or better scaling may be necessary.

Calculate the full monthly cost
Compare the hosting fee with additional charges for storage, bandwidth, backups, CDN services, extra websites, premium support, and resource upgrades.

The cheapest starting plan is not necessarily the cheapest option for a high-traffic publication. A plan with clear resource limits and the right built-in services may be more economical than a lower-priced plan that requires several paid add-ons.

Final Recommendation
There is no single hosting platform that is best for every high-traffic tech blog.

Kinsta is a practical option for WordPress publishers who prioritize managed hosting, integrated CDN delivery, caching, and backups.

Cloudways is worth evaluating if you want managed cloud hosting with server configuration options and application management tools.

WP Engine is a strong candidate for WordPress editorial teams that need staging environments, managed maintenance, and structured development workflows.

SiteGround Cloud Hosting suits publishers looking for dedicated cloud resources, automated scaling, and integrated WordPress tools.

DigitalOcean App Platform is relevant to developer-led publications that need a managed deployment platform for custom applications.

Before choosing, compare the current plans against your site's actual workload, security needs, recovery requirements, and operating budget. Test performance where possible, examine the provider's service terms, and confirm what technical support covers.

For a conventional WordPress technology blog, start by comparing Kinsta, Cloudways, and WP Engine. For a custom-built publication, evaluate DigitalOcean App Platform. Consider SiteGround if managed cloud resources and automated scaling are central to your requirements.

The objective is to select infrastructure that can support the publication's current needs while providing a clear path to handle future growth.

How AWS Is Designed: Regions, Availability Zones, and Where a Workload Runs

Wed, 21 Oct 2026 09:20:05 +0000

Most AWS learning starts with picking a service. The first decision is actually where the workload runs. Region and Availability Zone choices shape availability, speed, compliance, and cost for everything built on top of them.

WHAT

AWS is organized into Regions (geographic areas), each containing multiple Availability Zones (isolated data center groups), with edge locations placed close to end users.

AWS infrastructure has three layers:

Region: a geographic area such as Mumbai, Virginia, or Frankfurt. Each Region operates independently of the others.
Availability Zone (AZ): one or more physically separate data centers inside a Region, each with its own power, cooling, and networking. AZs in the same Region are connected by fast, low-latency links (very short delay between them).
Edge location: a small site close to end users that caches content for faster delivery. Amazon CloudFront, AWS's content delivery service, uses this network.

A Region decides where a workload lives. AZs decide how well it survives failures.

WHY

Any single data center can fail, and Regions exist because distance and law both matter.

Power, network, and natural events can take a data center offline. An application running in only one location goes down with it. Because AZs are physically separated, an application spread across two or more keeps running when one fails.

Regions solve a different problem. Choosing the Region affects:

Latency (response delay): users in Singapore experience real delay reaching a server in Virginia.
Data residency: some countries and contracts require data to stay within specific borders.
Service availability: not every AWS service or feature exists in every Region.
Compliance: regulatory frameworks may restrict where data can be stored and processed.
Cost: prices for the same service differ by Region.

These choices are hard to change later, and mistakes in Region choice or redundancy often surface only during an outage or an audit.

WHEN

Choose the Region by user location and compliance, then use multiple AZs whenever downtime has a business cost.

Region and AZ decisions come before most service decisions, because they constrain what follows. Some services, such as IAM and Route 53, are global, but most workloads are tied to the Region they are built in.

Region decision order

If a legal or contractual rule fixes the location, that decides it.
Otherwise, pick the Region closest to most users.
Then confirm the required services are available there, and compare cost.

Single AZ vs. multiple AZs

A single-AZ setup is simpler and cheaper but cannot survive a data center failure, so it suits development, testing, or a proof of concept (POC, a small trial build). For anything a customer depends on, multi-AZ is the baseline expectation rather than an upgrade. The distinguishing signal: if an hour of downtime would cost money, hurt customers, or breach an agreement, the workload should span at least two AZs.

Multi-AZ vs. multi-Region

Multi-AZ protects against a data center-level failure. Multi-Region protects against a whole Region becoming unavailable, but adds significant cost and complexity. Most workloads need only multi-AZ.

HOW

Run copies of each layer in two AZs, so a failure in one is absorbed by the other.

Example scenario: a production setup in the Mumbai Region.

EC2 instances (virtual servers rented from AWS) run in two AZs, one in each.
A load balancer (a traffic distributor that sends requests only to healthy servers) sits in front and receives all user requests.
The database runs on RDS (AWS's managed database service) in Multi-AZ mode, which keeps an automatic standby copy in the second AZ.
If one AZ fails, the load balancer stops sending traffic there, and the standby database is promoted to primary.
The web layer shifts almost instantly, and the database typically recovers within a minute or two.

Flow: Users → Load Balancer → EC2 instances in AZ-a and AZ-b → RDS primary (AZ-a) with standby (AZ-b)

A multi-AZ setup in one Region. If AZ-a fails, the load balancer routes all traffic to AZ-b and the RDS standby is promoted to primary.

Where the Services Fit

AWS services are building blocks in layers, and the Region and AZ layout decides how they are placed.

Memorizing every service matters less than knowing which layer each one covers:

Compute: EC2, Lambda, ECS, EKS
Storage: S3, EBS, EFS
Database: RDS, Aurora, DynamoDB
Networking: VPC (a private network inside AWS), Route 53, CloudFront
Security: IAM (access control), KMS, WAF, GuardDuty
Integration: SQS, SNS, EventBridge
Monitoring: CloudWatch, CloudTrail

A sound design answers six questions:

Where should the workload run?
How will it remain available?
How will users access it?
How will it scale?
How will it be secured?
What will it cost?

The first question shapes the other five. This is where cloud architecture moves beyond knowing individual services and starts designing solutions.

Takeaway

The one thing to remember: Choose the Region for compliance and latency, then spread across Availability Zones for resilience, because that one decision shapes every architecture choice that follows.

n8n Cloud vs Self-Hosted: Real Cost for Small Teams

Tue, 13 Oct 2026 15:50:00 +0000

For most small teams without a reliable server owner, n8n Cloud is the cheaper operational choice even when self-hosting has the lower monthly invoice. Self-host Community Edition when someone already manages Linux, Docker, backups, monitoring, and recovery, or when data location and infrastructure control justify that work.

Partner disclosure. This guide includes a partner link to n8n Cloud. If you sign up through it, I may earn a commission at no extra cost to you.

Short answer: choose Cloud Starter when 2,500 monthly executions and one shared project are enough. Choose Cloud Pro when the team needs 10,000 executions, more concurrency, and more collaboration. Choose Community Edition only after naming the person who owns updates, security, backups, and restores.

The cost at a glance

Option	Published price	What the team still owns
n8n Cloud Starter	€20/month, billed annually	Workflows, credentials, access decisions, and process ownership
n8n Cloud Pro	€50/month, billed annually	The same business ownership, with more execution and collaboration capacity
Self-hosted Community Edition	No software license fee for permitted internal use	Server, database, TLS, updates, backups, monitoring, security, and recovery
Self-hosted Business	€667/month, billed annually	Infrastructure operations plus the paid license and its execution quota

These are the prices shown on the official n8n pricing page on July 18, 2026. Prices and plan limits can change. Starter includes 2,500 workflow executions, one shared project, and five concurrent executions. Pro includes 10,000 executions, three shared projects, and 20 concurrent executions. Both list unlimited users and workflows.

What n8n Cloud actually buys

The Cloud subscription is not just a place to run the editor. n8n hosts the service, so your team does not need to provision a VPS, expose it safely to the internet, maintain Docker, or schedule application updates. This removes an operational job from a small team.

Cloud also makes collaboration easier. Starter includes one shared project, while Pro includes three and adds admin roles, global variables, workflow history, and execution search. Those features matter when more than one person builds or supports workflows. Community Edition does not include Projects or workflow and credential sharing, so a technically free server can be a poor fit for a team that expects clean access boundaries.

There are still limits. Cloud pricing is based on complete workflow executions, and retained execution data is constrained by plan. The current pricing table lists maximum execution-log retention of seven days on Starter and 30 days on Pro. Cloud also fixes the hosting location for standard hosted plans. n8n currently says hosted data is stored on servers in Frankfurt, Germany.

What self-hosted Community Edition costs

The Community Edition includes almost the complete core feature set, but the official feature list excludes Projects, sharing, environments, SSO, external secrets, external binary storage, log streaming, multi-main mode, and Git version control. Registered Community Edition adds folders, debug-in-editor tools, and custom execution data, but it does not turn the instance into the Business plan.

The first invoice is usually the VPS. DigitalOcean currently lists a basic 1 GB Droplet at $6/month and 2 GB at $12/month. Its weekly backup option costs 20% of the Droplet price, while daily backups cost 30%. That produces a simple infrastructure range like this:

Example	Compute	Weekly provider backup	Annual infrastructure
1 GB light-use VPS	$6/month	$1.20/month	$86.40/year
2 GB small production starting point	$12/month	$2.40/month	$172.80/year

The 2 GB example is a planning starting point, not an n8n sizing guarantee. Workflow concurrency, Code nodes, large payloads, AI tasks, binary data, execution retention, and PostgreSQL all change the requirement. Hetzner and other providers may price similar servers differently by region. Provider prices exclude the value of your time and may exclude tax, IPv4, object storage, or other services.

The hidden self-hosted bill

Updates

n8n publishes minor releases frequently. Its self-hosted update guidance recommends updating at least monthly, reading release notes for breaking changes, and testing before production. Current Docker documentation lists stable version 2.26.8 as of this review. A safe update therefore needs a backup, a maintenance window, a smoke test, and a rollback path.

Backups and credentials

A workflow JSON export is useful, but it is not a complete instance backup. n8n stores workflows, credentials, and execution data in SQLite by default or in PostgreSQL when configured. The persistent .n8n directory also contains important data such as the encryption key. n8n uses that key to encrypt credentials before storing them in the database. The self-hosted n8n backup and restore guide turns those parts into a tested recovery sequence.

A practical recovery plan protects the database, the persistent n8n volume or explicit encryption key, the deployment configuration, and any binary-data location. It also proves that a restore works on a separate instance. A backup that has never been restored is only a hope.

Security and monitoring

Self-hosting means owning operating-system patches, Docker exposure, HTTPS, firewall rules, account access, database health, disk usage, and alerts. n8n's own Docker guide says self-hosting requires server, container, scaling, security, and application knowledge, and recommends Cloud for people without that experience because mistakes can cause data loss, security problems, and downtime.

This is not a reason to avoid self-hosting. It is a reason to price the operator. If maintenance and verification take one hour per month and that person's time is worth $30/hour, the labor assumption adds $360 per year. Two hours per month adds $720. Use your real internal rate rather than treating this time as free.

When self-hosting wins

You already operate servers. n8n becomes another monitored container instead of a new discipline.
You need control over data location. You choose the region, database, retention, network, and backup destination.
You need self-host-only capabilities. Bash execution and custom nodes can justify the infrastructure.
Your workload is high or unusual. You can size compute and storage around the workflow instead of a standard hosted plan.
You have a tested recovery owner. One named person can update, restore, and troubleshoot the instance.

My self-hosted n8n deployment case shows what that responsibility looks like in practice: Docker, PostgreSQL, Cloudflare, persistent storage, monitoring, and backups, not just one container command.

When n8n Cloud wins

No one owns infrastructure. The team needs workflows, not a second job maintaining the workflow server.
The first automation matters to revenue. A predictable platform bill is easier to defend than an unplanned recovery incident.
Several people need access. Shared projects and collaboration are part of the requirement, not optional extras.
The execution allowance fits. Starter or Pro covers the expected runs with room for retries, testing, and growth.
You want the fastest start. The team can build the workflow without first deploying a public production service.

If you are still deciding whether n8n fits the process at all, start with the plain-English n8n guide for small businesses. If the choice is between automation platforms, the Zapier vs Make vs n8n pricing comparison explains how each one counts usage.

If Cloud appears to fit, use the 14-day n8n Cloud trial guide to test three small workflows, break them deliberately, estimate production executions, and make a keep-or-cancel decision before paying.

Do not ignore the self-hosted license

Community Edition is source-available under n8n's Sustainable Use License, not a conventional open-source license. The official license FAQ allows internal business use, personal use, and consulting around n8n. It does not allow white-labeling n8n or hosting it and charging customers for access without a separate agreement.

License check: automating your own company's processes is the normal allowed case. If customers will access n8n, provide their own credentials, or buy a product whose value substantially derives from n8n, confirm the arrangement with n8n before building the business around Community Edition.

A five-minute decision rule

Estimate complete workflow executions for a normal month and a busy month.
List how many people need to build, inspect, or support workflows.
Decide whether Projects, sharing, SSO, environments, or version control are required.
Price the server, backups, storage, domain, and monitoring.
Add real operator time for updates, testing, incidents, and restores.
Name the recovery owner. If no one accepts the job, choose Cloud.

For a one-person technical operation, Community Edition can be excellent value. For a small team with no infrastructure owner, Cloud Starter is often cheaper than one avoidable incident. Cloud Pro becomes the practical choice when production volume and collaboration outgrow Starter. Self-hosted Business is a different buying decision for companies that need paid governance features while keeping the infrastructure under their control.

Sources checked

n8n partner link

Want to start without maintaining the server?

I may earn a commission if you sign up through this link, at no extra cost to you.

Try n8n Cloud

Still unsure which n8n setup fits?

Send the expected workflow volume, team size, data constraints, and who can own the server. I will help turn those inputs into a defensible hosting decision.

Compare the options

Originally published on FloxoLab.

Zero-Cost Cloud & Agentic AI: How to Build Production-Grade Systems Without Burning Cash

Sat, 10 Oct 2026 16:08:24 +0000

A practical guide to leveraging Microsoft's free developer sandboxes, $100 student compute, and enterprise Copilot Studio orchestration.

When building systems like SwarmForge or open-source utilities like Win-Optimizer-Pro, one principle has guided my engineering: High leverage doesn't require high capital.

Most tutorials today make you believe that building AI systems requires paying $20/month for APIs, renting expensive GPU instances, and subscribing to multiple SaaS tools before writing a single line of useful logic.

In reality, Big Tech ecosystems offer massive, high-performance developer infrastructure for completely zero cost — if you know where to look.

Here is how you can set up a zero-cost production stack for cloud computing and enterprise agentic AI.

1. The Compute Layer: $100 Cloud Compute with Zero Billing Risk

Every developer needs somewhere to host databases, deploy containerized background workers, and run full-stack APIs.

Instead of paying for third-party VPS hosting out of your pocket, Microsoft runs the Azure for Students Program.

What you get:

$100 in annual cloud credits automatically renewed while you're a student.
No Credit Card Required: Verification is done solely via student email or institution ID, meaning zero risk of surprise overdraft fees.
Free Services Included:
- B-series Linux and Windows Virtual Machines
- Azure SQL Database & Cosmos DB instances
- Azure App Services for deploying web apps

If you haven't claimed your student allocation yet:
👉 Claim $100 Free Azure for Students

2. The Agentic Layer: Free Enterprise Copilot Studio Sandbox

Building toy chatbots with simple prompt wrappers is easy, but enterprise engineering demands:

Isolated RAG Knowledge: Restricting the AI strictly to internal Dataverse tables and company documentation with zero public internet leaks.
Deterministic State Workflows: Handling multi-branch decision trees (e.g., HR approvals, leave calculations) without unpredictable LLM hallucinations.
Automated Tool Calling: Triggering backend workflows (Power Automate) securely.

Normally, spinning up a Power Platform and Copilot Studio environment requires commercial Microsoft 365 licensing.

However, Microsoft provides a completely free, on-demand browser VM sandbox through its Applied Skills program:

👉 Launch the Free Microsoft Copilot Studio Assessment Lab

Why this is huge for builders:

You get an isolated, high-performance Windows VM directly in your browser.
You can design, ground, test, and deploy enterprise AI agents to Microsoft Teams.
Passing the automated lab awards you an official verifiable credential signed by Microsoft CEO Satya Nadella.

3. The Developer Tooling Layer: Cloud Workspaces & VS Code

If you work on large monorepos (like contributing to Microsoft PowerToys, Meta Docusaurus, or the Tauri ecosystem), local setup friction can slow you down.

Standardizing your toolchain with remote containers and cloud-ready extensions allows you to develop from low-spec laptops without compiling lag:

👉 Download the Latest Visual Studio Code & Remote Dev Tools

4. The Zero-Cost Systems Architecture Blueprint

Here is how all these pieces fit together into a production-grade pipeline:

┌────────────────────────────────────────────────────────┐
│                   DEVELOPER WORKSPACE                  │
│          Visual Studio Code + Remote Containers        │
└───────────────────────────┬────────────────────────────┘
                            │
                            ▼
┌────────────────────────────────────────────────────────┐
│                   BACKEND INFRASTRUCTURE               │
│          Azure for Students ($100 Free Credits)        │
│          • Node.js / Rust APIs on Linux VMs            │
│          • Cloud SQL / PostgreSQL Databases            │
└───────────────────────────┬────────────────────────────┘
                            │
                            ▼
┌────────────────────────────────────────────────────────┐
│                   AGENTIC REASONING CORE               │
│               Microsoft Copilot Studio                 │
│          • Grounded RAG (Internal Docs & Tables)       │
│          • Deterministic Topic Routing                 │
│          • Multi-Channel Teams & Web Deployment        │
└────────────────────────────────────────────────────────┘

Summary

You don't need a venture-backed budget to ship serious technology.

By stacking:

Azure for Students for free compute and databases
Microsoft Copilot Studio for grounded enterprise agent orchestration
VS Code & GitHub for streamlined development

You can build, deploy, and scale complex multi-agent architectures completely for $0.

What zero-cost tools are you using in your stack? Share your setup below.

Stop Selling Event-Driven Architecture. Start Selling Business Outcomes

Wed, 07 Oct 2026 13:28:48 +0000

Hi dev community! I originally published this on my LinkedIn Newsletter, but wanted to share it with the dev community here to get your thoughts on how your engineering teams handle infrastructure ROI. Let's discuss in the comments!

Why are the best EDA initiatives funded through business value, not technical excellence.

Many organizations view Event-Driven Architecture (EDA) as a technology modernization initiative focused on event streams, messaging platforms, and real-time integration. While these capabilities are important, they rarely justify investment on their own.

Business leaders fund outcomes, not architecture. One of the biggest challenges in securing EDA investment is focusing too heavily on technical benefits such as scalability, decoupling, and architectural elegance. These matter to technology teams, but business stakeholders are more interested in the impact on business performance, customer experience, operational efficiency, and risk management.

The key question is simple: "How will this improve business outcomes?" If we cannot answer that clearly, gaining support and funding becomes difficult.

For more insights, see my newsletter: "Event-Driven Architecture: Aligning Technology with Business Outcomes."

Start with the Business Problem

From my experience, successful EDA adoption starts with understanding business challenges, not promoting technology.

Conduct a thorough analysis of stakeholder use cases and current operational or system pain points. Then demonstrate how EDA can improve information flow, increase responsiveness, and remove bottlenecks.

This is where many IT teams and vendors fail. They focus on explaining the technology rather than the business value. Business leaders invest in outcomes, not architecture.

The technology remains the same, but the story becomes relevant to those making investment decisions.

Start Small and Prove the Value

Large-scale transformation programs often struggle because they ask stakeholders to commit significant investment before any value has been demonstrated.

A more effective approach is to identify a small but high-value use case and establish a proof of concept. When stakeholders see tangible improvements, support often expands organically to other business domains.

Four Lessons Learned from Driving EDA Adoption

Tailor the Conversation to Each Stakeholder

Not all stakeholders care about the same outcomes. Using the same narrative for every audience is rarely effective.

Instead, align the discussion with the objectives and success metrics that matter most to each stakeholder group.

Understand the Business and System Landscape Thoroughly

To build credibility, you need more than architectural knowledge.

You should understand:

Business processes
System interactions
Upstream and downstream dependencies
Data flows
Operational constraints
Regulatory considerations

Stakeholders frequently ask questions that extend beyond the immediate solution. Being prepared to discuss the broader landscape demonstrates ownership and builds confidence in the proposed approach.

Use Real Numbers Whenever Possible

One of the most effective ways to gain support is to quantify the expected benefits.

Avoid statements such as:

"The process will be much faster."
"The system will be more efficient."
"Users will have a better experience."

Instead, use measurable outcomes:

Reduce processing time from 4 hours to 5 minutes.
Improve customer notification speed from next-day to near real-time.
Reduce manual intervention by 60%.

Numbers transform assumptions into business cases.

Translate Technical Concepts into Business Language

Many stakeholders are not interested in technical architecture diagrams.

That does not mean they lack expertise.

In fact, they often possess a deep understanding of the business processes and systems they manage every day. When people understand the outcome, they become far more willing to support the solution.

They invest in better business outcomes that Event-Driven Architecture makes possible.

Takeaway

The fastest way to gain support for EDA is to stop selling architecture and start telling a business story. When stakeholders can clearly see the operational, financial, or customer benefits, funding conversations become significantly easier and adoption follows naturally.

Stay Ahead of the Curve If you enjoyed these practical insights on Event-Driven Architecture, APIs, observability, and digital transformation, consider subscribing to my newsletter on LinkedIn.

Subscribe to the LinkedIn Newsletter here to get the next framework directly in your feed!

What are your thoughts on EDA as a cost-saving mechanism? Let’s debate in the comments below!

The Real Value of Event-Driven Architecture Isn't Technology. It's Cost Reduction.

Wed, 07 Oct 2026 13:26:53 +0000

How Event-Driven Architecture (EDA) Significantly Reduces Enterprise IT Costs

In today's digital world, business users expect new features and services to be delivered faster than ever before, with minimal risk and within increasingly constrained budgets.

Unfortunately, traditional Request-Driven Architecture (RDA) often struggles to meet these expectations. As organizations scale, integration complexity, infrastructure costs, and operational overhead continue to grow, making even small changes expensive and time-consuming.

This is where Event-Driven Architecture (EDA) creates a significant advantage.

EDA is not just a modern integration pattern. It is a strategic approach that helps organizations reduce costs, accelerate delivery, and improve scalability while maintaining flexibility for future growth.

⚠️ Why Traditional Architecture Is Expensive

Complex Infrastructure

Traditional enterprise systems typically require multiple layers of infrastructure to support point-to-point communication between applications, including:

Firewalls
⚖️ Load Balancers
️ Proxy Servers
Multiple Network Segments
️ Dedicated Integration Components

As the number of applications grows, the supporting infrastructure grows with it. The result is higher hardware costs, increased operational effort, and greater complexity.

Integration Sprawl

Most traditional environments are built around tightly coupled point-to-point integrations.

Every application needs to know:

Who to connect to
Which protocol to use
What data format to exchange
How to handle failures and retry
How to buffer messages when a target system is unavailable

As new systems are added, the number of interfaces grows exponentially.

For an enterprise with hundreds of services, maintaining thousands of interfaces becomes a major cost driver.

Even a seemingly simple change to a single data element may require modifications and testing across many dependent systems.

Business Impact

Higher development costs
✅ Extensive No-Harm Testing Requirements
️ Increased maintenance effort
⚠️ Greater implementation risk
Longer project timelines
Additional monitoring and compliance costs
⏳ Slow and Expensive Change Delivery

Ultimately, the largest cost in enterprise IT is not infrastructure or software licenses. It is the cost of change.

The more connected and tightly coupled systems become, the more expensive every change becomes. EDA addresses this challenge by enabling systems to evolve independently, significantly reducing the cost, effort, and risk of change.

How EDA Reduces Costs

Organizations can address these challenges by implementing an Event Mesh, which securely connects applications across multiple network environments while providing a scalable foundation for self-service integration.

Combined with microservices, automation, and DevOps practices, EDA enables a much more efficient operating model.

The cost savings are substantial.

1. Reducing Integration Costs

Event-Driven Architecture (EDA) breaks down traditional, complex point-to-point integrations into multiple simpler, reusable interfaces. Events can be shared across different services rather than requiring dedicated integrations for each consuming application.

This approach significantly simplifies solution design, development, testing, support, and ongoing maintenance, resulting in a lower Total Cost of Ownership (TCO).

In large enterprises, integration development, testing, and maintenance can consume a significant portion of project budgets. By promoting reusable event streams and loose coupling, EDA can substantially reduce these ongoing integration costs.

️ 2. Lower Infrastructure Costs

EDA simplifies communication between systems through a shared messaging platform.

Using capabilities such as:

Exclusive Queues
Non-Exclusive Queues
Publish/Subscribe Messaging

Organizations can significantly reduce the need for:

Additional load balancers
Complex integration gateways
Dedicated communication infrastructure

Result

✅ Lower hardware investment
✅ Reduced software licensing costs
✅ Simplified infrastructure management

3. Optimizing Network and Bandwidth Usage

Many traditional integration solutions repeatedly exchange request/response messages or data synchronization payloads, even when only a small subset of data has changed.

EDA works differently.

Events are transmitted when changes occur.

Consumers subscribe only to the events they need.

Result

✅ Smaller message sizes
✅ Less network traffic
✅ Reduced bandwidth consumption
✅ Improved overall efficiency

This translates directly into lower infrastructure and operational costs.

⚙️ 4. Lower Operational Costs

In an EDA environment, applications communicate through the event mesh instead of maintaining direct relationships with numerous systems.

This greatly simplifies operations.

Teams no longer need to manage thousands of dependencies or firewall rules between applications.

Result

✅ Simpler architecture
✅ Reduced operational effort
✅ Faster problem resolution
✅ Lower support costs
✅ Improved system scalability

⏱️ 5. Faster Delivery Equals Lower Cost

Perhaps the most overlooked benefit of EDA is the reduction in delivery time.

Because systems are loosely coupled:

Teams can work independently.
New consumers can subscribe without impacting producers.
Changes can be deployed with less regression testing.
Business capabilities can evolve at their own pace

Result

✅ Faster time-to-market
✅ Lower project costs
✅ Reduced implementation risk
✅ Greater business agility

6. Increased Reusability and Economies of Scale

Events can be reused by multiple consumers simultaneously. As adoption grows, the cost of producing an event remains relatively constant while the business value generated by that event continues to increase. This creates economies of scale that are difficult to achieve with traditional point-to-point integrations. Each additional consumer can leverage existing events without requiring new integrations, allowing organizations to increase business value while minimizing incremental delivery costs.

Final Thoughts

When discussing Event-Driven Architecture, many people focus on technology. However, the real value of EDA is not technology alone. The real value is cost reduction.

By simplifying integrations, reducing infrastructure requirements, lowering operational overhead, minimizing network complexity, and accelerating delivery, EDA enables organizations to do more with less.

In our experience, EDA is not merely an architectural evolution. It is a practical strategy for reducing Total Cost of Ownership while increasing agility and scalability.

The true power of EDA is not simply connecting systems. It is enabling change at a lower cost, with lower risk, and at greater speed. In today's digital economy, the organizations that can change fastest are often the ones that win.

Stay Ahead of the Curve If you enjoyed these practical insights on Event-Driven Architecture, APIs, observability, and digital transformation, consider subscribing to my newsletter on LinkedIn.

Subscribe to the LinkedIn Newsletter here to get the next framework directly in your feed!

What are your thoughts on EDA as a cost-saving mechanism? Let’s debate in the comments below!

Upload Images With Node.js And Express To The Cloud Using Cloudinary

Wed, 07 Oct 2026 10:46:35 +0000

Today's websites and mobile applications worldwide requires the functionality to upload image somewhere to the cloud for storing and easy retrieval since in most cases images must be stored somewhere in the cloud for easy management (storing, retrieval, integration etc. ) and not directly into a database. Luckily Cloudinary comes to help us get this done in a few simple steps using Node.js and Express so

WHAT IS CLOUDINARY ?

Cloudinary is a SaaS technology company that provides a cloud-based image and video management services.

WHAT IS CLOUDINARY USED FOR ?

With Cloudinary, you can easily upload images and videos to the cloud and automate smart manipulations of those media without installing any other software. Cloudinary then seamlessly delivers your media through a fast content delivery network (CDN), optimized with the industry’s best practices. It basically enables users to upload, store, manage, manipulate, and deliver images and videos for websites and applications.

Additionally, Cloudinary offers comprehensive APIs and administration capabilities, which you can easily integrate with your web and mobile applications.

In this guide you'll learn how to upload images to Cloudinary with the Node.js SDK which provides simple, yet comprehensive image and video upload, transformation, optimization, and delivery capabilities that you can implement using code that integrates seamlessly with your existing Node.js application.

To begin, you must be able to use the terminal, familiar with Node.js , Npm , Github , you must also have Cloudinary account, if you don't, sign up for free here to get your API Key and other credentials needed for the project.

After signing up, it should take you to the dashboard, which looks like this

You can find your CLOUD NAME, API and SECRET KEY on the dashboard good!! that means we are ready, you can even go to the settings page to see how you can upgrade your current plan to suit your business needs, but for now we will go with the free plan which gives us 1,000 Transformations OR 1 GB Storage OR 1 GB Bandwidth.

To begin, you should get the exercise files from my Github, follow the instructions in the README.md file to get started. Everything is arranged for you so we can start integrating the image upload functionality.

After you're done, your project structure should look like this

Start the development server with npm:

npm start

And navigate to http://localhost:8080 in your internet explorer to see the starter project

CAUTION ⚠️

If you get any error with nodemon because it's not installed on your machine you can solve it by installing it via npm :

npm install -g nodemon

Or yarn :

yarn global add nodemon

Now we can install the rest of the packages needed for the project which is Cloudinary , multer( Node.js middleware for handling multipart/form-data, which is primarily used for uploading files ),multer-storage-cloudinary ( A multer storage engine for Cloudinary ) and dotenv to set up the environment variables, with npm :

npm install  cloudinary multer multer-storage-cloudinary dotenv

After these are done installing, we need to configure the storage, in the top level directory ( same as index.js) create a storage directory by running this command or create it manually :

mkdir storage

This will create storage directory, open and create a file storage.js inside the storage directory,
run this command to create it :

cd storage && touch storage.js

Good! now our project structure should look like this

Open storage.js and require Cloudinary and multer-storage-cloudinary

const cloudinary = require('cloudinary').v2;
const { CloudinaryStorage } = require('multer-storage-cloudinary');

Now we can use multer-storage-cloudinary' to create the storage engine to store images in Cloudinary , firstly we need to configure Cloudinary with the cloud_name, api_key and api_secret and use the configuration to create the storage engine with multer-storage-cloudinary', paste the following code below what you just required on storage.js :

cloudinary.config({
    cloud_name:process.env.CLOUD_NAME,
    api_key: process.env.CLOUDINARY_KEY,
    api_secret:process.env.CLOUDINARY_SECRET
});

const storage = new CloudinaryStorage({
    cloudinary,
    params: {
        folder: 'CloudinaryDemo',
        allowedFormats: ['jpeg', 'png', 'jpg'],
    }
});

module.exports = {
    storage
};

So the storage.js file should look like this

const cloudinary = require('cloudinary').v2;
const { CloudinaryStorage } = require('multer-storage-cloudinary');

cloudinary.config({
    cloud_name:process.env.CLOUD_NAME,
    api_key: process.env.CLOUDINARY_KEY,
    api_secret:process.env.CLOUDINARY_SECRET
});

const storage = new CloudinaryStorage({
    cloudinary,
    params: {
        folder: 'CloudinaryDemo',
        allowedFormats: ['jpeg', 'png', 'jpg'],
    }                                                              
}); 

module.exports = {
    storage
};

NOTE : Allowed formats can be omitted and cloudinary will deliver the image as the original image's format.

Since we are the using the environment variables to configure Cloudinary which does not exist yet in the project, we can create the file and embed the CLOUD_NAME, CLOUDINARY_KEY and CLOUDINARY_SECRET

In the same directory as index.js run :

touch .env

This will create .env file

Open and paste the following code in it :

CLOUD_NAME= your cloud name goes here 
CLOUDINARY_KEY=  your API KEY goes here 
CLOUDINARY_SECRET= your secrete goes here

Now go back to Cloudinary's dashboard to copy the details and set them in the .env file accordingly
[ **see below image**]

Once you are done, your .env file should copy this

Good now let's require multer and the storage we exported from storage.js so we can configure it to work with express to upload images, open index.js file and require (on the top level preferably below port):

const { storage } = require('./storage/storage');
const multer = require('multer');
const upload = multer({ storage });

If you check above, we are requiring multer and the storage configuration we exported from storage.js and passing the storage to multer to use the cloudinary storage instance. (Multer acts as a storage engine for Cloudinary).

Finally we can configure the route and pass upload to it so we can upload images, in the same index.js, paste this code (below " code goes here"):

//we used upload.single to tell "multer" to upload
// only single image 
 app.post('/upload', upload.single('image'), (req, res) => {
  console.log(req.file);
  res.send('Done');
});

NOTE: Cloudinary gives us the flexibility to specify the maximum and minimum number of images we wish to upload, here we specified single meaning to upload one image file, conversely we can pass upload.array to upload multiple images.

We need to make the environment variables available in the project so paste this on (line 1 top level) in index.js file:

require('dotenv').config();

The complete entry file index.js should look like this :

require('dotenv').config();
const express = require('express');
const app = express();
const path = require('path');
const ejs = require('ejs');
const port = 8080;

const { storage } = require('./storage/storage');
const multer = require('multer');
const upload = multer({ storage });

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

app.set('view engine', 'ejs');
app.set('views', path.join(__dirname, './views'));

app.get('', (req, res) => {
  res.render("home");
});

// code goes here
 app.post('/upload', upload.single('image'), (req, res) => {
  console.log(req.file);
  res.send('Done');
});

app.listen(port, () => {
  console.log(`Server is listening on port ${port}`);
});

In order to upload images, open home.ejs file, specify the form action to /upload and method to POST additionally set the form encoding data type attribute to accept multipart/form-data so we can upload images and files through the form.

 <form action="/upload" method="POST" enctype="multipart/form-data">
              <div class="mb-3">
                    <div class="form-file custom-file">
                        <input class="form-control" type="file" id="formFile" name="image" multiple>
                        <label class="form-file-label" for="image"
                        </label>
                    </div>
           </div>
     <button type="submit" class="btn btn-success">Upload</button>
</form>

The action attribute specifies where to send the form-data when a form is submitted. We specified in the post route to /upload and that is where the form will be submitted to.

The enctype attribute specifies how the form-data should be encoded when submitting it to the server. It can be used only if method="POST"
And the method specifies the HTTP verb we are using in this case POST.

Now with all this in place, we can we can start the server if not already started and navigate to http://localhost:8080 in your internet explorer, upload image through the form within the specified formats eg. jpeg, jpg, png.

If everything goes well , you should get something like this

Indicating that the image was uploaded successfully, and when you go to Cloudinary's dashboard, Media library you can see Cloudinary made a new folder for us as specified in the storage configuration CloudinaryDemo and stored the uploaded image in that folder.

And in the terminal since we logged req.file to the console, we can see what cloudinary gives us in return to the image upload.

{
  fieldname: 'image',
  originalname: 'download.png',
  encoding: '7bit',
  mimetype: 'image/png',
  path: 'https://res.cloudinary.com/evans646/image/upload/v1643114879/CloudinaryDemo/n496jz94zhwcte8kskhh.png',
  size: 5761,
  filename: 'CloudinaryDemo/n496jz94zhwcte8kskhh'
}

The path is the url of the image that we can use in our projects and or web applications. When you click on it (command+ left click) on mac or copy the path and paste it in your browser and you can see the same image that we uploaded to Cloudinary .

Summary

You have learned how to upload images with node.js and express to Cloudinary with the Node.js SDK which provides simple image and video upload, transformation, optimization, and delivery capabilities that you can implement using code that integrates seamlessly with your appliaction.

Cloudinary enables users to upload, store, manage, manipulate, and deliver images and videos for websites and applications. You can check out the documentation here

If you run into any issue, you can always comment here or connect with me on Linkedin Instagram Twitter

Congratulations

Thanks a lot for coming through and Good luck! to you as well.

Shared Hosting vs Managed Cloud Hosting: Which Is Right for Your Growing Blog?

Tue, 29 Sep 2026 13:22:40 +0000

Choosing web hosting is one of the first technical decisions a blogger makes, but it becomes more consequential as a site grows. A new blog may operate comfortably on an inexpensive shared hosting plan. As readership increases, however, heavier traffic, larger media libraries, more plugins, more frequent publishing, and higher expectations for page speed can put greater demands on the hosting environment.

The decision eventually becomes less about getting a website online and more about choosing an infrastructure that can support the website reliably.

Two options commonly considered at this stage are shared hosting and managed cloud hosting. They can both support a blog, but they differ substantially in how computing resources are allocated, how infrastructure is maintained, how easily resources can be increased, and how much technical responsibility falls on the site owner.

There is no universal hosting choice for every growing blog. The appropriate option depends on traffic patterns, technical requirements, budget, operational priorities, and how much server administration the publisher wants to handle.

What Is Shared Hosting?
Shared hosting places multiple websites on the same physical server and allows those websites to share resources such as CPU, RAM, storage, and other server infrastructure. Amazon Web Services describes shared hosting as a model in which customers share server resources, while noting that it is commonly suitable for personal websites, blogs, and small businesses.

The main attraction is cost.

Instead of paying for an entire server or a more sophisticated infrastructure, the customer pays for a portion of a shared environment. Hosting companies generally package server management, control panels, software installation, email services, backups, SSL options, and other functions into relatively simple plans.

For a new blogger, this can be practical. A website with modest traffic and relatively straightforward requirements does not necessarily need an elaborate hosting architecture.

Shared hosting also tends to require less technical knowledge. The provider manages the underlying server, allowing the blogger to concentrate on publishing rather than server administration.

The limitation is resource competition. Because multiple websites use the same underlying server, the available resources are not dedicated exclusively to one blog. Google Cloud describes shared hosting as economical but notes that individual websites are limited by the resources of the server they share.

That distinction becomes more important as traffic grows.

What Is Managed Cloud Hosting?
Cloud hosting uses a network of virtual or physical computing resources rather than relying exclusively on a single traditional server. Google Cloud describes cloud hosting as an environment that can add or remove resources as demand changes, while AWS describes cloud hosting as an approach that can use multiple servers and provide greater flexibility than traditional hosting.

The word "managed" adds another layer.

With managed hosting, the hosting provider takes responsibility for significant portions of infrastructure administration. Depending on the provider and plan, this can include server maintenance, security updates, backups, monitoring, software configuration, and technical support. Google Cloud identifies server maintenance, security, software and hardware upgrades, and backups among responsibilities that can be handled by a managed hosting provider.

Managed cloud hosting therefore combines two concepts:

Cloud infrastructure that can provide greater flexibility in resource allocation.
Provider management that reduces the amount of server administration required from the customer.
The exact services included vary considerably between hosting companies. A buyer should therefore examine the actual plan documentation rather than assuming that every product labeled "managed cloud hosting" provides identical resources or services.

Performance: What Actually Matters?
Hosting is only one part of website performance.

WordPress identifies several factors that can affect performance, including the hosting environment, WordPress configuration, software versions, image sizes, caching, and the use of a CDN.

That matters because upgrading from shared hosting to cloud hosting does not automatically make every page fast.

A poorly optimized blog with oversized images, excessive plugins, inefficient database queries, and inadequate caching can remain slow after moving to more powerful infrastructure.

Caching is particularly important. WordPress explains that caching allows previously generated results to be reused instead of requiring the server to perform the same processing for every request. Server-side caching, browser caching, object caching, and other caching layers can reduce processing requirements and improve response times.

A CDN can provide another performance layer. WordPress explains that a CDN distributes static content through multiple endpoints, allowing visitors to receive files from infrastructure closer to their location.

For a growing blog, therefore, the relevant question is not simply, "Which hosting type is faster?" It is, "Which infrastructure provides the resources and optimization options my website actually requires?"

Scalability Is Where the Difference Becomes More Significant
A small blog may have predictable traffic. A growing publication can have unpredictable traffic.

A new article may suddenly receive attention from social media, search engines, newsletters, another publication, or a large online community. Traffic can rise rapidly without the publisher having planned for it.

Traditional shared hosting generally operates within the limits of its server environment. If the blog outgrows the plan, the owner may need to upgrade to another hosting package or migrate to a different infrastructure.

Cloud infrastructure is designed around greater resource flexibility. Google Cloud identifies scalability as a central characteristic of cloud hosting, while AWS describes cloud hosting as capable of scaling resources according to changing requirements.

This does not mean every managed cloud package automatically scales without limits. Capacity, pricing, configuration, and provider policies still apply.

It does mean that cloud hosting can provide a more flexible foundation when a blog's resource requirements are becoming less predictable.

Reliability and Redundancy
Reliability is another important consideration for a commercial blog.

A website outage can prevent visitors from accessing articles, interfere with advertising or affiliate activity, and interrupt other online operations. The infrastructure behind the website therefore matters.

Cloud hosting can distribute workloads across multiple resources rather than relying entirely on one physical server. Google Cloud explains that cloud environments can use multiple servers and that another server can continue providing service when one server fails. AWS similarly describes cloud hosting as using clusters of servers and emphasizes availability and redundancy as benefits of the model.

The actual reliability of a particular hosting service still depends on its architecture and provider practices. "Cloud" alone is not a guarantee of uninterrupted availability.

For that reason, bloggers should examine uptime commitments, backup policies, recovery procedures, monitoring, and support arrangements before purchasing a plan.

Cost: Shared Hosting Usually Starts Lower
Price is often the strongest argument for shared hosting.

Because multiple customers share infrastructure, the cost can be distributed among them. AWS explicitly identifies low monthly fees as one of the benefits of shared hosting.

That makes shared hosting attractive for bloggers who are still establishing their audience or generating little revenue.

Managed cloud hosting generally costs more because the customer may be paying for additional infrastructure, resource flexibility, management, monitoring, support, or other services.

The important question is not simply whether cloud hosting costs more. It is whether the additional cost corresponds to a genuine business requirement.

If a blog has modest traffic, limited technical requirements, and no recurring resource problems, moving to a more expensive environment may provide little practical benefit.

If the site is regularly approaching resource limits or experiencing traffic surges, paying for additional infrastructure may be easier to justify.

Technical Management
Shared hosting is generally attractive to publishers who want minimal involvement with server administration.

The hosting company handles much of the underlying infrastructure, while the customer manages the website itself.

Managed cloud hosting can reduce the technical burden further, depending on the provider. Managed services may cover tasks that would otherwise require technical expertise, including system maintenance, security updates, backups, and infrastructure management.

This distinction is important for bloggers who are building businesses rather than technology projects.

A publisher may be capable of configuring a server but decide that server administration is not the best use of their time. In that situation, managed infrastructure can have value beyond raw performance.

When Shared Hosting Makes Sense
Shared hosting can remain a sensible option when a blog has:

Modest or relatively predictable traffic
Limited server resource requirements
A small media library
A relatively simple WordPress installation
No recurring resource-limit problems
A strong need to control hosting expenses
Little need for advanced server configuration
A blog does not become technologically inadequate merely because it uses shared hosting.

WordPress itself identifies shared hosting as the most common hosting type and notes that websites hosted in such environments share a server with other sites.

For many smaller publishers, that remains sufficient.

When Managed Cloud Hosting Becomes More Practical
Managed cloud hosting becomes more relevant when a blog has requirements such as:

Rapidly increasing traffic
Significant traffic spikes
More demanding WordPress configurations
Resource-intensive plugins
Large or expanding media libraries
Greater dependence on website availability
Requirements for additional server resources
A preference for managed technical operations
A business model where downtime carries meaningful financial consequences
The key is to identify an actual operational requirement rather than upgrading simply because cloud hosting sounds more advanced.

If the current hosting environment is stable, responsive, adequately resourced, and properly configured, there may be no urgent reason to migrate.

Do Not Ignore Caching and CDN Configuration
A hosting upgrade should not become an excuse to ignore basic optimization.

WordPress recommends caching as an important performance measure because cached content can reduce repeated processing by the server. Its documentation also identifies persistent object caching and CDNs as useful components of performance optimization.

A CDN can also reduce the amount of static content that must be delivered directly by the origin server. WordPress explains that CDNs can distribute images, JavaScript, CSS, and other static resources through geographically distributed servers.

Before migrating, a blogger should therefore review:

Page caching
Browser caching
Object caching
Image optimization
Plugin usage
Database performance
CDN configuration
PHP and WordPress versions
Server resource consumption
These checks can reveal whether the actual problem is hosting capacity or website configuration.

How to Decide
The simplest way to approach the decision is to start with evidence from the existing website.

Look at resource usage, traffic patterns, page performance, error logs, storage consumption, and any warnings from the current hosting provider.

If the blog consistently operates within its current limits and performs adequately, shared hosting may remain appropriate.

If the site regularly encounters resource constraints, experiences significant traffic fluctuations, or requires infrastructure capabilities that the existing environment cannot provide, managed cloud hosting deserves closer consideration.

Cost should then be evaluated against the business value of improved infrastructure.

A blog that earns meaningful revenue from advertising, affiliate marketing, digital products, memberships, sponsorships, or other channels may have different hosting requirements from a personal blog that receives a small number of visitors.

The Bottom Line for a Growing Blog
Shared hosting and managed cloud hosting serve different operational needs.

Shared hosting offers simplicity and lower costs by placing multiple websites on shared server resources. It can be entirely adequate for new and smaller blogs with modest requirements.

Managed cloud hosting provides a more flexible infrastructure model and can reduce the amount of technical administration required from the website owner. Cloud environments can also provide greater scalability and redundancy, although the exact capabilities depend on the hosting provider and plan.

For a growing blog, the decision should therefore be based on measurable requirements rather than the prestige of a particular hosting category.

Start with the site's current traffic, resource consumption, performance, reliability requirements, technical workload, and budget. If shared hosting continues to meet those requirements, there is little reason to upgrade solely for the sake of upgrading.

If the blog is encountering resource limitations or requires greater scalability and managed infrastructure, cloud hosting may provide the additional capacity and operational support needed for the next stage of growth.

The best hosting decision is ultimately the one that matches the website's actual workload today while providing a practical path for its expected development.

Kamatera 30-Day Free Trial Guide: $100 Credit Setup & Hidden Traps (2026)

Tue, 29 Sep 2026 10:49:51 +0000

Kamatera's 30-day free trial comes with significantly more nuance than the marketing headline suggests.

"30 days free" is technically accurate, but there are strict guardrails governing what that credit covers — and misreading them is the #1 reason developers end up with unexpected credit card charges.

Here is an unfiltered breakdown of how Kamatera's trial works in 2026, what hardware you actually get, the 3 traps that trigger surprise billing, and how to cancel safely.

What the Trial Actually Includes

According to Kamatera's official policy, the 30-day window covers:

1 Cloud Server (custom CPU, RAM, and SSD)
$100 Configuration Value Cap
1TB Combined Traffic (inbound + outbound)
Up to 1TB Cloud Block Storage
Full management console & 24/7 technical support

Parameter	Official Trial Terms	Developer Reality Check
Duration	Exactly 30 days from registration	Billing starts on minute 1 of Day 31
Active Instances	Max 1 server	Deploying a 2nd server bills immediately
Value Ceiling	Up to $100/month configuration	Exceeding $100 bills the difference
Bandwidth	1TB Combined	Outbound overages are metered
Storage	Up to 1TB Block Storage	Unattached volumes continue billing

The 3 Traps That Trigger Surprise Billing

Most complaints about "unexpected charges" stem from three specific platform behaviors:

1. Powering Off Does NOT Stop Billing

In enterprise cloud platforms, a stopped virtual machine still reserves CPU, memory allocations, public IPv4 addresses, and NVMe SSD space. Kamatera continues charging resources against your quota (or your card) even if the OS is shut down.

The Rule: If you are done testing, terminate/delete the server entirely. Simply turning it off won't stop the clock.

2. The $100 Cap Is a Monthly Rate, Not Spendable Cash

Kamatera does not hand you a $100 balance to spend freely. Instead, it allows you to run one server whose monthly price tag does not exceed $100.

If you spin up a high-spec server configured at $120/month, the $20 overage is billed immediately to your card.
Premium Windows Server licenses or commercial add-ons (cPanel/Plesk) are not covered by the baseline trial credit.

3. Lingering Cloud Block Storage

If you attach a secondary 500GB Block Storage volume during testing, deleting the VPS does not automatically wipe the storage volume. Storage volumes live independently and will continue accumulating monthly storage fees if left behind.

Safe Testing Specs (Stay Under the $100 Cap)

To safely maximize performance while guaranteeing zero charges, here is the recommended sweet-spot configuration:

CPU: 2~4 vCPU (Dedicated or Burstable)
RAM: 4GB ~ 8GB RAM
Storage: 50GB ~ 100GB SSD
OS: Ubuntu 24.04 LTS / Debian 12 (Free Linux distros)
Cost Estimate: Around $35 ~ $65/month — well inside the $100 ceiling.

You can verify and compare regional latency and hardware benchmarks against competitors using independent performance data from the VPS Rankings 2026 Cloud Benchmarks.

End-of-Trial Cleanup Checklist (Do This on Day 28)

If you do not plan to stay on Kamatera as a paid customer, follow this 4-step decommissioning checklist at least 48 hours before Day 30:

Backup Your Work: Download your source code, Docker configs, and database dumps via SFTP/rsync.
Terminate the Server: Navigate to Servers -> Select your instance -> Click Delete/Terminate (Confirm destruction).
Delete Detached Storage: Check Storage -> Block Storage and ensure no unattached disks remain.
Remove Networking Assets: Confirm any additional floating IPs are released back to the pool.

This guide was originally published with interactive hardware specs and live network latency comparisons at VPS Rankings. For real-world server speed tests across global data centers, check out our free global latency test tool.

The 2026 Student Tech Stack: What I Would Learn Before Graduation

Mon, 28 Sep 2026 15:38:52 +0000

Every year, I see students asking the same question:

"Which technology should I learn next?"

Python or Java?

Cloud or AI?

Data or DevOps?

Web development or automation?

The problem is not a lack of learning resources. The problem is trying to learn everything at once.

My view is simple:

Students don't need to learn everything. They need a coherent stack that lets them build.

A student who understands programming, uses modern developer tools, knows basic cloud concepts, can work with AI and data, automates repetitive work, validates skills, and participates in a community will usually move much faster than someone who spends years collecting disconnected tutorials.

The goal before graduation is not becoming an expert in every domain.

The goal is becoming capable of building useful things from idea to deployment.

This is the learning stack I would personally focus on in 2026.

1. Programming

Why it matters

Programming is still the foundation.

AI can generate code.

Low-code tools can accelerate development.

Templates can save time.

But none of these replace the ability to understand logic, debug problems, and design solutions.

When projects become even slightly complex, programming remains the skill that allows you to move independently.

I often meet students who jump immediately into advanced frameworks while struggling with basic problem solving. That usually creates frustration later.

Strong fundamentals age much better than trendy technologies.

What level I would target

I would not aim to become a programming-language historian.

Instead, I would target:

One primary language
Data structures and algorithms fundamentals
APIs
File handling
Git-based development
Basic software architecture concepts
Problem-solving confidence

For many students, Python, C#, or Java can serve this role effectively.

The exact language matters less than your ability to build with it.

One practical project

Build a student productivity application.

Examples:

Assignment tracker
Study planner
Expense manager
Event management tool

The project should store data, have authentication, and solve a real problem.

One Microsoft resource worth exploring

Microsoft Learn

https://learn.microsoft.com?wt.mc_id=studentamb_496381

It provides structured learning paths that help students build practical skills rather than simply memorizing syntax.

What I would not waste time learning

Memorizing obscure language features
Endless coding challenge collections without projects
Learning five programming languages simultaneously

You only need one language to become productive.

2. Developer Tools

Why it matters

Many students focus only on coding.

Professional developers spend enormous amounts of time using tools around the code.

Version control.

Debugging tools.

Editors.

Collaboration platforms.

Documentation systems.

The difference between a beginner and a productive builder is often tool proficiency.

A student using modern tooling can build significantly faster than someone fighting their environment every day.

What level I would target

Before graduation, I would be comfortable with:

Git
GitHub
Branches and pull requests
Debugging workflows
Extensions
Terminal basics
Reading logs

These are skills you will use repeatedly regardless of your career path.

One practical project

Create a team project with friends.

Use:

GitHub repository
Issues
Branches
Pull requests
Documentation

This teaches collaboration far better than solo practice.

One Microsoft resource worth exploring

VS Code

Learn how to use it properly rather than simply installing it.

Features like debugging, extensions, integrated terminals, and Git integration can dramatically improve productivity.

What I would not waste time learning

Customizing editors for weeks
Chasing every new extension
Endless comparisons between development environments

Tools should support your work, not become the work.

3. Cloud

Why it matters

Most modern applications eventually live somewhere other than a laptop.

Understanding cloud services helps students understand how software actually reaches users.

Cloud knowledge is no longer reserved for infrastructure specialists.

Developers, data professionals, AI engineers, and founders all benefit from understanding cloud concepts.

What level I would target

I would aim for:

Cloud fundamentals
Virtual machines
Storage
Networking basics
Authentication
Deploying applications
Monitoring fundamentals

You do not need advanced cloud architecture before graduation.

You need enough knowledge to deploy and operate a project.

One practical project

Deploy your student project to the cloud.

Make it accessible through a public URL.

This experience teaches more than dozens of cloud theory videos.

One Microsoft resource worth exploring

Azure for Students

It allows students to experiment with cloud services and gain practical experience building real applications.

You can combine it with learning paths on:

https://learn.microsoft.com?wt.mc_id=studentamb_496381

What I would not waste time learning

Extremely advanced enterprise architectures
Niche certification objectives without hands-on practice
Memorizing cloud service names

Understanding concepts is more valuable than memorizing product catalogs.

4. AI

Why it matters

Ignoring AI in 2026 would be a mistake.

Treating AI as magic would be another mistake.

The students gaining the most advantage are not necessarily building foundation models.

They are learning how to use AI effectively to solve practical problems.

AI is becoming a productivity multiplier.

Students who learn to work with it will often outperform students who either fear it or depend on it completely.

What level I would target

I would focus on:

Prompting fundamentals
Responsible AI concepts
AI-assisted development
Basic AI application development
Retrieval and knowledge-based solutions
Evaluating outputs critically

You do not need a PhD-level understanding of machine learning.

You need enough knowledge to integrate AI into useful projects.

One practical project

Build a study assistant for students.

Possible capabilities:

Question answering
Summarization
Note organization
Resource recommendations

This creates a realistic AI use case that solves an actual problem.

One Microsoft resource worth exploring

Microsoft Copilot

Use it as a learning partner.

Ask it to explain code, review ideas, generate documentation, and suggest improvements.

The goal is not replacing thinking.

The goal is accelerating thinking.

What I would not waste time learning

Every new AI framework
Daily AI hype cycles
Complex research papers far beyond your current goals

Focus on practical application before advanced specialization.

5. Data

Why it matters

Almost every decision today is influenced by data.

Products generate data.

Businesses analyze data.

AI systems rely on data.

Understanding data makes you a better developer, analyst, founder, and problem solver.

I often tell students that data literacy is becoming as important as coding literacy.

What level I would target

I would learn:

SQL
Data modeling basics
Data visualization
Basic analytics
Data cleaning
Reporting concepts

You do not need to become a data scientist.

You should be able to answer practical questions using data.

One practical project

Create a dashboard based on student performance, attendance, expenses, or productivity data.

Focus on finding useful insights.

Not just building charts.

One Microsoft resource worth exploring

Microsoft Fabric
It provides a modern environment for working with analytics, reporting, and broader data scenarios.

For guided learning:

https://learn.microsoft.com?wt.mc_id=studentamb_496381

What I would not waste time learning
Highly advanced mathematics before understanding basics
Massive datasets with no defined objective
Building dashboards that communicate nothing

The purpose of data is better decisions.

6. Automation and Low-Code

Why it matters

A lesson many students learn too late:

Not every problem needs custom code.

Sometimes automation is the smartest solution.

Sometimes a workflow tool is faster than building an entire application.

The ability to recognize that distinction is valuable.

The most effective builders choose the correct tool, not necessarily the most technical tool.

What level I would target

I would learn:

Workflow automation
Forms
App creation basics
Process design
Integration fundamentals

Enough to automate real-world tasks.

One practical project

Build a student club management workflow:

Registration form
Approval process
Notifications
Data collection
Reporting

This demonstrates immediate business value.

One Microsoft resource worth exploring

Power Platform

It introduces students to automation and low-code development while still encouraging problem-solving skills.

What I would not waste time learning

Rebuilding simple workflows from scratch
Creating complex solutions where automation already exists
Assuming low-code is somehow less valuable than traditional development

Real-world impact matters more than tool selection.

7. Career Validation

Why it matters

Learning alone is difficult to prove.

Employers often need evidence.

Projects matter.

Portfolios matter.

Demonstrated skills matter.

Having some form of objective validation can help students stand out.

What level I would target

I would aim to show:

Practical projects
Documented learning
Demonstrated capabilities
Public portfolio work

Not just certificates.

One practical project

Create a portfolio site that includes:

Projects
GitHub links
Write-ups
Screenshots
Lessons learned

The reflection is often as valuable as the project itself.

One Microsoft resource worth exploring

Microsoft Applied Skills

I like them because they focus on performing tasks rather than answering traditional multiple-choice questions.

They encourage students to demonstrate what they can actually do.

Explore opportunities through:

https://learn.microsoft.com?wt.mc_id=studentamb_496381

What I would not waste time learning

Collecting certificates without building
Chasing badges solely for social media posts
Treating credentials as replacements for experience

Projects remain the strongest proof of skill.

8. Community

Why it matters

Learning in isolation is harder than learning around builders.

Many opportunities come from conversations rather than courses.

Communities expose you to:

New ideas
Feedback
Mentorship
Collaborations
Career opportunities

Some of the fastest-growing students I know actively participate in technical communities.

What level I would target

I would aim to:

Attend events regularly
Ask questions
Share projects
Help others learn
Build professional relationships

You do not need thousands of followers.

You need meaningful connections.

One practical project

Present one project publicly.

This could be:

A technical session
A campus meetup
A student community event
An online demonstration

Teaching is one of the best ways to reinforce learning.

One Microsoft resource worth exploring

Microsoft Reactor

It provides access to developer-focused events, technical sessions, workshops, and communities where students can continue learning beyond coursework.

What I would not waste time learning

Obsessing over personal branding before building anything
Attending events without applying what you learn
Measuring success by follower counts

Skills create opportunities.

Visibility amplifies them.

A 90-Day Learning Routine I Would Personally Follow

If I were starting again as a student in 2026, this would be my next 90 days.

Month 1: Build Foundations

Practice programming daily
Learn Git and GitHub
Complete Microsoft Learn modules
Start one meaningful project
Use VS Code effectively
Publish weekly progress updates

Month 2: Add Cloud, AI, and Data

Deploy the project using Azure
Add an AI feature
Store and analyze project data
Create a dashboard
Learn cloud fundamentals through hands-on practice

Add workflow automation
Improve documentation
Complete an Applied Skills assessment
Present the project publicly
Participate in community events
Gather feedback and iterate

By the end of those 90 days, I would not expect to be an expert.

I would expect something more valuable:

A working project.

Real technical experience.

A portfolio.

A learning habit.

And, most importantly, a coherent stack that allows me to keep building.

Because in 2026, the students who stand out will not be the ones who tried to learn everything.

They will be the ones who learned enough of the right things to turn ideas into reality.

Cloud-Native Enterprise Platforms: What Developers Need to Know

Thu, 24 Sep 2026 05:02:17 +0000

Enterprise software has evolved dramatically over the last decade. Applications that ran on local servers and supported a limited number of employees now need to support distributed teams, cloud infrastructure, third-party integrations, mobile users, growing datasets and constantly evolving business requirements.

This shift has made cloud-native enterprise platforms a core component of modern software architecture.

But cloud-native is more than simply lifting and shifting an existing application from a physical server to the cloud. That means rethinking how you design, deploy, integrate, secure, monitor, and scale applications.

For developers, it is becoming increasingly important to understand this shift as organizations move away from isolated business applications toward interconnected and adaptable technology ecosystems.

Why It’s Getting Harder to Support Legacy Business Applications

Many traditional enterprise applications were based on a fairly simple model. Users accessed a central application in a controlled environment, with much of the processing and data remaining within an organisation’s own infrastructure.

That model worked well when businesses had less locations, less integrations, and relatively predictable workloads.

Today’s organizations work differently.

An enterprise application may need to connect to customer relationship management systems, accounting software, HR software, analytical tools, databases, APIs, authentication services, and other third-party applications. Employees can also access these systems from offices, homes, mobile devices and other geographic locations.

In tightly coupled applications even a small change can impact many parts of the system. Testing is more complicated, deployments are slower, and it becomes difficult to scale a particular function independently.

Developers have responded by looking into more modular and flexible ways of developing enterprise applications.

What is Cloud Native?

Cloud-native is not just hosting an application on AWS, Microsoft Azure, Google Cloud, or any other cloud provider.

The idea is wider.

Cloud native development is about building applications to take advantage of elasticity, automation, distributed infrastructure, resilience and continuous delivery.

Depending on the requirements, a cloud-native application might use containers, APIs, automated deployment pipelines, independent services, infrastructure automation, centralized logging, monitoring and elastic resource allocation.

But there is no one architecture which all organizations should follow.

But a business with a relatively simple app may not need dozens of microservices. In some cases a well-designed monolithic application can still be easier to operate and maintain.

We should be looking for an architecture that fits the requirements of the application, not for popular technologies.

A modular architecture makes change simple

Modularity is one of the central ideas in modern enterprise architecture.

For example, a business application that manages finance, human resources, inventory, customer management, reporting and document processing.

All these functions are tightly coupled so changing a part of the application might cause unanticipated effects somewhere else.

The modular architecture has clearer boundaries to the different responsibilities.

Developers are able to change individual components and not have to touch the entire application. Also, the testing can be more focused as individual components can be tested separately.

Modularity does not eliminate complexity. Distributed applications suffer from problems such as communication failures, data consistency, authentication, service discovery, and monitoring.

The value of modular architecture is thus a function of the quality of these boundaries.

APIs Connect the Enterprise Software Ecosystem

Today, very few organizations rely on a single software application.

Companies typically have many different systems, each designed to do a different job.

APIs offer a way for these systems to interact without forcing developers to duplicate functionality or reveal the internal implementation of an application.

For instance, an enterprise application might need to talk to a payroll system via an API instead of employees manually moving the data between the two systems.

This can reduce the need for repetitive work and help to create more consistent data flows.

Thinking in an API-first way also changes the way applications are designed for developers. The interface between systems becomes an explicit contract that can be documented, tested, versioned and maintained.

Scalability is more than simply the number of users

Scalability is frequently defined as the ability of an application to deal with more users or requests.Enterprise scalability is more complex.

Seasonal demand, sudden increase in transactions, expansion into new markets, other departments or rapidly growing data volumes can be a business reality.

Cloud-native architectures can provide resources that can be resized based on workload.

For example, a reporting or analytics component may use much more computing resources during a reporting cycle than during normal operations.

If components can scale independently, organizations don’t have to scale up the entire application just because one function has become more demanding.

This approach can enable more flexible resource management, but also calls for careful architectural planning.

Security Must Be Part of the Architecture

Cloud environments also change how developers need to think about securing applications.

Security is not a bolt-on feature!

Modern enterprise applications need to consider authentication and authorization, identity management, encryption, API security, control of access, audit trails, data protection and monitoring.

For example, role-based access control can limit the function and information available to users based on their responsibilities.

Communication between different services need some attention as well. Each API, integration, and external connection is another component that needs to be authenticated and access controlled correctly. This means that security needs to be thought of across the whole lifecycle of development and deployment in a cloud-native architecture.

Observability is now required

As applications get more distributed understanding what is going on inside of them is getting harder and harder.

Traditionally, application troubleshooting has been a case of looking at one server and its logs.

In a distributed environment, a single user request may pass through several services before generating a response.

This is why observability is so important.

Logs tell us about events, metrics tell us about changes in system behaviour, and traces help developers follow the requests across multiple services.

Monitoring and alerts can also help teams catch problems before they turn into larger operational issues.

It is not enough to know that something has gone wrong. Developers need sufficient information to know where and why the failure happened.

How Cloud-Native Development Is Changing the Game for Developers

Cloud native architecture is also changing the skills expected of modern developers.

While writing application code is still at the heart of software development, developers are increasingly required to understand what is going on around that code.

Developers who are familiar with APIs, databases, cloud infrastructure, containers, CI/CD pipelines, authentication, observability, security, and distributed systems can make better architectural decisions. That's not to say every developer needs to be a cloud architect or infrastructure specialist.

What that means is that the line between application development and infrastructure is starting to blur.

Developers today are expected to know how their applications are deployed, monitored, secured and scaled.

Microservices Aren't Always the Answer

The popularity of microservices has led some organizations to believe that simply breaking an application into many independent services makes it more modern or scalable.

That is not always so.

Microservices have the potential to provide benefits such as independent deployment and service level scalability, but also introduce operational complexity.

They have to take care of service communication, distributed data, monitoring, deployment, security and failure management.

For a small or medium complexity application these issues may outweigh the benefits.

The question is not whether an application should use microservices or not, but rather whether the architecture enables the organization to make changes effectively, while ensuring reliability and operational control.

The Future of Enterprise Software Is Getting More Connected

Enterprise applications are being integrated into broader ecosystems of technology, not as stand-alone systems.

Businesses need software that can talk to other applications, react to new demands, accommodate distributed users and deal with variable workloads.

Cloud-native architecture is one approach to this kind of flexibility.

But the technology itself is only half the story. A poorly designed distributed system can be a lot harder to maintain than a well -designed monolith.

Containers and microservices and APIs and cloud infrastructure don’t magically create better software. The real goal is to build systems that can be maintained, that are secure, observable and resilient, and that can evolve with the organization.

Beyond the Model: Why AWS Spent $8B on Generators & What It Means for Cloud Architecture

Fri, 18 Sep 2026 15:57:55 +0000

Hey Legions,

When we think about scaling AI, we usually talk about model params, CUDA kernels, or vector search efficiency. But there’s a physical limit we’re rapidly hitting: Raw Grid Power.

Amazon Web Services (AWS) just locked in an $8 Billion contract with Generac (plus a 2.6% stock warrant) to supply industrial backup generators for their AI data centers. First $2.4B in hardware lands in 2027-2028.

Here is a quick breakdown of the backend realities driving this deal:

Checkpoint Loss Risk: Training massive models takes weeks across thousands of GPUs. A 100ms grid flicker doesn't just drop a connection; it corrupts active memory states and destroys millions of dollars in compute time.

Continuous Heavy Load: A hyperscale AI data center draws 100 to 500 Megawatts continuously. That is equivalent to powering up to 375,000 homes.

Grid Upgrade Mandates: With legislation like the Ratepayer Protection Act passing, cloud providers are now forced to pay full infrastructure costs to upgrade public grids.

The Hardware Bottleneck: Power availability—not just GPU allocation—is now the primary bottleneck for scaling LLMs and cloud clusters.

As devs, are you seeing infrastructure costs or cloud provider power limits affecting how you architect your backends, or are you mostly relying on managed APIs?

👉 Read my full deep-dive analysis here: [https://www.thefluxread.com/2026/09/amazon-just-spent-8-billion-on.html]

Cloud #DevOps #SystemArchitecture #AWS #AI #Infrastructure

No More Midnight Deploys: The Magic of Blue-Green Deployments

Fri, 11 Sep 2026 02:43:30 +0000

No More Midnight Deploys: The Magic of Blue-Green Deployments

Imagine trying to upgrade the engine of a commercial airplane while it is mid-flight with passengers on board. In the software world, engineering teams face a similar challenge every day: they must update code, fix bugs, and release new features on live applications while millions of users are actively using them. This is where the concept of a blue-green deployment comes into play.

What is a Blue-Green Deployment?

A blue-green deployment is a software release strategy that uses two identical hardware or cloud environments to minimize downtime and risk during updates. One environment, designated "Blue," runs the active, publicly accessible version of the application. The other environment, designated "Green," remains idle or hosts the new, upcoming version of the software. Once the new version is thoroughly tested and ready, user traffic is instantly switched from Blue to Green.

The Bridge Analogy: Building a Parallel Path

To understand how this works, think of a heavily congested highway bridge that desperately needs repairs and structural upgrades. Traditionally, city planners would have to shut down the bridge, detour drivers through frustrating side streets, and cause massive traffic jams for days.

With a blue-green approach, engineers build an entirely new, identical bridge right next to the old one. Drivers continue to commute across the old bridge ("Blue") completely unaware of the construction happening next to them. Construction workers and inspectors spend weeks building, painting, and safety-testing the new bridge ("Green") in a controlled environment. Once the new bridge is certified safe, workers wait until a low-traffic moment to simply adjust the road signs and lane barriers. Instantly, drivers are guided onto the brand-new bridge ("Green"). If an unexpected safety issue is discovered on the new bridge an hour later, workers can quickly move the barriers back, routing traffic to the old, reliable bridge ("Blue") while they resolve the issue.

Why it Matters in Modern Software Engineering

In the tech industry, engineers use blue-green deployments to solve two major problems: service downtime and catastrophic deployment failures.

Historically, launching an update meant taking an application offline, displaying a "scheduled maintenance" page, upgrading the servers, and hoping everything worked when the system booted back up. If a bug slipped through, engineers faced a stressful race against the clock to fix it while users complained.

Blue-green deployments eliminate this panic. First, because the Green environment is isolated from actual users, developers can run comprehensive tests in a true production environment without any risk of breaking things for existing customers. Second, the release is instantaneous—it is a simple flip of a routing switch. Finally, if something does go wrong, the "rollback" process is just as fast. Instead of hours of frantic troubleshooting, reversing a bad deployment takes seconds: you just flip the traffic back to the stable Blue environment.

The "Switch" in Action: An Nginx Routing Configuration

In practice, this environment swap is often controlled by a reverse proxy or load balancer like Nginx. The load balancer acts as the traffic controller, directing incoming internet requests to either the Blue or Green servers.

Here is a simplified configuration demonstrating how an engineer flips traffic between the two environments by updating where the server points:

# /etc/nginx/conf.d/app.conf

# STEP 1: Routing to the Blue environment (Port 8080)
upstream production_app {
    server 10.0.0.10:8080; # Blue (Active)
    # server 10.0.0.20:8080; # Green (Idle)
}

# STEP 2: When Green is ready, the engineer modifies the configuration:
# upstream production_app {
#     # server 10.0.0.10:8080; # Blue (Now Idle/Fallback)
#     server 10.0.0.20:8080; # Green (Now Active!)
# }

server {
    listen 80;
    server_name myapp.com;

    location / {
        proxy_pass http://production_app;
    }
}

By changing a single line of configuration and reloading Nginx, the traffic instantly switches over without dropping a single active user connection.

The Takeaway

Ultimately, blue-green deployments shift our mindset from hoping a deployment goes well to knowing we can safely recover if it does not. It decouples the technical process of deploying code from the business decision of releasing it to customers. By turning stressful midnight releases into routine, daytime non-events, it keeps both engineering teams happy and users completely uninterrupted.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Project LifeOps (Part 1): Personal Operating System Architecture and FastAPI + Supabase Backend

Thu, 10 Sep 2026 17:06:49 +0000

How many apps do you use daily to manage your life? If you are anything like me, you probably have one app to track your runs and gym workouts, another to keep tabs on books, a website to rate movies and TV series, and two or three more tools for project management, tasks, and notes.

The result is almost always the same: scattered data silos, mounting monthly subscription bills, and zero ability to cross-analyze data or generate a consolidated monthly executive review.

With that problem in mind and true to the Datalaria ethos of "learning by building", I decided to design and engineer LifeOps: an All-in-One Personal & Professional Operating System (OS) that unifies under a single responsive dashboard:

🏃 Fitness & Performance: Track workouts (Running, Cycling, Gym), distance volume, Personal Bests (PB), and calories.
📚 Reading & Culture: Book progress tracking, percentage bars, and ratings.
🎬 Cinema & Entertainment: Movie, TV series, and documentary catalog organized by platform.
📋 Professional Portfolio & Tasks: Interactive Kanban board, budget tracking, and deliverable milestones.
📊 Reporting & Data Portability: Streaming Word (.docx) executive dossiers, multi-sheet Excel workbooks (.xlsx), and raw CSV exports with UTF-8 BOM.

Best of all: built on top of a rock-solid, production-grade architecture that is 100% Free ($0/month).

In this first installment of the series, we dive into the core foundations: cloud architecture design, PostgreSQL relational data modeling, and building the high-performance FastAPI backend. Let's get into it! 🚀

[!TIP]
Test the live app: You can explore the production build of LifeOps directly at https://datalaria.com/apps/lifeops/.

🗺️ LifeOps Series Roadmap

To understand how every architectural layer fits together, this series spans 5 structured installments:

🟢 Part 1 (This article): Personal Operating System Architecture and FastAPI + Supabase Backend.
⚪ Part 2: React Frontend with Glassmorphism, 360° Dashboard, and Design System.
⚪ Part 3: Core Interactive Modules: Fitness, Library, Cinema, and Professional Kanban Board.
⚪ Part 4: In-Memory Word Dossiers (.docx) & Multi-Sheet Excel Engine (.xlsx).
⚪ Part 5: 24/7 Zero-Cost Cloud Deployment ($0/month), Mobile UX, and PWA.

The $0/Month Architecture: How to Run Free at Scale 💡

When architecting LifeOps, the goal wasn't merely to run on localhost, but to deploy a production-grade system with zero recurring hosting costs using the most generous free tiers available:

┌─────────────────────────────────────────────────────────────┐
│                       CLIENT BROWSER                        │
│            [https://datalaria.com/apps/lifeops](https://datalaria.com/apps/lifeops)               │
└──────────────────────────────┬──────────────────────────────┘
                               │ (Proxy Rewrite 200)
                               ▼
┌─────────────────────────────────────────────────────────────┐
│               FRONTEND: Netlify CDN ($0/month)              │
│          React 18 + Vite 8 + i18n (ES/EN) + CSS Glass       │
└──────────────────────────────┬──────────────────────────────┘
                               │
                JWT Auth Token │  REST API Calls / Streaming
                               ▼
┌─────────────────────────────────────────────────────────────┐
│               BACKEND: FastAPI + Python 3.13                │
│    • In-Memory Docx Generator (python-docx + io.BytesIO)    │
│    • Multi-Sheet Excel Engine (openpyxl)                    │
│    • Anti-DoS Rate Limiting (slowapi) + Strict CORS         │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│              DATABASE: Supabase Managed Cloud               │
│       • PostgreSQL with isolated `lifeops` schema           │
│       • Row Level Security (RLS) & Supabase Auth            │
└─────────────────────────────────────────────────────────────┘

Frontend on Netlify ($0/mo): Single Page Application built with Vite and served globally with Gzip compression, route-level code splitting, and transparent proxy rewrite under Datalaria's domain.
Backend on FastAPI ($0/mo): Asynchronous Python 3.13 API with strict validation via Pydantic v2, hosted on Render with cold-start resilience.
Database on Supabase ($0/mo): Managed PostgreSQL database with 500 MB of storage, JWT authentication, and automated Row Level Security (RLS) policies.

[!NOTE]
Render Cold-Start Handling: Because the backend runs on Render.com's free tier, the container spins down after 15 minutes of inactivity. In the React frontend (detailed in Part 2), we implemented a preloader screen with automatic exponential-backoff retries, ensuring the user experiences zero abrupt connection drops.

Relational Data Modeling: Clean Polymorphism in PostgreSQL 🗄️

One of the biggest design challenges when combining disparate activities (a 10 km run, a 400-page book, or a Netflix movie) is avoiding monolithic tables cluttered with NULL columns.

To solve this, I designed a master table with specialized child extensions (1-to-1) inside an isolated lifeops schema:

-- 1. Master Activities Table
CREATE TABLE lifeops.activities (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
    activity_type VARCHAR(30) NOT NULL, -- 'sport', 'book', 'film'
    title VARCHAR(200) NOT NULL,
    date DATE NOT NULL DEFAULT CURRENT_DATE,
    duration_minutes INTEGER,
    rating INTEGER CHECK (rating >= 1 AND rating <= 5),
    description TEXT,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW()
);

-- 2. Fitness Extension (Workouts)
CREATE TABLE lifeops.workouts (
    activity_id UUID PRIMARY KEY REFERENCES lifeops.activities(id) ON DELETE CASCADE,
    workout_type VARCHAR(50) NOT NULL, -- 'running', 'cycling', 'gym', etc.
    distance_km NUMERIC(6,2),
    calories INTEGER,
    avg_heart_rate INTEGER,
    elevation_m INTEGER,
    personal_best BOOLEAN DEFAULT FALSE,
    notes TEXT
);

-- 3. Reading Extension (Books)
CREATE TABLE lifeops.books (
    activity_id UUID PRIMARY KEY REFERENCES lifeops.activities(id) ON DELETE CASCADE,
    author VARCHAR(150),
    pages_total INTEGER,
    pages_read INTEGER DEFAULT 0,
    status VARCHAR(30) DEFAULT 'reading', -- 'reading', 'completed', 'wishlist'
    genre VARCHAR(50)
);

-- 4. Cinema Extension (Films)
CREATE TABLE lifeops.films (
    activity_id UUID PRIMARY KEY REFERENCES lifeops.activities(id) ON DELETE CASCADE,
    media_type VARCHAR(30) DEFAULT 'movie', -- 'movie', 'series', 'documentary'
    director VARCHAR(150),
    platform VARCHAR(50), -- 'Cine', 'Netflix', 'HBO Max', 'Prime Video'
    genre VARCHAR(50),
    year INTEGER
);

Why this architecture shines:

Blazing Fast Global Aggregations: Dashboard queries hit only lifeops.activities for KPI counts and calendar activity heatmaps.
Cascading Referencial Integrity: When an activity is removed, PostgreSQL's ON DELETE CASCADE automatically purges child tables without orphaned rows.

Row-Level Security (RLS): Each user can only read and mutate their own data via declarative security policies:

CREATE POLICY "Users can only view their own activities"
ON lifeops.activities FOR SELECT
USING (auth.uid() = user_id);

[!IMPORTANT]
Isolated Schemas & Permissions in Supabase: When maintaining tables in a dedicated schema like lifeops (rather than default public), ensure you grant explicit schema access (GRANT USAGE ON SCHEMA lifeops TO authenticated;) and table permissions to the authenticated role. This isolates the operating system's database domain while honoring multi-tenant security.

The FastAPI Backend: Modular Architecture & Strict Typing ⚡

To organize the API cleanly as business domains expand, we follow FastAPI's Modular Router pattern:

lifeops-api/
├── main.py                  # Entrypoint, Middlewares, Rate Limiter & CORS
├── config.py                # Centralized environment settings with Pydantic Settings
├── middleware/
│   └── auth.py              # Supabase JWT validation & secure user_id extraction
├── models/
│   ├── activity.py          # Pydantic Schemas (Create, Update, Response)
│   └── project.py           # Schemas for Tasks and Projects
├── routers/
│   ├── activities.py        # CRUD endpoints for Fitness, Books & Cinema
│   ├── projects.py          # Endpoints for Kanban Board and Projects
│   ├── stats.py             # Global KPI aggregations & Dashboard metrics
│   ├── reports.py           # Word (.docx) generator & CSV/Excel streaming
│   └── alerts.py            # Intelligent reminder engine
└── services/
    ├── report_generator.py  # In-memory RAM .docx compilation
    └── data_exporter.py     # CSV (BOM) & multi-sheet Excel generator

Secure Supabase JWT Authentication

Every endpoint processing user data injects the get_current_user dependency:

from fastapi import Depends, HTTPException, status
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
import jwt

security = HTTPBearer()

async def get_current_user(credentials: HTTPAuthorizationCredentials = Depends(security)) -> AuthenticatedUser:
    token = credentials.credentials
    try:
        payload = jwt.decode(
            token,
            settings.supabase_jwt_secret,
            algorithms=["HS256"],
            audience="authenticated"
        )
        user_id = payload.get("sub")
        if not user_id:
            raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Token missing user identity")
        return AuthenticatedUser(id=user_id, email=payload.get("email"))
    except jwt.PyJWTError as e:
        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid or expired token")

Production Hardening: Rate Limiting & Anti-Abuse 🛡️

Because LifeOps supports on-the-fly Word report compilation and multi-sheet Excel generation, it was crucial to prevent resource starvation or DoS attacks.

We integrated Rate Limiting with slowapi directly into the FastAPI application:

from slowapi import Limiter, _rate_limit_exceeded_handler
from slowapi.util import get_remote_address
from slowapi.errors import RateLimitExceeded

limiter = Limiter(key_func=get_remote_address, default_limits=["120/minute"])
app.state.limiter = limiter
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)

# Strict rate limit for CPU/memory-intensive Word generation
@router.post("/generate")
@limiter.limit("10/minute")
def generate_report(request: Request, req: GenerateReportRequest, user = Depends(get_current_user)):
    ...

If a client exceeds the threshold, the API responds instantly with an HTTP 429 Too Many Requests, protecting compute resources.

Conclusion and What's Next 🎯

With our FastAPI backend hardened, the PostgreSQL relational schema operational on Supabase, and rate limiting active, we have a production-grade foundation running at $0/month.

In Part 2 of this series, we dive straight into the frontend:

Single Page App development with React 18/19, Vite, and Glassmorphism design.
Internationalization (i18n) system for instant Spanish 🇪🇸 / English 🇬🇧 locale toggling.
The 360° Dashboard architecture with aggregated KPI widgets and interactive Recharts visualizations.
Authentication context (AuthContext) and session synchronization with Supabase.

References & Useful Links 🔗

🚀 Production Application: Try the live app at datalaria.com/apps/lifeops.
🌐 Production REST API: Interactive OpenAPI Swagger documentation at lifeops-api.onrender.com/docs.
⚡ FastAPI Framework: Official documentation at fastapi.tiangolo.com.
🗄️ Supabase Cloud: Guide to PostgreSQL and Row Level Security at supabase.com/docs.
🛡️ SlowAPI: Rate limiting for ASGI/FastAPI at github.com/laurentS/slowapi.

See you in the next installment! Feel free to leave a comment below or connect on socials. 👇

Demystifying 'Cold Starts' in Serverless: Why Your App Sometimes Shivers

Wed, 02 Sep 2026 11:40:57 +0000

Have you ever clicked a button on a website, only to wait several agonizing seconds for the page to respond, even though it usually loads instantly? This annoying delay is often caused by a backend phenomenon known as a cold start. In serverless computing, a cold start is the delay that occurs when a cloud function is invoked for the first time or after a period of inactivity. Because there is no constantly running server, the cloud provider must spin up a new virtual container, load your code, and initialize the environment before running the function.

The Winter Car Analogy

Imagine driving your car in the dead of winter. If the car has been sitting in your driveway for days, you cannot just hop in and immediately speed down the highway. First, you must start the cold engine, wait for the fluids to warm up, and let the defroster clear the windshield—this initialization process is your "cold start."

However, if you drive to the grocery store, spend ten minutes shopping, and return to the car, the engine is still warm. You turn the key, put it in drive, and immediately head home. In the cloud, a "warm" function acts just like that warm car: it is already running in memory and ready to process your request instantly.

Why It Matters in the Tech Industry

Serverless computing is incredibly popular because engineers only pay for the exact milliseconds their code is executing, rather than renting an expensive virtual server to sit idle 24/7. However, if an e-commerce website experiences a sudden surge of traffic, or if a user accesses a rarely used feature, they will trigger a cold start.

This results in sudden, unpredictable lag spikes. If a customer clicks "Proceed to Checkout" and experiences a five-second cold-start delay, they might assume the website is broken and abandon their shopping cart. Software engineers must deeply understand cold starts to balance infrastructure cost-savings with a fast, reliable user experience.

Optimizing for Cold Starts in Node.js

To minimize cold start times, developers write code that initializes heavy resources (like database connections) outside the main execution handler. This ensures the heavy work is only done once during the "cold start," while subsequent "warm" requests bypass it entirely.

// 1. GLOBAL SCOPE: Runs ONLY during a cold start.
// This heavy database connection is established once and kept in memory.
const dbConnection = connectToDatabase();

exports.handler = async (event) => {
  // 2. HANDLER SCOPE: Runs on EVERY single request.
  // Warm executions skip the global setup above and run this instantly.
  const userId = event.pathParameters.id;
  const user = await dbConnection.find(userId);
  
  return {
    statusCode: 200,
    body: JSON.stringify({ user })
  };
};

The Takeaway

Cold starts are the natural "tax" of serverless architecture, representing the unavoidable trade-off between absolute cost efficiency and instant responsiveness. By optimizing how your code initializes, keeping your deployment packages small, and strategically choosing your runtime environment, you can harness the scale of the cloud without leaving your users waiting in the cold.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Building Offline-First Systems for Low-Connectivity Agritech

Wed, 02 Sep 2026 03:19:09 +0000

A note on the architecture

This article describes lessons from a production agricultural technology platform. Some implementation details, operational parameters, identifiers, and business-specific logic have been generalized or omitted to avoid exposing proprietary or sensitive information. The goal is to explain the engineering problems, decisions, and principles rather than the private details of the system.

The booking that happened twice

We had built idempotency to prevent duplicate tractor bookings.

Then duplicate tractor bookings started appearing.

The first thing I checked was the idempotency logic.

It looked correct.

That was the problem.

The mechanism wasn't necessarily broken.

The assumption underneath it was.

The platform was designed for field agents working in environments where network connectivity could be intermittent or unavailable. An agent might be in a village registering a farmer, selecting a tractor, choosing hiring dates, and completing a booking while the phone had no reliable connection to the backend.

The application therefore couldn't simply say:

"No internet? Come back later."

The agent needed to continue working.

The operation had to be stored locally and delivered to the server when connectivity returned.

That sounds like an offline mobile-app problem.

It quickly became a distributed-systems problem.

Once a device can create an operation without an active connection to the server, a much harder set of questions appears:

What happens when synchronization times out?
What if the server processes the request but the response never reaches the phone?
What if the mobile client retries?
How does the server know that two requests represent the same business operation?
What happens if two identical requests arrive at almost exactly the same time?
What happens if the database transaction succeeds but Kafka is temporarily unavailable?
What happens when the booking succeeds but nobody is notified?

These were the problems that shaped the architecture.

And, more importantly, they changed how I think about reliability.

1. Designing for a Network That Cannot Be Trusted

The fundamental requirement was simple:

A field agent should be able to continue important work even when the network disappears.

The workflow looked roughly like this:

Select or register a farmer.
Select a tractor or agricultural tool.
Select the relevant dates.
Complete the booking.
Store the operation locally if connectivity is unavailable.
Synchronize it when connectivity returns.

The backend had dedicated offline workflows for farmer onboarding, tractor hiring, and agro-tool hiring, with staging records that move through synchronization states such as pending, synced, and failed.

On the mobile side, the synchronization engine maintains offline forms, processes pending operations, retries failed synchronization, and recovers operations that may have been left in a syncing state after an interruption.

The important architectural distinction is this:

Creating an operation and delivering an operation are two different things.

The agent creates the booking locally.

The network eventually delivers it.

Those events may happen seconds, minutes, or much longer apart.

Caption:

The offline workflow separates creating an operation from delivering it to the backend.

This distinction becomes important once the same operation can be delivered more than once.

2. The First Line of Defense: Idempotency

The standard solution to repeated delivery is idempotency.

Each offline operation needs a stable identity.

In the mobile application, the offline form carries an idempotencyKey. During synchronization, that existing key is included with the request rather than creating a new identity for each network attempt. The synchronization request sends the value both as part of the request data and through the X-Idempotency-Key header.

The conceptual model is:

One logical booking
        │
        ▼
Operation ID: A
        │
   ┌────┼────┐
   ▼    ▼    ▼
Attempt 1  Attempt 2  Attempt 3
   │         │         │
   └─────────┼─────────┘
             ▼
        Same operation

If the server sees the same operation ID again, it should not perform the business operation from scratch.

Instead:

Request #1
    │
    ├── Key = A
    ▼
Create booking
    │
    ▼
Synced


Request #2
    │
    ├── Key = A
    ▼
Existing operation
    │
    ▼
Return existing result

That's what idempotency is supposed to provide.

But this is where an important assumption appears:

The server can only deduplicate requests that the client identifies as the same operation.

Code Snippet 1 — Persistent Idempotency Identity

final formData = Map<String, dynamic>.from(form.formData);
formData['idempotency_key'] = form.idempotencyKey;

final headers = {
  'Content-Type': 'application/json',
  'X-Idempotency-Key': form.idempotencyKey,
  'Accept': 'application/json',
};

Caption:

The idempotency key belongs to the logical offline form, not to an individual HTTP attempt.

3. The Idempotency Mechanism Wasn't Broken

Then we encountered duplicate bookings.

The natural question was:

"Why didn't idempotency stop this?"

The investigation revealed a more subtle failure mode.

An earlier client retry path could generate a fresh idempotency key for a retry of the same logical draft.

From the backend's perspective, that changes everything.

Imagine one booking being attempted three times:

             ONE LOGICAL BOOKING
                     │
          ┌──────────┼──────────┐
          ▼          ▼          ▼
       Retry 1    Retry 2    Retry 3
          │          │          │
        Key A      Key B      Key C
          │          │          │
          ▼          ▼          ▼
       Server     Server     Server
          │          │          │
          ▼          ▼          ▼
       Booking    Booking    Booking
          1          2          3

The server isn't seeing:

A
A
A

It is seeing:

A
B
C

There is therefore nothing for a key-based deduplication mechanism to connect.

The server isn't necessarily malfunctioning.

It has been given three different identities.

Diagram 2 — The Idempotency/Retry Failure

Caption:

The server can only deduplicate what the client identifies as the same operation.

This is one of the most important lessons I took from the incident:

Idempotency isn't simply a backend feature. It's a contract between the producer and consumer of an operation.

The identity of the business operation has to survive every delivery attempt.

4. When the Ideal Fix Isn't Immediately Available

The clean solution was to ensure that the operation receives its identity when it is created and that every synchronization attempt reuses that identity.

That is the model reflected in the current synchronization implementation: the form carries its persistent idempotency key, and synchronization reuses it.

But production systems sometimes require another consideration:

What can we safely do while an upstream problem is being corrected?

The backend still needed to protect the integrity of the booking workflow.

So I introduced a second layer of duplicate protection.

Not as a replacement for idempotency.

As a defensive backstop.

5. The Imperfect Safety Net

The backend gained an additional short-window, multi-attribute duplicate-detection mechanism.

The idea was straightforward:

If a new submission looks strongly like a recently synchronized booking, don't blindly create another business record.

This was intentionally not treated as the primary correctness mechanism.

It was a containment mechanism.

And that distinction matters.

A heuristic can have false positives.

It can have false negatives.

A legitimate booking might resemble another booking.

A retry might arrive outside the heuristic's window.

So the architecture remained:

Stable operation identity
          │
          ▼
      Idempotency
          │
          ▼
Database uniqueness
          │
          ▼
Concurrency handling
          │
          ▼
Defensive duplicate backstop

The backend implementation history explicitly describes this as a mitigation for the client retry problem rather than a substitute for correct idempotency semantics.

That led to an important engineering principle for me:

A defensive heuristic should contain a failure mode, not become the system's definition of correctness.

6. Then Concurrency Exposed Another Problem

Solving duplicate delivery introduced another question.

What happens when two requests with the same idempotency key arrive at almost exactly the same time?

A common implementation starts with:

Does this key already exist?

If not:

Create it.

Sequentially, this appears safe.

Under concurrency, it isn't.

Consider two requests:

Request A                  Request B
    │                          │
    ▼                          ▼
Check key                  Check key
    │                          │
"Not found"                "Not found"
    │                          │
    └──────────┬───────────────┘
               ▼
             INSERT
            /      \
           ▼        ▼
       Winner     Conflict

Both requests can perform the existence check before either one has committed its insert.

Then both attempt to insert the same unique value.

PostgreSQL correctly rejects one of them with a uniqueness violation.

At the database level, everything is behaving correctly.

At the application level, however, treating that as an ordinary failure would be misleading.

The losing request may simply be:

the second attempt to perform an operation that another request has just successfully created.

Diagram 3 — Concurrent Idempotency Race

Caption:

A check-then-insert sequence can race under concurrency; the database constraint becomes part of the application-level resolution.

7. Turning a Database Error Into a Business Outcome

The first implementation instinct might be:

UniqueViolation
      ↓
500 Internal Server Error

But that isn't necessarily what happened.

The database may simply be telling us:

"Another request got there first."

I introduced _is_idempotency_conflict() to distinguish the relevant uniqueness conflict from unrelated integrity errors.

Code Snippet 2 — Idempotency Conflict Detection

@staticmethod
def _is_idempotency_conflict(error: IntegrityError) -> bool:
    return "idempotency_key" in str(
        getattr(error, "orig", error)
    )

Caption:

The application distinguishes an idempotency-key conflict from unrelated database integrity errors.

But detecting the conflict wasn't the final step.

The service then needs to resolve the request against the operation that won the race.

Conceptually:

                 INSERT
                   │
             ┌─────┴─────┐
             │           │
          Success      Conflict
             │           │
             ▼           ▼
          Continue    Read winner
                         │
                   ┌─────┴─────┐
                   ▼           ▼
                Synced       Pending
                   │           │
                   ▼           ▼
             Return result   Retry later

If the winning operation is already synchronized, the losing request can receive that result.

If the winning operation is still being processed, the system can communicate a pending state rather than falsely declaring the business operation failed.

This changed how I think about database exceptions.

A database exception is sometimes evidence of a state transition elsewhere, not evidence that the user's business operation failed.

8. The Lost-Response Problem

There is another reason idempotency is essential.

Suppose the server successfully creates a booking.

Then the response is lost.

Mobile                       Server
   │                           │
   │──── Create booking ──────►│
   │                           │
   │                        Save ✓
   │                           │
   │◄──── Response ────────────X
   │          lost
   │
   │──── Retry ────────────────►│

The mobile application doesn't know whether the booking succeeded.

From its perspective:

"The request failed."

Retrying is therefore completely reasonable.

The danger is allowing that retry to create another booking.

Stable operation identity solves the uncertainty:

Retry
  │
  ▼
Same operation ID
  │
  ▼
Existing operation
  │
  ▼
Return existing result

The current synchronization engine also explicitly handles duplicate/conflict responses by treating the local form as synchronized rather than repeatedly attempting to create the operation.

This illustrates a broader distributed-systems principle:

The client doesn't always need to know what happened immediately. The system needs a safe way to resolve uncertainty when the client asks again.

9. A Successful Database Write Isn't the End

Once the booking is successfully stored, another problem appears.

Other parts of the system may need to know about the operation.

Maybe an event needs to be published.

Maybe another service needs to consume it.

Maybe a notification needs to be sent.

This creates another failure boundary.

Consider:

Save booking
     │
     ▼
PostgreSQL ✓
     │
     ▼
Publish event
     │
     X
Kafka unavailable

Now the business operation exists.

But the event doesn't.

If we reverse the order, we create the opposite problem:

Publish event
     │
     ▼
Kafka ✓
     │
     X
Database transaction fails

Now another system has received an event for an operation that never became durable.

We needed the database to become the durable source of truth for both the business state change and the intention to publish its event.

10. The Transactional Outbox

The solution was the transactional outbox pattern.

The business write and the corresponding outbox event are persisted within the same database transaction.

Conceptually:

┌────────────────────────────────┐
│        DB TRANSACTION          │
│                                │
│     Create booking             │
│            +                   │
│     Create outbox event        │
│                                │
└───────────────┬────────────────┘
                │
              COMMIT
                │
                ▼
          Relay Worker
                │
                ▼
              Kafka

The important guarantee is:

If the business transaction commits, the intent to publish the event is also durable.

The outbox implementation explicitly inserts the event as part of the same database transaction as the business operation and leaves publication to a later relay process.

Diagram 4 — Transactional Outbox → Kafka → Notifications

Caption:

The business change and the intent to publish its event become durable together; delivery happens asynchronously.

Code Snippet 4 — Transactional Outbox Write

For publication, use the actual relevant portion of your emit_event() implementation rather than a generic pseudocode transaction.

The important source-level pattern is:

db.execute(
    text("INSERT INTO outbox_events (...) VALUES (...)"),
    {...}
)

with the function called within the same database transaction as the business write.

Caption:

The outbox records the intent to publish while the business transaction is still the source of truth.

Important editorial rule: don't expose the full internal schema, private topic names, internal identifiers, or production payload structure.

11. Let the Relay Worry About Delivery

Once the event is safely stored, the original booking request doesn't need to remain responsible for delivering it to Kafka.

A background relay can take over.

The architecture becomes:

Business transaction
        │
        ▼
PostgreSQL + Outbox
        │
        ▼
Relay worker
        │
        ▼
Kafka

The relay can retry delivery independently.

This also introduces another concurrency question:

What happens if multiple relay workers are running?

We don't want two workers to claim the same pending event unnecessarily.

The relay uses database locking with FOR UPDATE SKIP LOCKED when claiming available outbox work.

Code Snippet 3 — Safely Claiming Outbox Events

Show the relevant query portion:

.with_for_update(skip_locked=True)

Caption:

FOR UPDATE SKIP LOCKED allows multiple workers to claim available events without waiting on rows already locked by another worker.

Again, the interesting part isn't the syntax itself.

It's the reason it exists.

The database is being used not just as storage, but as part of the coordination mechanism between concurrent workers.

12. Then We Found Something That Wasn't Broken

At this point, the system was getting much better at processing operations reliably.

But there was another question:

Who knows that the operation succeeded?

An offline hire could synchronize successfully.

The database could contain the booking.

The event could be persisted.

The downstream processing could work.

And yet the human workflow could still be silent.

There was no guarantee that everyone who needed to know would immediately receive useful confirmation.

This wasn't an idempotency bug.

It wasn't a database bug.

It was a workflow visibility gap.

Technical correctness and operational correctness aren't always the same thing.

13. Making Successful Synchronization Visible

The synchronization flow was extended with a notification path for successfully synchronized offline hires.

A successful synchronization can now lead to downstream communication such as:

SMS to the farmer;
in-app notification;
group push notification;
admin-facing notification;
relevant dashboard cache invalidation.

The notification implementation also treats those downstream actions as best-effort consequences rather than allowing a notification failure to invalidate the underlying business operation.

That distinction matters.

We don't want:

SMS provider unavailable
        ↓
Booking failed

The booking already succeeded.

The notification is another operation.

So the architecture becomes:

              Booking
                 │
                 ▼
               Commit
                 │
                 ▼
               Event
                 │
       ┌─────────┼─────────┐
       ▼         ▼         ▼
     Farmer     Agent     Admin
      SMS       Push     Dashboard

A system isn't fully successful merely because the database is correct.

The people who depend on that state need a reliable way to know what happened.

14. Testing the System Like the Network Hates You

This experience changed how I think about testing offline systems.

The conventional test looks something like:

Submit booking
      ↓
Receive response
      ↓
Booking created

That's necessary.

It isn't enough.

For an offline synchronization system, the interesting tests deliberately violate the assumptions.

What happens when connectivity disappears during synchronization?

What happens when the server processes the request but the response disappears?

What happens when the client retries?

What happens when a retry uses the wrong operation identity?

What happens when two requests arrive simultaneously?

What happens when the database commits but Kafka is unavailable?

What happens when the relay worker crashes?

What happens when notification delivery fails?

These scenarios aren't edge cases in the philosophical sense.

They are normal possibilities in distributed systems.

The architecture needs a deliberate answer for each one.

15. Build a Failure Matrix

One useful way to reason about these systems is to map failures to expected outcomes.

Failure	Desired outcome
No connectivity	Operation remains available for synchronization
Synchronization timeout	Operation can be retried
Response lost after server success	Retry resolves to the existing operation
Same operation ID submitted again	One business effect
Incorrect retry identity	Defensive duplicate protection may limit damage
Concurrent same-key requests	One wins; the other resolves against the winner
Database succeeds, Kafka fails	Outbox preserves the event
Relay worker fails	Event remains available for later processing
Multiple relay workers	Workers safely claim available work
Notification fails	Business operation remains successful
Dashboard data becomes stale	Relevant cache can be invalidated

The question changes from:

"Did an error occur?"

to:

"What state should the system eventually reach after this failure?"

That is a much more useful question for distributed systems.

16. Reliability Is a Chain, Not a Feature

Looking back at the architecture, there wasn't one feature called "offline reliability."

There was a chain of mechanisms.

                 Mobile App
                     │
                     ▼
             Offline operation
                     │
                     ▼
           Stable operation ID
                     │
                     ▼
                Idempotency
                     │
                     ▼
           Duplicate protection
                     │
                     ▼
          Database constraints
                     │
                     ▼
         Concurrency resolution
                     │
                     ▼
          Transactional outbox
                     │
                     ▼
               Relay → Kafka
                     │
                     ▼
          Notifications / Admin
                     │
                     ▼
          Observable outcome

Each layer exists because a different assumption can fail.

Local persistence doesn't prevent duplicate delivery.

Idempotency doesn't work if operation identity changes.

Database uniqueness doesn't automatically tell the application what a conflict means.

An outbox doesn't guarantee that every downstream consumer will process an event immediately.

Notifications don't automatically guarantee that a dashboard is current.

Reliability emerges from the interaction between these mechanisms.

17. What I Would Design Differently Today

If I were designing the system from scratch today, I would make operation identity a first-class concept from the beginning.

A booking is a business operation.

An HTTP request is merely one attempt to deliver that operation.

Those should not be treated as the same thing.

                 ONE BOOKING
                     │
               Operation ID
                   ABC123
                     │
          ┌──────────┼──────────┐
          ▼          ▼          ▼
       Attempt 1  Attempt 2  Attempt 3

Every attempt should preserve the same identity.

I would also explicitly model synchronization as a state machine:

CREATED
   │
   ▼
PENDING_SYNC
   │
   ├──────────────┐
   ▼              ▼
SYNCED          FAILED
   │
   ▼
NOTIFIED

The exact states can differ between systems.

The principle is more important:

Don't collapse uncertainty into failure.

A timeout doesn't necessarily mean failure.

A duplicate doesn't necessarily mean failure.

A uniqueness conflict doesn't necessarily mean failure.

A temporary broker outage doesn't mean the booking itself failed.

Those distinctions should exist in the architecture.

18. I Would Treat Heuristics as Defense in Depth

The duplicate-detection backstop was useful.

But I wouldn't want it to become the primary correctness mechanism.

The hierarchy should remain:

Stable operation identity
          ↓
Database uniqueness
          ↓
Concurrency handling
          ↓
Business-rule safeguards

The strongest guarantees should establish correctness.

Heuristics should help contain damage when something upstream behaves unexpectedly.

That distinction is important because a system can become dangerously complicated when a workaround gradually becomes the mechanism everyone assumes is responsible for correctness.

19. I Would Make Observability Business-Aware

Infrastructure monitoring is necessary.

CPU.

Memory.

Latency.

Database health.

Queue depth.

Worker health.

But an offline synchronization system needs business-level observability too.

I'd want to know:

How many operations are waiting?

How many have failed?

How many have retried?

How many duplicate attempts were detected?

How many idempotency conflicts occurred?

How long do operations remain pending?

How many outbox events are waiting?

How many downstream notifications failed?

Those metrics answer a different question.

Not:

"Are my servers running?"

But:

"Is the business workflow actually progressing?"

That distinction becomes increasingly important as systems become more distributed.

20. Idempotency Is a Contract, Not a Checkbox

It's easy to say:

"We have idempotency."

But that statement isn't enough.

You also need to answer:

What makes two requests the same business operation?

If the answer is an operation ID, that identity needs to survive retries.

If the identity changes with every attempt, the server cannot reliably connect those requests.

This is why mobile and backend components need a shared understanding of operation identity.

The mobile application owns the logical operation.

The network merely transports attempts to synchronize it.

21. Retries Change the Semantics of an Operation

A retry isn't simply another function call.

It may represent:

a request that never reached the server;
a request that reached the server but timed out;
a request that succeeded but whose response disappeared;
an operation whose outcome is genuinely unknown.

Those scenarios can all look similar from the client.

The backend therefore needs a way to answer:

"What happened to this operation?"

rather than simply:

"What should I do with this HTTP request?"

That's why stable identity, durable state, and idempotent processing matter.

22. Exactly Once Isn't Always the Goal

Distributed systems often make people think about "exactly once."

In practice, guaranteeing that a distributed operation is attempted exactly once can be extremely difficult.

A more practical objective is:

Allow repeated delivery while keeping the business outcome correct.

The network can deliver an operation more than once.

The client can retry.

A worker can restart.

An event can be processed again.

The architecture should still converge toward the correct business state.

This is particularly important in offline-first applications because intermittent connectivity naturally creates uncertainty around delivery.

23. Failure Needs Business Meaning

One of the strongest lessons from this work was that technical errors and business failures aren't always the same thing.

Consider:

Timeout

That doesn't necessarily mean:

Booking failed.

Consider:

Unique constraint violation

That doesn't necessarily mean:

Booking failed.

Consider:

Kafka unavailable

That doesn't necessarily mean:

Booking failed.

Consider:

Notification provider unavailable

That definitely doesn't necessarily mean:

Booking failed.

A robust system needs to distinguish:

transport failure

processing failure

business failure

and

uncertainty about the outcome.

When all four become:

500 Internal Server Error

the system loses important information.

24. Offline-First Is a Distributed-Systems Problem

This is probably the biggest lesson I took away from the project.

Offline-first is often described as a mobile development technique:

Store data locally and synchronize it later.

That description is technically true.

It doesn't capture the difficulty of doing it reliably.

Once synchronization becomes important, you quickly encounter:

Unreliable connectivity
        ↓
Uncertain delivery
        ↓
Retries
        ↓
Idempotency
        ↓
Concurrency
        ↓
Transactions
        ↓
Asynchronous events
        ↓
Recovery
        ↓
Eventual consistency

The device and server can temporarily disagree.

Two requests can arrive simultaneously.

A response can disappear.

A worker can crash.

A message can be delayed.

A downstream service can be unavailable.

These are distributed-systems problems.

The fact that one participant happens to be a mobile phone doesn't make them any less real.

25. Architecture Has to Account for the Human Environment

Perhaps the most important lesson wasn't technical.

It was architectural empathy.

A field agent doesn't experience:

"HTTP request timeout."

They experience:

"I don't know whether this booking went through."

A farmer doesn't experience:

"Eventual consistency."

They experience:

"Will the tractor actually come?"

An administrator doesn't experience:

"Cache invalidation."

They experience:

"Is the information I'm looking at accurate?"

Good architecture therefore cannot stop at the API boundary.

It has to account for the human workflow that the software exists to support.

26. What Reliability Means to Me Now

The experience changed the questions I ask when designing backend systems.

I don't only ask:

Does the API work?

I ask:

What happens if the network disappears?

What happens if the request succeeds but the response disappears?

What happens if the client retries?

What happens if the retry doesn't preserve operation identity?

What happens if two requests arrive simultaneously?

What happens if the database commits but the message broker is unavailable?

What happens if a worker crashes?

What happens if a notification fails?

What happens if different components temporarily disagree?

And perhaps most importantly:

What happens when the system doesn't know whether something succeeded?

Those questions naturally lead toward:

stable operation identities;
idempotency;
durable synchronization state;
database constraints;
concurrency-aware conflict handling;
transactional outboxes;
asynchronous recovery;
explicit failure states;
observability.

Conclusion: Design for the World as It Is

When we design software, it's natural to imagine the happy path:

User
  ↓
Request
  ↓
Server
  ↓
Database
  ↓
Response

Real systems rarely behave that cleanly.

The network disappears.

Requests time out.

Responses get lost.

Clients retry.

Workers crash.

Messages are delayed.

Two requests arrive at almost exactly the same time.

Dependencies become temporarily unavailable.

And users press the button again because they don't know what happened.

Reliable architecture doesn't eliminate those realities.

It gives them somewhere safe to go.

A timeout becomes a retry.

A duplicate request becomes an existing result.

A concurrency conflict becomes a resolvable state.

A broker outage becomes a pending outbox event.

A notification failure doesn't invalidate a successful booking.

An offline operation eventually converges with the backend's view of reality.

That is what building software for low-connectivity environments taught me:

The network may be unreliable. The system's understanding of the business outcome cannot be.

About the Author

Betini Akarandut is a Backend & Cloud Engineer focused on backend architecture, distributed systems, offline-first applications, cloud infrastructure, and DevOps.

His work involves building and operating production systems across technology-driven environments, with a particular interest in designing reliable software for situations where connectivity, infrastructure, and other assumptions cannot always be taken for granted.

He writes about lessons from building real-world software systems—especially the engineering problems that only become visible when architecture meets production reality.

Building Offline-First Systems for Low-Connectivity Agritech

Wed, 02 Sep 2026 03:19:03 +0000

A note on the architecture

This article describes lessons from a production agricultural technology platform. Some implementation details, operational parameters, identifiers, and business-specific logic have been generalized or omitted to avoid exposing proprietary or sensitive information. The goal is to explain the engineering problems, decisions, and principles rather than the private details of the system.

The booking that happened twice

We had built idempotency to prevent duplicate tractor bookings.

Then duplicate tractor bookings started appearing.

The first thing I checked was the idempotency logic.

It looked correct.

That was the problem.

The mechanism wasn't necessarily broken.

The assumption underneath it was.

The application therefore couldn't simply say:

"No internet? Come back later."

The agent needed to continue working.

The operation had to be stored locally and delivered to the server when connectivity returned.

That sounds like an offline mobile-app problem.

It quickly became a distributed-systems problem.

Once a device can create an operation without an active connection to the server, a much harder set of questions appears:

What happens when synchronization times out?
What if the server processes the request but the response never reaches the phone?
What if the mobile client retries?
How does the server know that two requests represent the same business operation?
What happens if two identical requests arrive at almost exactly the same time?
What happens if the database transaction succeeds but Kafka is temporarily unavailable?
What happens when the booking succeeds but nobody is notified?

These were the problems that shaped the architecture.

And, more importantly, they changed how I think about reliability.

1. Designing for a Network That Cannot Be Trusted

The fundamental requirement was simple:

A field agent should be able to continue important work even when the network disappears.

The workflow looked roughly like this:

Select or register a farmer.
Select a tractor or agricultural tool.
Select the relevant dates.
Complete the booking.
Store the operation locally if connectivity is unavailable.
Synchronize it when connectivity returns.

The important architectural distinction is this:

Creating an operation and delivering an operation are two different things.

The agent creates the booking locally.

The network eventually delivers it.

Those events may happen seconds, minutes, or much longer apart.

Caption:

The offline workflow separates creating an operation from delivering it to the backend.

This distinction becomes important once the same operation can be delivered more than once.

2. The First Line of Defense: Idempotency

The standard solution to repeated delivery is idempotency.

Each offline operation needs a stable identity.

The conceptual model is:

One logical booking
        │
        ▼
Operation ID: A
        │
   ┌────┼────┐
   ▼    ▼    ▼
Attempt 1  Attempt 2  Attempt 3
   │         │         │
   └─────────┼─────────┘
             ▼
        Same operation

If the server sees the same operation ID again, it should not perform the business operation from scratch.

Instead:

Request #1
    │
    ├── Key = A
    ▼
Create booking
    │
    ▼
Synced


Request #2
    │
    ├── Key = A
    ▼
Existing operation
    │
    ▼
Return existing result

That's what idempotency is supposed to provide.

But this is where an important assumption appears:

The server can only deduplicate requests that the client identifies as the same operation.

Code Snippet 1 — Persistent Idempotency Identity

final formData = Map<String, dynamic>.from(form.formData);
formData['idempotency_key'] = form.idempotencyKey;

final headers = {
  'Content-Type': 'application/json',
  'X-Idempotency-Key': form.idempotencyKey,
  'Accept': 'application/json',
};

Caption:

The idempotency key belongs to the logical offline form, not to an individual HTTP attempt.

3. The Idempotency Mechanism Wasn't Broken

Then we encountered duplicate bookings.

The natural question was:

"Why didn't idempotency stop this?"

The investigation revealed a more subtle failure mode.

An earlier client retry path could generate a fresh idempotency key for a retry of the same logical draft.

From the backend's perspective, that changes everything.

Imagine one booking being attempted three times:

             ONE LOGICAL BOOKING
                     │
          ┌──────────┼──────────┐
          ▼          ▼          ▼
       Retry 1    Retry 2    Retry 3
          │          │          │
        Key A      Key B      Key C
          │          │          │
          ▼          ▼          ▼
       Server     Server     Server
          │          │          │
          ▼          ▼          ▼
       Booking    Booking    Booking
          1          2          3

The server isn't seeing:

A
A
A

It is seeing:

A
B
C

There is therefore nothing for a key-based deduplication mechanism to connect.

The server isn't necessarily malfunctioning.

It has been given three different identities.

Diagram 2 — The Idempotency/Retry Failure

Caption:

The server can only deduplicate what the client identifies as the same operation.

This is one of the most important lessons I took from the incident:

Idempotency isn't simply a backend feature. It's a contract between the producer and consumer of an operation.

The identity of the business operation has to survive every delivery attempt.

4. When the Ideal Fix Isn't Immediately Available

The clean solution was to ensure that the operation receives its identity when it is created and that every synchronization attempt reuses that identity.

That is the model reflected in the current synchronization implementation: the form carries its persistent idempotency key, and synchronization reuses it.

But production systems sometimes require another consideration:

What can we safely do while an upstream problem is being corrected?

The backend still needed to protect the integrity of the booking workflow.

So I introduced a second layer of duplicate protection.

Not as a replacement for idempotency.

As a defensive backstop.

5. The Imperfect Safety Net

The backend gained an additional short-window, multi-attribute duplicate-detection mechanism.

The idea was straightforward:

If a new submission looks strongly like a recently synchronized booking, don't blindly create another business record.

This was intentionally not treated as the primary correctness mechanism.

It was a containment mechanism.

And that distinction matters.

A heuristic can have false positives.

It can have false negatives.

A legitimate booking might resemble another booking.

A retry might arrive outside the heuristic's window.

So the architecture remained:

Stable operation identity
          │
          ▼
      Idempotency
          │
          ▼
Database uniqueness
          │
          ▼
Concurrency handling
          │
          ▼
Defensive duplicate backstop

The backend implementation history explicitly describes this as a mitigation for the client retry problem rather than a substitute for correct idempotency semantics.

That led to an important engineering principle for me:

A defensive heuristic should contain a failure mode, not become the system's definition of correctness.

6. Then Concurrency Exposed Another Problem

Solving duplicate delivery introduced another question.

What happens when two requests with the same idempotency key arrive at almost exactly the same time?

A common implementation starts with:

Does this key already exist?

If not:

Create it.

Sequentially, this appears safe.

Under concurrency, it isn't.

Consider two requests:

Request A                  Request B
    │                          │
    ▼                          ▼
Check key                  Check key
    │                          │
"Not found"                "Not found"
    │                          │
    └──────────┬───────────────┘
               ▼
             INSERT
            /      \
           ▼        ▼
       Winner     Conflict

Both requests can perform the existence check before either one has committed its insert.

Then both attempt to insert the same unique value.

PostgreSQL correctly rejects one of them with a uniqueness violation.

At the database level, everything is behaving correctly.

At the application level, however, treating that as an ordinary failure would be misleading.

The losing request may simply be:

the second attempt to perform an operation that another request has just successfully created.

Diagram 3 — Concurrent Idempotency Race

Caption:

A check-then-insert sequence can race under concurrency; the database constraint becomes part of the application-level resolution.

7. Turning a Database Error Into a Business Outcome

The first implementation instinct might be:

UniqueViolation
      ↓
500 Internal Server Error

But that isn't necessarily what happened.

The database may simply be telling us:

"Another request got there first."

I introduced _is_idempotency_conflict() to distinguish the relevant uniqueness conflict from unrelated integrity errors.

Code Snippet 2 — Idempotency Conflict Detection

@staticmethod
def _is_idempotency_conflict(error: IntegrityError) -> bool:
    return "idempotency_key" in str(
        getattr(error, "orig", error)
    )

Caption:

The application distinguishes an idempotency-key conflict from unrelated database integrity errors.

But detecting the conflict wasn't the final step.

The service then needs to resolve the request against the operation that won the race.

Conceptually:

                 INSERT
                   │
             ┌─────┴─────┐
             │           │
          Success      Conflict
             │           │
             ▼           ▼
          Continue    Read winner
                         │
                   ┌─────┴─────┐
                   ▼           ▼
                Synced       Pending
                   │           │
                   ▼           ▼
             Return result   Retry later

If the winning operation is already synchronized, the losing request can receive that result.

If the winning operation is still being processed, the system can communicate a pending state rather than falsely declaring the business operation failed.

This changed how I think about database exceptions.

A database exception is sometimes evidence of a state transition elsewhere, not evidence that the user's business operation failed.

8. The Lost-Response Problem

There is another reason idempotency is essential.

Suppose the server successfully creates a booking.

Then the response is lost.

Mobile                       Server
   │                           │
   │──── Create booking ──────►│
   │                           │
   │                        Save ✓
   │                           │
   │◄──── Response ────────────X
   │          lost
   │
   │──── Retry ────────────────►│

The mobile application doesn't know whether the booking succeeded.

From its perspective:

"The request failed."

Retrying is therefore completely reasonable.

The danger is allowing that retry to create another booking.

Stable operation identity solves the uncertainty:

Retry
  │
  ▼
Same operation ID
  │
  ▼
Existing operation
  │
  ▼
Return existing result

The current synchronization engine also explicitly handles duplicate/conflict responses by treating the local form as synchronized rather than repeatedly attempting to create the operation.

This illustrates a broader distributed-systems principle:

The client doesn't always need to know what happened immediately. The system needs a safe way to resolve uncertainty when the client asks again.

9. A Successful Database Write Isn't the End

Once the booking is successfully stored, another problem appears.

Other parts of the system may need to know about the operation.

Maybe an event needs to be published.

Maybe another service needs to consume it.

Maybe a notification needs to be sent.

This creates another failure boundary.

Consider:

Save booking
     │
     ▼
PostgreSQL ✓
     │
     ▼
Publish event
     │
     X
Kafka unavailable

Now the business operation exists.

But the event doesn't.

If we reverse the order, we create the opposite problem:

Publish event
     │
     ▼
Kafka ✓
     │
     X
Database transaction fails

Now another system has received an event for an operation that never became durable.

We needed the database to become the durable source of truth for both the business state change and the intention to publish its event.

10. The Transactional Outbox

The solution was the transactional outbox pattern.

The business write and the corresponding outbox event are persisted within the same database transaction.

Conceptually:

┌────────────────────────────────┐
│        DB TRANSACTION          │
│                                │
│     Create booking             │
│            +                   │
│     Create outbox event        │
│                                │
└───────────────┬────────────────┘
                │
              COMMIT
                │
                ▼
          Relay Worker
                │
                ▼
              Kafka

The important guarantee is:

If the business transaction commits, the intent to publish the event is also durable.

The outbox implementation explicitly inserts the event as part of the same database transaction as the business operation and leaves publication to a later relay process.

Diagram 4 — Transactional Outbox → Kafka → Notifications

Caption:

The business change and the intent to publish its event become durable together; delivery happens asynchronously.

Code Snippet 4 — Transactional Outbox Write

For publication, use the actual relevant portion of your emit_event() implementation rather than a generic pseudocode transaction.

The important source-level pattern is:

db.execute(
    text("INSERT INTO outbox_events (...) VALUES (...)"),
    {...}
)

with the function called within the same database transaction as the business write.

Caption:

The outbox records the intent to publish while the business transaction is still the source of truth.

Important editorial rule: don't expose the full internal schema, private topic names, internal identifiers, or production payload structure.

11. Let the Relay Worry About Delivery

Once the event is safely stored, the original booking request doesn't need to remain responsible for delivering it to Kafka.

A background relay can take over.

The architecture becomes:

Business transaction
        │
        ▼
PostgreSQL + Outbox
        │
        ▼
Relay worker
        │
        ▼
Kafka

The relay can retry delivery independently.

This also introduces another concurrency question:

What happens if multiple relay workers are running?

We don't want two workers to claim the same pending event unnecessarily.

The relay uses database locking with FOR UPDATE SKIP LOCKED when claiming available outbox work.

Code Snippet 3 — Safely Claiming Outbox Events

Show the relevant query portion:

.with_for_update(skip_locked=True)

Caption:

FOR UPDATE SKIP LOCKED allows multiple workers to claim available events without waiting on rows already locked by another worker.

Again, the interesting part isn't the syntax itself.

It's the reason it exists.

The database is being used not just as storage, but as part of the coordination mechanism between concurrent workers.

12. Then We Found Something That Wasn't Broken

At this point, the system was getting much better at processing operations reliably.

But there was another question:

Who knows that the operation succeeded?

An offline hire could synchronize successfully.

The database could contain the booking.

The event could be persisted.

The downstream processing could work.

And yet the human workflow could still be silent.

There was no guarantee that everyone who needed to know would immediately receive useful confirmation.

This wasn't an idempotency bug.

It wasn't a database bug.

It was a workflow visibility gap.

Technical correctness and operational correctness aren't always the same thing.

13. Making Successful Synchronization Visible

The synchronization flow was extended with a notification path for successfully synchronized offline hires.

A successful synchronization can now lead to downstream communication such as:

SMS to the farmer;
in-app notification;
group push notification;
admin-facing notification;
relevant dashboard cache invalidation.

The notification implementation also treats those downstream actions as best-effort consequences rather than allowing a notification failure to invalidate the underlying business operation.

That distinction matters.

We don't want:

SMS provider unavailable
        ↓
Booking failed

The booking already succeeded.

The notification is another operation.

So the architecture becomes:

              Booking
                 │
                 ▼
               Commit
                 │
                 ▼
               Event
                 │
       ┌─────────┼─────────┐
       ▼         ▼         ▼
     Farmer     Agent     Admin
      SMS       Push     Dashboard

A system isn't fully successful merely because the database is correct.

The people who depend on that state need a reliable way to know what happened.

14. Testing the System Like the Network Hates You

This experience changed how I think about testing offline systems.

The conventional test looks something like:

Submit booking
      ↓
Receive response
      ↓
Booking created

That's necessary.

It isn't enough.

For an offline synchronization system, the interesting tests deliberately violate the assumptions.

What happens when connectivity disappears during synchronization?

What happens when the server processes the request but the response disappears?

What happens when the client retries?

What happens when a retry uses the wrong operation identity?

What happens when two requests arrive simultaneously?

What happens when the database commits but Kafka is unavailable?

What happens when the relay worker crashes?

What happens when notification delivery fails?

These scenarios aren't edge cases in the philosophical sense.

They are normal possibilities in distributed systems.

The architecture needs a deliberate answer for each one.

15. Build a Failure Matrix

One useful way to reason about these systems is to map failures to expected outcomes.

Failure	Desired outcome
No connectivity	Operation remains available for synchronization
Synchronization timeout	Operation can be retried
Response lost after server success	Retry resolves to the existing operation
Same operation ID submitted again	One business effect
Incorrect retry identity	Defensive duplicate protection may limit damage
Concurrent same-key requests	One wins; the other resolves against the winner
Database succeeds, Kafka fails	Outbox preserves the event
Relay worker fails	Event remains available for later processing
Multiple relay workers	Workers safely claim available work
Notification fails	Business operation remains successful
Dashboard data becomes stale	Relevant cache can be invalidated

The question changes from:

"Did an error occur?"

to:

"What state should the system eventually reach after this failure?"

That is a much more useful question for distributed systems.

16. Reliability Is a Chain, Not a Feature

Looking back at the architecture, there wasn't one feature called "offline reliability."

There was a chain of mechanisms.

                 Mobile App
                     │
                     ▼
             Offline operation
                     │
                     ▼
           Stable operation ID
                     │
                     ▼
                Idempotency
                     │
                     ▼
           Duplicate protection
                     │
                     ▼
          Database constraints
                     │
                     ▼
         Concurrency resolution
                     │
                     ▼
          Transactional outbox
                     │
                     ▼
               Relay → Kafka
                     │
                     ▼
          Notifications / Admin
                     │
                     ▼
          Observable outcome

Each layer exists because a different assumption can fail.

Local persistence doesn't prevent duplicate delivery.

Idempotency doesn't work if operation identity changes.

Database uniqueness doesn't automatically tell the application what a conflict means.

An outbox doesn't guarantee that every downstream consumer will process an event immediately.

Notifications don't automatically guarantee that a dashboard is current.

Reliability emerges from the interaction between these mechanisms.

17. What I Would Design Differently Today

If I were designing the system from scratch today, I would make operation identity a first-class concept from the beginning.

A booking is a business operation.

An HTTP request is merely one attempt to deliver that operation.

Those should not be treated as the same thing.

                 ONE BOOKING
                     │
               Operation ID
                   ABC123
                     │
          ┌──────────┼──────────┐
          ▼          ▼          ▼
       Attempt 1  Attempt 2  Attempt 3

Every attempt should preserve the same identity.

I would also explicitly model synchronization as a state machine:

CREATED
   │
   ▼
PENDING_SYNC
   │
   ├──────────────┐
   ▼              ▼
SYNCED          FAILED
   │
   ▼
NOTIFIED

The exact states can differ between systems.

The principle is more important:

Don't collapse uncertainty into failure.

A timeout doesn't necessarily mean failure.

A duplicate doesn't necessarily mean failure.

A uniqueness conflict doesn't necessarily mean failure.

A temporary broker outage doesn't mean the booking itself failed.

Those distinctions should exist in the architecture.

18. I Would Treat Heuristics as Defense in Depth

The duplicate-detection backstop was useful.

But I wouldn't want it to become the primary correctness mechanism.

The hierarchy should remain:

Stable operation identity
          ↓
Database uniqueness
          ↓
Concurrency handling
          ↓
Business-rule safeguards

The strongest guarantees should establish correctness.

Heuristics should help contain damage when something upstream behaves unexpectedly.

That distinction is important because a system can become dangerously complicated when a workaround gradually becomes the mechanism everyone assumes is responsible for correctness.

19. I Would Make Observability Business-Aware

Infrastructure monitoring is necessary.

CPU.

Memory.

Latency.

Database health.

Queue depth.

Worker health.

But an offline synchronization system needs business-level observability too.

I'd want to know:

How many operations are waiting?

How many have failed?

How many have retried?

How many duplicate attempts were detected?

How many idempotency conflicts occurred?

How long do operations remain pending?

How many outbox events are waiting?

How many downstream notifications failed?

Those metrics answer a different question.

Not:

"Are my servers running?"

But:

"Is the business workflow actually progressing?"

That distinction becomes increasingly important as systems become more distributed.

20. Idempotency Is a Contract, Not a Checkbox

It's easy to say:

"We have idempotency."

But that statement isn't enough.

You also need to answer:

What makes two requests the same business operation?

If the answer is an operation ID, that identity needs to survive retries.

If the identity changes with every attempt, the server cannot reliably connect those requests.

This is why mobile and backend components need a shared understanding of operation identity.

The mobile application owns the logical operation.

The network merely transports attempts to synchronize it.

21. Retries Change the Semantics of an Operation

A retry isn't simply another function call.

It may represent:

a request that never reached the server;
a request that reached the server but timed out;
a request that succeeded but whose response disappeared;
an operation whose outcome is genuinely unknown.

Those scenarios can all look similar from the client.

The backend therefore needs a way to answer:

"What happened to this operation?"

rather than simply:

"What should I do with this HTTP request?"

That's why stable identity, durable state, and idempotent processing matter.

22. Exactly Once Isn't Always the Goal

Distributed systems often make people think about "exactly once."

In practice, guaranteeing that a distributed operation is attempted exactly once can be extremely difficult.

A more practical objective is:

Allow repeated delivery while keeping the business outcome correct.

The network can deliver an operation more than once.

The client can retry.

A worker can restart.

An event can be processed again.

The architecture should still converge toward the correct business state.

This is particularly important in offline-first applications because intermittent connectivity naturally creates uncertainty around delivery.

23. Failure Needs Business Meaning

One of the strongest lessons from this work was that technical errors and business failures aren't always the same thing.

Consider:

Timeout

That doesn't necessarily mean:

Booking failed.

Consider:

Unique constraint violation

That doesn't necessarily mean:

Booking failed.

Consider:

Kafka unavailable

That doesn't necessarily mean:

Booking failed.

Consider:

Notification provider unavailable

That definitely doesn't necessarily mean:

Booking failed.

A robust system needs to distinguish:

transport failure

processing failure

business failure

and

uncertainty about the outcome.

When all four become:

500 Internal Server Error

the system loses important information.

24. Offline-First Is a Distributed-Systems Problem

This is probably the biggest lesson I took away from the project.

Offline-first is often described as a mobile development technique:

Store data locally and synchronize it later.

That description is technically true.

It doesn't capture the difficulty of doing it reliably.

Once synchronization becomes important, you quickly encounter:

Unreliable connectivity
        ↓
Uncertain delivery
        ↓
Retries
        ↓
Idempotency
        ↓
Concurrency
        ↓
Transactions
        ↓
Asynchronous events
        ↓
Recovery
        ↓
Eventual consistency

The device and server can temporarily disagree.

Two requests can arrive simultaneously.

A response can disappear.

A worker can crash.

A message can be delayed.

A downstream service can be unavailable.

These are distributed-systems problems.

The fact that one participant happens to be a mobile phone doesn't make them any less real.

25. Architecture Has to Account for the Human Environment

Perhaps the most important lesson wasn't technical.

It was architectural empathy.

A field agent doesn't experience:

"HTTP request timeout."

They experience:

"I don't know whether this booking went through."

A farmer doesn't experience:

"Eventual consistency."

They experience:

"Will the tractor actually come?"

An administrator doesn't experience:

"Cache invalidation."

They experience:

"Is the information I'm looking at accurate?"

Good architecture therefore cannot stop at the API boundary.

It has to account for the human workflow that the software exists to support.

26. What Reliability Means to Me Now

The experience changed the questions I ask when designing backend systems.

I don't only ask:

Does the API work?

I ask:

What happens if the network disappears?

What happens if the request succeeds but the response disappears?

What happens if the client retries?

What happens if the retry doesn't preserve operation identity?

What happens if two requests arrive simultaneously?

What happens if the database commits but the message broker is unavailable?

What happens if a worker crashes?

What happens if a notification fails?

What happens if different components temporarily disagree?

And perhaps most importantly:

What happens when the system doesn't know whether something succeeded?

Those questions naturally lead toward:

stable operation identities;
idempotency;
durable synchronization state;
database constraints;
concurrency-aware conflict handling;
transactional outboxes;
asynchronous recovery;
explicit failure states;
observability.

Conclusion: Design for the World as It Is

When we design software, it's natural to imagine the happy path:

User
  ↓
Request
  ↓
Server
  ↓
Database
  ↓
Response

Real systems rarely behave that cleanly.

The network disappears.

Requests time out.

Responses get lost.

Clients retry.

Workers crash.

Messages are delayed.

Two requests arrive at almost exactly the same time.

Dependencies become temporarily unavailable.

And users press the button again because they don't know what happened.

Reliable architecture doesn't eliminate those realities.

It gives them somewhere safe to go.

A timeout becomes a retry.

A duplicate request becomes an existing result.

A concurrency conflict becomes a resolvable state.

A broker outage becomes a pending outbox event.

A notification failure doesn't invalidate a successful booking.

An offline operation eventually converges with the backend's view of reality.

That is what building software for low-connectivity environments taught me:

The network may be unreliable. The system's understanding of the business outcome cannot be.

About the Author

Betini Akarandut is a Backend & Cloud Engineer focused on backend architecture, distributed systems, offline-first applications, cloud infrastructure, and DevOps.

He writes about lessons from building real-world software systems—especially the engineering problems that only become visible when architecture meets production reality.

From a Commodore 64 to AI Agents: Istvan Dobrentei's Journey Through Software Development

Wed, 26 Aug 2026 01:28:04 +0000

From a Commodore 64 to AI Agents: Istvan Dobrentei's Journey Through Software Development

From a black-and-white TV and BASIC programming to PHP, cloud migration, DevOps, and today's world of AI agents and LLMs.

Meet Istvan Dobrentei

Some programming journeys begin with a computer science degree.

Others begin with a job.

Istvan Dobrentei's started with a Commodore 64 connected to a small black-and-white television.

At around 10 years old, Istvan encountered something that felt almost magical: a machine communicating with him through a TV screen.

The messages weren't exactly encouraging.

They included things like:

Syntax error

and

Type mismatch error

He didn't understand them yet.

But he was fascinated.

That curiosity eventually led him from BASIC to Turbo Pascal, C, Delphi, PHP, JavaScript and Java, and ultimately into a long career as a software engineer.

His first computer-related job came in 2001, working in a library where he built a PHP application that made publications searchable online.

In many ways, his journey into software development started with books.

Since then, his career has taken him through real-world business systems, legacy applications, cloud migration, DevOps, and modern web development.

Today, he is also exploring AI, LLMs and agentic architecture, alongside his background as a Zend-certified full-stack PHP developer and Microsoft- and AWS-certified engineer.

But technology is only one part of Istvan's story.

His journey has also been shaped by continuous learning, ethical decisions, difficult experiences, resilience, and the realization that good software is about much more than simply making code work.

In this Developer Story, Istvan takes us from his first Commodore 64 to the modern world of AI and shares the lessons he has learned along the way.

Istvan's Story

It Started With a Commodore 64

When I was around 10 years old, I got my first computer: a Commodore 64.

At that time, it was something truly special.

Have you ever heard about the COCOM list?

Technology behind the Iron Curtain was very limited.

There was no monitor, so we had to connect the machine to a TV and manually adjust the frequency just to see the command prompt on the screen.

Back then, even having a color TV was a big deal.

I remember that I only had a small black-and-white TV called Junoszty.

In our country, there weren't many opportunities to learn English during that period.

Russian was mandatory in school, and I could choose only one additional language: German.

So when I saw the very first message appear on the black-and-white TV screen, I felt like I was on the moon.

The machine was communicating with me!

The messages were things like:

Syntax error

and

Type mismatch error

At that time, I had no idea what they meant, but I was fascinated.

Today, of course, I understand them well, but back then it felt almost magical.

Imagining a Worldwide Computer Network

I also clearly remember reading an article about computer networks in a magazine around 1992.

They wrote about CompuServe, and I imagined what a worldwide computer network could look like.

Today, of course, we know it as the World Wide Web.

My friends and I spent many hours playing games on that machine.

But eventually, programming became more interesting than simply using the computer.

One day, a friend gave me a book about the BASIC programming language.

I was probably 12 or 13 years old, and that was when I started learning programming by myself.

Later, in school, I chose computer science subjects to learn more.

We started with BASIC, then moved on to:

Turbo Pascal
C
Delphi

Around 2000, I learned PHP and JavaScript, and later Java.

Learning the Fundamentals

At the very beginning, in the classroom environment, I learned from mathematics teachers and solved basic mathematical and programming problems.

I also had a book full of beginner programming exercises.

Learning the fundamentals was very important.

I created many side projects that helped solve different kinds of problems.

For example, I developed an application that helped teachers organize class schedules and subject timelines for different classrooms.

Those projects gave me something that books alone couldn't provide:

practice.

You learn something differently when you actually have to make it work.

Interestingly, my first job was not related to computers.

I first worked as a book sales agent, and later I worked in a library.

That became my first computer-related job in 2001.

There, I learned PHP and created my first real application.

It helped organize different kinds of publications and made them searchable through an online interface.

Looking back, there is something quite fitting about this.

My journey into software development started with books.

Those early experiences eventually led me into professional web development and a career that would span many different technologies.

What Years of Experience Changed

It is really interesting because today I see myself in a different way.

After many years of experience, I see things very differently than I did when I was younger.

I have seen many kinds of code across many different domains and businesses of various sizes.

I have seen insightful and highly successful companies, but also shrinking businesses, along with the codebases behind those business models.

One surprising realization was that the best-designed and most well-organized code wasn't always the one that generated huge profits or became the most successful.

That was a key moment for me.

It taught me that business success depends on many factors beyond technology alone.

At the same time, I realized that good architecture and clean code are essential for long-term success.

Good code improves:

maintainability
scalability
teamwork
long-term sustainability

In many cases, poorly designed systems eventually become difficult to maintain and slow down innovation.

So I've learned that software engineering requires a balance.

Technology matters, but technology alone does not determine business success.

The Challenge of Working With Real Businesses

One of the biggest challenges in my developer journey was working as an independent contractor for SMEs.

I saw many interesting business ideas and situations from a business perspective.

However, I always stayed committed to ethical and legal principles.

Whenever I was asked to do something that was against the law or not ethical, I said no.

For me, that is an important part of being a professional.

Developers can have access to sensitive systems, information, business processes, and data.

Technical ability comes with responsibility.

Sometimes doing the right thing means refusing to do something.

Migrating a Legacy Webshop to the Cloud

Another major challenge was migrating a live legacy webshop to a cloud environment.

At that time, cloud technologies were a completely new area for me.

I had to quickly learn many different concepts and technologies.

To overcome the challenge, I invested a lot of time in learning, earned relevant certifications, and continuously improved my knowledge step by step.

In the end, the migration was successful.

The experience taught me the importance of:

adaptability
continuous learning
persistence

Technology doesn't wait for you to become comfortable.

Sometimes you have to learn something new because the project demands it.

There Is No Single Technology That Defines a Developer

I don't think there is a single technology that I could highlight as being responsible for my growth.

In the IT industry, continuous learning and self-learning are what really matter.

New solutions, tools, and resources appear every month, especially in today's world.

One thing is certain:

Continuous learning and adapting to change are essential.

Instead of focusing only on specific technologies, I would highlight:

critical thinking
curiosity
the ability to learn new things every day
adapting to change

Technologies will come and go.

The ability to learn remains.

The Job That Taught Me a Difficult Lesson

I remember a sad story, although today I can laugh about it.

At the time, however, it was a very difficult experience.

Once, I applied for a job where my task was to create an Android application to handle one of the company's logistics business functions.

It was an experimental project.

The company worked in a team environment without any version control and without applying object-oriented principles.

That really shocked me.

One day, the CEO asked for my opinion about their system.

I spent the weekend preparing a detailed and honest presentation and presented it to the team.

The next day, I was unemployed.

I did not criticize the system personally.

I only highlighted the main issues and shared what I had learned over the years.

It was a difficult experience.

But it taught me something important.

Life can sometimes be difficult and unfair.

Not every situation will reward honesty immediately.

But persistence and hard work eventually pay off.

Never give up.

Learning to Think Like a Developer

At the beginning of my learning journey, even solving simple tasks programmatically was a challenge.

Learning how to think like a developer and how to write code was not easy at first.

I believe software development is a step-by-step journey.

One of the most important lessons I learned is to try solving problems by yourself instead of always looking for shortcuts.

Practice is essential.

The more you build, experiment, and learn from mistakes, the more you improve over time.

You don't become a better developer simply by reading about programming.

You become better by actually doing it.

Nobody Is Perfect

Another important lesson is that nobody is perfect, and growth is a continuous process.

Mistakes are a natural part of learning.

The real question is whether you are able to reflect on them, adapt, and move forward.

This is especially important in technology because there is always something new to learn.

You will make mistakes.

You will sometimes choose the wrong technology.

You will sometimes write code that you later want to rewrite.

That's normal.

What matters is what you learn from the experience.

Resilience Matters

I also learned the importance of resilience.

Resilience is the ability to recharge yourself after challenges and keep going even when things become difficult.

Software development can be frustrating.

Projects can fail.

Jobs can end unexpectedly.

Technologies can change.

You can spend days solving a problem that seemed like it should take an hour.

The ability to recover and continue is therefore just as important as technical knowledge.

Learn From Other People

Having mentors and being open to feedback is extremely valuable.

Nobody can know everything or be an expert in every area.

It is essential to accept advice and learn from others.

Doing so can accelerate both personal and professional growth much faster than trying to figure everything out alone.

Don't hesitate to ask questions.

Don't be afraid to admit that you don't understand something.

Learning from other people is not a weakness.

It is one of the fastest ways to grow.

From PHP to Cloud and AI

My technology journey has continued to evolve.

PHP and web development remain an important part of my background, but my interests have expanded into cloud, DevOps, AI, and modern development tools.

I've earned certifications including:

PHP Zend Engineer
Microsoft Azure Fundamentals
Microsoft Azure Developer Associate
Microsoft DevOps Engineer Expert
AWS Certified AI Practitioner

Today, I'm particularly interested in LLMs, AI, cloud technologies, and AI agents.

The interesting thing about technology is that the learning never really stops.

The Commodore 64 I used as a child could not have prepared me for today's AI systems.

But the curiosity that made me want to understand that first computer is still useful.

The tools changed.

The curiosity didn't.

Advice for Developers Starting Today

As I mentioned earlier, in our rapidly changing world, the most important thing is to keep learning new things, stay curious, and never give up.

Hard work pays off in the long term.

Teamwork is also very important because nobody can know everything or be an expert in every area.

It is essential to truly understand what you are doing.

Don't just copy and paste code.

Understand why it works.

Understand why a feature works.

And understand why a feature may not work.

If you have doubts or don't understand something, ask for help.

There is nothing wrong with asking questions.

In fact, asking the right question can sometimes teach you more than finding the answer yourself.

My Experience With CoderLegion

It was a surprise for me when I was first invited to create publications.

It is an honor to share my thoughts with the community through a platform like CoderLegion.

It is also great to see the active community and the engagement behind the platform.

I have been able to read many interesting and up-to-date articles from talented people all around the world.

This gives me motivation to keep moving forward and continue learning.

Thank you for the opportunity, and I wish continued success to both the community and CoderLegion.

Looking Back

When I think about my journey, the contrast is quite remarkable.

I started with a Commodore 64 connected to a black-and-white television.

I learned BASIC from a book.

Then came Turbo Pascal, C, Delphi, PHP, JavaScript, and Java.

I worked in a library and built a searchable publication system.

I worked with real businesses and legacy systems.

I learned cloud technologies when they were new to me.

I earned certifications and continued developing my skills.

And today, I'm exploring AI, LLMs, cloud technologies, and AI agents.

The technology has changed dramatically.

But the fundamental lesson has remained the same:

Stay curious. Keep learning. Never give up.

About Istvan Dobrentei

Istvan Dobrentei is a Software Engineer and Zend-certified full-stack PHP developer based in Hungary.

His technical background includes PHP, JavaScript, Python, Docker, MySQL, Ansible, and GitHub, alongside extensive experience with Microsoft Azure and DevOps.

He holds certifications including PHP Zend Engineer, AWS Certified AI Practitioner, Microsoft Azure Developer Associate, and Microsoft DevOps Engineer Expert.

Today, Istvan is exploring LLMs, AI, cloud technologies, AI agents, and agentic architecture, continuing a technology journey that began with a Commodore 64 when he was a child.

You can learn more about Istvan and his work on his website and CoderLegion profile.

A Final Thought

Istvan's journey shows how dramatically a developer's world can change.

A child connecting a Commodore 64 to a black-and-white television could hardly have imagined today's cloud platforms, AI agents, and large language models.

But perhaps the most important skill was already there.

Curiosity.

The desire to understand how something works.

The willingness to learn.

The courage to ask questions.

And the resilience to keep going when things don't work out.

Technologies will continue to change.

Programming languages will evolve.

New AI systems will appear.

But the developers who continue to learn, adapt, and stay curious will always have somewhere to go next.

The technology changes. The learning never stops.

This Developer Story is part of CoderLegion's Developer Stories series, where we share the real journeys, experiences, challenges, and lessons of developers from our community.

No More Midnight Deploys: The Magic of Blue-Green Deployments

Tue, 25 Aug 2026 12:24:31 +0000

No More Midnight Deploys: The Magic of Blue-Green Deployments

What is a Blue-Green Deployment?

The Bridge Analogy: Building a Parallel Path

Why it Matters in Modern Software Engineering

In the tech industry, engineers use blue-green deployments to solve two major problems: service downtime and catastrophic deployment failures.

The "Switch" in Action: An Nginx Routing Configuration

Here is a simplified configuration demonstrating how an engineer flips traffic between the two environments by updating where the server points:

# /etc/nginx/conf.d/app.conf

# STEP 1: Routing to the Blue environment (Port 8080)
upstream production_app {
    server 10.0.0.10:8080; # Blue (Active)
    # server 10.0.0.20:8080; # Green (Idle)
}

# STEP 2: When Green is ready, the engineer modifies the configuration:
# upstream production_app {
#     # server 10.0.0.10:8080; # Blue (Now Idle/Fallback)
#     server 10.0.0.20:8080; # Green (Now Active!)
# }

server {
    listen 80;
    server_name myapp.com;

    location / {
        proxy_pass http://production_app;
    }
}

By changing a single line of configuration and reloading Nginx, the traffic instantly switches over without dropping a single active user connection.

The Takeaway

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Understanding the Consumption-Based Model in Cloud Computing

Sat, 22 Aug 2026 08:44:28 +0000

Cloud computing has transformed the way organizations build and deploy applications. Instead of purchasing expensive servers and networking equipment upfront, businesses can access computing resources on demand and pay only for what they use.

This pricing approach is known as the consumption-based model, and it is one of the biggest reasons cloud computing has become so popular.

What Is the Consumption-Based Model?

The consumption-based model (also called pay-as-you-go pricing) is a cloud billing approach where customers are charged based on their actual resource usage rather than paying a fixed amount in advance.

Whether you're using virtual machines, databases, storage, or networking services, your bill depends on how much you consume.

Think of it like your electricity bill—you pay for the units you use instead of paying a fixed amount every month regardless of consumption.

Traditional Infrastructure vs Cloud Consumption

Traditional Infrastructure	Consumption-Based Cloud
Large upfront hardware investment	No upfront infrastructure purchase
Pay even when servers are idle	Pay only for resources you use
Capacity planning required	Scale resources up or down instantly
Hardware maintenance is your responsibility	Cloud provider manages infrastructure

This shift helps organizations avoid spending large amounts of money before they even launch an application.

Key Benefits

1. No Upfront Costs

Traditional IT requires purchasing servers, storage devices, networking hardware, software licenses, and data center infrastructure before deployment.

With cloud computing, you can start using services immediately without these initial investments.

2. Pay Only for What You Use

You're billed based on actual resource consumption.

Examples include:

Compute hours for virtual machines
Storage space used
Number of database transactions
Data transferred over the network
Function executions in serverless platforms

If your application uses fewer resources, your costs decrease automatically.

3. Scale When Needed

Cloud platforms allow resources to grow or shrink depending on demand.

For example:

During a product launch, your application can automatically add more servers.
After traffic decreases, unnecessary resources can be removed to reduce costs.

This elasticity is difficult and expensive to achieve with traditional infrastructure.

4. Better Cost Optimization

Since billing reflects actual usage, organizations are encouraged to monitor workloads and optimize resources.

Common cost-saving practices include:

Shutting down unused virtual machines
Choosing smaller instance sizes
Using auto-scaling
Deleting unused storage
Monitoring cloud spending regularly

A Real-World Example

Imagine you're building an e-commerce website.

Traditional Approach

You purchase servers capable of handling holiday shopping traffic, even though most of the year they remain underutilized.

Result:

High upfront investment
Idle resources
Ongoing maintenance costs

Consumption-Based Cloud

You deploy your application in the cloud.

During normal days, only a few virtual machines run.
During festivals or sales, additional instances automatically start.
Once traffic returns to normal, extra resources shut down.

You only pay for the additional resources while they are actually running.

Consumption-Based vs Subscription-Based Pricing

Consumption-Based	Subscription-Based
Pay for actual usage	Fixed monthly or yearly fee
Flexible costs	Predictable costs
Ideal for changing workloads	Best for consistent workloads
Easy to scale	Usually includes predefined resource limits

Both pricing models have their place, and many cloud providers offer a combination of the two depending on the service.

Things to Keep in Mind

Although the consumption-based model helps reduce unnecessary spending, costs can increase unexpectedly if resources are left running or applications generate more traffic than expected.

To avoid surprises:

Set spending budgets
Enable billing alerts
Monitor resource usage
Remove unused services
Review monthly cloud bills

Cost management is an important part of working with cloud platforms.

Conclusion

The consumption-based model is one of the defining characteristics of cloud computing. Instead of making large capital investments, organizations can treat IT infrastructure as an operational expense, paying only for the resources they consume.

This flexibility allows startups to launch with minimal investment, helps enterprises optimize costs, and enables developers to build scalable applications without worrying about purchasing physical infrastructure.

Whether you're preparing for the AZ-900 Azure Fundamentals certification or simply learning cloud computing, understanding the consumption-based model is essential because it explains why cloud services are both flexible and cost-effective.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

High Availability vs. Scalability in Cloud Computing: Why Both Matter

Thu, 20 Aug 2026 08:44:28 +0000

When building applications for the cloud, two qualities determine whether users have a smooth experience: high availability and scalability.

While these terms are often mentioned together, they solve different problems.

What is High Availability?

High availability (HA) is the ability of a system to remain operational even when failures occur.

Hardware failures, network outages, software bugs, and maintenance are inevitable. Instead of preventing every failure, cloud platforms are designed to continue serving users by automatically switching workloads to healthy resources.

For example, imagine an online banking application. If one server crashes during a transaction, customers shouldn't lose access to their accounts. The cloud redirects traffic to another healthy instance, ensuring minimal or no downtime.

Cloud providers achieve this through:

Redundant infrastructure
Multiple data centers and availability zones
Automatic failover mechanisms
Service Level Agreements (SLAs) that define expected uptime

The result is a more reliable experience for users and reduced business disruption.

What is Scalability?

Scalability is the ability of a system to handle increasing or decreasing workloads by adjusting computing resources.

Unlike high availability, scalability focuses on capacity rather than uptime.

Consider an e-commerce website during a major sale. On a normal day, a few servers may be enough. During a festival sale, millions of users may visit simultaneously. Instead of slowing down or crashing, cloud platforms can allocate additional resources to meet demand.

There are two primary ways to scale:

Vertical Scaling (Scale Up)

Increase the power of an existing machine by adding:

More CPU
More RAM
Faster storage

This approach is simple but eventually reaches hardware limits.

Horizontal Scaling (Scale Out)

Add more server instances and distribute traffic among them.

This method is preferred for cloud-native applications because it provides greater flexibility and resilience. Modern cloud services often automate this process based on CPU usage, memory consumption, or request volume.

High Availability vs. Scalability

Although related, these concepts address different challenges.

High Availability	Scalability
Keeps services running during failures	Handles increasing workloads efficiently
Focuses on minimizing downtime	Focuses on increasing capacity
Uses redundancy and failover	Uses additional computing resources
Improves reliability	Improves performance under load

A modern cloud application should ideally provide both.

Why the Cloud Excels

Traditional on-premises infrastructure often requires purchasing hardware months in advance to prepare for future demand.

Cloud computing changes this model by offering:

On-demand resource allocation
Automatic scaling
Global infrastructure
Built-in redundancy
Pay-as-you-go pricing

Instead of buying servers that may remain idle most of the year, organizations pay only for the resources they actually use. This makes cloud environments both flexible and cost-efficient.

A Real-World Example

Imagine you're running a movie ticket booking platform.

On regular weekdays, traffic is relatively low.

When tickets for a blockbuster movie are released:

User traffic increases dramatically.
The application automatically launches additional servers to handle the load (scalability).
If one server fails during the booking rush, requests are redirected to healthy servers without interrupting users (high availability).

Customers experience a fast and uninterrupted service, even during peak demand.

Conclusion

High availability and scalability are fundamental pillars of cloud computing.

High availability ensures applications remain accessible even when failures occur.
Scalability allows applications to grow or shrink based on demand.
Together, they help organizations deliver reliable, high-performing services while optimizing costs.

Whether you're building web applications, APIs, or enterprise systems, designing for both high availability and scalability is a core principle of modern cloud architecture.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

Reliability and Predictability in Cloud Computing

Tue, 18 Aug 2026 08:44:28 +0000

Cloud computing isn't just about running applications online—it's about ensuring they continue working reliably while delivering consistent performance.

Two key principles that make this possible are

Reliability
Predictability

Understanding these concepts helps developers design applications that users can trust, even when unexpected failures occur.

What is Reliability?

Reliability is the ability of a cloud system to recover from failures and continue operating.

Hardware failures, network issues, software bugs, or even an entire data center outage can happen at any time.

Cloud providers design their infrastructure to minimize the impact of these failures through redundancy and automated recovery mechanisms.

Instead of relying on a single server, applications can run across multiple machines, availability zones, or regions.

For example:

A virtual machine crashes.
The cloud platform automatically starts another instance.
Users experience little to no interruption.

This resilience is one of the biggest advantages of cloud computing over traditional on-premises infrastructure.

Major cloud providers invest heavily in fault tolerance, backup systems, monitoring, and disaster recovery to improve service reliability.

What is Predictability?

Predictability means that cloud services deliver consistent performance and consistent costs over time.

When developers deploy an application, they expect:

Stable response times
Consistent application behavior
Predictable billing based on resource usage

Cloud platforms provide tools to monitor workloads, estimate costs, analyze performance metrics, and scale resources before bottlenecks occur.

This helps organizations plan infrastructure confidently instead of guessing future requirements.

Reliability vs Predictability

Reliability	Predictability
Focuses on recovering from failures	Focuses on delivering consistent results
Ensures applications stay available	Ensures performance and costs remain stable
Uses redundancy, failover, and recovery	Uses monitoring, autoscaling, and forecasting
Measures uptime and resilience	Measures consistency and expected behavior

Both are essential for delivering high-quality cloud applications.

How Cloud Providers Improve Reliability

Modern cloud platforms use several techniques to improve reliability:

Redundant infrastructure to eliminate single points of failure.
Availability Zones to isolate failures.
Load balancing to distribute traffic.
Automatic failover when a resource becomes unhealthy.
Regular backups and disaster recovery strategies.

These mechanisms allow applications to remain operational even during infrastructure failures.

How Cloud Providers Improve Predictability

Predictability is achieved through continuous monitoring and intelligent resource management.

Common practices include:

Monitoring CPU, memory, and network usage
Autoscaling based on workload demand
Capacity planning
Performance analytics
Cost estimation and budgeting tools

These features help teams maintain steady application performance while avoiding unexpected expenses.

Real-World Example

Imagine you're running an online shopping platform during a holidays season sale there are many discounts and offers going on.

Thousands of users suddenly visit your website.

A reliable cloud platform:

Detects a server failure.
Launches replacement instances automatically.
Keeps the website online.

A predictable cloud platform:

Automatically scales resources.
Maintains fast response times.
Keeps performance consistent despite increased traffic.
Lets you estimate infrastructure costs based on usage.

Customers continue shopping without noticing what's happening behind the scenes.

Why These Principles Matter

Reliability and predictability directly impact user experience and business success.

Organizations benefit from:

Reduced downtime
Better customer satisfaction
Stable application performance
Easier capacity planning
Lower operational risk
More accurate cost forecasting

Instead of reacting to failures, teams can proactively build resilient and consistent systems.

Conclusion

Reliability keeps your applications running when failures occur, while predictability ensures they continue delivering consistent performance and costs. Together, these principles form the foundation of modern cloud computing.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

Cloud Computing Is Just npm install for Infrastructure

Sun, 16 Aug 2026 08:44:28 +0000

Every time you pull in a third-party package instead of writing something yourself, you make a trade: you stop owning the internals, but you're still on the hook for how you use it, what it costs you at scale, and what happens the day it breaks in production.

Cloud computing runs on the exact same trade: you're just importing servers, storage, and networking instead of a library.

That mental model answers the questions that actually matter once you move past the marketing pitch: If I don't manage the OS anymore, what am I still on the hook for? Do I self-host, use a managed service, or mix both? Why does my bill move even when my code didn't change? And what happens when the dependency I'm relying on goes down, gets slow, or degrades under load?

In this article, I'll walk through the 5 concepts that answer those questions.

1. The Shared Responsibility Model

While cloud providers manage a significant portion of the infrastructure, customers still have important security and operational responsibilities.

This concept is known as the Shared Responsibility Model, and the division of duties changes depending on whether you're running workloads On-Premises, or using IaaS, PaaS, or SaaS.

On-Premises: You Manage Everything

When applications run in your own data center, every component—data, devices, accounts, IAM, applications, network controls, operating systems, physical servers, networking, and the datacenter itself is your responsibility.

Since there's no cloud provider involved, there's nothing to share.

Infrastructure as a Service (IaaS)

Examples include virtual machines such as Amazon EC2, Azure Virtual Machines, and Google Compute Engine.

The provider supplies the physical infrastructure, hosts, networking, and datacenters, while you manage data, devices, accounts, IAM, applications, network controls, and operating systems.

IaaS offers the most flexibility of the cloud models but also requires the most operational management.

Platform as a Service (PaaS)

Examples include Azure App Service, AWS Elastic Beanstalk, Google App Engine, and Heroku.

PaaS removes the burden of managing operating systems and infrastructure. Customers still manage data, devices, and accounts; IAM, applications, and network controls are shared; and the provider manages the OS, infrastructure, hosts, networking, and datacenters.

Software as a Service (SaaS)

Examples include Microsoft 365, Salesforce, Google Workspace, and Dropbox.

Customers manage data, devices, and accounts, with IAM as a shared responsibility.

The provider manages everything else—applications, network controls, OS, infrastructure, hosts, networking, and datacenters making SaaS the model with the lowest operational overhead.

Shared Responsibility at a Glance

Service Model	Customer Manages	Shared	Provider Manages
On-Premises	Everything	None	Nothing
IaaS	OS, applications, IAM, network controls, data	None	Infrastructure & physical hardware
PaaS	Data, devices, accounts	IAM, applications, network controls	OS, infrastructure & physical hardware
SaaS	Data, devices, accounts	IAM	Applications, OS, infrastructure & physical hardware

No matter which model you choose, your data, devices, and user accounts remain your responsibility.

As you move from On-Premises → IaaS → PaaS → SaaS, the provider takes on progressively more of the operational burden but protecting users, managing access, and securing data always stay in your hands.

2. Cloud Deployment Models: Public, Private, and Hybrid

A cloud deployment model defines where your infrastructure is hosted, who owns it, and who can access it.

The three primary models are:

Public
Private
Hybrid

each offer different levels of flexibility, security, and control.

Public Cloud

Owned and operated by a provider such as Microsoft Azure, AWS, or Google Cloud Platform, with infrastructure shared among multiple customers.

Characteristics: no hardware to maintain, rapid provisioning, pay-as-you-go pricing, high scalability, global availability.
Advantages: lower upfront costs, rapid deployment, automatic maintenance, high availability ideal for startups and growing businesses.
Limitations: less control, shared environment, potential compliance restrictions.
Best for: web applications, mobile backends, dev/test environments, AI/ML workloads, disaster recovery.

Private Cloud

Dedicated to a single organization, hosted either in-house or by a third-party provider, with no resource sharing across organizations.

Characteristics: dedicated infrastructure, greater administrative control, enhanced security, custom networking.
Advantages: better compliance support, increased data privacy, full infrastructure control, custom security policies.
Limitations: higher costs, requires skilled administrators, slower scaling.
Best for: government, healthcare, banking and finance, and other heavily regulated enterprises.

Hybrid Cloud

Combines public and private environments, letting workloads and data move between them as needed This keeps sensitive workloads private while using the public cloud for scale.

Characteristics: combines both models, flexible workload placement, supports gradual migration, optimizes cost and performance.
Advantages: flexibility, business continuity, disaster recovery, cost optimization, easier migration from on-premises.
Limitations: more complex architecture, requires strong networking and identity management, more governance overhead.
Best for: large enterprises, seasonal traffic spikes, backup/DR, organizations migrating to the cloud.

Quick Comparison

Feature	Public Cloud	Private Cloud	Hybrid Cloud
Ownership	Cloud provider	Single organization	Both
Cost	Low upfront	Higher	Moderate
Scalability	Excellent	Limited by hardware	Excellent
Security	Good	Very High	High
Maintenance	Provider	Organization	Shared
Flexibility	High	Moderate	Very High

Example: An e-commerce company might run its customer-facing website on the public cloud to handle traffic spikes during sales events, store payment records in a private cloud for compliance, and connect both through a hybrid architecture for secure data exchange balancing security, performance, and cost.

3. The Consumption-Based Pricing Model

Instead of purchasing expensive servers and networking equipment upfront, the cloud lets businesses pay only for what they use, an approach known as the consumption-based model, or pay-as-you-go pricing. Think of it like an electricity bill: you pay for the units you consume, not a fixed monthly amount.

Traditional Infrastructure vs. Cloud Consumption

Traditional Infrastructure	Consumption-Based Cloud
Large upfront hardware investment	No upfront infrastructure purchase
Pay even when servers are idle	Pay only for resources you use
Capacity planning required	Scale resources up or down instantly
Hardware maintenance is your responsibility	Cloud provider manages infrastructure

Key Benefits

No upfront costs : start using services immediately without buying hardware or licenses.
Pay only for what you use : compute hours, storage, database transactions, data transfer, and function executions are all billed based on actual consumption.
Scale when needed : resources grow during demand spikes and shrink afterward, an elasticity that's expensive to replicate on-premises.
Better cost optimization : usage-based billing encourages practices like shutting down unused VMs, right-sizing instances, using autoscaling, and monitoring spend regularly.

Consumption-Based vs. Subscription-Based Pricing

Consumption-Based	Subscription-Based
Pay for actual usage	Fixed monthly or yearly fee
Flexible costs	Predictable costs
Ideal for changing workloads	Best for consistent workloads
Easy to scale	Usually includes predefined resource limits

An e-commerce site running a consumption-based model keeps only a few virtual machines running on normal days, automatically adds instances during festival sales, and scales back down once traffic normalizes, paying only for the extra capacity while it's actually in use.

To avoid cost surprises, it's worth setting spending budgets, enabling billing alerts, monitoring usage, and reviewing bills regularly.

4. High Availability vs. Scalability

Two qualities determine whether users get a smooth experience in the cloud: high availability and scalability. They're often mentioned together, but they solve different problems.

High Availability

High availability (HA) is a system's ability to remain operational even when failures occur. Rather than preventing every failure, cloud platforms detect problems and automatically shift workloads to healthy resources. Providers achieve this through redundant infrastructure, multiple data centers and Availability Zones, automatic failover, and SLAs that define expected uptime.

Example: if a server crashes during an online banking transaction, traffic is redirected to a healthy instance so customers don't lose access to their accounts.

Scalability

Scalability is the ability to handle increasing or decreasing workloads by adjusting resources—a matter of capacity rather than uptime.

Vertical scaling (scale up): add more CPU, RAM, or faster storage to an existing machine—simple, but limited by hardware ceilings.
Horizontal scaling (scale out): add more instances and distribute traffic across them preferred for cloud-native applications, and often automated based on CPU, memory, or request volume.

High Availability vs. Scalability

High Availability	Scalability
Keeps services running during failures	Handles increasing workloads efficiently
Focuses on minimizing downtime	Focuses on increasing capacity
Uses redundancy and failover	Uses additional computing resources
Improves reliability	Improves performance under load

A movie ticket booking platform sees a traffic surge when tickets for a blockbuster release, the application automatically adds servers to handle demand (scalability), and if one server fails mid-rush, requests are redirected to healthy servers without interrupting users (high availability).

A well-designed cloud application aims for both.

5. Reliability and Predictability: Building Trust Beyond Uptime

Hosting an application in the cloud doesn't, by itself, guarantee a great experience. Users expect applications to stay available, recover quickly from failures, and perform consistently regardless of workload—expectations built on two more fundamental principles: reliability and predictability.

Understanding Reliability

Reliability is a system's ability to continue operating, or recover quickly, when failures occur. Hardware failures, software bugs, network outages, and even data center outages are inevitable in distributed systems, so rather than trying to eliminate every failure, cloud platforms are designed to detect, isolate, and recover from them automatically, distributing applications across multiple VMs, Availability Zones, or regions so that if one component goes down, another takes over with minimal disruption.

For example, if a virtual machine crashes unexpectedly, the platform can automatically provision a replacement and restore service without manual intervention.

Understanding Predictability

Predictability is about delivering consistent application behavior, performance, and costs over time, giving teams confidence that infrastructure will respond as expected under both normal and peak workloads. Providers support this through tools that monitor infrastructure health, analyze workload patterns, estimate future resource needs, and automatically adjust capacity. The result:

Stable application response times
Consistent system behavior
Predictable infrastructure costs
Better resource planning
Improved operational efficiency

Reliability vs. Predictability

Reliability	Predictability
Focuses on recovering from failures	Focuses on maintaining consistent performance and behavior
Ensures applications remain available	Ensures applications perform consistently over time
Achieved through redundancy, failover, and fault tolerance	Achieved through monitoring, autoscaling, analytics, and capacity planning
Measured by availability, resilience, and recovery	Measured by performance consistency, scalability, and cost stability

Reliability answers: "Will the application continue running if something fails?" Predictability answers: "Will the application continue performing as expected as demand changes?"<

The Benefits of Security and Governance in the Cloud

Fri, 14 Aug 2026 08:44:28 +0000

Cloud providers offer built-in security capabilities that help organizations protect their applications and infrastructure. Depending on the cloud service model you choose, the level of security management changes.

For example, with Infrastructure as a Service (IaaS), the cloud provider secures the physical infrastructure, while you are responsible for managing the operating system, software, and security updates. In Platform as a Service (PaaS) and Software as a Service (SaaS), the cloud provider handles more of these maintenance tasks, reducing the operational burden on customers.

Cloud providers also offer protection against common threats such as Distributed Denial of Service (DDoS) attacks, helping applications remain available even during large-scale attacks.

Cloud Governance

Cloud governance is the process of managing cloud resources so they comply with organizational policies and regulatory requirements. Cloud platforms provide built-in tools that make governance easier throughout the lifecycle of your resources.

Resource Templates

Templates allow organizations to deploy resources using predefined configurations. This ensures that every deployment follows the same technical standards and reduces the chances of configuration errors.

Compliance Auditing

Cloud platforms continuously monitor deployed resources to check whether they meet your organization's compliance requirements. If a resource doesn't comply with established policies, auditing tools can identify the issue and suggest remediation steps.

Automatic Updates

Many cloud services automatically apply software patches and platform updates. Keeping systems up to date improves both security and governance while reducing the administrative effort required to maintain infrastructure.

Why Governance Matters

Establishing governance early helps organizations maintain a secure, compliant, and well-managed cloud environment as it grows. By using templates, compliance checks, and automated updates, teams can ensure resources remain consistent and aligned with organizational standards.

Conclusion

Security and governance are foundational pillars of a well-run cloud environment. Cloud providers shoulder a significant portion of security responsibilities — from protecting physical infrastructure to defending against DDoS attacks — but the extent of that coverage depends on the service model you adopt (IaaS, PaaS, or SaaS).

Cloud governance tools such as resource templates, compliance auditing, and automatic updates give organizations the controls they need to enforce consistent policies, reduce human error, and stay aligned with regulatory requirements — all without heavy manual overhead.

The key takeaway is simple: start governance early. Organizations that establish clear policies and leverage built-in cloud tooling from day one are far better positioned to scale securely and confidently as their infrastructure grows.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

Understanding Cloud Manageability

Wed, 12 Aug 2026 08:44:28 +0000

When AWS launched its cloud computing model, where people can rent servers instead of owning them, it led to an increase in SaaS products.

But, similar to how Akamai solved the caching system, people wanted a similar way for their cloud model where they could build, deploy, and maintain applications on their own.

Valuable advantages of this cloud platform manageability. Similar to how we control the flame while cooking food, we manage how much flame should be used. If there is more flame, food gets roasted and gas gets wasted. But if we use very little flame, the food's taste may get spoiled based on how much we have added.

Similarly, through manageability, we can efficiently control, monitor, and automate cloud resources.

So, this automation is provided by many cloud platforms, which simplifies these operations and improves reliability.

Finally, cloud manageability is about more than just automation; it's about empowering users to manage their cloud resources effectively and efficiently.

Cloud manageability is viewed from two perspectives: management of the cloud and management in the cloud.

Management of the Cloud

Management of the cloud focuses on how cloud platforms help organizations manage their infrastructure and services more effectively. Modern cloud providers offer built-in capabilities that reduce operational complexity and increase system reliability.

Some key benefits include:

Automatic scaling: Resources can automatically increase or decrease based on application demand, ensuring optimal performance while avoiding unnecessary costs.
Template-based deployments: Infrastructure can be created using predefined templates, allowing teams to deploy consistent environments without manual configuration.
Health monitoring: Cloud services continuously monitor the health of resources and can automatically replace or recover failing components to maintain availability.
Real-time alerts: Administrators receive notifications when predefined performance metrics or thresholds are reached, enabling faster issue detection and response.

These features allow organizations to spend less time managing infrastructure and more time delivering value through their applications.

Management in the Cloud

Management in the cloud refers to the different ways users can interact with and control their cloud resources. Cloud platforms provide multiple management interfaces to suit different workflows and skill levels.

Common management options include:

Web Portal: A graphical interface that allows users to configure and monitor cloud resources through a browser.
Command-Line Interface (CLI): Enables developers and administrators to manage resources quickly using terminal commands.
Application Programming Interfaces (APIs): Allow applications and services to automate cloud operations programmatically.
PowerShell: Provides scripting capabilities for automating repetitive administrative tasks, particularly in Microsoft environments.

These management methods give teams the flexibility to choose the approach that best fits their operational requirements.

Conclusion

Manageability is one of the strongest advantages of cloud computing.

By combining intelligent resource management with flexible administration tools, cloud platforms enable organizations to automate routine tasks, improve reliability, respond quickly to issues, and maintain consistent deployments. Whether you're a developer, system administrator, or cloud engineer, understanding cloud manageability is an essential step toward building efficient and scalable cloud solutions.

Thanks for reading!

I'm Ganesh, and I'm building MakeSense, an AI tool that turns public GitHub pull requests into concise summaries, prioritized insights, and interactive quizzes. It's free, unlimited, and source-available. If you review open-source code, I'd love for you to give it a try and share your feedback.

Make Sense: https://makesensegithub.com/

DNS Records Explained: A, AAAA, CNAME, MX, TXT, NS, and SRV Records in Modern Internet Infrastructur

Mon, 10 Aug 2026 16:35:02 +0000

In the previous article, we explored how DNS translates human-friendly domain names into machine-friendly IP addresses and how a request travels through recursive resolvers, root servers, TLD servers, and authoritative nameservers before finally reaching a destination.

That journey answers how DNS works.

This article answers a different question:

What information is DNS actually returning?

When a DNS resolver contacts an authoritative nameserver, it isn’t simply asking for an IP address.

It is asking for a specific record.

Those records determine:

Where websites live
Where email should be delivered
Which servers are authoritative
How domains are verified
How email is authenticated
How cloud services are discovered
How SaaS platforms connect custom domains
How modern internet infrastructure operates

In other words:

DNS resolution is the process.

DNS records are the data.

Understanding DNS records is where DNS knowledge becomes practical.

Whether you’re launching a website, migrating infrastructure, configuring email, deploying applications, implementing custom domains, or troubleshooting production systems, DNS records sit at the center of the operation.

Why DNS Records Matter More Than Most People Realize

Many internet outages aren’t caused by application bugs.

They aren’t caused by database failures.

They aren’t caused by cloud providers.

They’re caused by configuration mistakes.

A misplaced DNS record can:

Take an entire website offline
Stop customer emails from arriving
Break API connectivity
Prevent SSL certificate issuance
Disrupt SaaS onboarding
Cause search engines to lose access to content

The irony is that DNS records are often configured once and forgotten until something breaks.

That’s why understanding them matters.

DNS Records: The Source of Truth Behind Every Domain

Imagine someone enters:

code

example.com

into a browser.

Eventually the request reaches the authoritative nameserver.

The authoritative nameserver responds with records stored inside its DNS zone.

Example:

code

example.com. IN A 203.0.113.10

The response tells the internet:

code

example.com
       ↓
203.0.113.10

That single record determines where traffic goes.

Without DNS records, DNS itself would be useless.

Understanding DNS Record Types

Modern DNS contains dozens of record types.

However, seven records power the overwhelming majority of websites, applications, email systems, and cloud services.

Record TypePrimary PurposeAMaps hostname to IPv4AAAAMaps hostname to IPv6CNAMECreates aliasesMXRoutes emailTXTVerification and securityNSDefines authoritative nameserversSRVService discovery

These records form the foundation of modern internet infrastructure.

A Record Explained

The A Record is the most fundamental DNS record.

The “A” stands for:

Address

It maps a hostname directly to an IPv4 address.

Example:

code

example.com. IN A 203.0.113.10

This creates a direct relationship:

code

example.com
       ↓
203.0.113.10

When someone visits the domain, traffic is sent to that server.

Why A Records Are So Important

Every website needs a destination.

The A Record provides that destination.

Without it:

Browsers cannot locate servers
Applications cannot connect
APIs become unreachable

Even the most advanced infrastructure eventually depends on records that resolve to an IP address.

Where A Records Are Commonly Used

Typical examples include:

code

company.com
portfolio.com
store.com
api.company.com
app.company.com

Almost every publicly accessible website uses A records somewhere in its architecture.

Production Reality: A Record Migrations

One of the most common infrastructure projects involves moving applications between servers.

For example:

code

Old Server
203.0.113.10

becomes:

code

New Server
203.0.113.50

Updating the A Record redirects traffic globally without requiring users to change anything.

This simple capability is one reason DNS remains such a powerful abstraction layer.

AAAA Record Explained

The AAAA Record performs the same role as an A Record.

The difference is the addressing standard.

RecordProtocolAIPv4AAAAIPv6

Example:

code

example.com. IN AAAA 2001:db8::1

Why IPv6 Exists

IPv4 provides approximately:

code

4.3 Billion Addresses

That seemed enormous in the early days of the internet.

Today, it isn’t.

Cloud computing, smartphones, IoT devices, smart vehicles, and connected infrastructure accelerated address consumption dramatically.

IPv6 was created to solve that limitation.

Should Every Website Use AAAA Records?

Not necessarily.

A common mistake is enabling IPv6 before infrastructure is ready.

If an application doesn’t properly support IPv6, users may experience connectivity problems despite having perfectly functioning IPv4 infrastructure.

IPv6 should be enabled deliberately rather than automatically.

CNAME Record Explained

CNAME stands for:

Canonical Name

Unlike an A Record, a CNAME does not point to an IP address.

Instead, it points to another hostname.

Example:

code

www.example.com IN CNAME example.com

Which effectively means:

code

www.example.com
         ↓
example.com
         ↓
203.0.113.10

Why CNAME Records Exist

Imagine managing:

code

www.example.com
blog.example.com
docs.example.com
shop.example.com

If each hostname pointed directly to an IP address, every infrastructure change would require updating multiple records.

CNAME records eliminate that complexity.

One change updates every dependent hostname.

Why Modern SaaS Platforms Love CNAME Records

Custom domains have become a standard expectation.

Instead of:

code

customer.platform.com

many organizations prefer:

code

app.customer-domain.com

The most common implementation relies on CNAME records.

This approach allows millions of customer domains to connect to shared infrastructure while maintaining branding and flexibility.

The Hidden Complexity of Custom Domains

What appears simple on the surface often requires:

Domain ownership verification
DNS validation
SSL certificate provisioning
Traffic routing
Edge network configuration

Behind a single CNAME record is often a surprisingly sophisticated platform architecture.

CNAME vs A Record

One of the most common questions is:

When should an A Record be used?

When pointing directly to an IP address.

Example:

code

example.com IN A 203.0.113.10

When should a CNAME be used?

When pointing to another hostname.

Example:

code

www.example.com IN CNAME example.com

Simple Rule

Use:

code

A Record

for servers.

Use:

code

CNAME

for aliases.

MX Record Explained

MX stands for:

Mail Exchange

MX records tell the internet where incoming email should be delivered.

Example:

code

example.com IN MX 10 mail.example.com

Without MX records:

Email cannot function.

Why MX Records Matter More Than Most Websites

A website outage is visible.

An email outage often isn’t.

Messages simply disappear.

Leads go unanswered.

Support requests never arrive.

Invoices are missed.

Customers assume silence means neglect.

The business impact can be substantial.

Understanding MX Priority

Example:

code

MX 10 mail-primary.example.com
MX 20 mail-backup.example.com

Lower values receive priority.

If the primary server becomes unavailable, mail automatically flows to the backup system.

This redundancy improves reliability.

TXT Record Explained

TXT records are among the most flexible records in DNS.

Historically they stored simple text.

Today they support critical infrastructure functions.

TXT Records Power Modern Trust Systems

Many internet services rely on TXT records to verify ownership.

Examples include:

Google Search Console
Microsoft 365
Cloud platforms
Email providers
SSL certificate services

Example:

code

google-site-verification=abc123

This proves domain ownership without requiring direct server access.

TXT Records and Email Security

TXT records are heavily used by:

SPF

Defines authorized email senders.

code

v=spf1 include:_spf.google.com ~all

DKIM

Provides cryptographic email signatures.

DMARC

Defines authentication policies.

Together these technologies help reduce:

Email spoofing
Phishing
Domain impersonation

In modern environments, email security is largely a DNS problem before it becomes an email problem.

NS Record Explained

NS stands for:

Name Server

NS records define which nameservers hold authority over a domain.

Example:

code

example.com IN NS ns1.cloudflare.com
example.com IN NS ns2.cloudflare.com

These records tell the internet:

These servers contain the source of truth.

Why NS Records Matter

Without NS records, the DNS hierarchy breaks.

Resolvers would have no way of discovering where authoritative information lives.

Every lookup ultimately depends on NS records.

They form the bridge between domain ownership and DNS management.

A Real Infrastructure Example

A domain may be registered through one provider.

DNS may be hosted elsewhere.

For example:

code

Registrar:
Namecheap

DNS Provider:
Cloudflare

NS records connect those two systems.

SRV Record Explained

SRV stands for:

Service Record

SRV records provide more information than a simple address.

They specify:

Service
Protocol
Port
Priority
Weight

Example:

code

_sip._tcp.example.com

Why SRV Records Exist

Many applications need more than an IP address.

Examples include:

SIP systems
VoIP platforms
Active Directory
Enterprise applications
Gaming infrastructure

SRV records allow applications to discover services automatically.

This reduces manual configuration and improves scalability.

DNS Records Every Modern Website Typically Uses

A standard production setup often includes:

Website Access

code

A Record
CNAME Record

Email Infrastructure

code

MX Record
SPF
DKIM
DMARC

Security and Verification

code

TXT Record

DNS Authority

code

NS Record

DNS Records During Website Migrations

Website migrations often fail because DNS planning is treated as an afterthought.

Common mistakes include:

Forgetting MX records
Removing TXT records
Incorrect TTL settings
Deleting verification records
Breaking email authentication

Successful migrations involve auditing DNS before moving infrastructure.

DNS Records and Search Visibility

DNS records do not directly improve rankings.

However, they influence systems that do.

Reliable DNS supports:

Website availability
Search engine accessibility
Email trust
User experience
Infrastructure reliability

Poor DNS management can undermine otherwise excellent websites.

Quick Reference Table

GoalRecord TypeHost a WebsiteAEnable IPv6AAAACreate an AliasCNAMEReceive EmailMXVerify Domain OwnershipTXTConfigure SPFTXTConfigure DKIMTXTConfigure DMARCTXTDelegate DNS AuthorityNSDiscover ServicesSRV

Key Takeaways

DNS records are the data layer of DNS.
A Records connect hostnames to IPv4 addresses.
AAAA Records connect hostnames to IPv6 addresses.
CNAME Records create aliases.
MX Records control email routing.
TXT Records power verification and security.
NS Records define authoritative nameservers.
SRV Records support service discovery.
Modern websites, cloud platforms, SaaS applications, and email systems depend on DNS records.
Understanding DNS records helps prevent outages, improve reliability, and simplify infrastructure management.

Secure VM Access Without Public IPs: A Beginner's Guide to Azure Bastion Service for Secure And Efficient VM Access

Thu, 16 Jul 2026 12:53:29 +0000

Introduction
Securely managing virtual machines is one of the major challenges of cloud engineering. Generally, administrators use SSH (for Linux VM) and RDP (for Windows) for accessing Azure VMs via public IP addresses. This is a good approach but exposing SSH (port 22) and RDP (port 3389) to the Internet increases the attack surface of your environment hence making it susceptible to brute force and other forms of attacks.

To overcome this problem, Azure Bastion is a Platform as a Service (PaaS) solution which provides secure SSH and RDP connection from Azure portal using HTTPS (Port 443). The use of Azure Bastion eliminates the need of assigning public IP addresses to virtual machines thus making them safer and easy to manage.

In this tutorial, you will be able to know how to implement Azure Bastion, set up the required networking configuration, connect to an Ubuntu virtual machine using SSH and understand Azure Bastion security benefits.

What is Azure Bastion?

Azure Bastion is a managed Azure service that provides secure browser-based SSH and RDP connectivity to Azure Virtual Machines without exposing them to the public internet.

Instead of connecting directly to your VM's public IP, you connect through the Azure Portal, and Azure Bastion establishes a secure session over Microsoft's private network.

Benefits of Azure Bastion
No Public IP required for Virtual Machines
Secure browser-based SSH and RDP access
No need to expose ports 22 or 3389
Reduced attack surface
Fully managed Platform as a Service (PaaS)
Uses HTTPS (Port 443)
Integrates seamlessly with Azure Virtual Networks

Project Objectives
In this project, you will:
Create an Azure Virtual Network
Create the AzureBastionSubnet
Deploy Azure Bastion
Create an Ubuntu Virtual Machine
Connect to the VM using Azure Bastion
Verify secure SSH connectivity
Understand Azure Bastion architecture and best practices

Prerequisites
Before beginning, ensure you have:
An active Microsoft Azure subscription
Access to the Azure Portal
Basic knowledge of Azure Virtual Machines
A web browser
Permissions to create Azure resources

Azure Bastion Architecture

Step 1: Sign in to Azure Portal
On your web browser, head over to https://portal.azure.com and log in with your Microsoft account. If you don't have an account, create or sign up and provide the required information so your Microsoft account can be activated.

A successful login bring you to this page below,

Step 2: Create a Resource Group
Search for Resource Groups on search bar and click on Resource group from the dropdown

-Click on Create

-Type the Resource group name, select the region and click Next

Apply tags to your resources to logically organize them by categories and click Next

Review the resources group and click on Create

-Click on Resource Group on the left to bring out the resource group you created

Step 3: Create a Virtual Network
1. .Click on Resource group created and right on the Resource Group page Search for Virtual Network on search bar and click on Virtual Network from the dropdown__

-Click on +Create(With a plus sign)

2. Define Basic Settings
Subscription: Select your subscription
Resource group: Choose an existing resource group or create a new one.
Region: Select the same region as your resource group
Virtual Network Name: Provide a name for the VNet.
Click on Next and Next under the Security

3. Define IP Address Space
Enter the IPV4 address 10.0.0.0/16 space for your virtual network. The image below shows that we can have 65,536 IP addresses within this network.

NOTE:Whenever you create your VNet there is a subnet that comes with it defaultly. We delete the subnet and create our own Subnet

-Click Next -> Review + Create -> Create

4.Click on Create, the deployment will be submitted, and the VNet will be created
Click on Go To Resources and you will see the the Created VNet

Step 4: Configure Subnets
Now, let's Create two subnets within the virtual network:
1 AzureBastionSubnet
2 Default Subnet( Create another subnet)

Type Subnet on the search bar and click on Subnet
Click +Subnet to define each subnet

Subnet Creation- Add a Subnet
AzureBastionSubnet
Azure Bastion requires a dedicated subnet named exactly:

Configure each subnet as follows:
Subnet purpose: Azure Bastion
Subnet Name: AzureBastionSubnet
Subnet Address range: Specify the subnet range within the virtual network address space (10.0.0.0/26).

Create another subnet(Default VM Subnet)

Step 5: Create an Ubuntu Virtual Machine
1.Type "Virtual Machines" in the search bar at the top. Once you see virtual machine display click on the virtual machine

When you click on "Create" button, it will display a drop down as shown below. You will then click on the Azure virtual machine highlighted in red arrow. This will take you to the project details where you will begin to fill the basic configuration for you to start creating your virtual machine.

2.Create virtual Machine under the BASIC CONFIGURATIONS by filling the following tabs as follows:

Subscription: Select your subscription(Azure Subscription 1)
Resource Group: Use an existing or create a resource group (FlackernProj-rg)
Virtual machine name: flackern-VM or use your desired username
Region: Select a nearby or preferred location ((Africa) South Africa North)
Availability options: Leave it as default (Availability Zone)
Availability Zone: Zone 1 (For this demo we select only one zone)
Security type: Standard
Image: Ubuntu Server 24.04 LTS - x64 Gen2
VM architecture: x64
Size: Standard_B2as_v2 - 2 vcpus, 8 GiB memory ($71.32)
Authentication type: SSH public key
Username: Use your desired username
SSH public key source: Generate new key pair
SSH Key Type: RSA SSH Format
Key pair name: flackern-vm_key or use your desired username
Public inbound ports: NONE
Select inbound ports: All traffic from the internet will be blocked by default.

3.Configure Disks

Click on Next : Disks >, this leads you to on the next page, you can use the default OS disk size, use the Image default (30 GiB) and for the OS disk type, use the default settings Premium SSD (locally-redundant storage) unless you have unique configuration requirements.

4.Networking Configuration
On the bottom of the last page in Step 3, Click on Next : Networking >, this leads you to the Network interface page below:
Create your virtual network and subnet or make use of the one you have created before
Virtual Network: flackernproj-Vnet (Vnet created before)
Subnet : Default Subnet (flackern-subnet)
Public IP: None
Public Inbound Port: None
Note: We are creating a private VM,so you select None for Public IP and Public Inbound Port.
**Remember: Public VM, Public IP

      **Private VM, No Public IP**

5.Management, monitoring, Advanced, Tags: These pages usually displayed as default. Find their default screenshots below. However, you may wish to skip them and proceed to “Review+Create”

Step 5. Generate SSH private Key
After a successful Linux VM deployment process, Azure will validate the configuration

Then Click "Review + create". This will show validation passed as highlighted below.

Then click “Create” . This will take you to the next page to generate new key pair. Then, you will click on “Download private key and create resource” as shown below. The new key pair will then be downloaded on your PC, while Azure will also provision your VM.

Download private key and create resource, afterwards, the deployment will start.

Wait for Deployment: Azure will now provision your virtual machine. This process may take a few minutes. Then the next page will show that your deployment is complete as shown below, then you will click on “Go to Resource”

Step 6: Deploy Azure Bastion
Type "Bastion" in the search bar at the top. Once you see Bastion display click on the Bastion

When you click on "Create" button,this will take you to the project details where you will begin to fill the basic configuration for you to start creating your Bastion

Advanced, Tags: These pages usually displayed as default. Find their default screenshots below. However, you may wish to skip them and proceed to “Review+Create”

Click on Create after validation is passed

Connect to VM using Bastion:
Once the VM is deployed, go to the VM in the Azure portal.
Click on the Connect button.
Select Bastion as the type of connection.
Enter the credentials (username and password or SSH key) and click Connect.

Provide your credentials to securely connect to the virtual machine using Bastion
Connection Settings:
Protocol: SSH
Port : 22
Authentication Type: SSH Private Key from Local File
Username: Use the username you created
Local File: Pick up the Key-pem you downloaded on your download,open the file on your azure portal and click on connect

Immediately you Click Connect, within a few seconds, Azure opens an SSH session directly in your browser.(We are connecting to our VM Securely)

Congratulations we have successfully SSH into our Virtual Machine securely using Bastion

Conclusion
Azure Bastion is one of the most effective ways to securely manage Azure Virtual Machines without exposing them to the public internet. By providing browser-based SSH and RDP access over HTTPS, it eliminates the need for public IP addresses and open management ports, significantly improving the security posture of your cloud environment.

In this step by step guide you created a Virtual Network, configured the required AzureBastionSubnet, deployed an Azure Bastion host, created an Ubuntu virtual machine, and successfully connected to it using a secure browser-based SSH session.

What is the shared responsibility model in cloud computing?

Tue, 14 Jul 2026 22:00:08 +0000

Hello, I'm Ganesh. I'm building git-lrc, an AI code reviewer that runs on every commit. It is free, unlimited, and source-available on Github. Star git-lrc on GitHub to help more developers discover the project. Do give it a try and share your feedback for improving the product.
One of the biggest misconceptions about cloud computing is that once you move to the cloud, the cloud provider takes care of everything.

While cloud providers manage a significant portion of the infrastructure, customers still have important security and operational responsibilities.

This concept is known as the Shared Responsibility Model.

The division of responsibilities changes depending on whether you're running workloads On-Premises, using Infrastructure as a Service (IaaS), Platform as a Service (PaaS), or Software as a Service (SaaS).

Let's break it down.

On-Premises: You Manage Everything

When your applications run in your own data center, every component is your responsibility.

Customer Responsibilities

Data
Devices
Accounts
Identity and Access Management (IAM)
Applications
Network controls
Operating systems
Infrastructure
Physical servers
Physical networking
Datacenter facilities

What the Provider Manages

Nothing.

Since everything is hosted on your own infrastructure, there is no cloud provider to share operational responsibilities.

Infrastructure as a Service (IaaS)

Examples include virtual machines such as Amazon EC2, Azure Virtual Machines, and Google Compute Engine.

With IaaS, the provider supplies the infrastructure while you manage everything running on top of it.

Customer Responsibilities

Data
Devices
Accounts
Identity and Access Management
Applications
Network controls (firewalls, security groups, etc.)
Operating systems

Cloud Provider Responsibilities

Physical infrastructure
Physical hosts
Physical networking
Datacenters

IaaS gives you the most flexibility among cloud service models, but it also requires the most operational management.

Platform as a Service (PaaS)

Examples include Azure App Service, AWS Elastic Beanstalk, Google App Engine, and Heroku.

PaaS removes the burden of managing operating systems and infrastructure, allowing developers to focus on building applications.

Customer Responsibilities

Data
Devices
Accounts

Shared Responsibilities

Identity and Access Management
Applications
Network controls

Cloud Provider Responsibilities

Operating systems
Infrastructure
Physical hosts
Physical networking
Datacenters

PaaS strikes a balance between flexibility and convenience, making it an excellent choice for many modern application deployments.

Software as a Service (SaaS)

Examples include Microsoft 365, Salesforce, Google Workspace, and Dropbox.

With SaaS, the provider delivers a complete application that users simply access through a web browser or client application.

Customer Responsibilities

Data
Devices
Accounts

Shared Responsibilities

Identity and Access Management

Cloud Provider Responsibilities

Applications
Network controls
Operating systems
Infrastructure
Physical hosts
Physical networking
Datacenters

SaaS offers the lowest operational overhead because the provider manages nearly the entire technology stack.

Shared Responsibility at a Glance

Service Model	Customer Manages	Shared	Provider Manages
On-Premises	Everything	None	Nothing
IaaS	OS, applications, IAM, network controls, data	None	Infrastructure & physical hardware
PaaS	Data, devices, accounts	IAM, applications, network controls	OS, infrastructure & physical hardware
SaaS	Data, devices, accounts	IAM	Applications, OS, infrastructure & physical hardware

Conclusion

No matter which cloud service model you choose, your data, devices, and user accounts remain your responsibility.

As you move from On-Premises → IaaS → PaaS → SaaS, the cloud provider takes on progressively more responsibility, reducing your operational burden. However, protecting your users, managing access, and securing your data always remain critical tasks.

Understanding where your responsibilities begin—and where the provider's responsibilities end—is essential for building secure, reliable cloud solutions.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

Understanding Cloud Computing Models: Public, Private, and Hybrid Cloud

Tue, 14 Jul 2026 06:42:54 +0000

Cloud computing has transformed how businesses build, deploy, and scale applications.

Instead of investing heavily in physical infrastructure, organizations can access computing resources on demand, paying only for what they use.

But not every cloud environment is the same.

Choosing between Public Cloud, Private Cloud, and Hybrid Cloud depends on factors such as security, compliance, performance, scalability, and cost.

Let's explore each cloud model and understand when to use them.

What Are Cloud Deployment Models?

A cloud deployment model defines where your infrastructure is hosted, who owns it, and who can access it.

The three primary deployment models are:

Public Cloud
Private Cloud
Hybrid Cloud

Each model offers different levels of flexibility, security, and operational control.

Public Cloud

A public cloud is owned and operated by a cloud provider.

Infrastructure such as servers, storage, networking, and databases are shared among multiple customers.

Popular public cloud providers include Microsoft Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP).

Key Characteristics

No hardware to purchase or maintain
Resources can be provisioned in minutes
Pay-as-you-go pricing
Highly scalable
Global availability

Advantages

Lower upfront costs
Rapid deployment
Automatic infrastructure maintenance
High availability
Ideal for startups and growing businesses

Limitations

Less control over infrastructure
Shared environment
Some organizations may have compliance restrictions

Best Use Cases

Web applications
Mobile backends
Development and testing
AI and machine learning workloads
Disaster recovery

Private Cloud

A private cloud is dedicated to a single organization.

The infrastructure can be hosted in the organization's own data center or managed by a third-party provider.

Unlike public cloud, resources are not shared with other organizations.

Key Characteristics

Dedicated infrastructure
Greater administrative control
Enhanced security
Custom networking and configurations

Advantages

Better compliance support
Increased data privacy
Full control over infrastructure
Custom security policies

Limitations

Higher infrastructure costs
Requires skilled administrators
Scaling can take longer

Best Use Cases

Government organizations
Healthcare systems
Banking and finance
Enterprises with strict regulatory requirements

Hybrid Cloud

A hybrid cloud combines public and private cloud environments, allowing applications and data to move between them when needed.

Organizations can keep sensitive workloads in a private cloud while using the public cloud for scalability.

Key Characteristics

Combines both deployment models
Flexible workload placement
Supports gradual cloud migration
Optimizes cost and performance

Advantages

Increased flexibility
Better business continuity
Improved disaster recovery
Cost optimization
Easier migration from on-premises systems

Limitations

More complex architecture
Requires strong networking and identity management
More monitoring and governance

Best Use Cases

Large enterprises
Seasonal traffic spikes
Backup and disaster recovery
Organizations migrating to the cloud

Quick Comparison

Feature	Public Cloud	Private Cloud	Hybrid Cloud
Ownership	Cloud provider	Single organization	Both
Cost	Low upfront	Higher	Moderate
Scalability	Excellent	Limited by hardware	Excellent
Security	Good	Very High	High
Maintenance	Provider	Organization	Shared
Flexibility	High	Moderate	Very High

Which Cloud Model Should You Choose?

There isn't a single "best" deployment model.

Choose Public Cloud if you want:

Fast deployment
Lower costs
High scalability

Choose Private Cloud if you need:

Maximum security
Regulatory compliance
Complete infrastructure control

Choose Hybrid Cloud if you want:

The best of both worlds
Flexible workload placement
A gradual path to cloud adoption

Real World Example of Cloud Computing Models

Imagine an e-commerce company.

Customer-facing website runs on the Public Cloud to handle millions of visitors during sales events.
Customer payment records are stored in a Private Cloud to meet compliance requirements.
Both environments work together through a Hybrid Cloud architecture, allowing secure data exchange while maintaining scalability.

This approach balances security, performance, and cost.

Conclusion

Cloud deployment models are not competitors—they solve different business challenges.

Public cloud focuses on speed and scalability.

Private cloud emphasizes security and control.

Hybrid cloud combines both, giving organizations the flexibility to place workloads where they make the most sense.

Understanding these models helps businesses make informed decisions as they modernize their infrastructure and prepare for future growth.

As cloud technologies continue to evolve, selecting the right deployment model becomes an important step toward building secure, scalable, and resilient applications.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

The day YouTube deleted our channel

Fri, 10 Jul 2026 22:53:46 +0000

It took a single notice. The YouTube channel that held years of a newsroom's video was gone in one action, and every video embed across the site was suddenly pointing at something that no longer existed. On a publication that runs video on most of its articles, that is the kind of morning that empties an archive.

It was a non-event. We changed one setting, the whole site began playing the same videos from our own storage, and readers never saw a gap. This is the story of why a deleted channel was a configuration change rather than a catastrophe, and the one decision, made years earlier, that made the difference.

Platforms can delete you, and you do not get a vote

Channels and accounts get removed for all sorts of reasons, and most of them are not a fair fight: an automated copyright match against music in the background, a strike from a report, a policy you did not know had changed, a regional block, an API quietly deprecated, or an entire product shut down (the internet is littered with services that were permanent right up until they were not).

The common thread is that the decision is theirs, the timing is theirs, and the content you put on their platform was never fully in your hands. For a newsroom, video on a third-party platform is best treated as reach, not as custody.

Publish to platforms, never store on them

The decision that turned the incident into a shrug was made on the very first upload: every video goes to our own S3 bucket first. The original lives with us. Pushing a copy out to YouTube, Dailymotion or Vimeo is a distribution step, for reach and free bandwidth, not the place the file actually lives.

It is the same discipline behind the rest of our media handling: the original is offloaded to storage we control on upload, and everything downstream is a copy or a rendition of it. So when a platform pulls a video, or an entire channel, nothing is actually lost. The master is still sitting in the bucket exactly where it was put.

One video, four sources

On top of that, each video is a single logical asset with up to four play sources linked behind it: YouTube, Dailymotion, Vimeo, and Amazon (the file served from our own S3 through CloudFront).

The important word is source. Where a video plays is a stored setting, not an embed baked into the post. That one distinction is the whole game. If the player reads a source field, you can change where every video comes from without editing a single article. If instead every post contains a literal YouTube iframe, changing the source later is a database migration under pressure, which is exactly when you do not want one.

The original lives in S3 and is published out to YouTube, Dailymotion and Vimeo. Where a video actually plays is a stored setting, a site-wide default with a per-video override, so any source can be the active one.

Two dials: site-wide and per video

The source has two levels of control.

Site-wide, you set the default that every video plays from. We ran on YouTube, because that is where the audience and the bandwidth were.

Per video, you can override that default, and in a newsroom that is not a rare event. Long videos that carry clips or background music from other sources get flagged by YouTube's automated Content ID constantly, often even when you hold the rights, and clearing a strike means filing an appeal and waiting days. So those videos are set, right there on the upload screen, to play from Dailymotion, Vimeo or S3 instead, while the rest of the library stays on YouTube. The content most likely to be struck simply sidesteps the platform that polices it hardest, with no downtime and no appeal queue.

The site-wide default sets the source for every video at once. A per-video override pins a single clip to another source: the long documentary full of licensed music is set to Vimeo on the upload screen, so it sidesteps the platform most likely to strike it, while everything else stays on the default.

The day the whole channel went

Then the version of the problem you cannot patch one video at a time: the entire channel was deleted at once. Every video that defaulted to YouTube would fail in the player, all at the same moment, across the whole archive.

Because the play source was a single site-wide setting, and because every original was already sitting in S3, the fix was to change the default from YouTube to Amazon and save. Every embed on the site re-pointed to the S3 copies, served through CloudFront, in one action. The outage was as long as it takes to change a dropdown.

Compare that to the alternative if YouTube had been the storage rather than a distribution target: the archive would simply be gone, and the "recovery" would be re-sourcing and re-uploading thousands of videos that no longer existed.

When the channel was deleted, every video that defaulted to YouTube failed at once. Because the play source is a single site-wide setting and the originals were already in S3, changing the default to Amazon restored the entire archive in one click.

The fallback is not a downgrade

It is worth being clear that playing from your own infrastructure is not a degraded emergency mode. The S3 source plays through CloudFront like any other CDN-backed asset. When you want adaptive streaming at scale, you put it in front: an upload event triggers AWS Elemental MediaConvert to produce HLS renditions, CloudFront caches the segments, and a standard player streams quality that adapts to each viewer's connection.

In other words, falling back to your own storage is not making do. For many sites it is the best source they have. The platforms were the convenience, not the foundation.

What it is really about: custody

Strip away the specifics and the principle is simple. Own the original, and treat every external platform as interchangeable reach you opt into, never as the home of your media. In practice that is three habits:

Keep the master somewhere you control (for us, S3).
Make playback a setting, not a hard-coded embed, so the source is a dial.
Publish to platforms for reach, but never let one become your only copy.

Do that, and platform risk (a strike, a deleted channel, a shut-down service, a pricing change you cannot stomach) stops being an existential event and becomes a switch you flip. The same thinking applies anywhere you syndicate: images to a CDN, articles to third-party readers, audio to podcast directories. Keep the master, keep the destination swappable.

A two-minute gut check for your own setup

Three questions worth answering honestly:

If your video platform closed your account tonight, do the originals exist anywhere else? If the only copy is on the platform, that is the first thing to fix, before anything clever.
Is playback a per-post embed, or a source you can change centrally? If it is baked into content, switching later is a migration, not a setting.
Do you publish to platforms, or store on them? Publishing is healthy. Storing is the trap.

If the answers make you uncomfortable, you already know where to start.

The bottom line

We keep newsrooms fast and unbreakable on WordPress and AWS, and this is the resilience side of the same coin as keeping the media pipeline lean: the originals never leave your control, so neither a platform nor a traffic spike can take the archive down.

If your video lives on someone else's platform and losing it would hurt, let's talk.

Reqrea’s 1 Million Passport Exposure Is a Reminder Why Cloud Security Audits Matter

Mon, 29 Jun 2026 19:49:17 +0000

Incident

On May 15, 2026, Reqrea, a Japan-based KYC company, reported that more than 1 million passports had been exposed to the public internet via a misconfigured S3 bucket.

This breach impacted over 1 million travelers from around the world. Exposed passports could potentially put all impacted individuals at risk of identity theft or sophisticated social engineering attacks in the future.

This recent incident further confirms my point in the past that AWS misconfigurations remain one of the top cybersecurity vulnerabilities in 2026.

How it was identified

The vulnerability was discovered by security researcher Anurag Sen, who detected that one of the S3 buckets used by the company was accessible to the public without authentication or an authorization process.

Why it matters

KYC platforms and companies handling PII are expected to have strong change control, security governance and regular security audits to prevent such costly mistakes from happening. In this instance, since Reqrea has collected such data, they may face regulatory, contractual, and reputational exposure.

Unfortunately, such cases could potentially harm companies' reputation among customers and partners, which could hurt future sales.

>[!NOTE]
>The average data breach costs U.S. organizations about USD 10.22 million, according to IBM’s 2025 Cost of a Data Breach Report.

Lesson Learned

Companies handling PII need to adopt "Secure by Design" policy and have strong cloud governance if they want to use cloud solutions.

A good preventive strategy would entail companies bringing in third parties (e.g. consultants) before going live or before getting involved in any activity involving protected data collection or transmission of such data.

Another lesson learned is that delegating KYC to third parties may actually reduce security if the KYC platform itself is insecure.

I can easily point out the fix needed for this vulnerability. However, we have to look at this incident or vulnerability rather as a question of governance and policies first and technical as second.

The Dirty Secret of Cloud Server Scaling!

Sun, 21 Jun 2026 19:22:50 +0000

The True Path to Cloud Scalability: Beyond the Horizontal Band-Aid

Introduction

In the rapidly evolving landscape of cloud computing, the mantra of “scaling horizontally” by simply adding more server instances has long been preached as the ultimate solution for handling increased load. While seemingly effective, this approach often masks a deeper, more pervasive issue: it’s frequently a costly band-aid, not a robust, efficient design. As we look towards 2026, the truly scalable backends are not those over-provisioned with redundant compute, but rather those meticulously crafted with intelligent database management intertwined with a serverless-first backend design. This tutorial will expose the “dirty secret” of cloud scaling and guide you towards building architectures that are not just bigger, but fundamentally smarter, more cost-effective, and performant.

Architectural Layout and Strategy Walkthrough

Moving beyond simplistic horizontal scaling requires a paradigm shift in how we conceive and construct our cloud backends. The “code” for true scalability lies less in boilerplate server logic and more in a strategic architectural layout that leverages cloud-native capabilities.

1. Embrace True Serverless Databases for Dynamic Scalability

The first critical step is to re-evaluate your database strategy. Traditional relational databases, even when hosted in the cloud, often require pre-provisioned capacity that leads to either over-spending or performance bottlenecks.

Conceptual Layout: Instead of defining fixed-size database instances or complex replication clusters you manage, envision a database layer that inherently scales with demand. For example, AWS Aurora Serverless v3 (or similar offerings like Azure Cosmos DB Serverless or Google Cloud Firestore) exemplifies this.
- You define your database schema and initial configuration.
- The database automatically scales compute and memory based on actual load, often down to zero capacity during idle periods.
- Your application’s database connection string remains static, abstracting the underlying scaling complexity.
Strategy: Configure your database to use auto-scaling policies, focusing on metrics like CPU utilization or active connections, but allowing the serverless engine to manage the details. This eliminates the need for manual scaling events and ensures you pay only for the resources consumed.

2. Implement Smart Sharding Based on Actual Access Patterns

Simply distributing data across multiple nodes (sharding) isn’t enough. The intelligence comes from understanding how your data is accessed and partitioning it accordingly to minimize hot spots and optimize query performance.

Conceptual Layout: Consider a microservices architecture where data ownership is clear. Each microservice might interact with its own shard or a specific sharding strategy.
- User Data Service: Shard by user_id to ensure all user-specific data (profiles, preferences) resides on a single shard, optimizing reads and writes for individual users.
- Time-Series Data Service: Shard by date or month to allow efficient querying of data within specific timeframes, often archived to cheaper storage after a period.
- Routing Logic: Implement a lightweight routing layer (e.g., an API Gateway and Lambda function, or a dedicated data access microservice) that intelligently directs queries to the correct shard based on the request’s key or content. This layer abstracts the sharding logic from the core application.
Strategy: Begin with thorough data access pattern analysis. Identify your most frequent queries, the data they touch, and potential hot keys. Use this analysis to design your sharding key and distribution strategy, ensuring even load distribution and minimizing cross-shard queries where possible.

3. Embrace Event-Driven Architectures for Offloading Synchronous Load

Synchronous processing often creates bottlenecks and increases latency. Adopting an event-driven architecture (EDA) allows you to offload non-critical or time-consuming tasks, improving frontend responsiveness and backend resilience.

Conceptual Layout:
- An API Gateway receives an inbound request.
- A lightweight Serverless Function (e.g., AWS Lambda, Azure Function) performs minimal validation and then publishes an event to a Message Queue (e.g., AWS SQS, Kafka, Azure Service Bus). It immediately returns a “202 Accepted” response to the client.
- Another Serverless Function or a dedicated worker processes the event from the queue asynchronously, performing heavy lifting like image processing, report generation, or complex business logic.
- Results are then stored in a database, S3, or pushed back to the client via WebSockets or another event.
Strategy: Identify parts of your application that don’t require immediate feedback to the user. Decouple these processes using message queues and serverless functions. This not only improves user experience by reducing latency but also enhances system resilience, as messages can be retried and processed even if the consumer temporarily fails.

Conclusion

The “dirty secret” of cloud server scaling isn’t about the inability to scale, but rather the industry’s default to inefficient, reactive over-provisioning. True scalability in 2026 demands a shift towards intelligent design: leveraging dynamically scaling serverless databases, implementing smart sharding strategies informed by real-world access patterns, and embracing event-driven architectures to decouple and optimize load. This proactive approach to anticipating bottlenecks and optimizing P99 latencies is where the real engineering magic happens. By moving beyond the horizontal band-aid, you build not just a bigger backend, but a smarter, more resilient, and significantly more cost-effective one—a true differentiator in today’s competitive digital landscape.

The Core Ideas Behind Cloud Native Development

Fri, 19 Jun 2026 17:41:28 +0000

Building applications for the cloud isn't just about running code on someone else's servers. It's about fundamentally rethinking how we design, build, and operate software. Cloud native applications are built for the cloud from the ground up, not adapted to it afterward. This matters because it changes everything about scalability, resilience, and how fast you can deploy.

What Cloud Native Actually Means
Cloud native means your application is designed to exploit cloud infrastructure. It's not a monolithic block that runs on one server. Instead, it's a collection of loosely coupled services that can scale independently, fail gracefully, and deploy without touching other parts of the system.

Think of it like the difference between a shipping container and a truck full of individually packaged items. With cloud native, you're thinking in containers. Each service is self-contained, portable, and can be replaced or updated without stopping the whole operation.

The Core Architecture: Microservices
Instead of one massive application, you build many small services. Each one does one thing. Your user service handles authentication. Your payment service handles transactions. Your notification service sends emails. They communicate through APIs or message queues.

The advantage is obvious: if your payment service needs scaling during holiday traffic, you scale just that service. If your notification service has a bug, it doesn't take down the entire application. Different teams can work on different services independently. Deployment cycles speed up because you're not redeploying the whole system for a single change.

Containerization: Consistency Across Environments
Every service runs in a container. A container packages your code, its dependencies, runtime, and configuration into one portable unit. Docker made this practical. You build a container image once, and it runs identically on your laptop, your testing environment, and production servers.

This solves the "works on my machine" problem. No more debugging configuration differences between environments. The container is your contract: whatever runs in your container will run the same way everywhere it's deployed.

Orchestration: Managing the Chaos
With dozens or hundreds of containers running, manually managing them is impossible. Kubernetes and similar orchestration platforms handle this. They decide which servers containers run on, handle failures by restarting crashed containers, manage networking between containers, and scale up or down based on demand.

Kubernetes is complex, but it solves hard problems. It lets you declare "I want three instances of this service" and automatically maintains that, replacing failed instances. It handles load balancing, rolling updates, and configuration management.

DevOps: Breaking Down the Silos
Cloud native requires developers and operations to work together. DevOps practices like continuous integration and continuous deployment (CI/CD) are essential. Code changes are automatically tested and deployed without manual intervention. This means faster feedback, fewer bugs making it to production, and the ability to deploy multiple times a day if needed.

Resilience and Scalability Built In
Cloud native applications expect failure. A container dies, but orchestration replaces it. A service is slow, so you add more instances. Traffic spikes, and the system automatically scales. You're not trying to prevent every failure, you're designing to recover from failures gracefully.

Scalability is automatic. If you get traffic, metrics trigger orchestration to spawn more containers. If traffic drops, excess containers shut down. You pay for what you use.

The Trade-offs
Distributed systems are harder than monoliths. Debugging is trickier. You need monitoring and logging across services. Operations becomes more complex. Your team needs to understand containerization, orchestration, and cloud infrastructure.

But the benefits are real. Faster development cycles. Independent scaling. Fault isolation. The ability to update one service without touching others. For most companies building applications that will grow and change, cloud native is the right foundation.

The cloud native ecosystem is mature now. The tools exist. The patterns are proven. The question is whether your organization is ready for the shift in thinking it requires.

Running Ollama Locally vs in the Cloud: A Practical Look at AI Infrastructure

Sat, 13 Jun 2026 19:35:17 +0000

Understanding the local AI experience

Running a machine learning model locally is one of the most interesting ways to understand what artificial intelligence really demands from infrastructure. When we use cloud-based AI platforms, most of the complexity stays hidden behind an API, a web interface, or a managed service. The response appears quickly, the model feels powerful, and the infrastructure seems almost invisible. But when we install something like Ollama on a personal machine and run a model directly from a notebook, the experience becomes much more concrete.

Ollama makes this local experience accessible because it allows developers and curious professionals to run open models on their own machines, including on Windows, macOS, Linux, and Docker environments. It also exposes a local API, which makes it possible to connect models to scripts, applications, and experiments without depending directly on an external AI provider for every request.

In my own test, I ran an Ollama machine learning model locally on a personal notebook with Windows 11, 12 GB of DDR4 memory, and a 12th generation Intel Core i3 processor running at 1.20 GHz. The result was very interesting from a learning perspective. The model worked, the interaction was possible, and it gave me a clear feeling of independence. At the same time, the model was noticeably slow compared with cloud-based models. That difference is not a failure of local AI. It is actually the main lesson.

Why local execution feels different from cloud execution

When a model runs locally, it depends entirely on the resources available on that machine. Memory, CPU, storage speed, GPU support, thermal limitations, and even background processes influence the experience. In a personal notebook with 12 GB of RAM and a Core i3 processor, the system can run smaller or optimized models, but it does not have the same computational power as a cloud environment equipped with modern GPUs and large amounts of high-bandwidth memory.

This is why the local model may answer slowly, especially when the notebook does not have a dedicated GPU capable of accelerating inference. Ollama supports GPU acceleration across different platforms and hardware configurations, but when acceleration is limited or unavailable, the workload can fall back heavily on the CPU. That makes the experience functional, but slower.

This slower performance is important because it reveals the hidden cost of AI. Every answer generated by a model requires computation. Every token has a price in processing time, memory usage, and energy consumption. In the cloud, this cost still exists, but it is absorbed by powerful infrastructure designed specifically for that workload. Locally, the same cost becomes visible in the form of waiting time, fan noise, heat, and system resource consumption.

The cloud as an environment built for performance

Cloud environments are different because they are designed to provide elastic computing power. Instead of depending on a single notebook, companies can use virtual machines, GPU instances, managed endpoints, autoscaling policies, monitoring tools, storage services, and security controls. For machine learning and generative AI, this difference is enormous.

AWS, for example, recommends GPU instances for most deep learning workloads because training and heavy model execution are faster on GPUs than on CPUs. Its documentation also highlights that model size should influence the choice of instance, especially when memory requirements exceed the available resources of a given machine.

For inference, the cloud also allows teams to think beyond a single user. A model running on a personal notebook may be enough for experimentation, but a business application may need to serve dozens, hundreds, or thousands of requests. In that scenario, the discussion changes from “can the model run?” to “can it run reliably, securely, and fast enough for real users?” AWS describes this as a process of right-sizing infrastructure and configuring autoscaling based on inference demand.

Azure Machine Learning follows a similar idea with endpoints for inference. A model can be exposed through a stable URL, protected by authentication and authorization, and backed by specific compute resources. This is closer to what companies need when they want to place AI inside a real application, workflow, or business process.

Local AI is not only about speed

Even though the cloud is usually faster, local AI has advantages that should not be ignored. Running a model locally gives the user more control. It can reduce dependency on external services, support offline experimentation, and help protect sensitive data during early tests. For developers, students, engineers, and technology professionals, this kind of environment is extremely valuable because it teaches the practical side of AI infrastructure.

A local Ollama setup is also useful for prototyping. Before investing in cloud resources, a professional can test prompts, evaluate model behavior, build small automations, experiment with retrieval-augmented generation, or understand how different models respond. The local environment becomes a laboratory. It is not always the fastest place to run AI, but it can be one of the best places to learn how AI really works.

This was exactly the value of running the model on a personal Windows 11 notebook. The experience was not about achieving enterprise-grade performance. It was about seeing the model operate inside a limited machine and understanding the relationship between hardware and AI performance. The fact that it ran slowly was not just a technical limitation. It was a practical lesson about why cloud infrastructure exists.

Why companies still need the cloud

For companies, speed is only one part of the decision. Cloud environments offer availability, scalability, centralized governance, access control, observability, integration with existing systems, and professional support. These elements are difficult to reproduce on personal machines or isolated local servers.

A company cannot usually depend on one notebook or one internal machine to serve an AI application that supports customers, employees, or production workflows. It needs infrastructure that can survive failures, handle traffic peaks, protect data, and integrate with identity systems, logging platforms, monitoring dashboards, and compliance requirements.

This is where the cloud becomes more than just “a faster computer.” It becomes an operational platform. Companies can deploy models as services, monitor latency, scale resources, rotate credentials, manage costs, and integrate AI into business applications. In other words, the cloud turns experimentation into production.

Will local AI become possible for companies in the future?

The future will probably not be purely local or purely cloud. It will be hybrid.

Local AI will become more powerful as personal computers, workstations, edge devices, and specialized chips improve. We are already seeing more interest in smaller, optimized, quantized, and domain-specific models. These models do not always need massive infrastructure to be useful. For certain business cases, local execution may become very attractive, especially when data privacy, low latency, offline operation, or cost control are more important than maximum model size.

However, this does not mean cloud AI will disappear. Large-scale models, high concurrency, enterprise integrations, and advanced workloads will still benefit from cloud infrastructure. The most realistic future is one where companies use local models for specific tasks and cloud models for workloads that require more power, scale, or centralized management.

For example, a company could run a local model inside a factory to analyze internal documents without sending data outside the network. At the same time, it could use cloud-based AI for customer service, large-scale analytics, model fine-tuning, or applications that require high availability. This hybrid approach gives companies flexibility instead of forcing them to choose only one side.

The practical comparison

The difference between running Ollama locally and running models in the cloud can be summarized through experience. Locally, the model feels closer, more private, and more educational. It gives the user control and helps build real understanding. But it is limited by the machine. On my personal notebook, the model can be interesting and useful for experimentation, but it will naturally feel slow when compared with cloud-based models running on optimized infrastructure.

In the cloud, the model feels faster, more scalable, and more production-ready. The user does not need to worry as much about local memory limits, CPU pressure, or hardware acceleration. But this comes with other responsibilities, such as cost management, security configuration, vendor dependency, and operational design.

This comparison shows that local and cloud environments are not enemies. They solve different problems. Local AI is excellent for learning, prototyping, privacy-focused experiments, and small automations. Cloud AI is better for production systems, enterprise workloads, large models, and applications that need consistent performance.

Challenges and opportunities

The main challenge for local AI is hardware. Many personal notebooks were not designed for heavy AI workloads. They can run models, but performance depends heavily on available memory, CPU power, GPU acceleration, and the size of the selected model. This means users need to choose models carefully and understand that not every model will behave well on every machine.

The main challenge for cloud AI is cost and governance. It is easy to scale resources, but it is also easy to create expensive architectures if teams do not monitor usage and right-size their deployments. Companies need technical maturity to use cloud AI responsibly.

The opportunity is that both worlds are becoming easier to access. Tools like Ollama make local AI more approachable. Cloud platforms make production deployment more structured. Developers who understand both environments will have a strong advantage because they will know when to experiment locally, when to scale in the cloud, and when to combine both approaches.

Final thoughts

Running an Ollama model on my personal notebook was a valuable experience because it made the infrastructure behind AI visible. The model worked, but it was slow compared with cloud-based models. That slowness was not simply a problem. It was a reminder that artificial intelligence depends on real hardware, real memory, real processing power, and real architectural decisions.

For companies, local AI will become increasingly possible, especially for focused, private, and lightweight use cases. But for large-scale production workloads, the cloud will remain essential. The future will likely belong to organizations that know how to balance both sides: using local AI where control and privacy matter, and using cloud AI where performance, scalability, and reliability are required.

The most important lesson is that AI is not only about choosing a model. It is also about choosing the right environment to run that model. And sometimes, the best way to understand that is not by reading documentation, but by installing the model, running it on your own machine, and seeing how the system behaves in practice.

From Dockerfile to Docker Hub: A Complete Beginner's Guide to Docker

Thu, 11 Jun 2026 10:31:28 +0000

Containerization has become a fundamental skill for DevOps Engineers, Cloud Engineers, and Software Developers.

However, many beginners learn Docker commands without understanding the complete workflow behind them.

Questions like:

What exactly is a Docker Image?
How does a Dockerfile become a running Container?
Why do we push images to Docker Hub?
How do organizations deploy the same application everywhere?

are common when starting out.

In this blog, we'll build a simple Node.js application, containerize it using Docker, run it locally, and finally push the image to Docker Hub.

By the end of this guide, you'll understand the complete Docker workflow used in real-world DevOps environments.

Docker Workflow Overview

flowchart LR
    A[Application Code] --> B[Dockerfile]
    B --> C[Docker Build]
    C --> D[Docker Image]
    D --> E[Docker Run]
    E --> F[Docker Container]
    D --> G[Docker Push]
    G --> H[Docker Hub]

This is the same workflow followed by developers and DevOps teams worldwide.

What Are We Building?

We'll create a simple Node.js application that returns:

A welcome message
Environment information
Container hostname

This helps us verify that our application is actually running inside a Docker container.

Project Structure

.
├── Dockerfile
├── app.js
└── package.json

Step 1: Create the Node.js Application

app.js

const express = require("express");
const app = express();

const PORT = process.env.PORT || 3000;
const ENV = process.env.ENV_VALUE || "No env set";
const HOSTNAME = process.env.HOSTNAME || require("os").hostname();

app.get("/", (req, res) => {
  res.json({
    message: "Hello from Simple App (Node)",
    env: ENV,
    container: HOSTNAME
  });
});

app.listen(PORT, () =>
  console.log(`Node Hello listening on ${PORT}`)
);

package.json

{
  "name": "hello-app-node",
  "version": "1.0.0",
  "main": "app.js",
  "dependencies": {
    "express": "^5.2.1"
  }
}

Application Architecture

flowchart LR
    User --> Browser
    Browser --> Container
    Container --> NodeJS
    NodeJS --> JSONResponse

Whenever a user accesses the application, the request reaches the containerized Node.js application, which returns a JSON response.

Step 2: Create the Dockerfile

A Dockerfile is simply a blueprint that tells Docker how to build an image.

FROM node:24-slim

WORKDIR /app

COPY package*.json ./

RUN npm install --production

COPY . .

EXPOSE 3000

CMD ["node", "app.js"]

Understanding the Dockerfile

Base Image

FROM node:24-slim

We start with an official lightweight Node.js image from Docker Hub.

Working Directory

WORKDIR /app

Creates and switches to the /app directory inside the container.

Copy Dependency Files

COPY package*.json ./

Copies package files first.

This is considered a Docker best practice because Docker can cache dependency layers and speed up future builds.

Install Dependencies

RUN npm install --production

Installs only production dependencies.

Copy Application Source Code

COPY . .

Copies the entire project into the container.

Expose Application Port

EXPOSE 3000

Documents that the application listens on port 3000.

Start the Application

CMD ["node", "app.js"]

Runs the Node.js application when the container starts.

What Happens During Docker Build?

flowchart TD
    A[Dockerfile]
    --> B[Pull Node Base Image]
    --> C[Create Working Directory]
    --> D[Copy package.json]
    --> E[Install Dependencies]
    --> F[Copy Application Code]
    --> G[Create Image Layers]
    --> H[Docker Image Ready]

Docker executes each instruction sequentially and creates image layers.

Step 3: Build the Docker Image

Run:

docker build -t madhavnaks/node-app .

Let's understand this command:

docker build

Builds an image from a Dockerfile.

-t

Assigns a tag to the image.

madhavnaks/node-app

Image name.

Current directory containing the Dockerfile.

After a successful build you'll see output similar to:

=> naming to docker.io/madhavnaks/node-app

Docker has now created an image locally on your machine.

Step 4: Verify the Image

To see locally available images:

docker images

Example output:

REPOSITORY                TAG       IMAGE ID       SIZE
madhavnaks/node-app       latest    42c936686000   234MB

At this point, the image exists but is not running.

Docker Image vs Docker Container

Many beginners confuse these two concepts.

Think of it like this:

Docker Image = Blueprint
Docker Container = Running Application

flowchart LR
    A[Docker Image]
    --> B[Container 1]

    A --> C[Container 2]

    A --> D[Container 3]

A single image can create multiple containers.

Step 5: Run a Container

Now let's start our application.

docker run -d --name node-app -p 3001:3000 madhavnaks/node-app

Understanding the Command

Detached Mode

-d

Runs the container in the background.

Container Name

--name node-app

Assigns a friendly name to the container.

Port Mapping

-p 3001:3000

Maps:

Host Port      → Container Port
3001           → 3000

The application listens on port 3000 inside the container but is accessible on port 3001 from your machine.

Step 6: Verify the Container

Check running containers:

docker ps

Expected output:

CONTAINER ID   IMAGE                 STATUS
10265698c978   madhavnaks/node-app   Up

Step 7: Access the Application

Open:

http://localhost:3001

Expected response:

{
  "message": "Hello from Simple App (Node)",
  "env": "No env set",
  "container": "10265698c978"
}

Notice the hostname.

The hostname corresponds to the container ID, proving the application is running inside Docker.

Before pushing images, authenticate with Docker Hub.

docker login

Example:

Login Succeeded

Docker is now authorized to push images to your repository.

Why Push Images to Docker Hub?

Docker Hub acts as a centralized registry.

flowchart LR
    A[Developer Machine]
    -->|Push| B[Docker Hub]

    B -->|Pull| C[Server]

    B -->|Pull| D[Kubernetes Cluster]

Once an image is pushed, it can be pulled and run from anywhere.

Step 9: Push the Image

Push the image to Docker Hub:

docker push madhavnaks/node-app

Example output:

latest: digest: sha256:d1081ec542f521e100bd886943c13ab7568c663b56e36200e6d846163975f979

This confirms that the image has been successfully uploaded.

Step 10: Verify on Docker Hub

Visit your Docker Hub account.

You should now see:

madhavnaks/node-app

listed under repositories.

You can even share the image with others.

Anyone can now run:

docker pull madhavnaks/node-app

docker run -p 3001:3000 madhavnaks/node-app

without needing your source code.

Real-World DevOps Workflow

In production environments, the workflow usually looks like this:

flowchart LR
    A[Developer Writes Code]
    --> B[GitHub]

    B --> C[CI/CD Pipeline]

    C --> D[Docker Build]

    D --> E[Docker Hub]

    E --> F[Kubernetes Cluster]

This is exactly how modern DevOps teams package and deploy applications.

Key Takeaways

✅ Dockerfile defines how an image should be built

✅ Docker Build creates an image

✅ Docker Run creates a container

✅ Docker Hub stores and distributes images

✅ One image can create multiple containers

✅ Containerization ensures consistency across environments

Conclusion

In this guide, we covered the complete Docker workflow from source code to Docker Hub.

We learned how to:

Create a Dockerfile
Build a Docker image
Run a container
Verify application execution
Push images to Docker Hub
Understand how images move through real-world DevOps pipelines

Mastering these Docker fundamentals is the first step toward advanced topics like Docker Compose, Kubernetes, Helm, CI/CD pipelines, and cloud-native deployments.

If you're learning DevOps, try containerizing applications in different languages such as Python, Java, Go, or .NET to strengthen your understanding of Docker.

Happy Containerizing!

Terraform CI/CD with Google Cloud: Plan on Pull Request and Apply with Approval

Thu, 28 May 2026 07:18:18 +0000

Simple Terraform codes on laptop is alright for learning. But, at some point, things gotten more complex and infrastructure changes need a more controlled workflow.

For my second Terraform x Google Cloud portfolio artifact, I had already built a production like web platform with:

VPC
app and DB subnets
Cloud NAT
private backend VMs
regional Managed Instance Group
HTTP/HTTPS Load Balancer
Cloud Armor security policy
remote Terraform state
modular Terraform structure

In this version, I added the next operational layer:

Terraform CI/CD

The goal was simple:

Pull Request -> Terraform Plan
Manual Approval -> Terraform Apply
No service account JSON key

This became:

v2.0 — Terraform CI/CD with GitHub Actions and Workload Identity Federation

Checkout my github! terraform-gcp-production-lite-web-platform

Why I Built This

Before this version, the workflow was still done locally:

local terminal
  -> terraform fmt
  -> terraform validate
  -> terraform plan
  -> terraform apply

That works for my learning, But i understand that in a team setting this way of doing things locally is not sustainable.

Thus, making me questions:

Can the change be reviewed before apply?
Can the Terraform plan run automatically on pull request?
Can apply require approval?
Can GitHub Actions authenticate to Google Cloud without a static key?
Can the workflow use remote state safely?

v2.0 was built to answer those questions.

What v2.0 Adds

This release adds:

GitHub Actions
Terraform plan on pull request
Terraform apply through manual workflow
GitHub environment approval before apply
Workload Identity Federation
Google service account impersonation
No service account JSON key
Remote state access from CI/CD
GitHub CLI-based release workflow

The important part is not just automation.

The important part is change control.

Final CI/CD Flow

The new workflow looks like this:

Pull Request
  -> terraform fmt -check
  -> terraform init
  -> terraform validate
  -> terraform plan
  -> upload plan artifact
  -> comment plan summary on PR

Manual Apply
  -> workflow_dispatch
  -> confirm input: APPLY
  -> GitHub environment approval
  -> terraform plan -out=tfplan
  -> terraform apply tfplan

This separates review from execution.

The PR workflow answers:

What will Terraform change?

The apply workflow answers:

Are we approved to apply this change?

Authentication Design

The most important security decision in this version:

No service account JSON key.

Instead of storing a long-lived Google Cloud key in GitHub Secrets, the workflow uses Workload Identity Federation.

The authentication flow is:

GitHub Actions OIDC token
  -> Google Workload Identity Provider
  -> Terraform CI/CD service account impersonation
  -> Google Cloud APIs

The GitHub repository is restricted through the Workload Identity Provider attribute condition:

assertion.repository == "abrahamparn/terraform-gcp-production-lite-web-platform"

That means the provider is only intended to trust this specific repository.

Why Workload Identity Federation Matters

A service account JSON key is a long-lived credential.

If it leaks, the impact can be serious.

Workload Identity Federation avoids that by allowing GitHub Actions to exchange its OIDC identity for short-lived Google Cloud access.

For this project, GitHub Actions impersonates a dedicated service account:

terraform-cicd@PROJECT_ID.iam.gserviceaccount.com

This gives the CI/CD pipeline a clear operational identity.

Workflow 1 — Terraform Plan on Pull Request

The first workflow runs on pull requests to main.

File:

.github/workflows/terraform-plan.yml

The workflow performs:

checkout repository
set up Terraform
authenticate to Google Cloud using WIF
set up gcloud
terraform fmt -check -recursive
terraform init
terraform validate
terraform plan
generate plain-text plan
upload plan artifact
comment plan summary on PR

In the screenshot, the Terraform Plan workflow successfully completed all steps:

Checkout repository
Set up Terraform
Authenticate to Google Cloud
Set up gcloud
Terraform format check
Terraform init
Terraform validate
Terraform plan
Generate plain-text plan
Upload Terraform plan artifact
Comment plan summary on PR

This is the main review workflow.

It allows infrastructure changes to be reviewed before they are applied.

Workflow 2 — Terraform Apply with Manual Approval

The second workflow is manually triggered.

File:

.github/workflows/terraform-apply.yml

The workflow uses:

workflow_dispatch
confirm_apply input
GitHub environment: terraform-apply
manual approval
terraform plan before apply
terraform apply using saved plan file

The apply workflow has two jobs:

Terraform Plan Before Apply
Terraform Apply

In the screenshot, the apply workflow succeeded after environment approval.

The deployment protection section shows:

Environment: terraform-apply
Approval: approved
Comment: go on

That approval gate is the main safety control before applying infrastructure changes.

Why I Still Run Plan Before Apply

One question I had while building this was:

Should the apply workflow reuse the PR plan artifact?

For this version, I decided not to.

Instead, the apply workflow creates a fresh plan immediately before apply.

Why?

Because PR plan artifacts can become stale.

Between PR review and manual apply:

state may change
main branch may change
plan artifact may expire
another workflow may run

So the safer v2.0 pattern is:

PR plan = review signal
Apply workflow plan = final execution plan

Then the apply step uses:

terraform apply tfplan

This keeps apply tied to the plan generated inside the apply workflow.

GitHub Environment Approval

The apply workflow uses this environment:

terraform-apply

This allows GitHub deployment protection rules to act as the approval gate.

The result is:

No automatic apply from pull request.
No apply just because code was pushed.
Apply only happens after manual workflow trigger and approval.

Repository Variables

I used GitHub repository variables for non-secret configuration:

GCP_PROJECT_ID
GCP_PROJECT_NUMBER
GCP_REGION
GCP_WORKLOAD_IDENTITY_PROVIDER
GCP_SERVICE_ACCOUNT
TF_WORKING_DIR
TF_VERSION

These values are not service account keys.

No Google Cloud JSON key is stored in GitHub.

Environment tfvars Strategy

One practical issue with Terraform CI/CD is variable management.

My project uses a lot of variables such as:

subnets
firewall_rules
service_accounts
Cloud Armor rules
managed SSL certificate domains

Passing all of that through TF_VAR_* environment variables would become messy.

So for this portfolio project, I used:

environments/dev.tfvars

This file contains non-sensitive environment configuration.

The workflow runs:

terraform plan -var-file=environments/dev.tfvars

and:

terraform plan -var-file=environments/dev.tfvars -out=tfplan

Important rule:

Only non-sensitive values should be committed.

Secrets should not be placed in dev.tfvars.

GitHub CLI Operations

I also tried to minimize GitHub UI usage.

For example, releases can be created using:

gh release create v2.0.0 \
  --title "v2.0.0 — Terraform CI/CD with GitHub Actions and WIF" \
  --notes "This release adds Terraform plan on pull request, manual approval before apply, and no key Google Cloud authentication through WIF."

The pull request can also be created using CLI:

gh pr create \
  --base main \
  --head feature/v2.0-terraform-cicd-wif \
  --title "add Terraform CI/CD with GitHub Actions and WIF" \
  --body "This PR introduces Terraform CI/CD with GitHub Actions, plan on pull request, manual approval before apply, and Workload Identity Federation with no SA JSON key."

And the manual apply workflow can be triggered with:

gh workflow run "Terraform Apply" \
  -f confirm_apply=APPLY \
  -f git_ref=main

The only part where UI is still useful is the environment approval step, because that is the point of having a manual deployment gate.

Evidence From the Workflow Runs

For this version, I captured two main screenshots.

Terraform Plan Workflow

The Terraform Plan workflow succeeded on pull request.

It completed:

authentication to Google Cloud
terraform fmt check
terraform init
terraform validate
terraform plan
plan artifact upload
PR plan comment

This proves the review workflow works.

Terraform Apply Workflow

The Terraform Apply workflow also succeeded.

It was manually triggered from main, required environment approval, then executed:

Terraform Plan Before Apply
Terraform Apply

The deployment protection section shows that the terraform-apply environment was approved before apply continued.

This proves the execution workflow works.

One Warning I Noticed

The workflow showed a warning related to Node.js 20 actions being deprecated.

This did not break the workflow.

The run still succeeded.

What This Version Does Not Solve Yet

v2.0 is intentionally focused.

It does not yet include:

policy-as-code
cost estimation
drift detection automation
custom least-privilege Terraform IAM role
multi-environment promotion
automatic rollback
scheduled plan

Those are future improvements.

For this version, the objective was:

reviewable plan
manual approved apply
keyless authentication

Final Architecture After v2.0

The infrastructure platform now has two layers:

Runtime platform

HTTPS Load Balancer
Cloud Armor
Backend Service
Regional MIG
Private backend VMs
Cloud NAT

Delivery platform

GitHub Pull Request
Terraform Plan workflow
GitHub Environment Approval
Terraform Apply workflow
Workload Identity Federation
No service account JSON key

That is the main improvement.

The project moved from:

I can provision infrastructure.

to:

I can manage infrastructure changes through a controlled delivery workflow.

Version Timeline

v1.0 — Production-Lite HTTP Platform
v1.1 — HTTPS and Custom Domain
v1.2 — Security Hardening with Cloud Armor
v2.0 — Terraform CI/CD with GitHub Actions and WIF

Next, I may continue with:

v2.1 — Drift Detection and Recovery

Because after CI/CD, the next important Terraform question is:

What happens when someone changes infrastructure outside Terraform?

5 Months Later: What Actually Happened (SRE/DevOps Self-Taught)

Wed, 20 May 2026 10:52:16 +0000

5 Months Later: What Actually Happened (SRE/DevOps Self-Taught)

Five months ago I wrote a post called "My First Steps in a Tough World". I was one month into this journey, I had just built a basic monitoring script, and I was equal parts excited and terrified.

I want to tell you what happened next. Not the polished version. The real one.

Where I was then

When I wrote that first post, my biggest achievement was a Python script that checked CPU and memory usage and sent a notification. I called it a "production-grade monitoring system" because I genuinely didn't know what production-grade meant yet.

I had WSL2, some basic Linux commands, and a lot of determination. That was it.

What the last 5 months actually looked like

I'm not going to pretend it was a clean upward line. There were weeks where I felt like I was drowning in concepts I didn't understand. There were moments where I questioned whether any of this was going anywhere.

But I kept building. And somewhere in between the confusion and the commits, things started to click.

Here's what I actually shipped:

Project 1: SRE Expense Tracker — from local to production on AWS

This started as a FastAPI + PostgreSQL application running on my local machine with Docker Compose. By the end, it was a production deployment on AWS EC2 with:

CI/CD pipeline via GitHub Actions + AWS SSM (no open SSH ports)
Database migrations with Alembic, applied cleanly in production
Automated S3 backups of PostgreSQL via a cron job inside the container
CloudWatch log streaming using the Docker awslogs driver
Prometheus metrics exposed and scraped
API key authentication on all endpoints
Elastic IP, IAM roles with least privilege, security groups locked down

I hit real production problems. At one point my FastAPI request-level logs weren't appearing in CloudWatch even though container startup logs were coming through fine — a uvicorn stdout routing issue I had to dig into and debug against a live system.

That kind of problem doesn't exist in tutorials. You only find it when something is actually deployed and you're staring at empty log streams at midnight wondering what's wrong.

Project 2: DevSecOps Security Pipeline — security as an independent layer

After getting the Expense Tracker into production, I realized I had real credentials in a live environment. A Docker image deployed on AWS. Python dependencies with specific versions. Code that could have insecure patterns. And no systematic way to know about any of it.

So I built a separate repository that implements a continuous security pipeline targeting the Expense Tracker as its subject.

The pipeline runs four independent layers in sequence:

Gitleaks — scans the full Git history for exposed secrets
Bandit — static analysis of the Python source code for insecure patterns
Trivy — scans the Docker image on Docker Hub for CVEs
pip-audit — scans declared Python dependencies for known vulnerabilities

Each job blocks the next. If Gitleaks finds a secret, nothing else runs.

The pipeline runs on push, on pull requests, and on a nightly schedule at 2AM UTC. That last part matters more than it sounds. Push triggers only catch what you broke. The nightly schedule catches new CVEs published in vulnerability databases even when your code hasn't changed. Security is proactive, not just reactive.

It found real things.

Trivy detected three CVEs in pip 25.0.1 inside the Docker image — with available fixes. I patched it. pip-audit found HIGH and MEDIUM severity CVEs in setuptools and pytest. I patched those too. Seven HIGH CVEs in the Debian base image had no available fix at the time — I documented them with a formal risk acceptance document covering attack surface analysis and review criteria. Then the nightly schedule detected newly published fixes for some of them weeks later. I patched within hours.

Total infrastructure cost: €0. GitHub Actions ephemeral runners on a public repo.

The architectural decision I'm most proud of: I kept the security pipeline in a separate repository from the application. It would have been simpler to put it inside the Expense Tracker. But that would make security look like a feature of the application. It isn't. It's an independent operational concern. Making that separation explicit was a deliberate engineering decision, and I documented why in an Architecture Decision Record.

Open Source Contributions: two merged PRs

This is the part that still feels surreal.

I submitted a pull request to bluewave-labs/Checkmate, an open source monitoring platform. The fix addressed a Docker non-root container permission issue. The maintainers gave feedback, I iterated, all CI checks passed. It's merged.

Then I submitted a PR to henrygd/beszel — a server monitoring hub with 22,000 stars on GitHub.

The issue: the hub image is distroless. No shell, no wget, no curl. Standard Docker healthcheck patterns fail silently because there's nothing to run them with. But the binary already exposes a health subcommand that performs an HTTP self-check. I added a HEALTHCHECK directive to the Docker Compose files using the binary itself:

healthcheck:
  test: ["/beszel", "health", "--url", "http://127.0.0.1:8090"]
  interval: 30s
  timeout: 10s
  retries: 3

Without this, containers running the distroless hub show as "running" indefinitely even when wedged — no orchestrator liveness signal, no automatic restart trigger. I also noticed the same file was included in a second Compose configuration and added the healthcheck there too, without being asked.

It was merged today.

AWS Certified Cloud Practitioner

I passed the AWS CCP exam. It unlocks a 50% discount voucher for the Solutions Architect Associate, which I'm currently studying for. Stephane Maarek's course and Tutorials Dojo practice exams if you're on the same path.

The certification matters less than what it represents in terms of learning sequence. You can't meaningfully study IAM, VPCs, security groups, and EC2 instance profiles in the abstract. Doing it while having a real application deployed in AWS makes every concept land differently.

What I actually learned

Not a skills list. The real things.

Production is a different planet. Everything works in development. Production has IAM roles that are slightly wrong, log drivers that need specific permissions, database connections that time out in ways that never happened locally. The only way to learn production is to deploy something to production and watch it break.

Documentation is engineering. I write Architecture Decision Records for every significant choice I make. Not because someone told me to — because six weeks later I genuinely couldn't remember why I made a decision, and having the reasoning written down is the difference between understanding your own system and being confused by it.

Security is not a checklist. It's an automated layer that runs independently, finds real problems, and forces you to make documented decisions about risk. The moment you have something deployed with real credentials, this stops being theoretical.

Open source is learnable. I was intimidated by the idea of contributing to real projects. The first PR felt impossible. The second one I found by using a tool, hitting a problem, tracing it to the root cause, and fixing it. That's just debugging. It's the same skill.

Where I'm going

AWS Solutions Architect Associate is next. Then more open source contributions. The Expense Tracker deployment still has open problems I want to solve. And somewhere in the longer arc, MLOps — which is where I wanted to be from the beginning.

Five months ago I was scared and excited in roughly equal measure. Today I'm still both. But the fear is smaller and the excitement is backed by things I actually built and shipped.

That's the only honest way I know how to measure progress.

I'm Iriome Santana, a self-taught SRE/DevOps learner based in Gran Canaria, Spain. You can find my projects on GitHub and connect with me on LinkedIn.

️ Getting Started with AWS: A Practical Guide from a Certified Cloud Practitioner

Mon, 18 May 2026 10:03:56 +0000

As a certified AWS Certified Cloud Practitioner, one thing I’ve learned is this: cloud computing isn’t just for experts—it’s for anyone willing to understand the fundamentals and start building.

If you’re new to AWS or preparing for the Cloud Practitioner certification, this guide will walk you through the essentials, with practical context to help you actually use what you learn.

What is AWS and Why Does It Matter?

Amazon Web Services (AWS) is a cloud platform that provides on-demand computing services like storage, databases, networking, and more—without needing to own physical infrastructure.

Instead of buying servers, you rent them.

Instead of guessing capacity, you scale on demand.

Learn more: What is AWS

Core Concepts Every Cloud Practitioner Should Know

**
If you're preparing for the Cloud Practitioner (CCP) exam, these are non-negotiable:

Global Infrastructure
**

AWS operates through:

Regions (e.g., us-east-1)
Availability Zones (AZs) (isolated data centers within regions)

This design ensures:

High availability
Fault tolerance

AWS Global Infrastructure

Compute: Amazon EC2
**

Amazon EC2 lets you launch virtual servers in minutes.

Use cases:

Hosting web apps
Running backend services
Testing environments

Amazon EC2

Storage: Amazon S3
**

Amazon S3 is one of the most widely used AWS services.

Key features:

Highly durable (99.999999999%)
Scalable storage
Ideal for static websites, backups, and media

Amazon S3

Pricing Model
**

AWS follows a pay-as-you-go model:

Pay only for what you use
No upfront costs
Scale up/down anytime

There’s also a Free Tier to help beginners experiment safely.

Shared Responsibility Model
**

One of the most important CCP concepts:

AWS is responsible for security of the cloud
You are responsible for security in the cloud

Example:

AWS secures data centers
You secure your applications and data

Shared Responsibility Model

A Simple Project to Get You Started

If you're learning AWS, don’t just read—build.

Host a Static Website with S3

Steps:

Create an S3 bucket
Upload your HTML/CSS files
Enable static website hosting
Make files publicly accessible

That’s it—you’ve deployed your first cloud app.

Check the steps here

How the Cloud Practitioner Certification Helps

The AWS Certified Cloud Practitioner certification validates your understanding of:

Cloud concepts
AWS services
Security and compliance
Billing and pricing

It’s not about deep technical skills—it’s about confidence and clarity.

Certification-Certified Cloud Practitioner

My Learning Approach

Here’s what worked for me:

Combining theory + hands-on labs
Using the AWS Free Tier consistently
Documenting what I learn (like this post)
Building small, real projects

Cloud isn’t something you memorize—it’s something you practice.

Why This Matters

As I continue learning and building with AWS, I’m also focused on:

Sharing beginner-friendly resources
Breaking down complex concepts
Encouraging more students to explore cloud computing

Final Thoughts

If you’re starting your AWS journey:

Start small. Stay consistent. Build often.

The cloud can feel overwhelming—but once you understand the basics, everything starts connecting.

Let’s Connect

If you're also learning AWS or preparing for the Cloud Practitioner exam, I’d love to connect and share ideas.

The OrbitFlare SDK: One Rust Crate Solana Transport

Thu, 14 May 2026 05:55:09 +0000

The OrbitFlare SDK: One Rust Crate Solana Transport

Introduction
Developer Challenges
How to Solve Them
Conclusion

1. Introduction

Software development is full of real-world challenges, especially when building systems that rely on reliability and correctness. Even small mistakes can break execution flow or cause unexpected behavior.

Note: Most developer challenges actually improve debugging skills and system thinking over time.

Why does this matter?

Because writing code is easy — but writing safe and predictable systems is where real engineering starts.

2. Developer Challenges

a. Division by Zero

A very common runtime issue:

int total = 50;
int number = 0;
int result = total / number;

This throws ArithmeticException.

b. Null Object Access

String username = null;
System.out.println(username.length());

Leads to NullPointerException.

c. Invalid Array Index

int[] marks = {70, 80, 90};
System.out.println(marks[5]);

This triggers ArrayIndexOutOfBoundsException.

Caution: Unhandled errors like these can crash apps and break user experience instantly.

3. How to Solve Them

a. Safe Division

try {
   int result = total / number;
} catch (ArithmeticException e) {
   System.out.println("Division by zero not allowed");
}

b. Null Checks

if(username != null) {
   System.out.println(username.length());
}

c. Array Validation

if(index < marks.length) {
   System.out.println(marks[index]);
}

4. Conclusion

In systems like OrbitFlare SDK or any real-world transport layer, these small mistakes can become critical failures.

So the key is simple:

validate inputs
handle exceptions
write defensive code

The more you deal with these issues, the more stable your systems become.

Happy coding

7 Signs Your Release Process Is More Fragile Than You Think

Mon, 11 May 2026 06:37:01 +0000

Most of the release processes look good. Pipelines are green, deploys happen all the time, and nothing is on fire. When something breaks, you only see the cracks, and suddenly you’re trying to answer questions that your tooling doesn’t help with.

DORA metrics are speed and stability. But they don't tell you if rollback works, if approvals live outside of Slack, or if the artifact you tested is the one in production. You can hit all four DORA metrics and still be one bad deploy away from a bad day.

Here are seven signs your process is more fragile than you realize. Individually they seem small. But they don't hold up well under pressure together.

1. "Who deployed this?" takes more than a minute to answer

This is asked almost as soon as a serious incident occurs. If you have to go through CI logs, scroll through Slack, or hope someone remembers to answer your questions, your release process has no operational memory.

The data is out there somewhere, but it’s not structured as a release record; it’s scattered across pipeline runs, commit messages, conversations. Running a CI does not capture the actual deployment intent, and so engineers open multiple systems during incident calls to reconstruct the course of events.

2. The rollback lives in a doc, not in the system

A healthy, well-defined rollback scenario is of utmost importance. Most teams talk about having a rollback plan, but hardly anyone’s ever put one to the test during a real incident.

Here’s what really happens: you scramble to find the old tag, kick off a pipeline, cross your fingers that the environment hasn’t changed, and try to remember all those finicky parameters. Every single step is a chance for things to break. As a post-mortem on approval-gated releases said it directly: “rollback instructions existed in documents, but not in executable policy.”

So you end up staring into that gap, right when pressure’s at its worst. And here is where you start to improvise chaotically.

3. Deployments come down to whoever’s got access and enough urgency to use it

Production access until there is an explicit process usually comes down to who can deploy and needs to do it quickly. That feels like efficiency on a normal day. In a risky change it gets murky: was this checked? this the correct artifact? who is responsible for the go decision?

If you’ve got repo access with deploy permissions, merging code is often synonymous with deploying it. That’s not really a decision, that’s just the way things happen when CI is doing the deployments.

From what I can tell this only happens when something has gone wrong and suddenly no one knows exactly who owns the release.

4. Your deployment metrics are good because you’re measuring the wrong thing

Pass/fail is the wrong indicator here. A release that makes it through on the third try after two reruns is not the same as one that went through cleanly the first time.

The rerun ratio is the most overlooked stat by most teams. Successful releases with high rerun rates are symptoms of brittle steps, flaky dependencies, and environment inconsistencies (even when the terminal status is green). One of the most underdiagnosed CI/CD challenges is environment inconsistency, as it doesn’t show up in pass/fail rates until a regression hits production.

If your deployment telemetry does not distinguish between first-run success and ultimate success, you are missing a reliability signal that is important.

5. Staging and production run different artifacts

If you rebuild per environment instead of promoting the same artifact, there’s no guarantee that what passed QA is what ends up in production. Dependencies can resolve differently, configs can drift, and suddenly you’re debugging a bug you’ve never seen in staging.

The fix is simple in theory: build once, promote the same artifact everywhere. But many CI setups rebuild by default, because that’s how the pipeline is wired.

Most of the time, no one notices until a regression makes it obvious.

6. A CI platform outage also freezes your ability to deploy a hotfix

If deployments run through your CI, a platform issue can take out both builds and releases at the same time. The moment you most need to ship a fix is the moment you can’t.

I can say that this isn’t rare. Looking at a set of failed or delayed releases, the root cause was often not bad code but a fragile setup: wrong variables in CI, missing approvals, and unclear artifact versions.

When your release layer depends entirely on your build layer, their availability becomes the same.

7. Deployment knowledge concentrates on one or two people

What I mean is that the team's ability to ship depends on specific individuals being present and available.

You’ll agree that this is the least dramatic sign. Yet I’m convinced it’s the most damaging long-term. Teams usually notice it when the "deployment person" takes a holiday and suddenly nobody is confident releasing anything. Or when someone leaves and takes the full process knowledge with them.

Deployment governance isn't just an operational concern. It's a scaling and team health concern.

Why do these signs appear together?

None of these is a crisis on its own. That’s why they tend to stick around for so long.

Together, they point to a release process optimized for the easy case: normal days, routine deploys, no incidents. The root cause is the same across all seven. CI and CD are different responsibilities. One is about validating code, the other is about governing how it moves. But most setups try to handle both with a single system designed for only one of them.

If you've gone deeper on the CI vs. CD distinction, the pattern becomes clear quickly. The coupling feels efficient early and becomes fragile at scale. Build maintainers end up carrying production risk decisions they don't own. Deployment logic accumulates inside pipelines built for something else.

As one measurement framework puts it: standardization doesn't remove complexity. It makes complexity measurable. And once it's so, you can improve it deliberately instead of reacting to isolated incidents.

How many did you recognize?

At first, you’ll most probably notice one or two. That’s nothing unusual; every team goes through this before major issues show up. But when you start seeing four or five, here’s where cracks are about to show. Things still run, but it’s clear the process is bogged down and getting heavier than it should be. Hit six or seven, and it’s trouble. The next incident is probably going to be costly for your team. Not just from any bug, but because all that extra process is slowing everything to a crawl. The faster you notice this pattern, the better.

What to do about it

The good news is that you don’t have to dismantle your pipeline to fix any of this.

The main change here is splitting up the purpose of CI and the purpose of releasing. CI’s whole job is to build, test, and create an artifact. And it does that just fine. Everything after that (what version goes where, who approved it, and what happens if it fails) is a whole other responsibility that needs its own structure.

So, when you put this in practice, you end up with four key things:

build the artifact once, then push it to each environment instead of rebuilding at every stage
make approvals a real, tracked step, not just a message in Slack
test your rollback process before you face an actual incident
keep your release history somewhere safe, so it doesn’t disappear on your CI failure

You don't actually need a large platform. This thin layer between build and release, which owns the release decision, captures the audit trail and handles rollback as policy rather than improvisation, removing most of the ambiguity without much overhead.

I’ve seen this pattern enough times to know the structure problem only reveals itself under pressure. Everything is fine until it isn't. And when something breaks at a bad time, the question is not whether you can fix the bug, but whether you can understand what happened quickly enough to do something about it.

Map Your Entire AWS Infrastructure in 60 Seconds

Fri, 08 May 2026 20:51:01 +0000

If you've ever inherited an AWS account, you know the feeling.

You open the console, click around a few services, and realise pretty quickly that you have no idea what's actually running. EC2 instances nobody recognises.

Security groups with ports open that shouldn't be. S3 buckets from three projects ago. RDS instances that might still be in use — or might not.

It’s nobody’s fault. AWS sprawl happens when a team moves quickly over time.

The problem is, most of the solutions people reach for don't actually solve it.

The usual approaches (and why they fall short)

Spreadsheets. Someone on the team volunteers to document everything
manually. It takes a week, and by the time it's done, it's already out of date.
AWS moves faster than any spreadsheet.

⚙️ AWS Config. Powerful, but complex to set up, expensive at scale, and it gives you raw data — not insight. You still have to do all the interpretation yourself.

️ Clicking through the console. This is what most people actually do.
Service by service, region by region. It works until you have 10+ services and multiple regions, at which point it becomes a full-time job.

The script someone wrote that one time. You know the one. It's in a private repo, untouched for 18 months, and three people have contributed to it without informing each other.

None of these gives you a complete, accurate, up-to-date picture of your infrastructure. And without that picture, everything else — compliance, security reviews, cost optimisation, incident response — becomes harder than it needs to be.

A different approach

Earlier this year, we built InfraMind to solve exactly this problem.

The idea was simple: connect it to your AWS account, and get a complete map of everything running — automatically, in under a minute, without writing a single line of code.

Here's what actually happens when you run it:

1. It crawls your account across 55 AWS services

EC2, RDS, Lambda, S3, VPC, EKS, ECS, IAM, CloudFront, Route53, and 45 more.
Every resource, every region, pulled in one pass.

2. AI analyses everything it finds

Not just listing resources — actually understanding them. It identifies misconfigurations, unused resources, security gaps, and relationships between services. The kind of analysis that would take a senior engineer hours to do manually.

3. ️ You get architecture diagrams across 6 views

Overview, network topology, security, database layer, serverless, and storage.
Auto-generated, always in sync with your actual infrastructure. No Lucidchart.
No draw.io. No manual updates.

4. Terraform HCL export for 42 services

This one is huge if you're trying to get your infra under version control.
InfraMind generates production-ready Terraform for what's already running — so you can start managing existing resources as code without writing it all from scratch.

5. ✅ Compliance reports across 8 frameworks

CIS Benchmarks · PCI DSS · HIPAA · SOC 2 · ISO 27001 · GDPR · NIST SP
800-53 · FedRAMP

One click, full report, no consultants required.

The bit that surprised us most

When we started testing it internally, we thought the Terraform export would be the most-used feature.

It wasn't. The thing people kept coming back to was the architecture diagram — specifically the moment when they saw their actual infrastructure mapped out visually for the first time.

For some teams, that was a "oh that's what we have" moment. For others, it was more like "wait, why is that connected to that?" Either way, it's information they didn't have before, and it changes how the team talks about the
infrastructure.

Change detection

One last thing worth mentioning: InfraMind doesn't just give you a one-time snapshot. It monitors your infrastructure for changes and sends alerts to Slack when something shifts.

Event	Alert
New resource spun up	✅ Slack and email notifications
Security group modified	✅ Slack and email notifications
Resource deleted	✅ Slack and email notifications
Compliance status changed	✅ Slack and email notifications

Try it

There's a free plan at app.solidstack.ae — no credit card needed. Connect your AWS account, and you'll have a full map of your infrastructure in under a minute.

If you're managing AWS at any scale and you've ever felt like you're flying blind, it's worth 60 seconds to find out what's actually there.

Questions or feedback welcome in the comments — especially if you've dealt
with AWS sprawl differently. Always curious how other teams handle it.

Architect's Log: Building the Bridges

Fri, 08 May 2026 16:04:00 +0000

ARCHITECT’S LOG | Episode 11: Building the Bridges

"Isolation is great for security, but eventually, your resources need to talk to the world."
Last time, i organized the VPC into "rooms" (subnets). This week, I mastered the two main ways we let traffic flow in and out: Internet Gateways (IGW) and NAT Gateways.

Think of it as the difference between a main entrance and a one-way security exit. Here is the technical breakdown:

The Internet Gateway (IGW): The Main Entrance
An IGW is a horizontally scaled, highly available component that allows communication between your VPC and the internet.
Two-Way Street: It allows both inbound and outbound traffic.

Architect's Requirement: To actually have internet access, you need three things simultaneously: an IGW attached to the VPC, a route in your subnet table pointing to that IGW, and a Public IP on your instance.

Elastic IPs: The Permanent Address
Sometimes a standard public IP isn't enough because it changes when you stop an instance.
Static & Persistent: An Elastic IP (EIP) is a static public IPv4 address that stays with you until you release it.
SURVIVAL: It survives instance stop and start cycles, making it perfect for fixed endpoints.
NAT Gateways: The One-Way Security Exit ️
What if your database needs to download a software update from the internet, but you don't want anyone on the internet to be able to reach the database?
Private Outbound: NAT Gateways allow instances in Private Subnets to connect to the internet while preventing the internet from initiating a connection with them.

Managed vs. Instance: While you could use an old-school NAT Instance, a NAT Gateway is a managed service that scales automatically and is much easier to maintain.

The Architect’s Take
A professional architect always follows the Principle of Least Privilege. If a server doesn't need to be reached from the outside, put it in a private subnet and use a NAT Gateway for its outbound needs. It’s the simplest way to reduce your "attack surface" while keeping your systems functional.

Next Episode: We head back to storage for the "Security Edition": S3 Security & Policies.
Watch this space every Friday.

Kindly like, share and follow.

Docker Concept that Most People Miss with a Real-Life Example

Thu, 07 May 2026 07:39:05 +0000

In this article, I will discuss some concepts that most tutorials rarely discuss in Docker containerization. So let’s get into the tutorial.

Most people are stuck on the idea of making a connection through the terminal, but it disconnects when you close the terminal, so we need to make the connection in the docker-compose.yml, and from the terminal. The benefit of using a terminal is that you can cross-check whether the container can communicate and verify the network being used.

Real-life Example of Communicating to the external network

I have taken a real example of nextjs app router and Django rest framework dockerization, and I created two separate docker-compose.yml files, but in this case, the bridge network won’t work since it isolates the Docker container from the outside network or even within the container, so it can’t communicate with the Docker container from the outside network unless you create a custom external network.

services: 
  frontend:
    container_name: frontend_container
    build: 
      context: .
      dockerfile: Dockerfile
    image: frontend-img
    ports:
      - "3000:3000"
    networks:
      - frontend
  
networks:
  frontend: 
    external: true
    name: custom_network

Just like I have created a custom_network. Which I can identify while running
docker network ls

Here is the output:-

C:\Users\CodewithDev>docker network ls
NETWORK ID     NAME             DRIVER    SCOPE
01cae74bfc4c   bridge           bridge    local
34e6d4067150   custom_network   bridge    local
0c1138bda19e   host             host      local
0b28bf0c69e4   none             null      local

C:\Users\CodewithDev>

It will list all the networks, then let’s inspect the custom_network.

If we inspect the custom network, we will see frontend_container, which means I can now fetch blog posts from the Django REST Framework backend since I set the external network as custom_network.

C:\Users\CodewithDev>docker network inspect custom_network
[
    {
        "Name": "custom_network",
        "Id": "34e6d4067150bf214d806e123f2901af1c7ac515162e3240ce37c5c24655916e",
        "Created": "2026-05-06T07:30:09.855048776Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv4": true,
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "IPRange": "",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Options": {
            "com.docker.network.enable_ipv4": "true",
            "com.docker.network.enable_ipv6": "false"
        },
        "Labels": {},
        "Containers": {
            "1d1da8a799c2ff20f27ebe3a7ac2e91f6cacd4f801d70bbb2b2434a3e8aefbbb": {
                "Name": "celery_container",
                "EndpointID": "4bc40c83581d46ea316862449ec91a5430367814463871bbecfbd92c48dafe34",
                "MacAddress": "e2:ac:cc:a9:f4:7b",
                "IPv4Address": "172.18.0.7/16",
                "IPv6Address": ""
            },
            "248a69f6ada00727f62392f7a0c8fb1f3eaa5cb6b11aeb89d6f154b53ca8aa3d": {
                "Name": "django_container",
                "EndpointID": "612b7b3b86bceeac83b132ac7d3fd93e20ffe6bdaff71f1ae30ad587de543621",
                "MacAddress": "26:60:3e:23:4d:44",
                "IPv4Address": "172.18.0.3/16",
                "IPv6Address": ""
            },
            "46fbda061d78516b9544e88bab899fa5a8edd0755eff92e7d3ddeff40975b865": {
                "Name": "frontend_container",
                "EndpointID": "3e27e396d5dd937f06fd1df841c86020c4587d25f924d0a6673c500d04a0ed28",
                "MacAddress": "02:bd:85:4e:04:ea",
                "IPv4Address": "172.18.0.5/16",
                "IPv6Address": ""
            },
            "8f76a7710f14a345497adf376b7d062f4a0e3786d1007e6185957941a9030597": {
                "Name": "celery_beat_container",
                "EndpointID": "ca6763da7562e18303c7a016c949c4fc8665eb60a4ae4c8637403dbcd2013954",
                "MacAddress": "3e:c5:e0:c0:e2:21",
                "IPv4Address": "172.18.0.6/16",
                "IPv6Address": ""
            },
            "e68c877f3befe4a83a76ccae9134780a3e72cb1162ec82f452e521fa64581418": {
                "Name": "redis_container",
                "EndpointID": "089ce4c9e7283361c14c126612cdafd416d9ba55feb265ec33d365bc12a26a53",
                "MacAddress": "ca:d8:88:35:12:4b",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            },
            "e9f11e99df4f75c6fd805ccd4289740faae0a03c8d12a514cfa6d30eb90c633a": {
                "Name": "postgres_container",
                "EndpointID": "e0d9cb031ee55ea11109a44887bb5e036bb810a2f06c9a3ddab2bea25e858a3c",
                "MacAddress": "e6:87:c3:72:53:6c",
                "IPv4Address": "172.18.0.4/16",
                "IPv6Address": ""
            }
        },
        "Status": {
            "IPAM": {
                "Subnets": {
                    "172.18.0.0/16": {
                        "IPsInUse": 9,
                        "DynamicIPsAvailable": 65527
                    }
                }
            }
        }
    }
]

C:\Users\CodewithDev>

Additionally, I have included a Postgres database, Redis for caching, and Celery and Celery Beat for the task queue. It is connected internally to the bridge network, which is isolated from the frontend network. Since we have created a custom network, the external container can now communicate with the blog network.

In this way, you can Manage Multiple Docker Compose files in the same Network, and this network will stay connected unless you delete the container, unlike the terminal. Now, let’s understand the default and custom bridge network. Since our blog network was used as a bridge network internally.

Default Network vs Custom Bridge Network

When you run multiple services in the docker-compose.yml file, it will automatically create a default network, which means it creates a bridge network so that each container can communicate with each other, even if you don’t mention the network in your services and define the network at the bottom.

Which you will see in the Windows PowerShell, open Windows PowerShell, and run this command,

docker network ls

Before running this command, make sure your Docker Desktop is up and running. Here you will see it’s creating a default network even if you don’t mention the network in your docker-compose.yml file.

Disadvantages of Default Network

The first disadvantage of using the default network is that your container can’t communicate with another container by name, but only with an IP address.
You can’t isolate any Docker container, which is inside the docker-compose.yml, as it will by default use the same default network, which was created when you ran the build and up commands.
Not scalable, since it will create a default network without proper naming, and it will be hard to identify when your application will scale up.

Benefits of using Custom Bridge Network

You can isolate Docker containers by creating a custom bridge
network, which means just take an example, your Redis running on the
shared infrastructure network, and your Django rest framework
running on the e-commerce network, which means both of the services
are isolated.
When you use a custom bridge network, the container can communicate
through both the container name and IP address. This is the benefit
of using a custom bridge network.
More scalable since you can easily identify which network is being
used among these containers.

A few days ago, I was working on the Same Architecture, and I successfully communicated with the different network containers. We use service names to fetch blog posts or products.

And that’s pretty much it for this article. If you learn something new, let me know your thoughts. Maybe I will see you in the next one. Peace out!

I passed 13 AWS certifications. Here's what I actually use at work (and what collects dust).

Sun, 03 May 2026 19:28:37 +0000

It was a random Tuesday. No warning, no gradual degradation. RDS just... stopped accepting connections. ECS tasks were crashing in a cascade, CloudWatch alarms were firing, and my Slack was lighting up with "is prod down?" messages from three different people simultaneously.

I didn't open a study guide. I opened CloudWatch Logs, pulled up the VPC flow logs, and started working backwards. Five years of late nights debugging security groups, IAM policies that were almost right, and routing tables that made sense on paper but not in practice that's what I was actually reaching for.

That incident got me thinking: I've spent a significant chunk of my career chasing AWS certifications 13 of them, including Professional and Specialty levels. But which ones actually matter when the pressure is on?

Here's my honest breakdown.

I work as an AWS Solutions Architect at Wipro, designing multi-account architectures, building Terraform modules, and managing CI/CD pipelines for enterprise clients. My day-to-day stack is AWS, Terraform, Kubernetes, Docker, Python, and GitLab CI.

The three that actually changed how I work

Solutions Architect – Professional

This one rewired how I think. Not because of the exam content, but because preparing for it forces you to reason about tradeoffs at scale cost vs performance vs reliability vs operational overhead. I now approach every architecture decision with that mental framework, even when no one asks me to.

DevOps Engineer – Professional

If you work with CI/CD, IaC, or anything that touches deployment pipelines, this cert is the closest to real work. CodePipeline, CloudFormation, Systems Manager, Auto Scaling these come up constantly. Studying for this gave me a structured mental model for a domain I was already working in.

Security – Specialty

Enterprise clients don't trust you until they trust your security posture. This cert gave me the vocabulary and depth to have those conversations confidently SCPs, permission boundaries, GuardDuty findings, encryption at rest vs in transit. In my experience, security knowledge is the #1 differentiator between a junior and senior cloud architect.

The honest truth about certification chasing

Certifications are a map. They are not the territory. The territory is production.

I've met engineers with 2 certs who could debug a broken VPC faster than people with 10. I've also met certified professionals who couldn't explain why their Terraform plan was destroying a resource it shouldn't.

Certs gave me:

A structured way to learn services I wouldn't have touched otherwise
Credibility in client conversations and job applications
A forcing function to fill gaps in my knowledge

What certs didn't give me:

The ability to stay calm at 2am when prod is down
Intuition for when not to use a managed service
Experience debugging a Kubernetes pod that won't start because of a misconfigured IAM role for service accounts

My recommendation if you're starting out

Don't do 13. Do these four, in order:

Cloud Practitioner — only if you're new to cloud entirely
Solutions Architect – Associate — your real starting point
DevOps Engineer – Professional — if you work in infra/platform/DevOps
Security – Specialty — regardless of your role, this pays off

Then go build something. Break it. Fix it. That's the cert that matters most.

I'm a Solutions Architect at Wipro and AWS Community Builder based in London. I write about real-world AWS, Terraform, and cloud architecture. Follow along if that's useful to you.

What cert has been most useful in your day-to-day work? Drop it in the comments.

Architect's Log: AWS VPC Subnet & Routing Learning

Fri, 01 May 2026 09:15:25 +0000

ARCHITECT’S LOG | Episode 10: Traffic Control inside the VPC

Previously i wrote on AWS VPC Basics, now I'm looking into the Traffic control of AWS VPC.

"A network without subnets is like a house without rooms. You need walls to stay organized and secure."
Last time, we looked at S3, but now we’re heading back into the VPC to talk about the "pipes and wires." Once you’ve defined your private network, the next step is to create Subnets and configure Routing.

Here is how we organize the flow of traffic:

Subnets: Carving the Space ️
A subnet is a slice of your VPC’s Range of IP addresses, carved from CIDR.
One AZ Only: While a VPC spans a whole region, a subnet is tied to a single Availability Zone (AZ).

Public vs. Private: This is the big architectural decision.
Public Subnet: Has a direct route to an Internet Gateway (IGW).
Private Subnet: Isolated from the direct internet perfect for databases or backend logic doesn't have an Internet Gateway (IGW).

The Routing Secret Sauce
How does a subnet know where to send its data? Through a Route Table.
Every subnet must be associated with a route table.
For a subnet to be "Public," its route table must have a entry pointing 0.0.0.0/0 (all traffic) to an Internet Gateway. Exactly one route table per subnet.
Without that specific route, your resources are effectively locked inside the VPC.
Internet Communication
Even if an instance is in a public subnet, it can’t talk to the world unless it has a Public IPv4 address or an Elastic IP. Think of the subnet as the "road" and the Public IP as the "passport" needed to cross the border.

The Architect’s Take
As an architect, your default should always be Private First. If a resource doesn't need to be reached from the internet, put it in a private subnet. Use public subnets only for your "front door" components like Load Balancers or NAT Gateways. Security starts with smart routing.

Next Episode: We dive into the ultimate security showdown Security Groups vs. NACLs. Which one do you need, and when?

Watch this space every Friday.

Kindly like, share and follow.

How many people are using localsend?

Sun, 26 Apr 2026 02:01:40 +0000

we are trying to build a tool like local send.

it support

cross-platform transfer
implement C2S data transfer for allowing background transfer
LAN-based transfer.
support text-chat, voice-mail, files, etc

Is there anyone interested, or has anyone had such needs?

Coder Legion - Recent posts tagged cloud

Cloudflare vs QUIC.cloud: Which CDN Best Improves Core Web Vitals?

From Big Data to the Cloud, Part 1 — Why One Machine Is Never Enough

Chapter 1: Where All This Data Comes From

Chapter 2: What Big Data Actually Means

Chapter 3: The Five V’s

Chapter 5: The Distributed Idea

Chapter 6: Enter Hadoop

What Comes in Part 2

References

Best Managed Cloud Hosting Platforms for High-Traffic Tech Blogs

How AWS Is Designed: Regions, Availability Zones, and Where a Workload Runs

WHAT

WHY

WHEN

HOW

Where the Services Fit

Takeaway

n8n Cloud vs Self-Hosted: Real Cost for Small Teams

The cost at a glance

What n8n Cloud actually buys

What self-hosted Community Edition costs

The hidden self-hosted bill

Updates

Backups and credentials

Security and monitoring

When self-hosting wins

When n8n Cloud wins

Do not ignore the self-hosted license

A five-minute decision rule

Sources checked

Zero-Cost Cloud & Agentic AI: How to Build Production-Grade Systems Without Burning Cash

A practical guide to leveraging Microsoft's free developer sandboxes, $100 student compute, and enterprise Copilot Studio orchestration.

1. The Compute Layer: $100 Cloud Compute with Zero Billing Risk

What you get:

2. The Agentic Layer: Free Enterprise Copilot Studio Sandbox

Why this is huge for builders:

3. The Developer Tooling Layer: Cloud Workspaces & VS Code

4. The Zero-Cost Systems Architecture Blueprint

Summary

Stop Selling Event-Driven Architecture. Start Selling Business Outcomes

The Real Value of Event-Driven Architecture Isn't Technology. It's Cost Reduction.

How Event-Driven Architecture (EDA) Significantly Reduces Enterprise IT Costs

⚠️ Why Traditional Architecture Is Expensive

How EDA Reduces Costs

Upload Images With Node.js And Express To The Cloud Using Cloudinary

WHAT IS CLOUDINARY ?

WHAT IS CLOUDINARY USED FOR ?

CAUTION ⚠️

Summary

Congratulations

Top Cloud Backup and Recovery Solutions for Small Business Websites

Shared Hosting vs Managed Cloud Hosting: Which Is Right for Your Growing Blog?

Kamatera 30-Day Free Trial Guide: $100 Credit Setup & Hidden Traps (2026)

What the Trial Actually Includes

The 3 Traps That Trigger Surprise Billing

1. Powering Off Does NOT Stop Billing

2. The $100 Cap Is a Monthly Rate, Not Spendable Cash

3. Lingering Cloud Block Storage

Safe Testing Specs (Stay Under the $100 Cap)

End-of-Trial Cleanup Checklist (Do This on Day 28)

The 2026 Student Tech Stack: What I Would Learn Before Graduation

1. Programming

Why it matters

What level I would target

One practical project

One Microsoft resource worth exploring

What I would not waste time learning

2. Developer Tools

Why it matters

What level I would target

One practical project

One Microsoft resource worth exploring

What I would not waste time learning

3. Cloud

Why it matters

What level I would target

One practical project

One Microsoft resource worth exploring

What I would not waste time learning