Coder Legion - Recent posts tagged devstories

I Didn’t Become the Hacker I Imagined: Rugero Tesla’s Journey into ICS/OT Security Research

Thu, 29 Oct 2026 10:51:02 +0000

Role: ICS/OT Security Researcher | Industrial Systems & Security Research

Location: Rwanda

Rugero Tesla once imagined becoming the kind of hacker he had seen in Hollywood movies: someone who could break into supposedly secure systems with a few commands and walk away victorious. Over time, that childhood fantasy gave way to a more grounded ambition. Today, his interests lie in understanding industrial systems, investigating their weaknesses, and contributing to the security of infrastructure that people depend on.

His journey into industrial control systems and operational technology (ICS/OT) security has been shaped by curiosity, practical experimentation, and a willingness to learn without waiting for perfect resources or formal opportunities. From exploring computers as a child to building security laboratories and researching industrial communication protocols, Tesla has gradually found a field that connects his technical interests with a purpose he cares about.

In this CoderLegion Developer Story, Tesla shares how his early fascination with computers developed into a commitment to industrial security research, why publishing his work changed his professional trajectory, and what he has learned about building a career when access to equipment, opportunities, and industry connections can be difficult.

A Childhood Built Around Figuring Things Out

Tesla's interest in technology began at home. His father worked as an IT technician and regularly repaired computers. What fascinated Tesla was not just the work itself, but the fact that his father had taught himself how to do it.

When Tesla was around five years old, his father showed him how to burn an operating system onto a CD and install it. At seven, he received his own computer after earning good grades. While hardware was not his primary interest, exploring software quickly became a favorite activity.

He spent hours investigating his computer, sometimes changing things he did not fully understand. Whenever something broke, however, there was a problem: he was afraid of his father's reaction if he could not fix it.

That fear pushed him to investigate what had gone wrong and find a solution. Over time, troubleshooting became more than a way to avoid getting into trouble. It became a habit of approaching unfamiliar problems with patience and determination.

Looking back, Tesla recognizes the connection between those childhood experiences and his current work.

He developed an early belief that spending enough time with a problem would eventually help him understand it. Even without knowing the correct approach at the beginning, he would search for alternatives until something made sense.

His formal introduction to computer science came later, during A-level studies in mathematics, computer science, and economics. The concepts came naturally to him, reinforcing his enjoyment of understanding how systems worked.

At that stage, he did not yet know where his interest would lead. He simply knew that computers offered an endless supply of things to investigate.

When Hollywood Hacking Met Reality

Tesla's first idea of hacking came from movies released during the 1990s and 2000s. In those stories, hacking appeared almost effortless. A character would sit in front of a computer, type a few commands, compromise a supposedly secure system, and achieve something extraordinary.

That portrayal shaped his early ambitions.

He even imagined that learning to hack banks would guarantee financial security. It was a childhood fantasy, but one that eventually developed into a genuine interest in cybersecurity.

During his final year of high school, a teacher introduced him to the subject. Something clicked, and Tesla decided he wanted to study cybersecurity at university.

His father supported the decision by enrolling him in Cisco's cybersecurity summer camp after high school. The experience became his first serious introduction to the field, but it also challenged his expectations.

Real cybersecurity was considerably more complicated than the movies suggested. Modern systems had security controls, and successful attacks were rarely as simple as the fictional scenarios he had grown up watching.

He also began reconsidering what he wanted to accomplish with his skills. Harming people, stealing their money, exposing their private information, or deceiving them was no longer part of the picture he wanted for himself.

He still wanted to understand how systems could be compromised. The difference was that he wanted to pursue that knowledge without using it to hurt others.

A mentor helped him take the next step by introducing practical exercises, homelabs, and scenarios that allowed him to experiment with systems he controlled.

This hands-on learning was valuable, but it exposed another limitation. Some exercises depended on deliberately weakening the security of a laboratory environment to make an attack succeed. That made it difficult for him to understand how easily the same attack might translate to real-world systems.

The experience taught him that following an attack scenario was not the same as developing a deeper understanding of security.

He wanted to investigate systems more realistically, understand their weaknesses, and build skills that could eventually support a legitimate professional career.

The Challenge of Entering Cybersecurity Without a Degree

After completing his initial training, Tesla began searching for internships and other opportunities. Many organizations turned him down because he did not have a university degree.

For someone who had invested time in learning, building laboratories, and developing practical skills, the repeated rejections were frustrating.

Rather than stopping, he continued creating projects.

He wrote scripts that could exploit systems in his own laboratories and published them on GitHub. He hoped others would discover the repositories, find them useful, and engage with his work.

Most received little attention.

The experience highlighted a difficult reality for an aspiring security researcher: building something technically interesting does not guarantee that anyone will find it. Even when the work requires substantial effort, a public repository can remain unnoticed.

Tesla was also becoming less interested in testing attacks against laboratory targets simply for the sake of completing an exercise. He wanted his work to lead somewhere meaningful, both professionally and personally.

An incident involving an attempted scam became a turning point in how he thought about the boundaries of security research.

After following a link sent by someone attempting to scam him, he discovered what appeared to be evidence of a larger scam campaign in his country. He took matters into his own hands, exported a database, deleted it from the servers, and contacted his country's investigation bureau.

He provided the information he had gathered and eventually met the head of the cybersecurity bureau, who thanked him.

Despite that response, Tesla was uncomfortable with his own actions. He recognized that he had crossed a line, even if his intentions had been to help.

The experience reinforced an important distinction: wanting to investigate security weaknesses does not automatically justify every action taken during an investigation.

Tesla had originally wanted to become a professional penetration tester. He wanted to apply his technical skills legitimately and earn a living doing the work.

After struggling to find opportunities without a degree, he decided that enrolling in university was the practical next step toward building that career.

Discovering Industrial Control Systems and Operational Technology

University did not immediately solve the problem of finding the right direction.

Tesla had already spent three years studying computer science at A-level, so revisiting much of the same material left him feeling stuck. He also began to question whether conventional software-focused work was what he wanted to pursue.

He enjoyed understanding systems, but reading code and hunting for software bugs did not interest him as much as he had expected.

What attracted him instead was infrastructure: the systems beyond the code editor, how they communicated, and how their components worked together.

That curiosity led him toward industrial control systems and operational technology, commonly known as ICS/OT.

Although he explored IoT security, it was industrial security that held his attention. He became interested in the fragility of industrial environments and the importance of protecting systems connected to real infrastructure.

Unlike a purely software-based problem, a weakness in an industrial environment can have consequences beyond the computer itself. That responsibility became an important part of the appeal.

Tesla did not want to abandon offensive security. He still wanted to understand how systems could be compromised and what might go wrong. However, he also wanted to use that understanding to help protect them.

The role of a security researcher offered a better fit than defining himself exclusively as a penetration tester. It allowed him to investigate systems, study weaknesses, and explore defensive implications while continuing to develop his technical knowledge.

For the first time, his interests seemed to fit together.

There was still a practical obstacle. ICS/OT security could be difficult to access without the money for industrial equipment, specialized training, or suitable research environments.

Tesla changed his university schedule to weekend classes to create more time for independent work during the week. He then decided to approach industrial security in the same way his father had approached computers: learn independently, use available resources, and figure things out.

Learning Industrial Protocols With the Resources Available

Tesla began by studying industrial protocols because he wanted to understand how industrial systems communicated.

His approach was practical rather than dependent on having access to an extensive industrial laboratory. He worked with the resources available to him, investigating systems and building projects that helped him develop his understanding.

One of his early projects in this direction was Modbus Exposure Analyzer. He published an article about the tool on DEV Community, initially treating it as another opportunity to document and share what he was learning.

He was not yet thinking seriously about personal branding or building a public professional identity. His focus was on learning, creating tools, and sharing the results.

Then an unexpected discovery changed his perspective.

While searching Google for his uncle's name, Tesla noticed that Google's AI Overview had surfaced information about his uncle, a well-known oncologist in Rwanda. He decided to search for his own name as well.

The result surprised him. Google's AI Overview described him as an ICS/OT Offensive Security Researcher known for Modbus Exposure Analyzer.

He had published an article about a tool and largely left it alone. Yet that single piece of public work had become part of how he was represented online.

The discovery made him reconsider the value of publishing.

Until then, he had spent considerable time trying to convince organizations of his abilities, often encountering rejection because he lacked formal qualifications. Public technical work offered another way for people to discover his skills and evaluate what he could actually do.

Publishing also served an educational purpose. Tesla wanted others to study his work, point out mistakes, and contribute their own ideas.

GitHub remained useful for sharing code, but articles could reach readers who might never discover a repository on their own.

He also wanted to demonstrate that beginning to learn ICS/OT security did not necessarily require a large budget. With available resources, a willingness to experiment, and a commitment to documenting findings, aspiring researchers could begin developing their knowledge.

That philosophy became the foundation of his industrial protocol research series.

How Public Research Created Unexpected Connections

For a long time, Tesla did not know what his published work might lead to. He continued investigating industrial protocols and sharing his progress.

Eventually, those publications began connecting him with people who had experience he could learn from.

One notable moment came when he posted a screenshot on LinkedIn showing his progress through his industrial protocol series. He was one protocol away from reaching IEC 61850.

Parker McCauley, a systems architect, commented on the post and suggested that Tesla contact Bruce Muschlitz to learn more about the history behind IEC 61850.

Muschlitz was a veteran utility communications engineer who had served for more than two decades as chairman of the UCA International Users Group Testing Committee, working with its device conformity testing program for IEC 61850.

Tesla reached out, and their conversation provided valuable input that informed his research.

The connection was significant because it emerged from something relatively simple: sharing progress publicly.

Tesla had not planned the interaction or known that the connection existed. He had been working through industrial protocols with the resources available to him and documenting what he learned. By making that work visible, he created an opportunity to connect with someone whose experience could deepen his understanding.

His public work also led to professional opportunities, including job offers that he ultimately declined. He has chosen not to disclose the details, but he sees the opportunities as evidence that people can recognize practical abilities through published work, even when those abilities are not backed by a degree.

The contrast with his earlier experience was striking. Repositories that once seemed to disappear into the internet were helping him build professional connections and attract interest in his work.

Publishing did not eliminate every barrier, but it gave him another way to demonstrate what he could do.

Why Ethical Security Research Matters

Tesla's journey has also changed his understanding of what it means to be a hacker.

His early interest came from fictional portrayals of people breaking into systems. As he gained practical experience, he began separating the technical challenge of understanding vulnerabilities from the decision to exploit them against others.

That distinction matters particularly in security research, where technical capability must be accompanied by judgment about authorization, potential harm, and responsibility.

His own experience with the suspected scam campaign made that lesson personal. Even though he intended to help, he recognized that his actions had crossed a boundary.

Today, he wants to understand how industrial systems can fail or be compromised while contributing to their protection.

This approach does not require abandoning offensive security knowledge. Instead, it means directing that knowledge toward legitimate investigation and better defenses.

For Tesla, ICS/OT security offers a way to combine curiosity about how systems work with an interest in protecting infrastructure that extends beyond the digital world.

It also gives his work a clearer purpose: learning about weaknesses is valuable not simply because a system can be broken, but because understanding those weaknesses can help people protect what matters.

Tesla's experience has shaped three principles he wants other aspiring researchers to understand.

Keep your curiosity alive

His first piece of advice is to preserve the curiosity that initially drew you to technology.

For him, that curiosity began with a computer at home and a desire to understand what happened when something broke. His professional direction changed over time, but his interest in investigating unfamiliar systems remained.

He recognizes that people who study computer science may eventually pursue entirely different careers, and there is nothing wrong with that. What matters is staying connected to the interests that genuinely motivate you.

Find another way forward

Tesla's second lesson is to look for a workaround when the expected path is unavailable.

He faced barriers involving formal qualifications, access to industrial equipment, training costs, and professional opportunities. Rather than waiting until every obstacle disappeared, he began working with the resources he already had.

That does not mean every limitation can be overcome through effort alone. Circumstances and access to resources matter. But when one route fails, finding another way to make progress can keep a long-term goal within reach.

For aspiring ICS/OT researchers, this means looking for practical learning opportunities that fit their circumstances, developing projects around available resources, and documenting the results.

Don't confuse hacking with harming people

His third lesson concerns ethics.

The idea that hacking was inherently cool or impressive came partly from the movies he watched growing up. Real experience taught him that technical ability does not make harmful behavior acceptable.

Stealing information, scamming people, or crossing legal and ethical boundaries is not justified simply because someone possesses the skills to do it.

Security professionals can investigate systems, understand vulnerabilities, and develop offensive capabilities without deliberately making other people's lives worse.

For Tesla, that distinction is central to the kind of researcher he wants to become.

Building a Career That Fits Who He Is

Tesla does not describe his current direction as the destination he originally imagined. He once wanted to become a penetration tester and thought professional hacking was the ultimate goal.

His interests have evolved, and he is comfortable with that change.

ICS/OT security research brings together several things he values: understanding unfamiliar systems, investigating how they work, discovering weaknesses, and contributing to their protection.

He also wants to make learning more accessible to others who encounter the same barriers he did. He knows what it feels like to enter a field and discover that the resources needed to progress can be expensive or difficult to obtain.

By publishing research, sharing tools, and explaining what he learns, he hopes to make it easier for others to begin exploring industrial security.

His plans will continue to develop as he gains experience, but his priorities remain consistent: doing work he cares about, building a sustainable career, protecting important systems, and helping other people learn.

The most meaningful change in Tesla's journey is not simply that he moved from one security specialization to another. It is that he found a way to direct his curiosity toward work that aligns with his values.

The child who once experimented with computers because he wanted to understand how they worked has become a researcher investigating the security of industrial systems. Along the way, he has learned that progress does not always depend on following the expected route, that sharing work can create opportunities that formal credentials alone may not provide, and that technical ability is most valuable when paired with responsibility.

He may not have become the hacker he imagined while watching Hollywood movies. Instead, he is building a career around something more meaningful to him: understanding the systems that shape the real world and helping make them more secure.

About Rugero Tesla

Rugero Tesla is an ICS/OT Security Researcher based in Rwanda, focused on industrial systems, security research, and industrial communication protocols. His technical interests include PLC security research, OT network architecture, industrial protocol analysis, and reproducible security research. He works with tools and languages including Python, Bash, C/C++, and PowerShell.

CoderLegion: Rugero Tesla
Website: rugerotesla.tech
Featured project: Modbus Exposure Analyzer, discussed in his interview.

Developer Stories is a CoderLegion interview series highlighting the people, ideas, projects, and engineering decisions behind software development.

Building Software for Failure: Roman on Python Networking Tools, AI-Powered Routers & Secure Systems

Thu, 29 Oct 2026 05:29:28 +0000

Interviewed by: Mehadi Hasan

Developer: Roman Madaev

Role: Full-Stack Developer and System Administrator

Location: Moscow, Russia

Website: neurocrat.io

Roman Madaev's approach to software development starts with a question many developers only consider after something goes wrong: what happens to the user if the application fails halfway through a task?

That question connects several of his projects, from AURA, a Windows VPN client written in Python, to REGENT, an AI-assisted tool for configuring OpenWrt routers. It also influences his security testing utilities, his approach to asynchronous programming, and the way he builds web applications.

With more than six years of experience across web development, system administration, and automation, Roman has worked on everything from landing pages and online stores to CRM dashboards and a food ordering platform. Alongside his professional work, he develops independent tools that address practical problems he has encountered himself.

His projects reveal a consistent engineering philosophy: software should not merely work when everything goes according to plan. It should explain failures, protect existing system state, and give users a reasonable way to recover when something breaks.

From Learning to Code at 14 to Managing Systems at 17

Roman began learning programming at 14. By 17, he was already working as a system administrator at Meridian College, where he was also studying.

His responsibilities included managing the college website and local network. The work gave him an early opportunity to understand technology beyond the code displayed in an editor. Websites depended on servers, networks connected systems, and problems often required investigating several layers before their causes became clear.

His vocational education covered JavaScript, HTML, CSS, PHP, client-server architecture, and relational databases. Coursework introduced him to building small web applications, but working on actual systems gave him a broader perspective on how software operates.

Roman was also involved in helping computer science teachers explain topics to students and prepare learning materials. That experience added another dimension to his technical work: understanding a system matters, but communicating how it works matters too.

An early milestone came during a programming olympiad involving three colleges. Participants had to build and debug a small web application in JavaScript or PHP under time pressure. Roman finished second.

The experience influenced the way he approaches deadlines today. Rather than attempting to perfect every detail immediately, he first establishes a working structure and then improves it.

That principle has remained useful throughout his career. When time is limited, a functioning foundation provides something concrete to test, debug, and refine.

How His Engineering Priorities Changed With Experience

Roman began professionally by building landing pages and online stores at a web studio. Later, he moved into CRM systems and dashboards, and he now works on a food ordering platform for Foodison.

The progression brought increasingly complex engineering concerns. Building a page that displays correctly is one challenge; maintaining a system as its dependencies, data flows, and business requirements change is another.

His priorities have shifted accordingly.

Migration work, for example, involves more than replacing one library with another. Roman has worked on moving from Vue 2 to Vue 3 and from Vuex to Pinia. Such changes need to preserve existing behaviour while making the application easier to maintain.

Payment processing presents a different problem. A payment flow must avoid creating duplicate orders, even when requests or responses do not behave as expected. API clients also need thoughtful handling of caching and data freshness, while deployments should minimise disruption for users.

These concerns have made maintainability a central criterion in his decisions. Tests, CI/CD, and clear boundaries between application layers are not simply additions to a project; they help make future changes safer and easier to reason about.

His background in system administration reinforces that mindset. Having worked with networks and servers, he naturally considers failures, recovery, and the possibility that an operation may stop before it finishes.

Choosing Technologies Based on the Problem

Roman does not begin a project by selecting his favourite framework. He starts with the requirements and chooses the technology accordingly.

For stores and location-based pages where search engine visibility and fast initial loading matter, he uses Vue with Nuxt.js and server-side rendering. For internal tools and dashboards, a Vue single-page application may be more suitable, particularly when live updates through WebSockets are needed.

Python is his choice for automation, networking tasks, and desktop utilities. For smaller REST backends in personal projects, he uses PHP with Slim 4.

This approach reflects the breadth of his work. Frontend development helps him understand how software feels to use, while backend and infrastructure experience helps him understand what happens behind the interface.

The result is a practical way of thinking about architecture: the technology stack should serve the project rather than dictate its design.

That philosophy becomes particularly clear in his independent tools, where he builds the components needed to solve a specific problem instead of relying entirely on existing applications.

AURA: Building a Windows VPN Client From Scratch in Python

AURA began with a practical problem at work. Roman needed to access a closed network through L2TP/IPsec, but the built-in Windows 10 VPN client refused to connect.

The immediate difficulty was not simply the failed connection. The client provided no useful logs explaining which stage had failed, leaving him without a clear way to diagnose the problem. He also could not find another program that handled the required task.

Rather than accepting those limitations, Roman decided to build his own VPN client, implementing the protocol stack down to key exchange.

AURA is written in Python and does not use Windows' built-in VPN connection mechanisms such as RAS or rasdial. Instead, it implements its own IKEv1/IPsec, ESP, L2TP, and PPP stack.

For network traffic, it uses a virtual adapter named AURA based on Wintun, the driver from the WireGuard project. On Windows, the application manages system routes and DNS records, requests administrator privileges through User Account Control at startup, and can store passwords and keys at the user's request, protecting them with Windows DPAPI.

The cryptographic primitives are implemented using PyCryptodome, while the protocol implementation follows the relevant RFC specifications.

Negotiating With Real VPN Servers

One of AURA's significant engineering challenges is negotiating connection parameters with the server.

The application offers 18 IKE variants involving AES or 3DES encryption, SHA-256 or SHA-1 hashing, and Diffie–Hellman groups 14, 5, or 2. It then uses the variant selected by the server.

Authentication must also match the server's requirements. Depending on the configuration, this may involve MS-CHAPv2, CHAP, or PAP, followed by IPCP negotiation for IP addressing and DNS. MPPE is supported where required by the server.

Implementing the protocols was only part of the challenge. Roman also needed to make connection failures understandable to someone who might not know how IKE negotiation works.

Turning Silence Into a Useful Error Message

The built-in VPN client gave Roman little information about why a connection failed. AURA approaches the problem differently by displaying each connection stage in its status bar and log.

One particularly difficult case involves an incorrect pre-shared key. The server may not explicitly reject the key; it may simply stop responding.

AURA interprets that silence in the context of the negotiation sequence. During IKEv1, the first four messages are sent unencrypted, while the fifth is the first protected by a key derived from the pre-shared key.

If the server responds to the first four messages but becomes silent at the fifth, AURA reports that the pre-shared key may be incorrect and highlights the relevant field. If the server does not respond to the early messages, the application instead reports that the server is not responding.

When the server explicitly rejects a login, AURA highlights the username and password fields.

Roman also takes care not to treat an undecipherable or unverifiable response as definitive evidence of an authentication failure. A forged packet should not be allowed to determine the application's verdict.

This is a useful distinction between detecting an error and explaining it responsibly. A good diagnostic system must interpret the evidence it receives without claiming more certainty than that evidence supports.

Recovering Cleanly From System Changes

A VPN client can change more than its own internal state. It may modify routes, DNS settings, and virtual network adapter configuration.

If the application crashes halfway through these operations, the remaining system state could affect connectivity even after the application stops running.

AURA addresses this by recording each system change before applying it and reverting changes in reverse order. If the program crashes, the next launch attempts to remove changes left behind.

Roman also encountered differences between protocol specifications and the behaviour he could verify on real servers. For example, although RFC 3948 permits a zero checksum for the inner UDP packet, zero is the default in AURA because that was the value he verified against a real server. An automatic mode calculates the checksum but is not enabled by default.

The project remains under development. Connections can occasionally drop, and in-place rekeying and automatic reconnection are planned improvements.

AURA illustrates one of Roman's recurring lessons: implementing a protocol according to its specification is not the end of the work. Real systems must be tested, failures need to be interpreted carefully, and changes to the operating system must be reversible.

Explore the project: AURA on GitHub

REGENT: Giving AI a Safer Way to Configure OpenWrt

REGENT emerged from another frustrating experience with infrastructure.

Roman's own OpenWrt router once stopped passing traffic even though its web interface appeared normal. It took him an entire day to identify four separate configuration errors, none of which was visible on the page where he had made the settings.

He saw an opportunity for an AI assistant that could inspect the router's configuration as a whole, explain what was broken, and apply requested changes expressed in ordinary language.

Rather than exposing a general-purpose command line to the model, he chose the Model Context Protocol (MCP). The idea is to give the model a defined collection of tools with controlled behaviour, rather than leaving it to construct arbitrary commands for every task.

Defined Tools Instead of Unrestricted Commands

REGENT runs as an MCP server that communicates with its client through standard input and output. It does not open network ports.

Its implementation uses FastMCP to expose tools for router topology, UCI configuration changes, backups, diagnostics, firewall settings, Wi-Fi, package management, and other operations. AsyncSSH maintains a reusable SSH session with the router.

Roman separates command construction and output parsing from the transport layer, keeping these responsibilities in independent functions.

He also implements composite operations for tasks that require several coordinated changes. For example, giving clients internet access involves six changes across three configuration files in a defined order. That procedure lives on the server instead of requiring the model to reconstruct the sequence each time.

The routerTopology tool reads the full configuration in one pass and attempts to explain what is broken and why traffic is being blocked. When a change is requested, REGENT translates it into ordered UCI and ubus commands, executes them over SSH, and restarts the service responsible for the affected configuration.

The emphasis is on actionable diagnosis rather than a long list of disconnected settings.

Protecting the Router When Something Goes Wrong

AI-assisted infrastructure management introduces a particular risk: a configuration change can disconnect the very network connection needed to reverse it.

Roman designed REGENT with this failure scenario in mind.

Read operations are always available, while configuration changes require the OPENWRT_ENABLE_WRITE=1 environment variable. Reboots, firmware flashing, and factory resets require separate confirmation for each call.

Before a risky change, REGENT copies the relevant configuration files and schedules a restoration on the router itself, normally after 90 seconds. If the router responds after the change, the restoration is cancelled. If it does not, the previous settings are restored.

Running the rollback on the device matters because the process can survive an SSH session dropping. REGENT also supports dry runs.

The project includes safeguards for sensitive information. Wi-Fi passwords, VPN keys, and subscription links are removed before information reaches the model or command log. REGENT uses a dedicated SSH key rather than a personal one.

Roman is also explicit about the limitations. The safeguards reduce risk but do not eliminate it. The routerExec tool can execute arbitrary commands outside the special tools' guardrails, so he recommends keeping backups and maintaining physical access to the router.

At the time of his interview, REGENT had been tested on one device: a TP-Link Archer C59 running OpenWrt 23.05.4. Its tests use a fake SSH transport with data copied verbatim from a live router, including the configuration state from when the router was broken.

Those boundaries are important. REGENT demonstrates an approach to AI-assisted infrastructure management, but its current testing scope should not be mistaken for broad device compatibility.

Explore the project: REGENT on GitHub

Security Tools That Explain More Than They Discover

Roman has also built two security testing utilities: CR4SH3R and NR34P3R. Both are intended for educational use and authorised testing.

CR4SH3R focuses on arbitrary file upload vulnerabilities in WordPress plugins. Roman developed it after a friend asked him to examine a WordPress site and he discovered plugin vulnerabilities that exposed sensitive data.

The tool automates vulnerability discovery and data extraction, provides a graphical interface, and supports extensible payloads. The experience also demonstrated how much sensitive information a single exposed file, such as wp-config.php, can reveal.

NR34P3R grew out of a different experience. After conducting a penetration test on his own laptop, Roman found himself manually searching the web to identify the purpose of each discovered port.

That process led him to a broader question: what value does a scanner provide if its results still require substantial manual research before they become useful?

NR34P3R is a multithreaded port scanner that attempts to identify the services associated with discovered open ports. Its built-in database contains more than 3,000 services, allowing results to include contextual information instead of just port numbers.

The scanner uses 100 concurrent threads and supports 34 common ports, individual ports, arbitrary ranges such as 8000–9000, and the full port range from 1 to 65535. Results appear in tables with a real-time progress indicator, and users can export them for further analysis.

Balancing Speed With Honest Results

NR34P3R highlights a familiar trade-off in network tooling. More concurrent threads can accelerate scanning, but they also increase the load on both the machine running the scanner and the target.

Timeouts create another limitation. A short timeout makes scanning faster but may cause slow hosts to be missed.

Roman chose threads because port scanning largely involves waiting for network responses, and threads offered a straightforward way to run many blocking socket checks in parallel. He did not initially consider asyncio, but he would consider it for a rewrite because asynchronous I/O can handle thousands of pending connections with lower overhead than 100 threads.

His most important lesson concerns the interpretation of results. A closed port and a slow host skipped because of a short timeout can look identical to a scanner. A reliable tool must acknowledge that uncertainty instead of presenting every result as conclusive.

Planned improvements include host reachability checks, JSON reports, ASN lookup, and a ping sweep to avoid wasting time scanning unreachable hosts.

Explore the projects: CR4SH3R on GitHub and NR34P3R on GitHub

Asynchronous Python and the Importance of Responsiveness

Roman is continuing to develop his understanding of Python's asynchronous capabilities, including asyncio, aiogram, and asyncpg. His progress has been slower than he originally planned, but REGENT has provided practical experience with asynchronous systems through AsyncSSH, FastMCP, and tests using pytest-asyncio.

The project taught him several important lessons: avoid blocking the event loop, share a reusable SSH session safely between tool calls, and put a timeout on every command. REGENT uses a default command timeout of 30 seconds.

For Roman, responsiveness is not just a performance metric. A program that hangs can frustrate users even if its underlying operations are eventually correct. Predictable execution and clear failure handling are equally important.

The same concern appears in VOCIS, his unreleased Windows microphone-processing tool.

VOCIS is designed to process microphone input locally in real time. Its planned processing pipeline includes RNNoise noise reduction, a digital signal processing chain with a low-pass filter, noise gate, compressor with automatic gain, and limiter. It also mutes output when the user is silent and sends the processed audio to a virtual microphone.

The processing runs on the CPU without recording audio to disk. Roman reports that noise reduction and processing take approximately one millisecond of CPU time per 10-millisecond audio block inside the audio callback. The voice-detection model runs in a separate thread, limited to two PyTorch threads so that it does not take time away from audio processing.

The project is not yet released. A Microsoft-signed driver and the higher-quality DeepFilterNet noise suppressor are among the planned improvements.

Although REGENT and VOCIS address very different problems, both demand predictable behaviour. A router tool must not leave the network inaccessible, and an audio tool must not interrupt the sound it is processing. In each case, responsiveness and recovery are part of the product's core requirements.

The Principle Behind His Projects: Design for Failure First

When asked what changed his approach to development most, Roman pointed to a principle that connects his work across networking, infrastructure, and application development: design for failure earlier than the happy path.

AURA records system changes before making them so that leftover changes can be reverted after a crash. REGENT schedules a rollback on the router itself before applying a risky configuration change.

Both decisions begin with the same question: what will remain for the user if this breaks halfway?

It is a practical way to think about reliability. Instead of treating recovery as a feature to add once the main functionality is complete, the developer considers failure scenarios while deciding how the feature should work.

Roman's recommendations for developers building Python applications, networking utilities, or security tools follow directly from that experience:

Test against real devices and servers. Specifications and test data cannot fully represent the behaviour of every real environment.
Use fake transports and recorded outputs. Most tests can run without physical hardware when external interactions are represented accurately.
Make dangerous operations explicitly opt-in. Risky actions should require deliberate permission rather than being available by default.
Use security tools responsibly. Testing should be limited to systems you own or have permission to assess.

These lessons are particularly relevant to tools that interact with operating systems, network infrastructure, and security-sensitive data. A failure can affect more than the application's own process, so developers need to account for the state their software changes.

Building neurocrat.io and Bringing His Interests Together

Roman's current main focus is neurocrat.io, his bilingual Russian- and English-language IT blog covering programming, security, anonymity, and technology news.

He develops both the frontend and backend himself. The frontend uses Vue 3, while the REST API is built with PHP and Slim 4.

The site includes role-based access control with CASL, a TipTap editor that stores content as plain Markdown, and AI-assisted translation. It also supports light and dark themes, loading skeletons, and stale-while-revalidate caching with ETag revalidation.

Security and maintainability receive attention alongside the visible features. Roman has worked on a strict Content Security Policy, sanitised Markdown, automated tests, and search engine optimisation.

The blog brings several parts of his experience together. Frontend development shapes the reading and editing experience, backend development supports the site's functionality, and his systems and security background informs decisions about caching, access control, and application protection.

Looking ahead, he wants to keep developing in web engineering, system administration, and cybersecurity. He also wants to explore AI-driven infrastructure management further, building on what he has learned from REGENT.

His plans for AURA include in-place IPsec rekeying and IKEv2 support, while REGENT is expected to gain support for OpenWrt 24.10.

These are continuing projects rather than finished destinations, and Roman's approach leaves room for further testing, refinement, and learning.

About Roman Madaev

Roman Madaev is a full-stack developer and system administrator based in Moscow, Russia, with more than six years of experience in web development and enterprise systems. His technical work spans Vue.js, Nuxt.js, JavaScript, PHP, Python, networking, and automation.

His independent projects include AURA, a Python-based Windows VPN client; REGENT, an MCP server for AI-assisted OpenWrt configuration; and the security testing tools CR4SH3R and NR34P3R. He is also developing VOCIS, a real-time microphone-processing tool, and maintaining his bilingual IT blog, neurocrat.io.

Roman is interested in the intersection of web development, system administration, cybersecurity, and AI-assisted infrastructure management.

CoderLegion: Roman Madaev's profile
Website: neurocrat.io
LinkedIn: Roman Madaev
AURA: GitHub repository
REGENT: GitHub repository
CR4SH3R: GitHub repository
NR34P3R: GitHub repository

Developer Stories is a CoderLegion interview series highlighting the people, ideas, and engineering decisions behind software development.

From Bug Reproduction to Reliable Regression Tests: How Orhan Is Building Practical QA and AI Tools

Wed, 28 Oct 2026 09:24:06 +0000

Interviewed by: Mehadi Hasan

Developer: Orhan Eren Kara

Role: QA Engineer and Chrome Extension Developer

Location: Türkiye

Featured Project: QA Bug → Regression Test

Technologies: TypeScript, React, WXT, Chrome Extension Manifest V3, Playwright

Software testing is about more than finding bugs. It is about understanding why something failed, documenting the conditions that caused the failure, and ensuring the same problem does not return.

For Orhan Eren Kara, a QA engineer and independent developer from Türkiye, this principle has shaped his approach to software quality, browser automation, and AI-assisted development. After graduating in Management Information Systems from İzmir Democracy University in 2025, he began exploring ways to make testing workflows more practical and less repetitive.

His work focuses on a familiar challenge for QA engineers: turning a manually reproduced bug into a useful, maintainable automated regression test. Rather than treating automation as a replacement for engineering judgment, Orhan sees it as a way to preserve information, reduce repetitive work, and help developers concentrate on the decisions that matter.

That philosophy is reflected in two of his Chrome extension projects, QA Bug → Regression Test and MindContext. Although they address different workflows, both explore the same broader idea: tools should make existing work easier without hiding the limitations or decisions behind the results.

From Software Testing to Browser Automation

Orhan's professional journey began in software testing. While working with different types of software, including web applications, APIs, chatbots, and AI-powered workflows, he became interested in understanding how applications behave beyond their expected scenarios.

A feature might work perfectly under normal conditions but fail when users provide unexpected input, interrupt a conversation, encounter a timing issue, or move through an unusual sequence of actions.

These experiences shaped his understanding of software quality. Testing is not simply a matter of checking whether a feature works. It also involves identifying edge cases, understanding user behavior, reproducing problems clearly, and giving developers enough information to investigate and fix them.

AI-powered applications made this challenge particularly visible. In conversational and voice-based workflows, small changes in wording, context, timing, or user input can lead to different outcomes. Interruptions, ambiguous responses, and unexpected silence may expose problems that ordinary happy-path tests fail to catch.

Orhan also learned to distinguish between an API returning a successful response and an application delivering the experience users actually expect.

This perspective led him to focus on reproducibility, clear expected and actual results, and appropriate evidence. A bug report should explain what happened and provide the conditions necessary to reproduce it. A fix should then be verified rather than assumed to be correct.

As he explored ways to reduce repetitive testing work, he became interested in tools such as Selenium and Playwright. Eventually, his interest in browser automation led him to Chrome extension development.

Browser extensions offered a practical advantage: they could fit directly into an existing workflow. Instead of asking developers and testers to adopt an entirely separate process, he could build tools that assist them while they were already working.

The Problem Behind QA Bug → Regression Test

One of the most repetitive tasks in software testing happens after a bug has been discovered.

A QA engineer reproduces the problem, documents the steps, records the expected and actual results, and may capture screenshots or other evidence. Later, when the issue needs automated regression coverage, those same interactions must be translated into test code.

The process creates two related but separate tasks: documenting the bug and writing the automated test.

Information can be lost during that transition, and the same work may need to be repeated. Even when the original reproduction is well documented, the person writing the test must reconstruct the sequence of actions and determine which assertions will demonstrate the failure.

Orhan wanted to explore whether a single browser session could serve as the starting point for both tasks.

That idea became QA Bug → Regression Test, a Chrome extension designed to record supported browser interactions and help generate two editable outputs: a Markdown bug report and a Playwright TypeScript test draft.

The objective is not to eliminate the judgment required from a QA engineer. It is to reduce repetitive work and preserve more of the information gathered during the original bug reproduction.

This distinction is central to the project's design. Recording what a user did is useful, but the recording alone does not explain why the behavior was incorrect or prove that a future test will detect the same problem.

The extension aims to make the transition from manual reproduction to automated testing more efficient while keeping the engineer responsible for validating the result.

How the Extension Turns Browser Activity Into Test Drafts

The workflow begins in Chrome's side panel. The user starts a bug session and grants the extension access to the relevant website when recording begins.

The user then reproduces the issue naturally. During the session, the extension records supported interactions, including clicks, form inputs, selections, and navigation.

The user can also add assertions, provide a bug title, document the expected and actual results, and optionally capture screenshot evidence.

After recording, the collected information can be reviewed and used to generate two artifacts.

1. A structured Markdown bug report

The first output organises the captured information into a report containing reproduction steps, expected and actual results, technical notes, and available evidence.

The report can serve as a starting point for issue trackers such as GitHub Issues, Jira, or Linear.

Its value is not simply that it saves time writing documentation. By preserving the sequence of interactions and relevant context, it can help another developer understand what happened and investigate the problem more effectively.

2. An editable Playwright TypeScript test

The second output is a Playwright TypeScript .spec.ts test draft based on the recorded interactions and assertions.

Instead of reconstructing the entire interaction sequence from scratch, the engineer receives an initial structure that can be reviewed, edited, and adapted to an existing Playwright project.

However, the generated code is not automatically a complete regression test. The engineer still needs to check whether the locators identify the correct elements, whether the assertions represent the original failure, and whether the test has the necessary data and environment configuration.

This review stage is essential. A test that replays the recorded clicks may execute successfully without verifying the behavior that originally made the scenario a bug.

For Orhan, the generated artifacts are starting points rather than guarantees. The purpose is to make the work easier without suggesting that recording alone can replace careful test design.

Engineering Decisions: Keeping the Workflow Local and Inspectable

QA Bug → Regression Test is built with TypeScript, React, WXT, and Chrome Extension Manifest V3. Its side-panel interface allows users to work with the extension without leaving the website they are testing.

The architecture must support several connected responsibilities: observing interactions on an active website, maintaining a recording session, collecting relevant information, and generating downloadable artifacts.

Browser permissions are another important consideration. The extension requests site access when a recording begins rather than treating every page as an unrestricted source of data.

Orhan also chose to keep the core recording and generation workflow local. The current version does not require an account, backend, or cloud synchronisation to create reports and test drafts.

This simplifies the architecture and keeps recorded sessions in the browser. It also means users must remain attentive to what they review and share, particularly when reports contain screenshots or captured information from real applications.

For test generation, Orhan favours predictable and inspectable output over claims of complete automation. Playwright provides the target framework for a TypeScript-based browser regression-testing workflow, while the generated test remains editable so engineers can make the necessary adjustments.

These decisions reflect a deliberate trade-off: a focused tool with clear boundaries can be more useful than a broader tool that implies it supports every possible browser interaction.

The Hardest Part: A Recording Is Not the Same as a Test

One of the most important lessons from developing QA Bug → Regression Test is that browser actions alone do not provide everything needed for reliable automated testing.

A recording may accurately capture a sequence of interactions while still producing an unreliable test.

For example, a test might depend on a particular authentication state, user permission, backend record, or initial application configuration. Those conditions may not be recreated by recording clicks and form inputs.

Locators introduce another challenge. If several elements have similar labels or appearances, a generated selector may identify the wrong element. Timing-sensitive behaviour and interactions involving frames, popups, or new tabs can also be more complicated than straightforward actions in the main browser frame.

Orhan approaches these problems by separating the stages of the workflow:

Capture: What interactions and information did the browser session actually record?
Representation: How is that information stored and described within the session?
Generation: How does the extension transform the captured information into a report or test?
Validation: Does the resulting output accurately represent the original behaviour?

Separating these stages helps identify where information may have been lost or where an assumption has been introduced.

When a scenario cannot be represented reliably, Orhan prefers to expose the limitation or require manual adjustment rather than silently generate code that appears more complete than it is.

Meaningful assertions matter more than replaying clicks

A regression test should preserve the reason a bug matters, not merely reproduce the sequence of actions that preceded it.

Consider a form that accepts an invalid value when it should reject it. A recorded test might successfully fill out the form and click Submit. But unless the test checks the relevant validation behaviour, it has not established that the original problem is covered.

The same principle applies to locators, test data, and starting conditions. A reliable test must identify the intended elements, represent the expected behaviour, and run in the environment where it will be maintained.

Before incorporating a generated test into a suite, Orhan recommends reviewing:

Starting conditions: The base URL, user state, permissions, fixtures, and required test data.
Locators: Whether selectors consistently identify the intended elements.
Assertions: Whether they verify the behaviour that originally made the scenario a bug.
Test data and privacy: Whether placeholders have been replaced with safe values and real credentials have been excluded.
Execution and stability: Whether the test runs successfully in the intended project and remains stable when repeated.

Warnings, unsupported steps, skipped interactions, and data placeholders also deserve attention. Some scenarios will require additional setup or manual correction before the generated code becomes useful.

This approach recognises an important boundary in automation: reducing repetitive work is valuable, but hiding uncertainty can undermine the reliability of the entire testing process.

Privacy and Responsible Browser Recording

Recording browser activity introduces privacy considerations, particularly when sessions involve real users, accounts, or sensitive application data.

QA Bug applies rule-based redaction to recognised sensitive values and uses placeholders where appropriate. However, Orhan acknowledges that detection is not exhaustive. Screenshots and other captured information may still contain private data.

That limitation makes review an essential part of the workflow. Users should inspect both the generated report and the test draft before sharing them or incorporating them into a project.

Keeping the core workflow local reduces the need for a backend or cloud synchronisation to generate the artifacts, but it does not eliminate every privacy risk. Information can still be exposed when users export, copy, or share captured material.

The practical lesson is to treat generated reports and tests as material that requires inspection, not as automatically safe outputs.

For tools that operate inside a browser, permissions, data handling, and transparent limitations are part of product quality—not details that can be postponed until later.

Using AI Without Giving Up Engineering Judgment

Beyond test automation, Orhan uses ChatGPT, Claude, and Gemini to explore implementation options, clarify unfamiliar concepts, brainstorm solutions, draft documentation, and accelerate repetitive coding tasks.

He finds these tools particularly useful when developing an initial implementation or comparing different approaches.

However, he does not treat an AI-generated answer as proof that a solution is correct. Generated code still needs inspection, execution, and testing. An API can be used incorrectly, an edge case can be missed, or an implementation that appears reasonable in isolation may not fit the actual project.

For tasks involving precise behaviour, security, browser permissions, or test reliability, he relies on conventional engineering practices: examining the code, running the application, verifying the result, and reproducing failures.

His approach is straightforward: AI can accelerate development, but responsibility for technical decisions and validation remains with the developer.

The level of verification should also match the risk. A draft used for brainstorming does not require the same scrutiny as code intended to run in a production testing pipeline.

This principle connects his work on AI-assisted development with his work on QA automation. In both cases, a generated output is useful only when it can be evaluated against the actual requirements and supporting evidence.

MindContext: Making AI-Assisted Research More Organised

Orhan's interest in practical browser tools extends beyond software testing. Another Chrome extension he built, MindContext, addresses a different problem: managing research context across webpages and AI conversations.

Developers and researchers often collect information across browser tabs, notes, files, and conversations with different AI assistants. Over time, it becomes difficult to remember where a claim originated, why a source was useful, or which details should be reused in a new conversation.

Copying raw URLs into an AI assistant does not guarantee that the assistant will retrieve the correct information. A page may be inaccessible, dynamically rendered, behind authentication, or too large to process as expected. Even when some content is retrieved, important details can be missed and findings can become separated from their original sources.

Orhan's preferred alternative is to provide deliberately selected context: relevant notes, important excerpts, source URLs, the task's goal, and any constraints that matter.

MindContext is intended to help keep research notes and quotations connected to their original webpages and make relevant context easier to reuse across ChatGPT, Claude, and Gemini.

The approach is guided by four principles.

Source traceability keeps notes and quotations connected to the webpages they came from.

Purposeful organisation groups information by research project or task rather than allowing everything to accumulate in one large collection.

Controlled context transfer lets users choose which information is relevant to a particular AI conversation.

User control allows people to review the context before sending it, rather than automatically submitting information they have not checked.

These principles reflect the same concern that shapes his testing tools: information becomes more useful when its origin and limitations remain visible.

For AI-assisted research, organised context can improve the quality of the work, but it does not remove the need to verify sources or evaluate conclusions.

Evaluating Accuracy Through Evidence

Whether generating a regression test or organising research for an AI assistant, Orhan starts by identifying what the output is supposed to establish.

For QA Bug, the questions are practical: Does the report accurately describe the observed behaviour? Does the generated test verify the expected result? Are any interactions missing or unsupported? Have sensitive values been handled appropriately?

For AI-assisted research, the equivalent process involves checking factual claims against relevant sources, revisiting original quotations, and distinguishing evidence from interpretation.

Fluent wording is not a substitute for proof. If a claim cannot be supported by the available information, it should not be treated as established merely because an AI assistant presents it confidently.

This approach also reinforces the importance of traceability. A useful report should help someone return to the original reproduction, while a useful research note should help someone return to the original source.

In both situations, the goal is not to eliminate human review. It is to make that review more focused and effective.

Lessons From Building Focused Developer Tools

One lesson has had a particularly strong influence on Orhan's approach to software development: a useful tool does not need to automate every possible case.

A tool can deliver value by handling a clearly defined set of scenarios, explaining its limitations, and helping users complete a task more quickly or consistently.

That principle has shaped how he positions QA Bug. Rather than presenting generated code as a guaranteed, ready-to-run solution, he treats it as an editable draft that needs review.

It also influences how he thinks about product development more broadly. The implementation should be clear about what it captures, what it generates, and what the user must still verify.

For developers building similar tools, his experience highlights several practical lessons:

Start with a specific, repetitive problem rather than attempting to automate an entire workflow at once.
Preserve the context needed to understand and reproduce the original problem.
Make generated outputs editable and inspectable.
Distinguish recorded actions from verified behaviour.
Surface unsupported cases and uncertainty instead of hiding them.
Treat privacy and data handling as part of the workflow.
Match the depth of validation to the consequences of an error.

These are not arguments against automation. They are reasons to design automation around the actual needs of the people using it.

Advice for QA Engineers Moving Toward Automation

For QA engineers who want to move beyond manual bug reproduction, Orhan recommends starting with a small, repeatable scenario rather than trying to automate an entire application immediately.

When a bug is discovered, the first step is to document the conditions required to reproduce it. The expected and actual results should be clear, and the engineer should identify the assertion that would demonstrate the failure.

From there, the next steps are to build a focused regression test, verify its locators and test data, and run it in the environment where it will be maintained.

This sequence keeps the test grounded in an understandable problem.

Automation works best when the underlying test case is clear. It cannot replace knowing what should happen or why a particular behaviour is incorrect.

The same advice applies when AI tools are involved. Generated code can provide a useful starting point, but the engineer must still determine whether the implementation is correct, whether the test proves the intended behaviour, and whether the result is reliable enough for its intended use.

What's Next: Less Repetitive QA and Better Engineering Workflows

Orhan remains interested in making the transition from manual QA to automation less repetitive, improving how browser interactions are captured and represented, and handling scenarios that require more context than a simple sequence of actions.

He also wants to continue exploring tools that combine browser workflows with practical AI-assisted development and research.

His broader goal is to build focused tools that solve real problems in everyday engineering work, learn from people using them, and improve them through concrete feedback.

As an independent developer, he sees each project as an opportunity to learn. Publishing tools and discussing their limitations openly are part of that process.

His work offers a useful perspective on the relationship between automation and engineering judgment. The best tools do not necessarily remove every manual step. They preserve useful information, reduce unnecessary repetition, and help people make better-informed decisions.

For QA engineers, that means turning bug reproduction into a stronger foundation for regression testing. For developers using AI, it means organising context and checking evidence rather than relying on plausible-sounding answers.

Across both areas, Orhan's guiding principle remains consistent: automation should make work more efficient without making the results harder to trust.

About Orhan Eren Kara

Orhan Eren Kara is a QA engineer and independent Chrome extension developer based in Türkiye. He graduated in Management Information Systems from İzmir Democracy University in 2025 and works with software testing, test automation, browser workflows, and AI-assisted tools.

His projects include QA Bug → Regression Test, which helps turn recorded browser interactions into Markdown bug reports and editable Playwright TypeScript test drafts, and MindContext, which helps organise research notes and quotations connected to their original webpages for reuse across AI assistants.

CoderLegion: Orhan Eren Kara
Featured project: QA Bug → Regression Test on the Chrome Web Store

Developer Stories is a CoderLegion interview series highlighting the people, ideas, projects, and engineering decisions behind software development.

From Real Customer Failures to Reliable AI Agents: How Yaoshen Luo Approaches Agent Evaluation

Tue, 27 Oct 2026 07:56:42 +0000

Interviewed by: Mehadi Hasan

Developer: Yaoshen Luo

Role: AI Agent Evaluation and Benchmarking Specialist

Location: Shanghai, China

Website: aiagentbenchmark.com

AI agents are becoming increasingly capable, but capability alone does not guarantee that they will perform reliably in real-world environments. An agent may pass a benchmark, use tools correctly, and produce convincing answers, yet still fail when confronted with ambiguous customer requests, changing conditions, or workflows that require judgment.

For Yaoshen Luo, understanding these gaps has become a central part of his work. His approach to AI agent evaluation is grounded in practical experience: testing systems, investigating failures, working with real customer data, and turning difficult-to-reproduce problems into repeatable benchmarks.

His perspective is shaped by a career that began with traditional software engineering and expanded through mobile internet, location-based services, machine learning, and modern AI applications. Along the way, he has learned that evaluating an AI agent requires more than measuring whether it can complete an isolated task. It requires understanding what successful performance actually means to the people who depend on it.

In this CoderLegion Developer Story, Yaoshen shares how his engineering background influenced his approach to AI, why real customer failures can reveal weaknesses that conventional benchmarks miss, and what developers need to reconsider as AI agents move into increasingly complex workflows.

From a Pascal Program to Applied AI

Yaoshen's interest in programming began when he was just ten years old. During a computer class, he followed his teacher's instructions to write a Pascal program that used a for loop to calculate the sum of the numbers from 1 to 100.

That small exercise sparked an interest that eventually shaped his academic and professional direction. When it came time to choose a university major, he decided to study software engineering.

After graduating in 2014, he joined Baidu and experienced China's rapid mobile internet expansion firsthand. He worked on Baidu Maps, a product comparable to Google Maps in China, where practical business requirements gave him opportunities to work across front-end engineering, back-end engineering, and algorithms.

This exposure to different areas of engineering helped him develop a broader understanding of how software systems operate beyond individual components.

His work also coincided with the growth of big data in location-based services. During this period, he began studying machine learning through Andrew Ng's courses on Coursera and found opportunities to apply what he learned to real map-related problems, including navigation and positioning.

His entry into AI was therefore not driven by theory alone. It emerged from the combination of technical curiosity, changing technology, and the practical problems he encountered at work.

That experience continues to influence how he approaches AI today: the value of a technology becomes clearer when it is tested against the demands of an actual business problem.

Discovering the Potential of AI Agents

When large language models first began attracting widespread attention, Yaoshen was not immediately convinced that they would fundamentally change software development.

He initially found the technology relatively slow and wondered whether ChatGPT might follow the trajectory of earlier applications built around language models.

His perspective changed in 2024, when he began using AI coding tools, particularly Cursor and Windsurf.

Through hands-on experimentation, he started to believe that the way software was developed could change fundamentally within six months to a year.

Rather than relying on predictions, he chose to learn by using the tools himself. He was willing to pay for tools that worked well because he wanted to understand their capabilities through direct experience.

This early-adopter mindset also shaped his later interest in evaluation. As AI systems became more capable, it became increasingly important to distinguish between what they could demonstrate in a controlled test and what they could reliably accomplish in practice.

Building or using an AI agent can reveal its potential. Evaluating it properly requires a different discipline: identifying where it fails, understanding why those failures occur, and determining whether the system is useful enough for its intended purpose.

Why Traditional AI Benchmarks Can Miss the Real Problem

A benchmark provides a structured way to assess an AI system. It defines tasks, establishes evaluation criteria, and measures performance against expected outcomes.

However, Yaoshen believes many benchmarks simplify the conditions under which real work happens.

He illustrates this through his investigation of procurement agents. When he discussed automation with procurement professionals, he found that purchasing highly standardized products could be relatively suitable for agents.

But procurement involves more than selecting a product from a list. Supplier relationships introduce additional considerations, and the appropriate decision can depend on the situation.

A benchmark might treat a task as though there were one correct answer. Actual procurement work may allow several reasonable choices, each appropriate under different circumstances.

This distinction matters because a technically correct answer does not necessarily represent a good business decision.

The same issue appears across many knowledge-work workflows. Tasks that look straightforward when isolated can become much more complicated when they involve incomplete information, competing priorities, or decisions that depend on context.

For Yaoshen, the challenge is not simply to create harder benchmarks. It is to design evaluations that reflect the real structure of the work an agent is expected to perform.

When an Agent's Capabilities Do Not Translate Into Product Success

One of Yaoshen's important learning experiences came from an agent developed for an elderly-care project associated with a foundation.

The agent was designed to process follow-up conversations with elderly people and generate answers to a questionnaire.

Working with this project helped him recognise a distinction that remains central to his thinking: an agent's capabilities are not the same as successful product deployment or successful business results.

A system may demonstrate that it can perform a particular function without establishing that it can deliver the expected outcome consistently in a real setting.

Deployment introduces additional considerations. The surrounding workflow, the quality of the available information, the cost of operating the system, and the consequences of an incorrect result can all influence whether the solution is useful.

This experience reinforced the importance of evaluating AI agents in the context of the problems they are meant to solve.

The question is not merely whether an agent can generate an answer. It is whether the complete system can perform its intended job under the conditions that matter to its users.

Building a Voice Agent Benchmark From Real Customer Failures

One of Yaoshen's clearest examples of practical evaluation comes from his work on a voice agent.

When he first took over the project, he was not yet fully familiar with how its relevant functions had been implemented. He had tested the system himself, and its voiceprint and speaker-recognition performance appeared promising.

The turning point came through customer feedback.

The customer explained that several people would deliberately interrupt one another during conversations. They would also ask questions about the conversation's history, such as who had said what.

These scenarios exposed a gap between the system's apparent performance during testing and the customer's expectations for real conversations.

Yaoshen's first objective was straightforward: reproduce the customer's problem.

He did not begin by designing a comprehensive benchmark or searching for an existing dataset. Instead, he focused on making one specific failure repeatable.

Once he could reproduce the interaction, he had a runnable example of the problem. That individual case then became the starting point for a repeatable annotation tool, and the benchmark developed from there.

Importantly, he did not rely on an existing speaker-recognition benchmark dataset. He built the benchmark by annotating real service data.

This process reflects a practical principle in his work: a useful benchmark can begin with a single well-understood failure. The important step is to transform that failure into something that can be reproduced, examined, and tested repeatedly.

Turning Real-World Failures Into Measurable Tests

For Yaoshen, the process of building a benchmark does not necessarily begin with a fully defined evaluation framework.

Instead, it begins with repeated interaction with the system.

He uses the relevant functionality until he develops a better understanding of how it behaves. Then he asks two fundamental questions: Does the information needed to evaluate the problem actually exist, and what elements are involved?

Once the main elements become clearer, he begins designing the annotation process.

He starts by performing the annotation manually. This gives him a more concrete understanding of the information an evaluator must examine and the judgments they must make.

Only after that does he consider turning the process into a batch-processing tool.

The goal is to help an operator focus on a group of tasks and annotate them carefully within a concentrated period of time.

He then implements the tool and creates a small dataset. He typically annotates the initial samples himself and, when possible, asks other people to annotate similar examples.

Comparing those annotations can reveal disagreements and contradictions.

Rather than treating every disagreement as a simple annotation mistake, Yaoshen sees it as a signal that the underlying problem may need further investigation. The evaluation criteria might be unclear, the task may involve ambiguity, or the annotators may have different interpretations of the same evidence.

Resolving these differences helps establish clearer standards. Once the main disagreements have been addressed, he uses real daily data logs for annotation.

The resulting process can be understood as a progression:

Reproduce a real failure.
Identify the information needed to evaluate it.
Define the annotation process manually.
Build a tool that supports repeatable annotation.
Compare annotations and investigate disagreements.
Refine the evaluation criteria.
Annotate real operational data.

The key is that the benchmark develops alongside the team's understanding of the problem. It is not simply a collection of predefined tests; it is a structured way to preserve and investigate failures discovered through actual use.

Evaluating More Than Correct Answers

Correctness is important, but Yaoshen pays particular attention to two additional aspects of agent behaviour.

The first is the consistency between an agent's words and its actions.

An agent might explain that it is going to perform an operation, but the important question is whether its subsequent actions actually align with that explanation.

The second is how the agent responds when an operation fails.

Does it simply repeat the same unsuccessful action, or does it change its approach and attempt a different method?

These behaviours matter because real workflows rarely consist entirely of successful, straightforward steps. Agents must often respond to unexpected results and decide what to do next.

For Yaoshen, these observations are also relevant to management. Evaluating an agent involves examining not only the final result but also the behaviour that leads to it.

This broader perspective can help reveal weaknesses that a simple success-or-failure score might overlook.

Why Good Evaluation Still Does Not Guarantee Production Success

In his writing about AI agent evaluation, Yaoshen has explored why promising evaluation results do not automatically translate into successful production deployment.

He identifies two major differences between controlled evaluation environments and real-world operation.

The evaluation may simplify the actual problem

Evaluation tasks are often abstracted to make them easier to define and measure. In doing so, they may exclude factors that determine whether the agent can solve the complete business problem.

Even when an agent successfully completes a task, the result may not be useful if the cost of achieving it is unacceptable to the customer.

This introduces an economic dimension to evaluation. A system's technical ability must be considered alongside the resources required to use it and the value of the outcome it produces.

The operating environment continues to change

Real-world systems do not necessarily operate under the same conditions indefinitely.

The surrounding environment can change, and those changes may affect an agent's performance over time. Yaoshen notes that there are still important unanswered questions about how such changes influence the sustainability of an agent's performance.

This makes ongoing evaluation particularly important.

A benchmark result describes performance under particular conditions. It does not, by itself, establish that the system will continue to perform well as the environment evolves.

For developers, the implication is clear: evaluation should not be treated as a one-time milestone that permanently establishes an agent's reliability.

What a Real Issue-Resolution Bot Revealed

Another important example comes from an issue bot Yaoshen has been working on for a voice-agent product.

The bot helps identify issues based on customer feedback. When customers report a problem, their descriptions are often vague.

A customer might simply say, "The audio playback was stuttering."

To investigate the issue, the bot must use the customer's text, images, and videos as context, reason about the available evidence, and repeatedly use tools to identify the relevant problem.

During traditional capability testing, the bot performed well on tasks such as demonstrating its ability to use tools.

But when it was applied to real data, the results exposed weaknesses that those tests had not captured.

A single set of clues could correspond to multiple possible cases. The bot might miss relevant possibilities or select the wrong issue.

These failures mattered because identifying the correct issue was the actual business objective. Demonstrating tool use was only one part of the larger task.

Yaoshen initially assumed that once the bot had located an issue successfully several times, it would be able to handle future cases in a similar way.

That assumption proved incorrect.

When the system was exposed to a broader range of real cases, different execution failures appeared at different stages. It could not identify every issue as expected.

The response was not simply to keep testing the same capability. He had to redesign the business workflow, introduce human fallback mechanisms, and add newly discovered problems to the benchmark so that they could guide further development.

This example highlights a recurring theme in his work: successful demonstrations do not establish that a system can handle the full range of cases it will encounter.

Real data can expose combinations of clues, ambiguities, and failure patterns that a carefully controlled test may never represent.

Making Real Data Part of the Benchmark

The issue-resolution bot also illustrates why Yaoshen believes the real data environment should become part of the evaluation process.

Traditional benchmarks can be useful for testing specific capabilities, but the data encountered in actual workflows may be much more varied.

For an issue-resolution system, a task might require interpreting an incomplete description, examining several pieces of evidence, considering multiple possible cases, and using tools to narrow down the answer.

A test that checks only whether the system can call a tool does not adequately represent that full process.

Yaoshen argues that real-world data can be a unique asset for each company. It reflects the specific problems its customers encounter, the information available to investigate those problems, and the practical requirements of its business.

Turning that data into a benchmark creates a way to test the system against problems that genuinely matter.

Customer behaviour plays a central role in this process. In Yaoshen's view, customers are generally the most important validators because their behaviour, within a particular context, helps reveal whether a problem has actually been solved.

A benchmark formalises those behaviours so they can be tested repeatedly.

The objective is not to replace every traditional benchmark. It is to connect capability testing with the real conditions under which an agent must deliver value.

Defining When an Agent Is Ready for Production

If real-world failures are inevitable, how should developers decide when an AI agent is good enough to deploy?

Yaoshen believes there is a threshold at which an agent becomes suitable for production. At the current stage of AI development, he suggests that the performance of human workers already performing the same job can provide a useful reference point.

He also draws on the economic concept of marginal benefit.

The practical question is whether the additional value provided by the agent justifies the remaining cost and risk.

This perspective avoids treating production readiness as a purely technical milestone.

An agent might perform impressively on a benchmark but still be unsuitable for a particular workflow if its mistakes are too costly, its operating expenses are too high, or its performance does not meet the requirements of the business.

Conversely, an agent does not necessarily need to eliminate every failure to be useful. Its suitability depends on the work being performed and the balance between the value it creates and the risks that remain.

This is why Yaoshen emphasises appropriate evaluation design, economic considerations, and stable, repeatable evaluation as three important directions for making agent evaluation practical.

AI Agents Require Continuous Evaluation, Not One-Time Testing

Traditional deterministic software is generally built around behaviour that developers expect to remain relatively predictable under defined conditions. AI-agent software presents a different challenge because its observed effectiveness must be monitored continuously.

Yaoshen considers the ability to understand how well an agent is performing over time one of the most important aspects of working with this type of software.

Deployment may also reveal capabilities that developers did not initially anticipate. Once an agent begins operating in real workflows, teams may discover that it can perform a broader range of tasks than expected.

That discovery can create opportunities, but it can also mean that existing workflows need to change fundamentally.

Developers therefore need to think beyond the initial implementation. Building an agent may be the beginning of an ongoing process in which the system, its evaluation criteria, and the surrounding business workflow evolve together.

This approach also helps explain why new failures should be added back into the benchmark. Every newly discovered problem can become a test case that helps the team understand the system more deeply and assess whether subsequent changes have improved its behaviour.

Human Responsibility Remains Central

As AI agents become more capable, they are also being asked to take on more work. But Yaoshen makes an important distinction: greater capability does not mean that responsibility has been transferred to the agent.

People remain responsible for the systems they build and deploy. In some circumstances, the amount of responsibility carried by individuals may even increase as agents take on more consequential tasks.

He also believes that the responsibility assigned to any individual should have limits.

This observation has practical implications for agent evaluation. If an agent is going to perform work previously handled by people, developers must understand not only whether it can complete that work but also where human judgment, oversight, or intervention remains necessary.

Human fallback mechanisms, such as those introduced in his issue-resolution workflow, are one way to account for cases the agent cannot reliably handle.

For Yaoshen, the future of AI agent evaluation will depend in part on designing appropriate evaluations, understanding the economic realities of deployment, and ensuring that performance can be assessed consistently and repeatedly.

These considerations become especially important as agents move into knowledge-work roles that have traditionally been performed by people without technical backgrounds.

Advice for Developers: Start With a Problem That Matters

For developers who want to begin building AI agents, Yaoshen recommends a practical starting point: build an agent on top of an existing large language model.

He believes that direct experimentation is the fastest way to develop a practical understanding of the technology.

When the goal is learning, he follows a principle that has also shaped his benchmark-building process: do the work manually first.

Before automating a task, it is useful to understand what the task actually involves. The experience of doing it yourself can reveal details and requirements that might otherwise remain hidden.

He is currently working on a series of hands-on projects built around this approach.

One example involves a friend who works in human resources. He coached her as she explored building something around a workflow that had been troubling her for a long time: reconciling accounts with the finance department and distributing monthly salaries to employees.

Her project focused on learning through vibe coding, but Yaoshen sees a broader lesson in the experience.

A real problem provides a concrete reason to learn. It gives the developer a goal, a way to recognise progress, and a practical environment in which to discover what the technology can and cannot do.

His advice is to avoid waiting until you feel sufficiently prepared to begin.

As he explains, people learn fastest through interaction with their environment. Building something around a genuine problem creates opportunities to test assumptions, encounter limitations, and develop a deeper understanding through practice.

That principle applies to both AI development and evaluation. The process becomes much more meaningful when learning is grounded in real work rather than hypothetical tasks alone.

Conclusion: Reliability Begins With Understanding Failure

Yaoshen Luo's approach to AI agent evaluation is shaped by a simple but demanding idea: a system must be judged against the work it is expected to perform.

His engineering background, experience with applied machine learning, and hands-on work with AI agents have led him to focus on the gap between demonstrated capability and practical reliability.

By reproducing real customer failures, building annotation tools, formalising evaluation criteria, and feeding newly discovered problems back into benchmarks, he has developed a practical way to investigate that gap.

His examples also show why no single benchmark score can tell the entire story. Real workflows introduce ambiguity, economic constraints, changing conditions, and failure patterns that may not appear in controlled tests.

For developers building AI agents, the lesson is to start with real problems, investigate failures carefully, and treat evaluation as an ongoing part of system development.

The objective is not simply to build agents that can succeed on a test. It is to understand when they work, where they fail, what their performance is worth, and when people still need to intervene.

As AI agents take on more responsibility in real-world workflows, that understanding will be essential to making them genuinely useful.

About Yaoshen Luo

Yaoshen Luo is a software engineer and AI practitioner based in Shanghai, China, whose work focuses on AI agent benchmarks, evaluation, and real-world workflows. He graduated in software engineering and joined Baidu in 2014, where he worked on Baidu Maps across front-end engineering, back-end engineering, algorithms, and location-based services.

His experience with applied machine learning and modern AI coding tools has informed his work on evaluating AI agents through real customer data, reproducible failures, and practical business requirements. His CoderLegion articles include How I Built a Voice Agent Benchmark from Real Customer Failures, How to Evaluate AI Agent with Benchmark in Practice?, Why AI Agent Benchmark Matters, and Why Good Agent Evaluation Wasn’t Enough for Production.

CoderLegion: Yaoshen Luo
Website: aiagentbenchmark.com

Developer Stories is a CoderLegion interview series highlighting the people, ideas, projects, and engineering decisions behind software development.

How Built Galaxy Movies: From a Side Project to a Cost-Conscious Streaming Discovery Platform

Mon, 26 Oct 2026 09:08:18 +0000

Interviewed by: Mehadi Hasan

Developer: Nick Fasulo

Role: Software Developer

Location: Los Angeles, California

Featured Project: Galaxy Movies

A Side Project That Became Something More

Building a software product is not just about getting features to work. It is also about keeping infrastructure costs under control, protecting external API quotas, making reliable data available, and helping users discover something genuinely useful.

For Nick Fasulo, these challenges became central to the development of Galaxy Movies, an open-source movie and TV discovery application designed to answer two practical questions: What can I watch on the streaming services I already pay for, and what should I watch before it disappears?

What began as a personal experiment with infinite scrolling eventually evolved into a production application with streaming availability data, watchlists, email alerts, an AI chatbot, and a paid Plus tier. Much of the application's current functionality was built during an intensive month of development after Nick returned to a project that had remained dormant for years.

His journey offers a useful perspective on independent software development, particularly for developers working with limited budgets. Rather than treating cost, infrastructure, and technical constraints as problems to solve later, Nick has made them part of the application's design from the beginning.

An Unconventional Path Into Software Development

Nick did not follow the traditional computer science degree route. His formal software development training began with Fullstack Academy's Software Engineering Immersive in 2021, followed by General Assembly's Web Development Immersive in 2023.

He gained practical experience through a React frontend internship at Yup.io and a Shopify development internship at Stasis, where he worked on real storefronts. These early experiences helped him move from learning web development concepts to building software that people could actually use.

His subsequent career took him beyond conventional frontend development. At UBS, he worked as a data specialist in SQL-heavy financial systems. Later, at Bright Data, he became a technical support manager responsible for handling complex issues involving web scraping APIs and proxy networks.

Although the move into technical support might have appeared to be a detour, it became one of the most influential parts of his engineering journey.

At Bright Data, Nick spent his time investigating HTTP requests, reproducing integration bugs, and understanding how traffic moves through the web. That experience gave him a practical understanding of API behavior, rate limits, and crawler defenses.

Those lessons would become especially valuable when Galaxy Movies began handling real traffic and faced abuse against its API endpoints.

His experience illustrates how skills gained outside a traditional software engineering role can translate directly into stronger engineering decisions. Understanding how systems behave under difficult conditions can be just as important as knowing how to build their features.

Galaxy Movies: Solving a Problem Every Streaming Viewer Understands

Galaxy Movies began approximately four years ago as a small experiment. Nick wanted to explore infinite scrolling, and a movie catalog provided a convenient way to experiment with the interface.

The project eventually stalled as other responsibilities took priority.

When Nick returned to it, he had a clearer idea of what the application could become. He was frustrated by the difficulty of tracking movies across multiple streaming platforms. Titles appeared on one service, disappeared from another, and often left viewers unaware that something they wanted to watch was about to become unavailable.

Instead of building another movie catalog, he decided to focus on streaming availability and the changes happening across different services.

Galaxy Movies now tracks additions, removals, and upcoming departures across 16 US streaming services, with a growing catalog for India. It also provides provider-specific catalogs, genre browsing, title search, streaming availability information, and monthly lists of arriving and departing content.

The underlying idea is straightforward: streaming availability should be useful, current data rather than a detail buried inside individual entertainment platforms.

The application brings that information together so viewers can make better decisions about what to watch and when to watch it.

Building a Connected Set of Features Around One Data Pipeline

Galaxy Movies includes several features designed to help users discover content and keep track of changes.

Users can browse catalogs by streaming provider and genre, search for titles, and check where movies and shows are available. They can also maintain watchlists and receive email alerts when titles become available on their selected services.

The application includes an AI chatbot for conversational movie discovery, an RSS feed covering streaming departures, shareable lists, and an embeddable widget that other websites can integrate into their pages.

A paid Plus tier adds additional alert types, including departure notifications, taste-based recommendations, and episode air dates. It also offers Trakt and Letterboxd synchronization and higher usage limits.

However, the feature Nick is most proud of is not a user interface or a standalone product capability. It is the streaming-changes engine that powers much of the application.

A scheduled job takes snapshots of each provider's catalog and calculates the differences between them. Those changes become the foundation for the site's departure pages, streaming availability indicators, alert matching, RSS feed, social cards, embedded widget, and even a Reddit trivia game.

This approach means that multiple features can rely on the same underlying dataset rather than implementing separate mechanisms for tracking availability.

As Nick explained in his interview, getting the data right once allows downstream features to benefit from that work.

For a small independent project, this is an important architectural choice. A reliable shared data pipeline can provide more long-term value than building each new feature as an isolated system.

Why Nick Moved From Vercel to Cloudflare Workers

As Galaxy Movies developed into a more substantial application, infrastructure costs became harder to ignore.

Nick had initially hosted the application on Vercel, but its pricing no longer made sense for a side project with growing usage. Cloudflare Workers had become a viable alternative for running Next.js through the OpenNext adapter, making migration possible.

The move involved more than transferring the main application. Nick migrated server-side rendering, scheduled jobs, incremental static regeneration, and dynamically generated Open Graph images.

The process exposed several runtime differences that required specific solutions.

Cloudflare Workers does not provide Node.js's TCP networking stack, so code that depends on it needs an alternative. Services such as Upstash, OpenAI, and Resend were easier to retain because they support REST APIs.

Other components required more substantial adjustments. Runtime WebAssembly and filesystem restrictions broke the Open Graph image route until Nick conditionally imported the edge-compatible build. Vercel cron jobs also needed to be replaced with a generated scheduled handler, while the incremental static regeneration cache required an R2 bucket.

These issues demonstrated that compatibility at the framework level does not guarantee that every part of an application will work in a different runtime.

One of Nick's most important lessons came from testing. Standard builds passed, but the application could still crash when executed inside the actual Workers environment.

He responded by changing the deployment checks. His CI process now boots the built application under the real Workers runtime and runs smoke tests before deployment.

That test has caught more problems than the other checks combined, according to Nick.

The experience reinforces a practical principle for developers adopting edge platforms: test the application in the environment where it will actually run, not just in a development environment that may behave differently.

Designing Around Free-Tier Limits Instead of Hoping for Low Bills

Nick's approach to infrastructure extends beyond choosing a cheaper hosting platform.

He prefers hard usage limits over unpredictable, open-ended pricing. A service that stops at a defined limit may be easier to manage than one that continues charging as traffic grows, especially when an independent project can receive unexpected bursts of requests.

Galaxy Movies uses several techniques to keep that risk manageable.

For Redis, where billing counts individual commands, including commands inside pipelines, Nick designed request paths to reduce unnecessary operations. The implementation uses hash storage instead of per-key fan-out, batches counters in memory before flushing them, and avoids casual reads on frequently accessed paths.

The application also tracks upstream API quota information from response headers. It begins refusing requests when usage reaches 90 percent of the available quota and suspends scheduled jobs at 99 percent, helping preserve the remaining capacity.

Nick has also built a private usage dashboard that aggregates worker, database, AI, and billing information. This gives him a way to monitor consumption rather than discovering unexpected costs after receiving a bill.

These decisions reflect a broader engineering philosophy: cost control is not simply a finance task. It influences how software should be designed.

For developers building side projects, this can mean thinking carefully about how often a request touches a database, whether a background job is necessary, how repeated work can be cached, and what happens when an endpoint receives far more traffic than expected.

Nick's experience suggests that these questions are worth addressing before a project becomes expensive to operate.

When API Abuse Becomes an Infrastructure Problem

One of the most difficult challenges Nick encountered was abuse against Galaxy Movies' API endpoints.

Some endpoints made requests to upstream providers, so excessive traffic could consume external API quotas and database resources. The incoming requests appeared to come from clean browser user agents and rotating IPv6 addresses, with traffic distributed across proxies.

Traditional per-IP rate limits were insufficient because the requests did not originate from a single stable address.

Nick drew directly on his experience at Bright Data to address the problem.

Rather than relying on one defensive measure, he introduced multiple layers of protection.

The application persists provider quota headers and stops making upstream calls when usage crosses the 90 percent threshold. A global hourly budget acts as a circuit breaker for the flood of requests.

When that budget is exceeded, each edge isolate denies requests for 60 seconds without contacting Redis. This detail matters because the system should not incur additional database costs merely to reject requests generated by the abuse itself.

A crawler policy at the CDN layer provides another level of protection.

The approach prioritizes keeping the application operational within its resource limits rather than assuming that more server capacity will solve every traffic problem.

For a small application running on free or low-cost infrastructure, protecting quotas and preventing unnecessary database operations can be just as important as protecting the application server.

Adding an AI Chatbot Without Sacrificing Accuracy

Galaxy Movies' AI chatbot emerged from the way people naturally search for entertainment.

A user might ask, "What's a good heist movie on Hulu?" or "Is The Bear still streaming?" These questions are conversational, and ordinary filters can make them awkward to express.

The chatbot lets users describe what they want in natural language. But implementing the feature introduced a significant technical challenge: keeping answers grounded in current streaming data.

A language model might confidently claim that a movie is available on a service even after the title has left its catalog. For a streaming discovery application, that kind of error undermines the product's main purpose.

Nick designed the chatbot so it does not rely on the model's memory to answer availability questions. Instead, it detects intent patterns, adds real provider data and the site's own streaming-change records to the context, and resolves titles mentioned in responses into actual links in the interface.

Responses stream token by token to make the interaction feel faster.

Cost and predictability also influenced the implementation. Since AI API calls represent a real expense for a hobby-budget application, Nick introduced layered rate limits and caching for repeated generations.

He deliberately kept some parts of intent detection simple. In his view, predictable behavior is more valuable than unnecessary cleverness for a feature that needs to return useful, trustworthy answers.

This is a useful distinction for developers building AI-powered products: a model can generate the response, but the application still needs to control which facts the model receives and how those facts are used.

Another interesting design decision involved the watchlist alert system.

Nick wanted users to receive notifications when a title on their watchlist became available on a selected streaming service. Building this feature around a conventional account system would have required passwords, sessions, and account recovery flows.

Instead, he used email confirmation as the starting point for establishing identity.

After confirmation, the application generates a synchronization key and returns it in the URL fragment. Because the fragment is handled by the browser rather than included in the HTTP request sent to the server, the key stays out of ordinary server request logs.

Devices store the key to synchronize watchlists, ratings, and streaming-service preferences. A device-linking flow allows users to add a second device, while a separate token supports unsubscribing from individual lists.

When Nick later introduced the paid tier, the existing email-and-key mechanism became the basis of the login experience. The payment provider also associates the email address at checkout, preventing a different payment email from unintentionally creating a separate account identity.

Nick acknowledges that this design would not necessarily be appropriate for an enterprise application. For a consumer product, however, it removes a major point of friction: requiring someone to create an account before they can use a useful feature.

The lesson is not that conventional authentication is unnecessary. It is that authentication and account infrastructure should match the product's actual requirements rather than being added automatically.

Launching the Product Was Only the Beginning

Galaxy Movies had technically existed for years, but its recent transformation was effectively a relaunch.

Nick brought the application to Product Hunt and several other launch platforms and directories. The experience reinforced an important distinction: shipping software and getting people to discover it are separate challenges.

At the time of the interview, Galaxy Movies had received roughly 160 organic clicks over the preceding 28 days, with most landing on a single title page that had gained search visibility.

The alert system and Plus tier were implemented and tested end to end, but the only accounts using them were Nick's own test accounts. Real customer conversion therefore remained unproven.

Rather than treating launch-day attention as the ultimate measure of progress, Nick focused on improving the application's organic search visibility.

His work included refining sitemaps so they contained only pages worth indexing, sending IndexNow notifications when catalogs changed, and using Google Search Console to understand which pages Google had actually discovered.

Early visitors also exposed issues that testing had missed, particularly region-specific streaming data and unusual title matches.

Those problems highlighted something that can be easy to overlook when building a technically sophisticated product: users primarily care whether the information is correct and useful.

A well-designed architecture matters, but stale availability information can quickly damage trust.

Nick's next priorities reflect this reality. Rather than adding features simply to expand the product, he plans to concentrate on distribution, monetization, and making the existing dataset more valuable.

That includes expanding affiliate opportunities, improving provider-specific discovery pages, reaching out to librarians and entertainment writers, and extending regional coverage beyond the United States and India.

What Nick Learned From Building on a Limited Budget

Across Galaxy Movies' development, several lessons emerge from Nick's experience.

First, keeping infrastructure costs low requires more than selecting free services. It requires understanding how the application consumes those services and designing around their limits.

Second, architectural decisions should remain reversible whenever possible. Nick keeps his previous hosting environment configured as a cold standby, so returning to it would require only a DNS change. This provides a practical fallback if the new deployment encounters serious problems.

Third, reliability work deserves attention early. The streaming-change engine, deployment smoke tests, usage dashboard, and working unsubscribe flows all protect important parts of the product. They may be less visible than new features, but their value compounds over time.

Finally, unfinished projects can still have potential. Galaxy Movies remained dormant for years before Nick returned to it and turned it into a more complete product during a focused month of development.

His advice to developers who feel overwhelmed by hosting costs, infrastructure choices, or the amount of work required to launch is straightforward: start with the smallest version that solves a real problem, use a familiar technology stack, and treat free-tier limits as part of the design specification.

Building within constraints can force developers to understand the real cost of each request, the purpose of every background job, and the trade-offs behind their infrastructure decisions.

Those habits remain useful even when a product grows beyond its initial budget.

Looking Ahead: Turning Streaming Changes Into a Valuable Dataset

Nick's longer-term ambitions for Galaxy Movies extend beyond becoming another place to browse movies and television shows.

Over the next year, he hopes to establish consistent revenue from the monetization options already wired into the application, bring the alert list closer to 1,000 subscribers, and improve the search visibility of its programmatically generated pages.

Over a three-to-five-year horizon, he sees potential in the accumulated history of streaming availability changes.

A growing record of what arrived, disappeared, and changed across streaming platforms could eventually become useful to studios, distributors, entertainment publications, and other organizations interested in understanding streaming catalogs.

That possibility depends on whether the dataset proves valuable to audiences beyond ordinary viewers. Nick is self-funding the project and intends to maintain that approach, with operating costs kept near zero.

The strategy connects the technical architecture to the business model. The same underlying data that powers the consumer application could also support feeds, widgets, and other distribution channels that bring the information to additional audiences.

Rather than relying exclusively on new features to attract users, Nick is building around a dataset that becomes more useful as its history grows.

The Bigger Lesson: Constraints Can Improve Engineering

Nick Fasulo's journey with Galaxy Movies demonstrates how a side project can evolve when a developer returns to it with a clearer problem to solve and a willingness to address the operational details.

His experience spans frontend development, SQL-heavy systems, API integrations, edge deployment, AI grounding, abuse mitigation, and product distribution. These are not separate concerns in an independent application. They interact, and decisions in one area can affect the reliability and cost of everything else.

His most important lesson is that constraints can lead to better engineering decisions.

A limited budget encourages careful resource usage. Strict quotas make request costs visible. Runtime differences expose assumptions hidden by familiar development environments. And real users reveal problems that internal tests cannot always anticipate.

As Nick puts it, unfinished projects are not necessarily dead projects. Sometimes the next step is not to find a better idea, a more expensive stack, or the perfect time to begin. It is to take an existing idea seriously, build the smallest useful version, and keep improving it based on what actually happens.

Galaxy Movies is still working toward its longer-term business goals, but its development already illustrates an approach that other independent developers can learn from: build around a real problem, keep the architecture practical, and make every technical decision earn its place.

About Nick Fasulo

Nick Fasulo is a software developer based in Los Angeles, California. His experience includes React and Shopify development, SQL-heavy financial systems, and technical support for web scraping APIs and proxy networks. His technical interests include Next.js, JavaScript, serverless infrastructure, and AI-powered applications.

He is the developer behind Galaxy Movies, an open-source movie and TV discovery application that tracks streaming availability, helps users manage watchlists, and provides alerts when titles become available or are about to leave a service.

CoderLegion: Nick Fasulo's profile
LinkedIn: Nicholas Fasulo
Featured project: Galaxy Movies

Developer Stories is a CoderLegion interview series highlighting the people, ideas, and engineering decisions behind software development.

Making the AI Agent Ecosystem Easier to Navigate, Compare, and Build With

Mon, 26 Oct 2026 08:58:09 +0000

Interviewed by: Mehadi Hasan

Developer: AgDex.ai Team

Role: AI Agent Ecosystem Researchers and Tool Curators

Website: AgDex.ai

From a Growing Collection of Tools to a Practical Engineering Resource

Building an AI agent involves much more than choosing a language model. Developers must also decide how to orchestrate tasks, manage memory, execute code safely, monitor performance, and control operational costs. With new frameworks and infrastructure tools appearing constantly, understanding how these components fit together can become a project of its own.

That challenge inspired the team behind AgDex.ai, a curated directory and knowledge hub for AI agent technologies. What began as a way to organize useful engineering resources has evolved into an effort to help developers evaluate tools, compare architectural approaches, and make better-informed decisions before committing to an implementation.

In its interview with CoderLegion, the AgDex.ai team explained how they approach tool curation, why architectural trade-offs matter more than popularity alone, and what developers should consider when building reliable AI agents. Their experience offers a practical perspective on an ecosystem where choosing the right combination of technologies can be just as important as choosing the model itself.

The Engineering Problem Behind AgDex.ai

AgDex.ai started with a problem the team encountered while building autonomous workflows: understanding how the different parts of an agent system fit together was taking too much time.

Orchestration frameworks, memory systems, sandboxed execution environments, and observability tools each solve different problems. However, selecting one component does not guarantee that it will work well with the rest of the stack.

As the team explained, developers regularly face questions such as how a framework handles cyclic state retries, how context grows over multiple turns, and whether a particular setup can be deployed in a self-hosted environment to satisfy privacy requirements.

These questions are difficult to answer by browsing isolated project repositories or reading product landing pages. A framework might look promising on its own but introduce unnecessary complexity when combined with a particular memory store or execution environment.

The team initially responded by creating a curated directory. They soon recognized that a list of links was not enough. Developers needed information about compatibility, deployment choices, operational costs, and architectural trade-offs.

That realization shaped AgDex.ai's direction: instead of simply helping people discover tools, the platform aims to help them understand which tools fit their requirements and why.

Helping Developers Choose an AI Agent Stack

One of AgDex.ai's central features is its 3-Step Stack Finder, designed to make initial architecture decisions more straightforward.

Rather than beginning with a long list of frameworks, developers select three things:

Workload: Enterprise RAG, coding agents, or multi-agent workflows.
Deployment preference: Managed cloud or self-hosted and sovereign infrastructure.
Primary constraint: Cost, data privacy, or latency.

Based on these selections, the finder recommends one of 18 reference architecture blueprints. Each recommendation includes an estimated total cost of ownership (TCO) breakdown, an architecture diagram, and a starter container template.

The goal is to give developers a more structured starting point. Instead of assembling a stack from unrelated recommendations, they can begin with an architecture aligned with their workload and constraints.

The platform also includes a feature called Copy AI Context. Developers can export a blueprint's dependency notes and configuration structure as plain text and provide that context to coding assistants such as Cursor, Claude Code, or Windsurf.

This connects research with implementation. Developers can use the selected architecture as a starting point when scaffolding a project, rather than manually translating information from multiple documentation pages into their coding workflow.

AgDex.ai also offers a Versus Arena for comparing tools and orchestration approaches. Together, these features reflect the team's broader objective: making technical evaluation more useful before developers invest time in a proof of concept.

Why an Open and Neutral Directory Matters

The AI agent ecosystem includes tools built around different models, programming languages, deployment environments, and commercial strategies. The AgDex.ai team believes developers should be able to choose among them without being pushed toward a single vendor's ecosystem.

They describe their approach using an analogy involving AI as a fundamental utility. If foundation models form the underlying infrastructure, then frameworks, memory engines, MCP servers, and sandboxes are the components developers connect to build applications.

From that perspective, an open ecosystem should let builders select the components that best suit their needs.

AgDex.ai approaches this principle in three ways.

First, its tool profiles aim to document meaningful pros and cons rather than relying on promotional language. These include considerations such as token overhead, learning curves, and the complexity of self-hosting.

Second, its reference blueprints, comparison tables, and architectural notes are freely accessible without logins or paywalls.

Third, the platform maintains machine-readable technical resources through its llms.txt and llms-full.txt files. These provide structured information that AI assistants and search systems can use when discovering technical resources.

The common thread is accessibility. Developers should have enough information to evaluate their options, including the limitations of a tool, instead of relying exclusively on popularity or marketing claims.

Curating More Than 730 AI Agent Tools

According to the team's interview responses, AgDex.ai indexes more than 738 active tools from a broader tracking backlog.

The team does not aim to include every project that happens to use an AI model. Instead, it looks for components with a meaningful architectural role, such as deterministic execution, persistent state, standard protocols, isolated environments, or established domain workflows.

This distinction helps keep the directory focused on the infrastructure developers need to build agent systems.

Each tool profile is organized around four dimensions:

Deployment type: Open-source and self-hosted, managed cloud, or local-first.
Pricing and cost model: How the tool is priced and what developers should consider when evaluating its cost.
Target audience and maturity: Who the tool is intended for and how mature it is.
Ecosystem compatibility: Which other components it works best with.

The platform's Quick View Drawer is another practical part of this approach. When developers open a tool card, they can inspect its information without immediately leaving the current page. A simple history stack lets them return to previously viewed tools while exploring related dependencies.

For example, someone evaluating an agent framework can inspect an associated vector store or observability tool and follow the relationships between components without repeatedly switching browser tabs.

AgDex.ai organizes the ecosystem into eight categories: Agent Orchestration Frameworks, Memory & Vector Systems, Coding & SWE Agents, MCP & Tool Integrations, Compute & Sandboxes, Observability & Evals, Voice & Multimodal Agents, and Enterprise Workflows.

It also provides seven role-specific portals for developers, teachers, doctors, lawyers, marketers, designers, and researchers. These are intended to help people explore tools relevant to their work without requiring them to navigate every low-level infrastructure category.

Categorization itself is not always straightforward. The team highlights MCP servers and unified AI databases as particularly challenging examples. MCP servers can bridge APIs, local runtimes, and permission gateways, while databases such as PostgreSQL with pgvector combine transactional data with vector embeddings. In cases like these, a rigid single-category system can obscure how a component is actually used.

Flexible multi-tagging helps the directory represent those overlapping roles more accurately.

Keeping Technical Information Reliable

AI tooling changes quickly. Framework APIs evolve, model providers adjust pricing, and services sometimes deprecate endpoints. A directory that was accurate several months ago can become misleading if its information is not maintained.

The AgDex.ai team uses a combination of community feedback, architectural analysis, and automated checks to address this problem.

Project maintainers and community readers send updates through GitHub or email. The team verifies reported changes against upstream documentation before updating its records.

It also distinguishes between details that change frequently and architectural concepts that tend to remain more stable. Pricing and API versions may need frequent updates, while fundamental differences between directed acyclic graphs, cyclic graphs, and actor-based models can remain useful for longer.

Automated scripts periodically check link health, identify schema inconsistencies, and verify alignment across translated content.

These practices do not eliminate the difficulty of maintaining a fast-changing catalog, but they establish a process for keeping information useful rather than treating publication as a one-time task.

Making the Knowledge Hub Accessible Across Languages

AgDex.ai uses a lightweight static architecture built with vanilla HTML, Tailwind CSS, and client-side JavaScript. According to the team, this approach helps keep pages fast, reduces hosting overhead, and supports access across different network conditions.

The platform's Multi-Language Parity Pipeline is intended to maintain consistent structural and technical coverage across five languages: English, Spanish, German, Japanese, and Arabic.

The team's stated coverage includes more than 738 tools, 74 technical guides, and 18 Stack Finder presets.

Maintaining technical accuracy across languages requires more than translating every word. Code snippets, API names, and standard acronyms such as RAG, MCP, OLTP, and GDPR remain unchanged, while explanatory content is localized.

Arabic introduces another implementation consideration. The interface uses a right-to-left layout, but code blocks, JSON configurations, and architecture diagrams remain left-to-right so that technical syntax stays readable.

The team also sees localization as a way to address different engineering priorities. Some European teams may prioritize self-hosting and GDPR compliance, while developers in Japan may rely on local technical communities. Organizations in the Middle East may pay particular attention to data residency requirements.

By making technical resources available in multiple languages, AgDex.ai aims to help developers evaluate their options in a context that is more accessible to them.

Choosing Frameworks Based on the Problem

One of the team's clearest recommendations is to start with a workflow's control-flow and state requirements rather than choosing a framework based on GitHub stars.

Different frameworks address different needs, and the most elaborate option is not always the best one.

For a straightforward pipeline involving two or three sequential steps, a simple script or a lightweight SDK such as Vercel AI SDK may be sufficient. Introducing a complex orchestration framework into such a workflow can add abstractions that make debugging harder without delivering much benefit.

Cyclic state-machine frameworks such as LangGraph become more relevant when a workflow needs persistent state, iterative retries, or human approval steps. A coding agent that generates code, runs tests, examines error output, and attempts a correction is one example of a workflow that benefits from explicit state and control flow.

Role-based orchestrators such as CrewAI may be appropriate when a task naturally divides into specialist responsibilities, such as research, analysis, and writing. AutoGen or AG2 can suit workloads that benefit from open-ended discussion, multiple perspectives, or simulations between agents.

For teams working primarily in TypeScript and Next.js, the team also points to TypeScript-native frameworks such as Mastra as an option for typed workflows and native Node.js execution.

The underlying lesson is that frameworks should follow requirements, not the other way around. Developers should understand what their application needs before adding infrastructure to support it.

Memory Architecture: Why a Large Context Window Is Not Enough

Memory is another area where the team urges developers to think beyond the model's capabilities.

A large context window can accommodate substantial amounts of information, but repeatedly sending entire conversation histories and static documentation to a model can increase token consumption and make instructions harder to follow.

The cost problem becomes more noticeable in multi-step workflows. An agent that performs 10 to 15 tool calls may repeatedly process growing amounts of context, especially when tool outputs change between steps.

The team recommends thinking about memory in distinct layers:

Working memory: Information needed for the immediate task.
Episodic memory: Summaries of previous interactions, potentially managed with tools such as Mem0 or Zep.
Semantic memory: Domain knowledge retrieved when needed through techniques such as retrieval-augmented generation (RAG).

Separating these responsibilities can help avoid carrying unnecessary information through every step of a workflow.

The choice of storage infrastructure also depends on the workload. For collections containing fewer than a few hundred thousand vectors, the team suggests that pgvector or pgvectorscale may be sufficient when an application already uses PostgreSQL.

Keeping embeddings alongside relational data can simplify infrastructure and allow them to participate in the same transaction model.

Dedicated vector engines such as Qdrant, Milvus, or Pinecone become more attractive when an application requires capabilities such as high-throughput hybrid search, very low tail latency across millions of records, or scalable multi-tenant isolation.

Rather than treating one database choice as universally correct, the team emphasizes matching the memory layer to actual scale and performance requirements.

When Multiple Agents Make Sense

Multi-agent systems can help manage complexity, but the team cautions against adopting them simply because several agents appear more sophisticated than one.

A single agent may become less reliable when it must choose among too many tool definitions. With 15 to 20 or more tools available at once, instructions and tool descriptions can compete for attention, potentially reducing tool-selection accuracy.

Splitting work across specialized agents can help in specific situations.

One is context and tool isolation. A specialized agent receives only the narrow context and small set of tools needed for its task, then returns a compact result to the main orchestrator.

Another is independent validation. A maker-checker pattern separates the agent generating code or queries from another agent responsible for evaluating the result against safety or formatting criteria before execution.

Cost-aware routing is a third use case. A capable reasoning model can handle high-level planning while faster, lower-cost, or locally hosted models perform routine parsing, extraction, or validation.

These approaches introduce their own coordination overhead, so the benefit depends on whether specialization genuinely improves reliability, context management, or cost efficiency.

The team recommends using deterministic code for predictable logic and reserving model reasoning for tasks that require flexible interpretation.

Security Must Be Part of the Architecture

The risks associated with AI agents change significantly when they move from generating text to executing commands, querying databases, or interacting with external services.

The AgDex.ai team identifies three areas that deserve particular attention.

Indirect prompt injection occurs when an agent encounters malicious instructions embedded in external content, such as a webpage, email, or support ticket. If that untrusted content influences an agent with broad execution privileges, it can affect how the agent uses its tools.

Scoped permissions and human approval help limit the consequences of compromised or unintended actions. The team recommends applying the principle of least privilege to tool integrations, including MCP-connected services, and requiring explicit approval for irreversible operations such as changing production data, sending emails, or performing financial actions.

Sandboxed execution is equally important when agents generate code. Running agent-generated scripts directly on a developer's workstation can expose credentials, environment variables, and private keys if the code behaves unexpectedly. Ephemeral containers or isolated sandboxes can help contain the potential damage.

These are not features that should be added only after an agent has become complex. They belong in the design process from the beginning, particularly when an application has access to sensitive data or external systems.

Researching Tools Beyond Their Feature Lists

The team approaches tool comparisons as engineering design exercises rather than simple feature inventories.

A product homepage can explain what a framework offers, but developers also need to understand what it costs to operate, how it behaves when something fails, and whether its abstractions introduce unnecessary complexity.

AgDex.ai's technical comparisons aim to address those questions by examining architectural trade-offs, approximate operating expenses, and situations where a tool may not be the right choice.

The team applies the same philosophy to its knowledge hub. Technical guides are connected to relevant directory categories, while tool profiles link back to the guides that discuss them. This creates a two-way relationship between discovery and deeper technical research.

The result the team is working toward is a resource that supports more of the decision-making process: discovering a component, understanding its trade-offs, comparing it with alternatives, and finding a starting configuration for implementation.

The team also notes that AI coding assistants and AI search systems have become part of how developers discover technical information. Structured content and exportable context notes are intended to support these workflows alongside conventional browsing.

Lessons From Building AgDex.ai

The team's experience points to several lessons that extend beyond AI agent tooling.

The first is that a catalog alone does not provide enough decision context. Developers need explanations of compatibility, trade-offs, and implementation requirements before they can confidently select a tool. Features such as the Stack Finder, Quick View Drawer, and Versus Arena are responses to that need.

The second is that technical information must be maintained as a continuing responsibility. Automated validation helps, but community feedback and verification against upstream documentation remain important when APIs, pricing, and projects change.

The third is that there is no universal AI agent stack. Different applications face different constraints involving privacy, latency, cost, deployment, and operational complexity. A managed cloud architecture may work well for one team, while another may need self-hosted infrastructure to meet its requirements.

Finally, the team sees reliability as more important than impressive demonstrations. Autonomous workflows need dependable error recovery, idempotency, predictable latency, and clear debugging tools if they are to work consistently beyond a demo environment.

These lessons reinforce a common engineering principle: the value of a system depends not only on what it can do under ideal conditions, but also on how well its components work together and how it behaves when things go wrong.

Advice for Developers Starting With AI Agents

For developers entering the field, the AgDex.ai team recommends learning the basic agent loop before adopting a heavyweight orchestration framework.

A useful exercise is to build a simple tool-calling loop in roughly 50 lines of Python or TypeScript. Start with a basic model API, a loop, structured schemas using Pydantic or Zod, and two local functions.

Manually parsing tool calls, executing functions, and updating the message array makes the underlying process easier to understand. Once developers know how the loop works, they are better positioned to judge which framework abstractions are useful and which ones they can avoid.

The team also recommends adding basic observability early. Logging or a tracing backend such as Langfuse or Arize Phoenix can help developers understand token usage, tool payloads, and retry loops before unexpected requests become expensive.

Most importantly, developers should resist overengineering. A deterministic script and a single structured model call may solve a problem more reliably than a five-agent system. Autonomy introduces latency and variability, so it should be used where it provides a meaningful advantage.

What Comes Next for AgDex.ai

The team's roadmap focuses on developing AgDex.ai from a discovery catalog into a more comprehensive evaluation resource for production-oriented engineering decisions.

One priority is expanding head-to-head comparisons and RFC-style technical playbooks, including more detailed total-cost-of-ownership estimates, failure-mode analysis, and discussions of state persistence.

Another is expanding the role-specific portals with domain-focused prompt patterns, compliance considerations, and practical combinations of tools for different professional workflows.

The team also plans to continue maintaining its growing catalog, improving categorization, and preserving access to technical resources across English, Spanish, German, Japanese, and Arabic.

These goals reflect the same challenge that motivated the platform: as the AI agent ecosystem grows, developers need more than another list of tools. They need reliable information that helps them understand the trade-offs and make sound architectural decisions.

The team is also encouraged by the movement toward shared tool standards such as the Model Context Protocol (MCP), which can reduce the need for developers to build custom integrations for every combination of tools.

At the same time, it believes the ecosystem needs to pay more attention to security, token economics, error recovery, and production reliability. These less visible engineering concerns often determine whether an autonomous workflow remains useful after the initial demonstration.

AgDex.ai's work is an attempt to make those considerations easier to explore before implementation begins. Its central message is straightforward: start with the problem, understand the architecture, and choose the simplest combination of tools that meets the requirements.

About AgDex.ai

AgDex.ai is a curated directory and knowledge hub focused on AI agent tools, frameworks, LLM APIs, cloud infrastructure, and memory systems. The platform helps developers and teams discover technologies, compare architectural approaches, and explore reference blueprints for building AI agent applications.

Its areas of focus include agent orchestration, vector databases and memory, coding agents, MCP integrations, sandboxed execution, observability, and multi-agent workflows.

CoderLegion: AgDex.ai Team Profile
Website: AgDex.ai
Stack Finder: Explore AI Agent Architecture Blueprints
Tool Comparisons: AgDex.ai Versus Arena
Technical Resources for AI Assistants: llms.txt

Developer Stories is a CoderLegion interview series highlighting the people, ideas, and engineering decisions behind software development.

How Alice Severi Gonçalves Is Building Deterministic AI Infrastructure for Animal Welfare

Mon, 26 Oct 2026 08:47:32 +0000

Interviewed by: Mehadi Hasan

Developer: Alice Severi Gonçalves

Role: Independent Data Scientist and Computational Ethologist

Location: São Paulo, Brazil

Featured Project: The Transparency Project — EthoPipe

Website: The Transparency Project

What if the biggest challenge in understanding animal behaviour isn't a lack of observations, but the way those observations are recorded?

A shelter worker might describe a dog as stubborn, aggressive, or guilty. A trainer might interpret a particular posture as evidence of confidence or disobedience. These descriptions may reflect human interpretation rather than measurable biological events. When such judgments become part of research datasets, the distinction between observation and opinion becomes difficult to recover.

For Alice Severi Gonçalves, an independent data scientist and computational ethologist based in São Paulo, Brazil, this problem sits at the intersection of science, software engineering, and animal welfare. Through The Transparency Project, she is developing EthoPipe, a data pipeline designed to transform unstructured canine behaviour observations into standardized, machine-readable records.

Her approach is built around a straightforward principle: scientific data should be reproducible, verifiable, and resistant to subjective interpretation. That principle also shapes how she uses artificial intelligence. Rather than allowing an AI model to make unconstrained judgments, Alice treats it as a tightly controlled component within a larger system of validation, biological boundaries, and auditable data processing.

Her journey into this work has been anything but conventional. It spans legal education, language, digital operations, hands-on experimentation with computers, and years of experience working with dogs. The result is a project that explores an important question for AI-assisted science: how can we make software more useful without allowing its uncertainty to become someone else's risk?

From Exploring Windows 95 to Building Data Systems

Alice's relationship with computers began in the late 1990s, with a beige Compaq Presario running Windows 95. Early exploration involved navigating desktop menus, experimenting with software, and learning through interactive programs.

One experience that particularly influenced her problem-solving approach was playing Zoo Tycoon. Because the original game was in English and localized walkthroughs were unavailable to her in Brazil, she had to work out how the simulation behaved through experimentation.

She changed environmental variables, adjusted exhibit boundaries, observed animal responses, and examined how those changes affected guests and animals. Instead of relying on instructions, she learned by forming hypotheses, changing inputs, and observing outcomes.

That experimental mindset continued through hardware modifications and operating-system testing, including participation in the Windows Insider Program. Long before she began developing software formally, she was learning to investigate complex systems by exploring their behaviour.

Her academic and professional path took several turns. She earned a Bachelor of Laws from UNIP and later spent three years at Elife working with high-volume consumer text auditing and digital customer operations for major retail platforms.

These experiences gave her exposure to structured rules, interpretation, consistency, and the consequences of handling information incorrectly.

Alice eventually recognized a connection between these disciplines and software engineering. Legal reasoning depends on rules, defined boundaries, and the interpretation of exceptional cases. Language follows grammatical structures. Reliable software similarly depends on explicit rules, controlled inputs, and predictable handling of errors.

Her transition into generative AI began in 2024 and early 2025 through a visual branding project for her local pet-care service. She was creating graphic assets based on photographs of her American Pit Bull Terrier, Morgan, and encountered recurring problems with image models producing inaccurate bully-breed anatomy.

Rather than accepting the results, she experimented with cleaner backgrounds, consistent square images, and more precise prompts informed by anatomical and art-historical parameters. One of the resulting designs was selected for the official Adobe Firefly gallery.

The project introduced her to a broader possibility: if AI could help translate detailed instructions into visual outputs, perhaps it could also help translate complex, non-linear ideas into executable Python code and structured data systems.

That realization helped establish the direction of her current work.

Why Animal Behaviour Needs Better Data

Alice's interest in canine behaviour and welfare developed through years of practical involvement with dogs. Since 2012, she has worked in dog walking, pet sitting, and canine facility care in São Paulo.

Through that experience, she saw how behavioural information was often recorded in narrative form. Shelter intake notes, training records, and observational reports frequently relied on subjective descriptions instead of standardized measurements.

Words such as stubborn, spiteful, dominant, and guilty can make a record sound meaningful while leaving important questions unanswered. What exactly did the animal do? What was its posture? How long did it take to recover from a stressful event? What environmental conditions were present?

Different observers may interpret the same behaviour differently. When researchers or organizations attempt to compare those records, inconsistent terminology can make the underlying evidence difficult to reproduce or analyze.

For Alice, this is not merely a data-cleaning problem. It affects how people understand animals and make decisions about their welfare.

Her own perspective on animal training also changed over time. After initially participating in traditional balanced-training circles, she moved toward force-free, positive-reinforcement methodologies by 2020, influenced by what she observed about the behavioural consequences and ethical compromises associated with aversive control.

Quantitative observation offers a way to examine animal behaviour without relying exclusively on human interpretation. Instead of treating a label as a biological fact, researchers can document observable movement, response latency, physiological measurements, and environmental context.

This distinction matters when behavioural assessments influence training decisions, animal-care practices, and welfare policy.

Alice's legal background adds another dimension. She sees standardized, reproducible evidence as a way to strengthen accountability and support evidence-based animal protection standards.

The challenge is to create a system that preserves the complexity of biological behaviour while making the resulting information consistent enough to analyze.

EthoPipe: Turning Observations Into Structured Evidence

EthoPipe is the central technical project within The Transparency Project, Alice's open-science initiative.

Its purpose is to transform unstructured canine behavioural observations into standardized records that can be validated, stored, and shared in formats suitable for scientific analysis.

The project approaches this challenge as a data-engineering problem. Rather than treating a behavioural note as a finished record, the pipeline processes it through a series of validation and normalization steps.

1. Replacing subjective labels with observable actions

Consider a note stating that a dog was stubborn and refused to move.

The word stubborn is an interpretation. It does not describe a directly measurable physical event.

EthoPipe's approach is to identify observable motor patterns, using standardized terminology grounded in ethological literature. Depending on the actual observation, a record might describe a behaviour using terms such as balking or posture_freeze.

The objective is to preserve what can be observed while removing unsupported conclusions about the animal's intentions or emotions.

This distinction makes the resulting data more suitable for comparison across observers and studies.

2. Enforcing strict schemas at ingestion

EthoPipe uses Pydantic v2 to validate incoming data. Alice describes a strict configuration using ConfigDict(strict=True, extra="forbid").

Strict validation helps prevent unexpected types and unrecognized fields from silently entering the system.

Incoming observations are expected to include information such as timestamps, standardized behavioural classifications, environmental context, and physiological measurements.

The principle is to reject invalid or incomplete records at the system boundary rather than allow their problems to spread into later processing stages.

3. Applying biological limits

Software validation alone cannot determine whether a value makes biological sense. A record may have the correct data type and still contain an impossible measurement.

To address this, EthoPipe applies physiological boundaries informed by veterinary references, including the Merck Veterinary Manual.

Alice gives canine heart rate as an example, with a configured range of 30 to 250 beats per minute. A reported value of 400 BPM is flagged and isolated rather than accepted as an ordinary observation.

These boundaries are part of the pipeline's validation policy. They are intended to catch suspicious measurements, not replace veterinary interpretation or clinical judgment.

4. Standardizing records for interoperability

After validation, EthoPipe maps records to Darwin Core, an international standard used to facilitate the sharing of biodiversity information.

The project uses the MeasurementOrFact structure and distinguishes HumanObservation from MachineObservation through dwc:basisOfRecord.

This step matters because a dataset becomes more useful when its structure can be understood beyond the system that created it.

By combining controlled terminology, strict validation, biological boundaries, and standardized metadata, EthoPipe aims to make behavioural observations easier to audit and reuse.

Why Determinism Matters More Than Creative AI

Generative AI is often valued for its flexibility. It can interpret ambiguous requests, produce alternative explanations, and generate varied responses to similar prompts.

In scientific data processing, however, that flexibility can become a liability.

Alice's definition of determinism is simple: for an unchanged input state, the pipeline should produce the same structured output.

If an animal shelter processes an intake note on Monday and processes the same note again on Friday, the resulting normalized record should not change simply because the model produced a different interpretation.

This requirement shapes how EthoPipe uses large language models.

The pipeline interfaces with foundation-model APIs, with Alice's responses describing a migration from Gemini 1.5/2.5 Pro to Gemini 3.5 Flash. The model's generative temperature is set to zero, and structured JSON schemas are enforced through the SDK.

The model is not intended to act as an independent behavioural authority. Instead, it functions as a constrained text-normalization component within a system that applies explicit rules and validation.

Temperature zero alone does not guarantee perfect reproducibility across all model versions, API changes, or service configurations. Alice's broader engineering approach therefore places importance on schema enforcement, tests, documented changes, and validation outside the model itself.

The distinction is important: predictable formatting is not the same as scientifically correct interpretation. A structured record can still be wrong, which is why the rest of the pipeline must be able to detect and isolate invalid data.

For Alice, deterministic engineering is a way to reduce uncertainty at the points where software can control it.

Designing for Reliability as an Independent Developer

Maintaining a complex data pipeline as an independent developer creates a different set of constraints from working within a large engineering team.

Alice has responded by imposing strict architectural limits on EthoPipe.

The project maintains a Python-only runtime and a maximum budget of eight runtime packages. Its current dependency budget is reported as eight out of eight. It also avoids external broker daemons such as Redis and Celery, keeping state transitions within a leaner architecture.

These decisions reflect a preference for minimizing operational overhead and limiting the number of moving parts that must be maintained.

Dependencies can introduce maintenance burdens, compatibility problems, and potential supply-chain risks. Background services can also add deployment and debugging complexity. For a solo maintainer, every additional component represents something that must be understood, tested, and kept operational.

The project also emphasizes automated testing. Alice reports 27 passing tests and at least 90% code coverage.

Those figures are useful indicators of the project's current testing discipline, although passing tests and code coverage do not, by themselves, establish scientific validity or guarantee the absence of defects.

The larger goal is to make failures visible and manageable.

Alice identifies three particularly concerning failure modes:

Anthropomorphic contamination: Human judgments enter the dataset as though they were objective biological observations.
Silent schema drift: Changes to upstream APIs or incoming data formats cause fields to become misaligned without an obvious error.
Physiological boundary violations: Biologically implausible measurements are accepted as valid records.

Her approach combines strict Pydantic validation, hardcoded biological thresholds informed by veterinary references, immutable audit records, and quarantine mechanisms for malformed payloads.

Rather than silently accepting questionable data, the system is designed to make the problem visible and preserve the record for manual review.

This is a broader engineering lesson from EthoPipe: reliable systems should not merely process valid inputs efficiently. They should also fail clearly when an input cannot be trusted.

Where AI Can Help Animal Welfare—and Where It Must Stop

Alice sees several promising applications for AI in animal welfare, particularly when the technology is used to organize information that would otherwise be difficult to analyze at scale.

One example is extracting structured information from shelter records. Frontline staff and veterinary professionals may document observations in short, informal notes rather than complete extensive database forms. A carefully constrained language model could help transform those notes into standardized records for subsequent review.

Computer vision offers another opportunity. Markerless pose-estimation frameworks such as DeepLabCut and SLEAP can track body landmarks across video footage. Those coordinates can support calculations involving movement, gait asymmetry, and other kinematic characteristics.

Bioacoustic analysis may also help identify patterns in kennel audio, including vocalizations that warrant closer attention from animal-care staff.

These applications share a common principle: AI can help identify, organize, or measure information, but its outputs must remain subject to appropriate validation.

Alice is equally clear about where automated systems should not be granted independent authority.

Behavioural assessments and life-or-death decisions

Animal shelter decisions can be affected by historical bias, inconsistent records, and breed-related stereotypes. An automated system that labels an animal aggressive or unadoptable based on poorly calibrated notes could amplify those problems.

For that reason, Alice argues against allowing an unmonitored algorithm to determine an animal's fate.

Physical interventions

She also rejects connecting autonomous AI agents directly to aversive training devices, electric fences, or physical containment systems.

An algorithm should not independently trigger interventions that affect an animal's bodily integrity or physical freedom.

Veterinary diagnosis and treatment

AI-generated veterinary recommendations require particular caution. Alice argues that generative systems should not prescribe medication dosages or direct veterinary interventions without deterministic validation boundaries and approval from a qualified veterinary professional.

These limits are consistent with the architecture behind EthoPipe. The more consequential the decision, the less appropriate it is to rely on an unconstrained model output.

AI can assist with data processing, but assistance should not be confused with accountability.

Building With AI Without Giving Up Engineering Judgment

Alice's use of AI extends beyond EthoPipe's data pipeline. She also uses AI tools as part of her own development workflow.

As a self-taught developer working across several disciplines, she has found that AI can help organize complex ideas, interpret technical documentation, and explore architectural alternatives.

She describes using AI as an active technical sparring partner: challenging assumptions, discussing design choices, and translating complicated material into more manageable mental models.

Her workflow also involves experimenting in controlled environments, building command-line tools, and examining error logs to understand how systems behave when something breaks.

This approach reflects the same mindset that informs her data pipeline. Rather than treating an AI-generated answer as authoritative, she uses it as an input to a process of investigation.

That distinction is particularly relevant for developers working with rapidly changing AI tools. A generated solution may be useful, but it still needs to be understood, tested, and evaluated against the system's actual requirements.

Alice also speaks candidly about the personal difficulty of navigating technology work without a conventional software-engineering background. She describes ongoing imposter syndrome, uncertainty about employability, and the challenge of keeping pace with changing dependencies and model versions.

Her experience suggests that technical progress is not always a straight line from formal education to employment. It can involve combining knowledge from different disciplines, learning through experimentation, and building systems while still questioning whether one's skills are sufficient.

The engineering challenge is to turn that learning into software that remains maintainable when the tools and assumptions around it change.

Learning Cloud Architecture Without Prematurely Moving to the Cloud

Alice's interest in infrastructure also developed through formal learning opportunities.

As a fellow in the 2026 Google Cloud Launchpad Program with Mentor Me Collective, she explored concepts including VPC networking, identity and access management, containerization, Pub/Sub message queues, and Cloud Run architectures.

She also applied concepts from a full-stack development sprint at EBAC to her work on EthoPipe.

Rather than building disconnected exercises for every new technology, she adopted what she calls the "One-App Rule": channeling coursework concepts into the architecture of one ongoing project.

This approach allows new ideas to be evaluated in the context of an actual engineering problem.

Cloud concepts helped her think about component boundaries, stateless interfaces, and future scaling requirements. However, EthoPipe itself has not yet been deployed to cloud infrastructure.

That distinction reflects an intentional choice. Learning how to design for cloud environments does not mean every application needs an immediate cloud deployment.

For a project still undergoing architectural hardening, adding infrastructure prematurely could introduce complexity before the core data-processing workflow is sufficiently stable.

Alice's current direction emphasizes a lean local application while preparing its architecture for future requirements.

The goal is to understand the available tools without allowing infrastructure decisions to distract from the underlying scientific problem.

The Hardest Lesson: Owning the System

For Alice, one of the most difficult parts of developing EthoPipe has been learning to take full ownership of the system.

Model deprecations, changing dependencies, and architectural complexity have forced her to reconsider implementation choices while preserving the project's reliability.

Migrating the extraction pipeline to a newer model, keeping the runtime dependency budget limited, and maintaining the existing test suite have all contributed to her understanding of what sustainable engineering requires.

Her conclusion is that engineering competence cannot be measured simply by how much syntax someone remembers or whether their educational background follows a conventional route.

It also depends on whether they can understand a system's constraints, make deliberate trade-offs, test their decisions, and respond to change without allowing the architecture to collapse under its own complexity.

That lesson extends beyond independent development. Every software system eventually encounters changing requirements, aging dependencies, and unexpected failures. The difference lies in whether the system has been designed to expose those problems and recover from them.

Alice's approach is to document the process rather than hide the uncertainty.

The Transparency Project records architectural milestones, test failures, and design decisions in public project logs. Its documentation and scientific lineage are part of the effort to make the work auditable.

The project has also established a permanent Zenodo DOI, 10.5281/zenodo.21211371, and maintains a documented scientific identity through persistent identifiers.

EthoPipe is currently in Phase 4, focused on journal publication preparation, Zenodo minting, and architectural hardening. Alice reports that the project has received open-source documentation sponsorship from GitBook.

She remains candid about its uncertain future. She does not know whether EthoPipe will ultimately work as intended or gain wider adoption in the research community.

But she believes that documenting the work is valuable regardless of the outcome. Even if the project does not achieve its intended reach, a transparent record of the experiments, decisions, and limitations can still contribute to open science.

What Comes Next for EthoPipe

Alice's current development priorities focus on strengthening the data layer and preparing the project for more demanding workloads.

Her plans include:

Localized storage adapters: Developing dedicated adapters to persist normalized records into a query-optimized Kimball Star Schema.
Data-layer hardening: Improving Supabase PostgreSQL real-time replication endpoints and binding direct Google AI Studio response schemas at the ingestion gateway.
Stress testing: Building automated end-to-end tests for scenarios involving multiple dogs.
Scientific publication: Preparing the project's paper.md and paper.bib files for formal submission to the Journal of Open Source Software.

Longer term, she wants to automate Darwin Core Archive exports, connect Zenodo citations with international biodiversity repositories, and deploy a public telemetry portal for The Transparency Project.

These goals remain part of the project's ongoing development rather than completed outcomes.

The intended direction is clear: make it easier to collect, validate, preserve, and share structured evidence about animal behaviour.

If the infrastructure works as intended, standardized data could help researchers compare observations more consistently, evaluate interventions more objectively, and support animal welfare discussions with stronger empirical evidence.

Achieving that goal will require more than AI alone. It will depend on sound data standards, careful validation, reproducible methods, and appropriate human oversight.

Data Over Dogma

EthoPipe brings together several strands of Alice Severi Gonçalves's experience: the rule-based thinking of legal training, the structure of language, the experimental mindset of early computer exploration, and years of practical involvement with dogs.

Its central idea is that software can help separate observable evidence from subjective interpretation, provided the system is designed to enforce clear boundaries and expose uncertainty rather than conceal it.

The project is still developing, and its wider scientific impact remains uncertain. Yet its engineering priorities offer a useful perspective on responsible AI development: use models where they can help, constrain them where consistency matters, and never confuse a well-formed output with a verified fact.

For developers working on scientific, healthcare, or other consequential data systems, that distinction is especially important. Reliable automation is not simply about getting a model to produce an answer. It is about building the surrounding infrastructure that determines whether the answer can be trusted, reproduced, reviewed, and corrected.

Alice's work is a reminder that some of the most important technology projects begin with an unglamorous problem: information is being collected, but the way it is recorded makes it difficult to trust.

Her response is to build the structure that the data is missing.

Data over dogma.

About Alice Severi Gonçalves

Alice Severi Gonçalves is an independent data scientist and computational ethologist based in São Paulo, Brazil. Her work focuses on canine behaviour, quantitative animal welfare science, deterministic data processing, and reproducible research infrastructure.

She is developing EthoPipe through The Transparency Project, an open-science initiative that aims to transform subjective canine behavioural records into standardized, auditable datasets. Her technical interests include Python, data engineering, AI-assisted text normalization, strict schema validation, biodiversity data standards, and cloud architecture.

CoderLegion: Alice Severi Gonçalves
Website: The Transparency Project

Developer Stories is a CoderLegion interview series highlighting the people, ideas, projects, and engineering decisions behind software development.

RayLabs: Building Reliable Software Through Curiosity, Practical Engineering, and Human Judgment

Mon, 26 Oct 2026 05:36:45 +0000

RayLabs: Building Reliable Software Through Curiosity, Practical Engineering, and Human Judgment

Interviewed by: Mehadi Hasan

Developer: Raylabs

Role: Software Developer and Independent Technology Builder

Location: Jakarta, Indonesia

Website: raylabs.app

From Curiosity to Building Real Software

Not every software developer begins their journey with a passion for programming. For Raylabs, the journey started with curiosity, a change of direction, and the experience of building something that actually worked.

During college, coding seemed complicated enough that Raylabs considered pursuing a career outside programming. That perspective began to change around 2018, when a friend encouraged him to join Binar Academy. Although he initially had an interest in project management, he eventually chose Android Engineering.

The experience introduced him to mentors who made programming easier to understand. Building his first Android application became an important moment: seeing something he had created work in practice gave him a reason to explore further.

That curiosity eventually extended beyond Android development into backend systems, automation, and artificial intelligence. Rather than following a single technology throughout his career, Raylabs continued exploring how different systems work and how they could be used to solve practical problems.

Looking back, he does not attribute his continued involvement in software development to being naturally gifted at coding. Instead, he points to a more enduring motivation: wanting to understand how things work and discovering whether he can build them himself.

That mindset runs through his approach to engineering today. For Raylabs, development is not simply about learning a framework or mastering a programming language. It is about turning curiosity into something useful.

Moving Beyond Courses to Learning Through Building

Like many developers, Raylabs initially invested considerable time in courses, training programs, and certifications, including several from Google. These experiences helped him establish a technical foundation, but they also taught him an important distinction: understanding a concept during a course is not the same as applying it to a real problem.

His perspective began to change when he encountered practical challenges at work. Knowledge that had seemed theoretical months earlier suddenly became relevant to a problem that needed solving. In some cases, the solution was simpler than he had expected.

These experiences changed the questions he asked himself.

Instead of constantly wondering which technology to learn next, he began asking what problems he could solve with the knowledge he already had.

The distinction is significant. Developers can spend years collecting certifications and exploring new tools without becoming comfortable applying what they know under real constraints. Building something forces them to confront the details that courses cannot always anticipate: unexpected behavior, practical limitations, trade-offs, and the need to make decisions with incomplete information.

Raylabs' approach does not dismiss formal learning. Rather, it places learning in a broader cycle of experimentation and application.

Knowledge becomes more valuable when it helps solve a problem. Each project then creates new questions, encouraging further learning.

RayLabs: Turning Technical Experiments Into Practical Knowledge

RayLabs grew out of a habit of experimenting with technology and documenting what those experiments revealed.

When working on projects or exploring unfamiliar systems, Raylabs often discovers something worth sharing. Sometimes it is a simpler solution. Other times, it is an explanation of why a particular approach failed or behaved differently than expected.

He wanted a place to document these discoveries through experiences he had actually encountered, rather than relying exclusively on theoretical explanations.

That motivation shaped the principles associated with RayLabs: clear thinking, practical engineering, minimal unnecessary complexity, and evidence grounded in real work.

The emphasis on practical knowledge is especially relevant in software development, where tutorials and technical articles can make a solution appear straightforward without explaining the conditions under which it works.

A developer who has tested an approach, encountered its limitations, and documented the results can offer readers something more useful than a description of how a technology is supposed to behave.

Raylabs also recognizes that useful knowledge does not have to come from a major breakthrough. A small discovery can help another developer overcome a problem that has been slowing them down.

Failure is part of that process. An unsuccessful experiment can reveal assumptions that were incorrect, expose limitations in a tool, or show why a seemingly reasonable solution is not suitable for a particular situation.

For Raylabs, the goal is not to present every experiment as a success. It is to share what was learned.

He has also been experimenting with automation that turns conversations, notes, and technical explorations into potential article ideas. This connects his interest in automation with his broader goal of documenting practical engineering knowledge.

The idea is to make it easier to capture useful discoveries while keeping the underlying experiences and lessons at the centre of the content.

Choosing Technologies by the Problems They Solve

Raylabs does not approach technology selection by simply following whatever happens to be trending.

Instead, he starts with something he wants to build or a problem he wants to solve. Only then does he consider which technologies might help.

This approach became particularly interesting as AI tools grew more accessible. Previously, moving from an idea to a working experiment could require substantial time and effort. Some ideas were not necessarily impossible, but the technical challenges or the amount of work involved made them difficult to explore independently.

AI has changed that equation for him by making experimentation faster and allowing him to test ideas that might previously have felt too complicated or time-consuming.

However, the main benefit is not merely that AI can generate code quickly. It is the freedom to explore more possibilities and turn ideas into testable implementations with less friction.

When considering a technology, Raylabs returns to three practical questions:

What am I trying to build?
What problem am I solving?
Can this technology actually help me get there?

These questions provide a useful way to evaluate new tools without becoming distracted by their popularity.

A framework might be technically impressive but still introduce unnecessary complexity into a small project. An AI-powered solution might accelerate implementation without addressing the underlying problem. A popular developer tool might fit another person's workflow perfectly while making someone else's work slower.

Technology selection, in this view, is an exercise in judgment rather than trend-following.

The tool should serve the goal, not determine it.

Reliability Before Features

One of the strongest themes in Raylabs' engineering philosophy is reliability.

He believes that a system with numerous impressive features has limited value if it breaks frequently or if its features do not genuinely help the people using it.

For him, a simple system that works consistently is often preferable to a complicated one that looks impressive on paper.

This perspective influences how he thinks about both professional work and personal projects at RayLabs. Stability is not something to consider only after development is complete. It is part of the decision-making process from the beginning.

That does not mean every potential failure can be predicted before implementation. Raylabs acknowledges that some problems become visible only during development or after a system starts running.

What matters is how the developer responds when reality differs from the original plan.

Rather than immediately trying to repair every problem within the existing design, he prefers to evaluate the approach itself. Is the original solution still worth pursuing? Does it make sense to continue? Or would changing direction produce a better result?

These questions introduce an important distinction between fixing an implementation and reconsidering the decision that led to it.

Sometimes a problem can be solved with a small correction. In other cases, the underlying approach may no longer be appropriate.

Knowing when to continue, when to change direction, and when to stop is part of good engineering judgment.

For Raylabs, reliability is not simply about preventing errors. It is about building systems that remain useful, manageable, and dependable when conditions do not match expectations.

Keeping Architecture Flexible When Things Go Wrong

Anticipating failure is important, but Raylabs does not assume that every failure scenario can be identified in advance.

Some issues emerge only when implementation begins. Others become apparent when a system runs under real conditions. These discoveries are part of the development process, and they can require developers to reconsider their original assumptions.

This is one reason he prefers simple, flexible approaches to architecture.

A design that is unnecessarily complicated can make it harder to understand problems or adapt when requirements change. A simpler approach can provide more room to evaluate what is actually happening before committing to additional complexity.

The key is not to eliminate planning. It is to avoid treating the original plan as something that must be preserved at all costs.

Raylabs' go-or-no-go mindset offers a practical way to approach uncertain projects. When a solution encounters difficulties, the developer should assess whether continuing is justified rather than automatically investing more effort in the same direction.

This principle also applies to scaling, automation, and infrastructure decisions. A solution that works under current conditions may need to change as requirements evolve, but the appropriate response depends on the problem and the available evidence.

Sometimes optimization is sufficient. Sometimes additional resources are justified. Sometimes a different design is necessary.

The important thing is to make those decisions deliberately.

Refactoring Legacy Code Without Creating New Problems

Legacy code presents another challenge that has shaped Raylabs' engineering approach.

When an existing codebase works, changing it can feel risky. Even when the code is difficult to maintain, a large refactor may introduce regressions that are more costly than the original problem.

At the same time, leaving every imperfect implementation untouched creates its own long-term difficulties.

Raylabs has learned to approach refactoring gradually instead of trying to clean up an entire codebase at once.

Unit tests and instrumentation tests help him make changes with greater confidence. They provide ways to check whether existing behavior remains intact while the implementation evolves.

Testing does not eliminate all risk, and regression testing can feel never-ending. Nevertheless, it creates a stronger basis for making incremental improvements.

The objective is not to make every part of a codebase perfect immediately. It is to understand the behavior that already exists, improve it in manageable steps, and verify that those changes do not introduce unwanted consequences.

This approach recognizes that maintainability and stability must be considered together.

A refactor that produces cleaner code but breaks important behavior is not necessarily an improvement. Equally, avoiding every change can leave developers dealing with increasing complexity over time.

Gradual refactoring provides a middle ground: make progress while preserving confidence in the system.

Developer Tools Should Reduce Friction, Not Add It

Raylabs' preference for practical engineering also influences how he chooses developer tools.

He has tried tools recommended by other developers because they promised to make development faster. Yet some of those tools ended up slowing him down instead.

Those experiences helped him recognize that popularity and capability do not automatically make a tool useful for every developer.

A genuinely useful tool should make the existing workflow easier. If using it introduces unnecessary steps, complicates routine tasks, or creates more work than it removes, its benefits may not justify the cost.

This is a straightforward principle, but it challenges a common habit in software development: adopting tools because they are popular rather than because they address a specific need.

Raylabs is comfortable removing a tool when it does not fit his workflow and moving on.

The same reasoning applies to frameworks, abstractions, automation, and AI-assisted development. Additional technology is valuable when it solves a real problem. Otherwise, it can become another layer that developers must understand, configure, and maintain.

The objective is not to use the largest collection of tools. It is to complete the work effectively.

SEO Is Also an Engineering Problem

Raylabs' interest in search engine optimization began much earlier than his work on RayLabs. During his junior high school years, in the era of Blogger, he became interested in helping people find information through search engines.

His understanding of SEO has evolved as he has learned more about how websites work.

Discovering schema markup and structured data helped him see that search visibility is not determined by content and keywords alone. Website structure, the organization of information, and the way search engines interpret a page also matter.

When building RayLabs, he brought that understanding into the project.

Rather than treating publishing as a separate activity from technical implementation, he considered how the website itself could help make the content easier to discover and understand.

This perspective connects SEO with several familiar engineering concerns: how information is organized, how pages are structured, and how systems communicate meaning to other systems.

It also reflects his broader philosophy of building for a practical outcome. Publishing useful information is only part of the process. Making that information accessible to the people who need it is another important consideration.

For Raylabs, SEO is not simply about increasing traffic. It is about helping people find the information they are looking for through a combination of useful content and sound engineering.

Scaling Without Losing Control of Cost and Reliability

As projects grow, engineering decisions become more complicated.

A system that works well at a small scale may require different operational choices when it handles thousands of pages, jobs, or automated operations. Performance, reliability, and cost must be considered together.

Raylabs approaches these decisions as a balancing exercise.

Before expanding infrastructure, he prefers to examine whether the existing setup can be optimized. Improving what is already available may be more appropriate than immediately adding resources.

However, optimization is not an absolute rule. If additional infrastructure provides sufficient benefits to justify its cost, scaling up can be the right decision.

The important distinction is between spending resources because growth makes it necessary and adding complexity without a clear reason.

Reliability also requires preparation for situations in which systems or external services do not behave as expected. Raylabs believes in having backup plans, particularly when dealing with automation or systems that need to operate continuously.

This is consistent with his preference for evaluating failure scenarios and keeping architecture flexible.

Scaling is not simply a matter of handling more requests or increasing the number of operations. It also means keeping a system understandable, dependable, and economically manageable as its demands change.

There is no single solution that fits every project. The right choice depends on current constraints, expected benefits, and the consequences of failure.

AI Agents Are Collaborators, Not Decision-Makers

Raylabs sees practical value in AI agents for speeding up development, exploring ideas, and handling repetitive work. These capabilities can reduce the effort required to move from a concept to an initial implementation.

However, he believes human judgment remains essential.

One limitation he has observed is that AI-generated solutions can follow familiar patterns without fully addressing the specific context of a problem. The output may be technically reasonable while lacking originality or a deeper understanding of what the developer is trying to accomplish.

He connects this observation with the idea of AI slop: output that can appear acceptable at first glance but may lack the thoughtfulness or contextual fit needed to make it genuinely useful.

Raylabs also draws on an Indonesian concept known as ATM: Amati, Tiru, Modifikasi, meaning Observe, Imitate, and Modify.

AI can be effective at recognizing and reproducing existing patterns. But applying those patterns appropriately often requires a further step: modifying the approach to fit the actual problem.

His own experience has reinforced this point. He has encountered situations in which an AI tool suggested several reasonable solutions, but the approach he considered most suitable appeared only in its fourth or fifth recommendation.

The lesson is not that AI suggestions should be dismissed. Rather, developers should avoid assuming that the first answer is necessarily the best one.

They need to compare alternatives, challenge recommendations, understand trade-offs, and determine whether a proposed solution fits the requirements.

AI can accelerate exploration, but it does not remove the need to evaluate the results.

That distinction becomes especially important when AI is used to generate code, recommend architectural changes, or automate tasks that must run reliably. A plausible output is not the same as a verified solution.

For Raylabs, AI agents are powerful collaborators, but they should not replace the developer's responsibility to make the final engineering decisions.

What Other Developers Can Learn From Raylabs

Raylabs' experiences point to several practical principles for developers working across different technologies and stages of their careers.

Start with a problem, not a trend. Choose technologies based on what you want to build and the constraints you need to address. A popular tool is useful only when it fits the task.

Turn learning into practical experience. Courses and certifications can establish a foundation, but building real things helps reveal how knowledge applies under actual conditions.

Treat reliability as part of engineering quality. Features have limited value when a system is unstable or difficult to depend on. Consider what happens when the expected path fails.

Make changes incrementally. When working with legacy code, understand existing behavior, use appropriate tests, and improve the implementation in manageable steps.

Be willing to change direction. Not every problem should be solved by adding another fix to the original design. Sometimes the better decision is to reconsider the approach.

Use tools selectively. Frameworks, automation, and AI should reduce friction rather than introduce unnecessary complexity.

Verify AI-generated solutions. Explore alternatives, evaluate trade-offs, and use engineering judgment to decide whether a recommendation is appropriate.

Share what you learn, including failures. Useful technical knowledge does not require every experiment to succeed. Honest documentation of what worked, what failed, and why can help other developers make better decisions.

These principles reinforce one another. Practical learning generates experience. Experience informs better decisions. Testing and verification help preserve reliability. Sharing the resulting lessons allows others to benefit from the process.

Conclusion: Better Engineering Begins With Better Questions

Raylabs' journey from an initially hesitant college student to a developer exploring backend systems, automation, AI, and technical publishing illustrates how curiosity can become a lasting foundation for software development.

His approach is not built around using the newest technology or making every system as sophisticated as possible. It is built around understanding problems, experimenting with possible solutions, and evaluating the results.

That mindset becomes especially valuable as AI makes it easier to generate code and explore ideas. When implementation becomes faster, the ability to choose the right problem, recognize a poor approach, verify behavior, and make thoughtful trade-offs becomes even more important.

The same principle applies to reliability, refactoring, developer tools, SEO, and scaling. In each area, the goal is not complexity for its own sake. It is to make deliberate decisions that produce useful, maintainable outcomes.

For developers who are still learning, Raylabs' experience offers a reassuring perspective: you do not need to know everything before you start building. You need the curiosity to investigate, the willingness to test your ideas, and the judgment to learn from what happens next.

Sometimes, the most valuable engineering skill is not knowing the answer immediately. It is knowing which questions to ask, how to evaluate the available options, and when to reconsider your assumptions.

About Raylabs

Raylabs is a developer based in Jakarta, Indonesia, who explores backend development, serverless systems, DevOps, developer tools, AI agents, and automation. His technical journey began with Android Engineering and expanded through practical experience and experimentation with different technologies.

Through RayLabs, he documents technical discoveries, practical implementations, and lessons learned from working with software. His approach emphasizes clear thinking, reliable systems, minimal unnecessary complexity, and evidence grounded in real experience.

He is also exploring ways to use automation and AI to support technical experimentation and article ideation while maintaining the importance of human creativity and engineering judgment.

Developer Stories is a CoderLegion interview series highlighting the people, ideas, projects, and engineering decisions behind software development.

What AI Coding Loses That Git History Can't Tell You — B Donald Harris

Mon, 26 Oct 2026 03:02:57 +0000

AI coding agents can generate code faster than ever—but can developers still explain why that code exists?

In this CoderLegion Developer Spotlight, software architect and founder B Donald Harris shares how AI-assisted development has reignited his passion for coding, why traditional Git history doesn't tell the whole story, and how he's building Hindsight to help developers understand the decisions, conversations, and reasoning behind their software.

From discovering programming on a Commodore 64 at age nine to exploring AI agentic workflows after more than two decades in software development, Donald offers a thoughtful perspective on the changing role of developers—and why the joy of building still matters.

Meet B Donald Harris

B Donald Harris is a software architect and founder based in Tulsa, Oklahoma. He has spent more than two decades building software, primarily in .NET and cloud technologies, and continues to explore new ways to build, learn, and collaborate.

His current work includes Hindsight, a workflow intelligence tool intended to help developers reconstruct how a software project evolved and understand why particular decisions were made.

Donald is also behind BIT Voices, a podcast and builder network created to showcase Black technology professionals, share their stories, and help builders discover and connect with one another.

Connect with Donald on CoderLegion:
https://coderlegion.com/user/BDonaldTechnologist

Why AI coding can leave developers with unanswered questions

AI coding agents can produce substantial amounts of code, help implement features, and accelerate development. But faster implementation creates another challenge: keeping track of everything that has changed and understanding the reasoning behind those changes.

As Donald explains in the interview, experienced developers might not remember every line of code they've written, but they can often recall the overall system design and know where to look when something goes wrong.

With agentic development, that understanding can become harder to maintain as changes and commits accumulate quickly.

A Git history may show that a feature was added, a bug was fixed, or an implementation changed. What it might not explain is why the team made that decision in the first place.

The missing context can live in conversations with AI agents, prompts, discussions between developers, terminal sessions, notes, and the reasoning behind architectural choices.

That is the problem Donald wants Hindsight to help solve.

Hindsight: Remembering the story behind software

Hindsight is a workflow intelligence tool Donald is building to help developers make sense of how their projects have evolved.

The concept is to connect a repository and bring together relevant development information, including:

Repository history, pull requests, and issues.
AI conversations and prompting sessions.
Terminal sessions and development notes.
Decisions made during implementation and debugging.
Changes in direction and the reasoning behind them.

By working with this information and a language model, developers could build a more complete account of their project's development journey.

Rather than relying exclusively on commit messages and code changes, the aim is to preserve the context that helps explain what happened, why it happened, and how the software reached its current state.

Donald describes the project as being in development and experimentation during the interview, with an MVP alpha targeted for the beginning of the following year.

He also emphasizes that privacy and control over what information is captured and seen are important considerations for builders using such a tool.

How AI has changed Donald's approach to development

One of the most interesting moments in the conversation comes when Donald reflects on how AI coding agents have affected his relationship with programming.

He started coding at nine after his father brought home a Commodore 64. Using a book of BASIC programs, he would type instructions and watch something happen on the computer. That experience felt like magic, and it sparked a lifelong interest in building software.

Over time, professional responsibilities and other commitments meant that coding sometimes became something he did primarily for work rather than for fun.

AI-assisted development has helped bring back that original excitement.

With agents taking on much of the implementation work, Donald can spend more time thinking about architecture, orchestrating agents, and bringing ideas to life—ideas he might not previously have had the time to explore.

His perspective is not that developers should abandon their skills or stop understanding their code. Instead, he sees AI as another step in the evolution of computing, much like earlier layers of abstraction that made software development possible.

The challenge now is learning how to work effectively with these tools while maintaining control over the systems being built.

From a .NET developer tool to AI agentic workflows

Donald also discusses a VS Code extension he created to simplify .NET project creation.

While exploring a move from Visual Studio to VS Code at work, he found that starting certain .NET projects required a sequence of command-line operations: creating solutions, creating projects, and adding projects to solutions.

He wanted a simpler workflow, so he explored the VS Code extension architecture and built a tool that automated those steps.

That project also became his first introduction to agentic workflows and helped shape his interest in how new tools could make development easier.

The experience reflects a recurring theme in Donald's work: identifying friction in the development process and finding practical ways to remove it.

Building BIT Voices and creating opportunities for others

Donald’s work extends beyond software tools.

Through the BIT Voices podcast, he provides a platform for Black technology professionals to share their experiences, work, and ideas. The accompanying BIT Voices Network is intended to help builders discover one another, showcase projects, connect, and potentially collaborate.

Donald explains that representation matters because technology professionals can be geographically and organizationally dispersed, sometimes with few opportunities to connect with others who share similar experiences.

He wants younger generations to see people like themselves building careers and creating opportunities in technology.

His decision to become more visible as a public builder is part of that mission: sharing his own work while helping others tell their stories.

Lessons from building products and communities

The interview touches on several lessons that extend beyond AI tooling.

1. Don't lose the joy of building.

Donald says the advice he wishes he had received earlier in his career was to never stop doing software development for fun. Personal projects can provide room to experiment and rediscover the excitement that first drew someone to programming.

2. Understand the reasoning, not just the implementation.

Code and commits tell part of a software project's story. The discussions, decisions, and context behind them can be equally important when maintaining or extending a system.

3. Learn to embrace technological change.

Donald sees AI as another stage in computing's evolution. Rather than dismissing it, developers can learn how to use it effectively while continuing to apply their own judgment.

4. Pay attention to how the software actually works.

Building and testing a product can reveal workflows that differ from the assumptions made during initial design. Donald emphasizes learning from those discoveries rather than rigidly following preconceived ideas.

5. Start building.

His advice to developers who want to create their own products is straightforward: start. Developers already have many of the skills needed to outline a project and begin turning an idea into something real.

Watch the full conversation

In this CoderLegion Developer Spotlight, Donald and host Tom Smith explore AI-assisted development, software architecture, Hindsight, the importance of developer context, and the work involved in building products and communities.

The conversation is particularly relevant for developers experimenting with AI coding agents and thinking about how to maintain a deeper understanding of increasingly complex codebases.

About the interviewer: Tom Smith

Tom Smith hosts this CoderLegion Developer Spotlight, speaking with developers and builders about their experiences, technical decisions, projects, and lessons learned along the way.

Connect with Tom on CoderLegion:
https://coderlegion.com/user/Tom Smith

About CoderLegion Dev Stories

CoderLegion Dev Stories shares the experiences behind software development: the people building products, the problems they solve, the decisions they make, and the lessons they learn.

If you're building something interesting, exploring a new technology, or learning from real-world development challenges, consider sharing your story with the CoderLegion community.

Explore CoderLegion: https://coderlegion.com

Keep building, keep sharing, and keep learning from one another.

The Test Passed. The Attack Worked: Marco Bellingeri on Building More Trustworthy AI Systems

Sun, 25 Oct 2026 11:57:49 +0000

Interviewed by: Mehadi Hasan

Developer: Marco Bellingeri

Role: Cloud & AI Security Engineer | Python, AWS, AI Agents, and DevSecOps

Location: Italy

Website: marcobellingeri.dev

A security test can pass while the vulnerability it was designed to catch remains exploitable. For Marco Bellingeri, discovering that failure in his own AI system became one of the most important lessons of his engineering journey.

Bellingeri works at the intersection of cloud infrastructure, artificial intelligence, and application security. His projects explore practical challenges in building AI-powered systems, from protecting language models against prompt injection to independently monitoring the actions of coding agents. Rather than treating AI security as a collection of restrictions placed around a model, he approaches it as an engineering problem involving permissions, evidence, observability, and verification.

His background spans systems programming, networking, Linux, cybersecurity, infrastructure monitoring, and production Python applications. That experience shaped a perspective that now runs through his work: a system should not be trusted simply because its implementation looks correct or its monitoring dashboard is green.

Through projects such as MonferrinoAI, Agentic-OS, and LLM-Council, Bellingeri explores a central question for modern AI engineering: How do we prove that an AI system behaves as intended, rather than merely assuming that it does?

From Systems Engineering to AI Security

Bellingeri did not begin his career with artificial intelligence. His path developed gradually through systems, programming, networking, Linux, and cybersecurity. He also spent several years teaching computer science and security-related subjects, an experience that helped him understand how different components interact within a larger system.

Infrastructure and observability became important parts of that foundation. His earlier work involved Linux environments, virtual machines, VPNs, Prometheus, Grafana, and cloud monitoring. From there, cloud engineering became a natural progression.

He later moved deeper into Python-based software engineering, contributing to production systems that included 24/7 monitoring software for Rome Metro Line C, enterprise network diagnostics, and a larger backend platform built around Python and AWS.

That backend platform involved serverless components, asynchronous processing, databases, queues, and interconnected services. Working with these systems introduced challenges that extended beyond writing code: handling failures, managing permissions, tracking state, and understanding what happens when individual components behave unexpectedly.

Artificial intelligence added another layer of complexity.

When Bellingeri began building applications with large language models (LLMs), retrieval-augmented generation (RAG), and AI agents, he became interested not only in their capabilities but also in what happened when they were treated as components of real software systems.

Models introduce uncertainty. They process untrusted inputs, retrieve external information, interact with tools, and sometimes produce incorrect results with considerable confidence. Those characteristics make security inseparable from the broader engineering challenge.

Today, Bellingeri approaches an AI system with two questions:

What can it do, and what evidence do I have that it behaves the way I think it does?

That distinction between capability and evidence provides a foundation for understanding his projects.

MonferrinoAI: Making RAG Useful Beyond Retrieval

One of Bellingeri's projects, MonferrinoAI, began with a practical business problem: how can a local business website remain current and visible over time, particularly for search engine optimization (SEO), answer engine optimization (AEO), competitor awareness, and local Google search?

A website can be redesigned and launched successfully but gradually become outdated as competitors publish new material and search behavior changes. Bellingeri wanted to explore a system that could maintain an evolving knowledge base about a business website and its competitive environment, identify content gaps, and help determine what should be developed next.

MonferrinoAI uses scheduled GitHub Actions workflows to automate much of that preparation.

Firecrawl collects web content from the business website and competitors. Voyage AI generates embeddings, while Supabase, PostgreSQL, and pgvector provide the storage and retrieval foundation.

The system maintains information at multiple levels, including competitor snapshots, vectors representing complete pages, and more granular vectors representing individual website sections.

Each week, it selects a candidate from an SEO backlog, queries the RAG system to understand what the website already covers, and produces a structured briefing.

An important architectural decision is that MonferrinoAI does not autonomously write and publish content. It gathers information, refreshes the knowledge base, identifies opportunities, and prepares context. The actual writing remains a human responsibility, as does the decision to merge or publish the resulting work.

This human-in-the-loop approach reflects a broader principle in Bellingeri's engineering philosophy: automation can assist with analysis and preparation without automatically receiving authority over the final action.

Why retrieving information is not enough

Building MonferrinoAI also challenged a common assumption about RAG systems: that successful retrieval necessarily produces useful results.

Bellingeri tested retrieval against the actual database and found that a keyword-only query could rank the wrong page above the page that was genuinely relevant. Including additional context, such as the keyword cluster and target page, improved retrieval behavior.

The problem was not that the database returned invalid results. It returned results that were less useful for the decision the system needed to make.

That experience changed how he thought about retrieval. Instead of treating it as a simple step in which relevant-looking material is passed to a model, he began focusing more explicitly on evidence, relevance, provenance, and the limits of what the retrieved information could justify.

Freshness presented another challenge. Even if every subsequent component worked correctly, recommendations based on outdated website or competitor information could still lead to poor decisions.

Security introduced a related concern. Web pages are untrusted inputs, and their contents can include text that resembles instructions. Such text should not acquire authority over the AI system merely because it was retrieved successfully.

In MonferrinoAI, the information passed from RAG into the final briefing is deliberately constrained. Rather than passing retrieved page bodies directly into that handoff, the system exposes information such as URLs, headings, and similarity distances.

The objective is to limit how much untrusted content crosses into the next stage.

For Bellingeri, these are not separate problems to solve independently. Information quality, reliability, and security are connected because each depends on knowing where information originated, how relevant it is, and what the system is permitted to do with it.

Agentic-OS: Observing What AI Agents Actually Do

While MonferrinoAI focuses on retrieving and preparing information, Agentic-OS addresses another challenge: understanding what happens when AI agents interact with development environments.

Bellingeri noticed a gap between what a coding agent reports doing and what actually occurred in the environment. An agent might describe an action, but its own account is not necessarily sufficient evidence that the action happened as reported.

This matters because coding agents can do considerably more than generate text. They can call tools, modify files, execute commands, consume resources, and make decisions across multiple steps.

Traditional observability already helps engineers understand complex software. Agentic systems make independent observation even more important because the system being monitored can also produce the narrative used to explain its behavior.

Agentic-OS was built to explore this problem.

The platform collects telemetry from Claude Code through an OpenTelemetry Collector, using Prometheus and Grafana for metrics and a private Loki pipeline for logs. It also includes a small status API, while its public-facing surface exposes only a sanitized subset of aggregate information.

The architecture separates internal operational visibility from the information made publicly available.

Observability is also a privacy boundary

Building the platform revealed another issue: telemetry can contain sensitive information that engineers might not expect to appear in monitoring data.

Bellingeri found that the real Claude Code client could send identity attributes, including an email address. This meant that the observability pipeline itself needed to be treated as a privacy boundary.

Simply assuming that sensitive attributes would never arrive was not sufficient.

His approach uses allow-lists and tests against the actual telemetry structure so that unknown attributes are dropped rather than stored by default.

This is an important distinction. Monitoring systems are often designed to collect enough information to explain failures, but collecting more information can also introduce privacy risks. Observability therefore needs its own controls and verification.

For Bellingeri, monitoring an AI agent means looking beyond its final answer. Engineers need to understand which actions occurred, which tools were involved, what failed, how long operations took, and whether the system has independent evidence that reported actions actually happened.

As AI agents become more capable, observability becomes part of the trust model rather than merely a debugging convenience.

LLM-Council: Why Multiple Models Do Not Guarantee the Truth

Bellingeri's LLM-Council project explores a different problem: how to evaluate AI-generated answers without treating a single model as an unquestionable authority.

Language models can produce useful reasoning, but they can also be confidently wrong, follow a user's framing too closely, or reinforce assumptions that deserve to be challenged.

LLM-Council responds by asking three different models to generate answers independently. Their responses are anonymized, and the models blindly rank one another's answers. A separate chairman model then produces a final synthesis while surfacing meaningful disagreements.

The approach introduces additional perspectives rather than relying on a single response.

However, Bellingeri is careful about what this process can establish.

Multiple models can share the same blind spots. Agreement does not automatically make an answer correct, and a consensus can still be wrong.

The value lies in obtaining additional evidence through independent attempts, disagreement, and potentially different failure modes. The evaluation process itself must also be tested.

To assess the system, Bellingeri uses unit tests, adversarial cases, live end-to-end checks, and mutation testing. These methods help examine not only the answers but also whether the mechanisms intended to evaluate them can detect failures.

That last concern leads to one of the most revealing experiences in his work: a security test that passed while the attack it was meant to prevent still worked.

The Prompt-Injection Test That Passed While the Attack Worked

The incident occurred inside LLM-Council, where output from one model stage becomes input to another. Bellingeri needed to separate untrusted model-generated content from the instructions surrounding it.

His initial implementation used static delimiters to mark the boundaries of untrusted content. He wrote tests that checked marker placement and string structure.

Those tests passed.

But they were checking whether the Python string had been assembled correctly, not whether the next model would interpret the boundaries safely.

A hostile model output could reproduce the static closing delimiter, making the downstream model interpret the untrusted block as though it had ended. The tests remained green because the structure matched their expectations, even though the intended security property was not enforced.

The distinction was fundamental: a test can verify that a mechanism exists without proving that the mechanism provides the protection engineers expect.

Testing the property, not the implementation

Bellingeri changed the delimiter mechanism to use a fresh random nonce for each prompt. This made the authentic closing marker unpredictable to an untrusted response. He also identified another boundary requiring similar protection: model-generated rankings entering the chairman stage were untrusted as well.

He then tested the protections by deliberately breaking them.

He replaced the random nonce with a static value and expected the relevant tests to fail. He also removed the ranking boundaries and expected their tests to detect the change.

This approach provided stronger evidence that the tests were observing the security properties they were intended to protect.

There is an important limitation, however. A random nonce helps protect the structural boundary against delimiter forgery; it does not make the content inside that boundary trustworthy or solve prompt injection as a whole.

Bellingeri's lesson extends beyond this particular implementation. A test's name describes a claim about the system, but its assertions determine what evidence the test actually provides.

Before trusting a security test, he now wants to see that it can detect a known-bad state.

The practical question is simple: if the protection stops working, what should cause the test to fail?

Penetration Testing Your Own Systems

Bellingeri applies a similar approach when evaluating the security of systems he has built himself.

In his work on Agentic-OS, he treated the project as a security engagement, combining a read-only review of code and configuration with dynamic testing of the full stack in a controlled local environment.

The goal was not simply to discover unusual vulnerabilities. It was to identify cases where a security control existed and appeared reasonable but provided less protection than its documentation or his assumptions suggested.

Agentic-OS was particularly useful for this exercise because the observability pipeline has its own security requirements, including authentication, privacy filtering, storage boundaries, separation of public and private telemetry, and monitoring of the monitoring system.

The exercise reinforced the limits of reviewing configuration files in isolation. Some security properties can only be established by exercising the actual running component or testing against the real shape of its data.

He also distinguishes between a potential concern and a demonstrated security failure.

When prioritizing an issue, he considers the complete path from an initial condition to an exploitable action, a resulting state change, and the actual impact. A weakness that demonstrably crosses an authorization boundary or exposes sensitive information deserves a different priority from a configuration improvement without a realistic path to harm.

This evidence-based approach helps avoid two mistakes: overlooking a meaningful vulnerability because the configuration appears correct, and overstating a potential weakness before demonstrating its consequences.

The question remains consistent throughout his work: which security property failed, and what evidence proves that it failed?

A Security Model Built Around Boundaries and Least Privilege

These experiences inform how Bellingeri approaches security for RAG applications and AI agents.

His central principle is that content is not authority.

A retrieved document may provide useful evidence without being allowed to control the system. The same principle applies to web pages, memory entries, tool outputs, and responses generated by other models.

To maintain this separation, he distinguishes among system instructions, retrieved data, model reasoning or recommendations, and authorized actions. Prompt injection becomes more dangerous when those boundaries collapse and untrusted content can influence decisions beyond its intended role.

He applies least-privilege thinking to agents in much the same way he would to conventional software services.

An agent that needs to read a resource does not automatically need permission to modify it. An agent that can recommend an action does not necessarily need authority to execute it. Higher-impact actions should have stronger policy boundaries and, where appropriate, human approval.

He also prefers architectures in which a single model does not simultaneously act as the sensor, interpreter, executor, and final authority on whether an action was correct.

Instead, he favors a flow in which untrusted input is processed through reasoning, policy and authorization, tool execution, and independent observation.

Security cannot depend solely on instructing a model to avoid dangerous behavior. The surrounding system must enforce the boundaries that matter.

Making security compatible with development speed

Bellingeri does not advocate adding maximum friction to every operation. He sees two unhelpful extremes: treating security as something to add at the end, and imposing so much manual overhead that developers begin working around the controls.

His preference is to move security into architecture, safe defaults, and automation.

Dependency checks, secret scanning, static analysis, workflow security, and build validation can run automatically in continuous integration (CI). This reduces the need for developers to remember and apply every control manually.

The level of control should also reflect the capability and potential impact of an operation. A read-only retrieval task and an agent capable of modifying files or executing commands should not necessarily follow the same approval process.

The objective is to make security proportional to capability, uncertainty, and impact, so that it becomes part of the development workflow rather than an obstacle added from outside.

Production Readiness Means Testing What Actually Ships

Bellingeri's approach to verification extends beyond AI systems. His production website combines security controls, accessibility, automated testing, and performance optimization.

One example involves his use of a strict, hash-based Content Security Policy (CSP) without unsafe-inline. During implementation, syntax highlighting introduced inline styles that were incompatible with the intended policy.

The straightforward solution would have been to weaken the policy. Instead, Bellingeri removed the incompatible behavior and added tests against the built output.

Those CI checks verify the HTML that will actually reach production, including inline scripts, hashes, style attributes, and other relevant properties that may only become apparent after the build process.

The decision reflects a broader engineering principle: verifying source code alone is not always enough. Production behavior depends on the generated output, dependencies, browser behavior, network conditions, unexpected inputs, and operational boundaries.

His website also uses a broader set of engineering practices, including automated testing, dependency checks, secret scanning, static analysis, accessibility validation, performance optimization, supply-chain controls, and observability.

The common thread is meaningful verification. A control that remains green because it never observes the property that matters can create a false sense of security.

Why Public Technical Writing Improves Engineering

Bellingeri initially shared projects because he enjoyed building software and documenting what he learned. Over time, publishing technical work changed how he approached engineering.

When code stays private, it can be easy to assume that an implementation works for the reasons the developer believes it does. Publishing a technical claim introduces a more demanding question: how can that claim be demonstrated?

For Bellingeri, that pressure is useful. Public technical writing and open-source work have encouraged greater attention to assumptions, reproducibility, scope, and evidence.

He also sees value in documenting failures rather than publishing only successful implementations.

His prompt-injection article came from a test that was wrong. He could have corrected the test privately and moved on, but explaining why the protection appeared correct while failing to provide the intended security offered a more useful engineering lesson.

Public work also creates opportunities for peer review. Other developers can challenge assumptions, propose better experiments, and identify failure modes that the original author may have missed.

That feedback can influence how a system evolves.

For Bellingeri, sharing technical findings is therefore not separate from building software. It is another way to examine whether the claims made about that software can withstand scrutiny.

What Comes Next: Identity, Trust, and Agent Security

Bellingeri is particularly interested in the security of increasingly capable AI agents.

As AI systems move beyond generating text to using tools, maintaining memory, calling APIs, modifying files, executing code, and coordinating with other agents, their trust requirements become more complicated.

One question concerns the relationship between an agent's identity and its changing configuration.

If an agent has a durable identity but its model, system prompt, toolset, permissions, or runtime configuration changes, should the trust previously associated with that identity automatically carry over?

Bellingeri does not think it should.

He wants to explore systems in which trust remains scoped to the configuration and conditions under which the supporting evidence was earned. A change in capabilities or runtime conditions should not automatically inherit every previous assurance.

The Model Context Protocol (MCP) is another area of interest. It can make it easier to connect models to capabilities, but each additional capability introduces another trust boundary that needs to be considered.

He is also interested in adversarial testing and observability for agent systems. Ideally, he wants to reconstruct the full path from a claim to a decision, a tool invocation, the actual action, the resulting state, and the evidence that confirms what happened.

The agent's own explanation should not be the only source of truth.

The Engineering Lesson: Confidence Is Not Evidence

Looking back at his projects, Bellingeri finds it difficult to select just one. MonferrinoAI, Agentic-OS, and LLM-Council each represent a different stage of his exploration of AI, cloud systems, and security.

But the lesson he values most comes from LLM-Council and the prompt-injection test that passed while the underlying security property remained broken.

The significance was not simply that he found a weakness. It was that the implementation and its test suite had both appeared correct to the person who designed them.

That experience forced him to challenge his own assumptions and reconsider what a successful test actually proved.

His advice to developers building AI applications is to treat an LLM as one component in a larger system, surrounded by familiar engineering principles: input validation, least privilege, observability, failure handling, testing, and separation of responsibilities.

He also encourages developers to treat externally sourced information as untrusted, avoid giving models more authority than necessary, and test security properties rather than only happy paths.

If a system claims to prevent an attack, deliberately weaken the protection and check whether the tests notice. If an agent is supposed to be unable to access a resource, test the boundary directly.

This approach brings together the ideas behind his projects. A green test does not automatically prove that a system is secure. Multiple models agreeing does not guarantee that an answer is correct. A model reporting that it performed an action does not prove that the action occurred.

The responsibility of an engineer is to understand exactly what a control claims to guarantee, identify the evidence supporting that claim, and verify that the mechanism used to evaluate it can itself detect failure.

That is the thinking behind Bellingeri's personal engineering motto:

“I build it. I break it. I secure it.”

About Marco Bellingeri

Marco Bellingeri is a Cloud & AI Security Engineer based in Italy, working across Python, AWS, DevSecOps, LLM engineering, RAG, AI agents, and cloud infrastructure. His work focuses on building and evaluating systems where AI capabilities meet practical security and operational requirements.

His projects include MonferrinoAI, which explores AI-assisted website knowledge management and content planning; Agentic-OS, which focuses on observability for AI-assisted development; and LLM-Council, which explores multi-model response evaluation and adversarial testing.

He is particularly interested in AI agent security, MCP security, prompt injection, LLM observability, and testing that demonstrates whether security properties actually hold.

CoderLegion: Marco Bellingeri's profile
Website: marcobellingeri.dev

Developer Stories is a CoderLegion interview series highlighting the people, ideas, and engineering decisions behind software development.

From Code Quality Metrics to Explainable Insights

Sun, 25 Oct 2026 11:03:02 +0000

Interviewed by: Mehadi Hasan

Developer: Fady Desoky Saeed Abdelaziz

Role: Data Analyst | Data Analytics & Data Engineering | Enterprise Systems & Process Optimization

Location: Cairo, Egypt

Software engineering, data analytics, and research are often treated as separate disciplines. For Fady Desoky Saeed Abdelaziz, however, they are closely connected by a common question: how can technical information be transformed into something people can understand and use to make better decisions?

Fady's experience spans backend development, software testing, business process optimization, enterprise reporting, machine learning, and applied research. These experiences have shaped an approach to engineering that goes beyond implementation. He is interested in understanding the problem behind a system, evaluating the reliability of its data, and presenting technical findings in a way that supports practical decisions.

That perspective is evident in his projects. Qualioro explores how software quality metrics can become understandable recommendations. His Retail FMCG Sales Analytics project connects transaction data, predictive modelling, dashboards, and an interactive application. His distributed-systems work investigates how services behave when failures occur, while his research examines the relationship between computing resources and energy consumption.

Across these projects, a consistent theme emerges: technical measurements become more valuable when their meaning, limitations, and implications are made clear.

A Journey Across Software Engineering and Data

Fady's journey began with his studies in Software Engineering at Sadat Academy for Management Sciences. He was interested in understanding how software systems work, how applications are designed, and how technology can solve practical problems.

His early experience with backend development and software testing introduced him to application logic, database interactions, and reliability. These foundations helped him understand the importance of building systems that behave correctly and remain maintainable.

His subsequent work in business process optimization and data analytics expanded that perspective. Instead of considering software only in terms of its implementation, he began examining how systems operate within larger organizations, how workflows affect the people who use them, and how data influences operational decisions.

Research added another dimension. His work on sustainable software engineering encouraged him to investigate computing resource consumption, while his urban mobility research introduced him to the challenges of analysing real-world location data.

Each area contributed a different way of approaching technical problems. Software engineering emphasizes architecture, correctness, and maintainability. Process optimization focuses on workflows and inefficiencies. Data analysis encourages evidence-based reasoning and the validation of assumptions.

As Fady explains, working across these disciplines has helped him avoid jumping straight into implementation before understanding the actual problem.

This approach also influences how he develops new skills. Rather than studying technologies in isolation, he prefers applying them to practical projects that connect with his existing experience and the problems he wants to solve.

What Enterprise Operations Taught Him About Data Reliability

One of the most influential parts of Fady's experience has been his work with Nestlé Business Solutions across HR Operations Process Optimization and Data Analytics & Reporting.

In his earlier process optimization work, he contributed to a regional HR system migration initiative involving multiple countries. The work required an understanding of existing workflows, dependencies between systems, and the consistency of historical employee records.

His profile also describes involvement in the MENA Gratuity Project, a regional migration initiative across six Gulf Cooperation Council countries. His responsibilities included mapping workflow dependencies, identifying data gaps and potential migration risks, documenting processes, and supporting historical data validation.

These activities illustrate a practical challenge in enterprise technology: moving information between systems is not simply a matter of transferring records. The information must remain accurate and consistent as processes and systems change.

Later, his work in data analytics and reporting involved business data used in dashboards and operational reporting across regional markets. His profile describes reporting frameworks and data workflows supporting operations across more than ten MENA and Gulf markets, alongside reporting activities involving more than 60 retail partners.

Together, these experiences reinforced an important lesson about reliability.

A system can function correctly from a technical perspective and still produce unreliable results if its underlying data is inaccurate, incomplete, or inconsistent. An error in a source record or transformation can propagate into a report and potentially influence a business decision.

For Fady, data validation, process documentation, and consistency checks are therefore essential parts of reliable systems, not secondary tasks to be considered after implementation.

This principle carries through to his independent projects, where he places particular emphasis on explaining what data represents and what conclusions can reasonably be drawn from it.

Qualioro: Making Software Quality Metrics More Useful

Software quality tools can generate measurements that indicate potential weaknesses in a codebase. However, a score by itself does not necessarily explain what caused the result, why it matters, or what a developer should investigate first.

That gap motivated Fady to build Qualioro, a software quality intelligence prototype designed to connect measurable indicators with understandable explanations and targeted recommendations.

Qualioro evaluates indicators such as bug density, code complexity, test coverage, and team output. It then presents an overall quality score, a risk classification, a confidence indicator, and recommendations based on weaker areas.

The idea is to move beyond simply displaying metrics. Developers should be able to explore the factors contributing to an assessment and identify areas that may deserve further attention.

Fady built the application using Next.js 15, TypeScript, Tailwind CSS, shadcn/ui, and Recharts. Its workflow separates the evaluation of input metrics from the interpretation of results and their presentation in an interactive interface.

The interface includes quality scoring, risk breakdowns, visualizations, and a what-if simulation. By adjusting inputs, users can observe how the overall assessment changes. An advanced view also exposes more of the scoring breakdown, making the result easier to inspect.

These features reflect a central design consideration: an assessment is more useful when users can understand how it was produced rather than being asked to accept a number without context.

Explainability Without False Confidence

One of the challenges in designing Qualioro is balancing simplicity with accuracy.

A simplified scoring model can demonstrate how different quality indicators might contribute to an assessment. However, its output should not be mistaken for a definitive evaluation of an actual software repository.

Fady makes this distinction clear in the current implementation. The analysis runs entirely on the client side, without sending or storing user data through an external service. The model uses simplified inputs to demonstrate the concept; it does not automatically analyse arbitrary repositories as a production-grade system.

He sees real repository integrations, automated metric extraction, CI/CD data, and broader evaluation as possible future extensions.

His approach to explainability can be summarized through three questions:

What was detected?
Why might it matter?
What should the developer investigate next?

These questions provide a useful framework for turning technical measurements into actionable information.

The broader lesson is that transparency is not simply a matter of presenting information clearly. It also means communicating uncertainty and avoiding claims that the available evidence cannot support.

Understanding Failure in Distributed Systems

Fady's interest in reliability also extends to distributed applications.

His Mobile Cloud System project explores a simulated cloud-native microservices architecture through an interactive observability dashboard. Rather than treating a distributed application as a single unit, the project examines how multiple services communicate, how failures can affect connected components, and how developers can understand what happens when a workflow breaks down.

The project explores several important distributed-systems concepts, including the Saga pattern, retries, circuit breakers, and observability.

Each addresses a different aspect of reliability. Retries can help recover from temporary errors. Circuit breakers can prevent repeated calls to a service that is experiencing problems. Distributed transaction patterns such as Saga help coordinate operations across service boundaries.

These mechanisms are important because distributed applications introduce failure points that are less straightforward than those in a single component. A service may become unavailable, communication may fail, or an error in one part of a workflow may affect subsequent operations.

For Fady, the project reinforced a fundamental engineering principle: reliable distributed systems must be designed with failure in mind rather than assuming every request will succeed.

Observability is equally important. When a workflow fails, developers need enough information to identify where the failure occurred, understand its effects on other services, and determine whether the system recovered as expected.

The project provides a way to explore these behaviours in a simulated environment. Its value lies in making distributed-system interactions and failure-handling concepts easier to examine.

Together with Qualioro, it demonstrates Fady's interest in building tools that help developers understand not just what a system does, but also how and why it behaves in particular ways.

From 100,000 Retail Transactions to Business Insights

Fady's Retail FMCG Sales Analytics project approaches a different problem: how to transform a large transaction dataset into information that can support business decisions.

The project uses a dataset containing 100,000 transactions and 21 variables covering products, customers, revenue, costs, stores, sales channels, payment methods, and inventory.

Rather than focusing on a single analytical technique, Fady combined data preparation, exploratory analysis, visualization, predictive modelling, and an interactive application into one workflow.

He used Python, Pandas, and NumPy for data preparation and analysis, Scikit-learn for machine learning, Power BI for interactive dashboards, and Streamlit to make the analytical workflow accessible through an application.

The analysis explored questions such as which product categories and brands performed best, how sales channels compared, what patterns appeared in customer behaviour, and where inventory or profitability issues might warrant further investigation.

The project also included a Linear Regression model designed to predict margin using available business and transactional features.

The documented evaluation results were:

R²: 0.863
Mean Absolute Error (MAE): 19.28
Root Mean Squared Error (RMSE): 27.64

These metrics describe different aspects of the model's performance. R² indicates how much of the variation in the target variable is explained by the model under the evaluation setup, while MAE and RMSE measure prediction errors using different approaches.

The figures provide an evaluation of this particular model and dataset; they should not be interpreted as a guarantee of performance on other retail datasets or in other business environments.

For Fady, one of the project's most valuable aspects was connecting the stages of the analytical process. Data preparation established a foundation for analysis, exploratory work helped reveal patterns, dashboards made the results easier to investigate, and predictive modelling demonstrated how a model could become part of an interactive workflow.

The underlying lesson is that analytics is most useful when technical methods remain connected to a clear business question and the results are presented in a form people can understand and use.

The project is available on GitHub.

Sustainable Software Engineering: Looking Beyond Application Performance

Another important area of Fady's work is sustainable software engineering.

Software depends on physical computing infrastructure, and its execution consumes resources through components such as CPUs, GPUs, and memory systems. Understanding these demands can help engineers investigate resource efficiency alongside more familiar concerns such as correctness and performance.

Fady's research, Analyzing Software Resource Consumption Data for Sustainable Green Software Development: A Predictive Model to Reduce and Optimize Energy Consumption in Software, examines the relationship between resource-consumption variables and energy usage using a dataset covering 2020–2025.

The project reflects his interest in whether resource-consumption data can help explain and predict energy use, potentially supporting more informed engineering decisions.

This work connects software engineering with environmental responsibility. Developers may not control every aspect of the infrastructure on which their applications run, but they can measure resource consumption, investigate inefficiencies, and evaluate the effects of potential optimizations.

Fady's profile identifies him as the first author and presenter of this research at the AII 2025 International Conference in Washington, DC. The abstract was published in the conference proceedings, with the full paper described as being under preparation for publication through Springer Nature.

His interest in sustainability also complements his work on software quality and distributed systems. Each involves understanding system behaviour through evidence rather than relying solely on assumptions or surface-level indicators.

In software quality, the objective is to interpret metrics responsibly. In distributed systems, it is to understand failures and recovery. In sustainable software engineering, it is to investigate the relationship between resource use and energy consumption.

Across all three, measurement is a starting point for investigation rather than an end in itself.

Learning from Real-World Data: Urban Mobility Research

Fady's research interests also extend beyond software systems into urban mobility.

During his research internship at Nile University, he worked on GNSS-based urban traffic modelling using real-world GPS traces from urban traffic networks. The work involved processing large-scale mobility datasets and applying statistical and time-series modelling to forecast traffic congestion patterns.

He co-authored Graphing the Jam: GNSS-Powered Modeling and Forecasting of Urban Traffic Congestion, which was presented at the NetMob 2025 Data Challenge & Conference in Paris, France.

Working with real-world location data introduces challenges that differ from those encountered with simplified examples. The data must be processed and analysed carefully before it can support useful conclusions about traffic patterns.

This research complements Fady's wider interest in transforming complex datasets into predictive insights. Whether the subject is software resource consumption, retail transactions, or urban traffic, the analytical challenge is similar: identify meaningful patterns, evaluate them using appropriate methods, and communicate the results without overstating what the evidence establishes.

The publication is available through ResearchGate.

A Consistent Engineering Philosophy

Although Fady's projects cover different domains, several principles connect them.

Understand the problem before choosing the implementation. His experience in process optimization taught him to examine existing workflows, dependencies, and the people affected by a system before jumping into development.

Treat data quality as part of system quality. Enterprise reporting and migration work demonstrated how inaccurate or inconsistent information can undermine otherwise functional systems.

Make technical results understandable. Qualioro explores how measurements can be connected to explanations and recommendations, while the Retail FMCG project brings analytical results into dashboards and an interactive application.

Design for failure and uncertainty. His Mobile Cloud System project explores failure-handling mechanisms and observability, while Qualioro emphasizes the limitations of simplified scoring models.

Connect technical work to practical outcomes. Across analytics and research, Fady aims to investigate questions that matter beyond the implementation itself, whether they concern business performance, system reliability, or energy consumption.

These principles reflect an approach to engineering in which the technology is only one part of the solution. Understanding the context, validating the evidence, and communicating the results are equally important.

What Comes Next

Fady's current interests include sustainable and green software systems, environmental data analysis, predictive modelling, data engineering, and cloud-based data systems.

His projects also suggest several directions for further exploration. Qualioro could be extended through real repository integrations and automated metric extraction. The Retail FMCG analytics workflow demonstrates how data preparation, machine learning, and interactive reporting can work together. His sustainability research investigates how computing resource measurements may contribute to better-informed decisions about software efficiency.

These areas provide opportunities to deepen the connection between software engineering, data, and measurable outcomes.

Rather than treating software development, analytics, and research as isolated activities, Fady's work brings them together around a shared objective: understanding complex systems and making the information they produce more useful.

The result is an engineering perspective that values not only building a system, but also examining its behaviour, validating its outputs, and helping people make informed decisions based on what the evidence actually shows.

About Fady Desoky Saeed Abdelaziz

Fady Desoky Saeed Abdelaziz is a Cairo-based data analyst with a background in software engineering. His interests span data analytics, data engineering, enterprise systems, process optimization, machine learning, sustainable software engineering, and applied research.

His projects include Qualioro, a software quality intelligence prototype; Mobile Cloud System, a simulated microservices architecture with an observability dashboard; and Retail FMCG Sales Analytics, which combines transaction analysis, predictive modelling, Power BI, and Streamlit. His research includes sustainable software resource consumption and GNSS-based urban traffic modelling.

CoderLegion: Fady Desoky Saeed Abdelaziz's profile
Portfolio: Fady's professional portfolio
GitHub — Qualioro: github.com/Fadydesoky/Qualioro
GitHub — Retail FMCG Sales Analytics: github.com/Fadydesoky/Retail-FMCG-Analytics
Research — Urban Traffic Modelling: Graphing the Jam on ResearchGate

Developer Stories is a CoderLegion interview series highlighting the people, ideas, and engineering decisions behind software development.

From Sensors to Smartwatches: How Reetain Raina Explores the Technology Behind Everyday Experiences

Sun, 25 Oct 2026 02:26:58 +0000

Interviewed by: Mehadi Hasan

Developer: Reetain Raina

Role: Technical Writer Exploring the Future of Technology

Location: India

Technology often feels simple on the surface. A smartwatch displays your heart rate, tracks your sleep or reminds you to stand up. Behind that seemingly straightforward experience, however, lies a complex combination of sensors, signal processing, algorithms and software.

For Reetain Raina, that gap between what technology does and how it actually works is where the most interesting questions begin. A technical writer whose interests span the Internet of Things (IoT), embedded systems, artificial intelligence, cybersecurity and consumer technology, Raina enjoys investigating the engineering behind familiar devices and translating it into explanations that readers can understand without losing the technical detail.

His recent writing has increasingly focused on wearable technology and health data, including how AI models interpret information collected by everyday devices and what happens when people follow every notification their smartwatches send them.

But his approach goes beyond explaining features. Through hands-on experiments, technical research and persistent curiosity, Raina explores a deeper question: what does technology actually mean for the people who use it?

From Circuits and Sensors to Technical Writing

Raina did not initially set out to become a technical writer. His interest in technology began much closer to the hardware, with a fascination for circuits, sensors and the way small components could interact with the physical world.

That curiosity led him toward IoT and embedded systems, where hardware and software work together to sense, process and respond to their surroundings.

Writing gradually became another way to explore these subjects. When Raina tried to explain how a sensor worked or how an embedded system made decisions, he often found himself developing a deeper understanding of the technology in the process.

Over time, that connection between learning and explaining became one of the most rewarding parts of his work.

As his interests expanded, he began exploring artificial intelligence, cybersecurity, cloud computing and consumer technology. More recently, health technology has become a particularly interesting area, bringing together several subjects that have long captured his attention: sensors, connected devices, AI and data.

The progression makes sense. A smartwatch, for example, might present a single heart-rate measurement to its wearer, but producing that number involves several underlying processes. Understanding those processes requires looking beyond the interface to the hardware, algorithms and software working behind it.

For Raina, technical writing provides a way to investigate that complexity and make it accessible.

The goal is not simply to make difficult concepts sound easy. It is to understand them well enough to explain what is happening underneath while preserving the details that make the technology worth understanding.

Choosing Technologies Through Curiosity, Not Hype

With new technologies emerging constantly, deciding what deserves attention can be difficult. Raina's approach is to begin with a question rather than a trend.

As he puts it:

"Okay, but how does this actually work?"

That question helps him look beyond the headlines surrounding a technology. He is interested in whether it solves a real problem, changes how people interact with something or makes an experience possible that was previously difficult to achieve.

The engineering itself also matters. Sensors, algorithms, systems and AI may operate largely out of sight, but they often determine whether a product feels intuitive and useful.

Sometimes, however, there is no elaborate selection process. A technology simply captures his attention and continues occupying his thoughts long after he has finished reading about it.

When that happens, it may become the subject of an article.

This approach gives his writing a distinctive focus. Rather than merely documenting what a product or technology claims to do, he looks for the underlying mechanisms and the questions that remain unanswered.

It also creates room for topics that might otherwise be overlooked. A familiar feature can become an interesting technical story when examined from the perspective of the systems that make it possible.

Why Wearable Technology Has Become a Focus

Wearable technology brings together many of the subjects that initially attracted Raina to technology: sensors, connected devices, software and the physical world.

A smartwatch, smart ring or pair of smart glasses can collect information continuously while remaining small enough to fit naturally into everyday life. The challenge is not simply collecting that information. It is determining what it means and how it can be used.

Raina is particularly interested in how AI and health technology are changing the possibilities for wearable devices.

Companies are exploring capabilities beyond traditional fitness tracking and notifications. Examples he highlights include Huawei's work on blood-pressure monitoring in smartwatches and Meta's exploration of neural interfaces that could enable interaction through subtle signals from the hand.

These examples represent different approaches to wearable interaction, but they point toward a broader question: how much more can these devices do beyond the features people already use?

For Raina, the excitement lies partly in the fact that the category is still evolving. New sensors, AI applications and approaches to health monitoring continue to create opportunities for investigation.

That uncertainty is valuable to a writer who enjoys asking questions. Rather than treating wearable technology as a finished category, he sees it as an area where the next useful application may emerge from an unexpected combination of hardware, software and data.

What AI Can Reveal About Wearable Data

In his CoderLegion article, What Happens When ChatGPT, Gemini and Claude Analyze Your Wearable Data?, Raina explores how AI models might help people make sense of information collected by everyday devices.

One of the most interesting aspects of this subject is how much context can be hidden inside seemingly ordinary measurements.

A wearable might report reduced sleep, an increase in resting heart rate or a decline in activity. Considered individually, each measurement offers only a limited view. Examined together, they may reveal a pattern worth investigating.

AI can potentially help connect these separate observations and explain their relationships in more accessible language.

For Raina, the comparison between ChatGPT, Gemini and Claude raised another interesting question: what happens when different AI models examine the same information?

Models may identify similar changes but emphasise different patterns or offer different explanations. This makes the comparison more interesting than a simple contest over which model is smarter. It raises questions about how AI systems interpret information and why their conclusions can differ.

However, identifying a pattern is not the same as establishing its cause.

If someone sleeps less and experiences an increase in resting heart rate, an AI model might highlight the relationship between those observations. That does not automatically establish why either change occurred.

This distinction is especially important when wearable data concerns health. An explanation generated from measurements should not be confused with a confirmed medical conclusion.

Raina's central interest is therefore not simply whether AI can generate another health score. It is whether AI can help people understand the information they already collect and ask better questions about what it might mean.

That perspective also highlights the importance of communicating uncertainty. Making data easier to interpret is valuable, but a useful explanation must preserve the difference between an observed pattern and a proven explanation.

Explaining the Engineering Without Losing the Detail

Wearables offer a useful example of why technical communication requires more than simplifying terminology.

A device may rely on sensors to collect measurements, Bluetooth to communicate with another device and software to process the resulting information. Explaining the overall process requires understanding how these components fit together.

Raina believes the first step is to understand the technology properly before attempting to explain it.

"I can’t simplify something I haven’t properly understood myself."

That principle influences how he approaches technical subjects. Rather than stopping at a statement that a wearable uses sensors to measure something, he tries to explain what the sensor detects, what happens to the data afterwards and why the final result matters to the person using the device.

He describes the task as translating between two languages.

On one side is the language of sensors, algorithms, Bluetooth and data processing. On the other is the reader's practical question: what is happening, and why should I care?

Good technical writing connects those perspectives without sacrificing accuracy.

This is particularly relevant when explaining sensor fusion, where information from multiple sensors can be combined to produce a more useful interpretation of a person's activity or environment. Readers do not necessarily need to understand every implementation detail, but they benefit from knowing that a displayed result is often the outcome of several processing steps rather than a direct, self-explanatory measurement.

The same principle applies to the scores presented by wearable applications.

A readiness score of 82 or a sleep summary may be convenient, but the number alone does not explain how it was produced. Understanding something about the sensors, algorithms and processing behind it can change how a person interprets the result.

Raina does not expect every consumer to become an engineer. Instead, he wants readers to understand enough of the underlying technology to look at familiar features with greater awareness.

For him, a concept that remains difficult to explain is also a signal to return to the research and develop a better understanding before publishing.

A Smartwatch Experiment That Became More Than a Feature Review

Not all of Raina's writing begins with technical research. Sometimes, the most revealing questions emerge from using a device in an unusual way.

His article I Followed Every Notification My Smartwatch Sent Me. It Got Weird Fast grew out of a simple experiment: he decided to follow every notification his smartwatch sent him for an entire day.

If the watch told him to stand, he stood. If it sent a stress alert, he stopped what he was doing. When it reminded him to complete his step goal, he even found himself walking around at night to finish it.

The experiment began as a fun way to explore the device's features. But following the notifications throughout the day changed his perspective on the relationship between the user and the technology.

Initially, the reminders felt motivating. As the day progressed, however, he began to feel as though the watch was managing him rather than helping him.

That shift was the most revealing part of the experience.

A product description can explain that a smartwatch sends reminders, encourages movement or provides alerts. It cannot fully capture how those features feel when they begin directing a person's behaviour throughout the day.

By following the instructions rather than merely observing them, Raina discovered a different angle from which to examine the technology.

The experiment raised questions about how easily people begin trusting the numbers and instructions produced by their devices, even when they may not stop to consider whether every prompt is useful in that particular moment.

It also demonstrated the value of first-hand observation in technical writing. Using a product can reveal practical consequences that specifications and feature lists leave out.

For Raina, that is often where a stronger story begins: not with what a technology promises to do, but with what happens when someone actually lives with it.

Why Human Judgment Still Matters in the Age of AI

As AI systems become increasingly capable of explaining technical concepts, summarising research and generating detailed content, technical writers face an important question about their role.

Raina does not believe human writers remain valuable simply because they can produce better sentences than AI. Modern AI tools can already generate clear explanations and organise complex information effectively.

Instead, he places greater importance on the questions that come before the writing.

Is a technology genuinely useful? What is happening underneath its interface? What happens when someone uses it in real life? Are there limitations or unintended consequences that deserve attention?

These questions require judgment about what matters and what deserves further investigation.

Raina's own work illustrates the distinction. Comparing AI interpretations of wearable data goes beyond asking a model to describe a feature. Following every smartwatch notification turns a familiar product capability into an experiment about human behaviour. Investigating the sensors behind wearable measurements reveals the engineering hidden beneath a simple display.

In each case, the value comes from looking beyond the obvious explanation.

He sees AI as another tool in the research and writing process rather than something that necessarily removes the need for human technical writers.

As he explains:

"AI can help explain the technology, but the human still has to decide what is worth asking, what is worth questioning and what the reader actually needs to understand."

That perspective places curiosity and critical thinking at the centre of the work. AI can assist with generating explanations, but a writer still needs to decide whether those explanations are useful, sufficiently supported and relevant to the reader.

Researching Emerging Technology Without Taking Claims at Face Value

Writing about rapidly changing technology creates another challenge: the information available today may become outdated surprisingly quickly.

Raina approaches unfamiliar subjects by combining hands-on exploration with deeper research. Whenever possible, he starts by using the technology himself. Direct experience often produces questions that would not emerge from reading a product description alone.

Those questions then guide his research.

He spends time reading research papers and material from reputable research institutions and publications, particularly when investigating AI, sensors, wearables and health technology. A single technical detail can lead him into a much deeper investigation as he tries to understand what is actually happening underneath the surface.

He also cross-checks important claims instead of accepting the first explanation he encounters.

This is particularly important in areas where new products and capabilities are announced frequently. Even a credible source may become outdated as the technology develops.

His approach is iterative rather than strictly linear: use the technology, read the research, question what appears to be happening, and investigate further.

The process also provides a useful test of understanding. If he cannot explain a concept clearly after researching it, he treats that as a reason to keep learning rather than simply simplifying the explanation until it sounds convincing.

For technical writers, this is an important distinction. Clear language is only useful when it accurately represents the underlying subject.

The Future of Computing: Technology That Asks Less of Our Attention

Among the areas Raina is currently exploring are AI agents, ambient computing, smart glasses and Edge AI.

He is particularly interested in the possibility that AI agents and ambient computing could work together to make technology less visible in everyday life.

Today, interacting with technology often requires deliberate action. People open an application, look at a screen, type a request or issue a command. Raina sees an alternative in systems that can understand more of their surrounding context and assist without constantly demanding attention.

Smart glasses could bring computing more directly into a person's surroundings, while Edge AI could allow some processing to happen closer to the device. AI agents that can take actions, rather than merely respond to questions, could further change how people interact with digital systems.

These technologies are still developing, and their practical impact will depend on how effectively they work together. Nevertheless, the direction interests Raina because it shifts attention away from simply adding more features and toward changing the nature of the interaction itself.

As he sees it, the most significant change may not be the amount of technology people have around them. It may be how naturally that technology fits into their lives.

This interest also connects with his work on wearables. In both cases, the challenge is to make increasingly sophisticated systems useful without requiring people to understand or manage every underlying process.

Advice for Aspiring Technical Writers

For people who want to start writing about technology, Raina's advice is straightforward: do not wait until you feel you know everything.

His own journey began with curiosity about circuits, sensors, IoT and embedded systems. Understanding developed through the process of investigating subjects and trying to explain them.

He encourages aspiring writers to follow the questions that arise during research. A small detail that initially seems unimportant can lead to a more interesting article if the writer takes the time to understand it.

He also advises writers not to simply repeat what a company or another publication has already said.

Even when a technology has been covered many times, there may still be an opportunity to offer something new through a personal observation, a practical experiment or a different question.

That does not necessarily mean building something entirely new or conducting an elaborate experiment. It means approaching the subject with enough curiosity to discover what is genuinely interesting about it.

Raina's smartwatch experiment demonstrates how this can work. A familiar feature became the starting point for an article about the experience of following a device's instructions. His exploration of AI and wearable data similarly moves beyond the existence of health measurements to examine how those measurements might be interpreted and where that interpretation has limits.

His advice reflects the same principle that guides his own work: expertise can develop through writing, provided the writer remains willing to investigate, question and learn.

What Comes Next: Building at the Intersection of AI, Wearables and Health

Looking ahead, Raina wants to explore the intersection of AI, wearable devices and health technology more deeply.

He has spent considerable time examining sensors and the information these devices collect. His next area of interest is what happens after that data has been gathered: how AI can interpret it and potentially make it more useful to the people who generate it.

He also wants to investigate AI agents, ambient computing, smart glasses and Edge AI further.

Beyond writing, he hopes to spend more time building things himself. Technical writing has taught him how to investigate problems and understand the systems behind them, and he would like to apply that curiosity to creating technology as well.

He does not have one fixed project or destination in mind yet. Instead, he remains drawn to ideas involving health, connected devices and AI.

That openness is consistent with the approach that brought him into technical writing in the first place. Rather than committing to a subject simply because it is popular, he follows the questions that continue to interest him and allows the research to reveal where they lead.

For Raina, the process of learning is not separate from the work of writing. It is a central part of it.

His experience offers a useful reminder that understanding technology involves more than knowing what a product does. It means looking beneath the interface, testing assumptions, investigating limitations and asking what the experience actually means for the people using it.

Whether he is examining a sensor, comparing AI interpretations of wearable data or following every notification from a smartwatch, the underlying motivation remains the same: to understand the technology well enough to explain not only how it works, but why it matters.

About Reetain Raina

Reetain Raina is a technical writer based in India who writes about emerging technologies, AI, consumer technology and digital trends. His interests include IoT, wearable technology, health tech, cybersecurity, cloud computing and the engineering behind connected devices.

He enjoys turning complex technical topics into practical explanations that help readers understand how technology works and how it shapes everyday experiences. He is currently exploring AI agents, ambient computing, smart glasses and Edge AI, with a growing interest in building technology related to AI, connected devices and health.

CoderLegion: Reetain Raina
LinkedIn: Reetain Raina

Developer Stories is a CoderLegion interview series highlighting the people, ideas, projects, and engineering decisions behind software development.

The Truck Driver Building His Own Programming Language- Meet Bryan MacLee

Thu, 15 Oct 2026 22:44:46 +0000

Developer Spotlight: Meet Bryan MacLee — The Truck Driver Building His Own Programming Language

🚀 What if you worked long shifts in the oil and gas industry and spent your spare time building your own programming language?

Meet Bryan MacLee, the creator of Scriml, a programming language designed to rethink how web applications are built.

In this CoderLegion Developer Spotlight, host Tom Smith talks with Bryan about his programming journey, the ideas behind Scriml, and the challenges of building a compiler independently.

💡 What You'll Discover

The beginning: How Bryan discovered programming as a teenager.
Life beyond coding: Balancing oilfield truck-driving work with software development.
Building Scriml: Why he decided to create his own programming language.
Simplifying web development: His approach to frontend-backend communication, validation and compilation.
The compiler challenge: What it's like to build a programming language largely on your own.
What's next: Bryan's hopes for Scriml and getting other developers to try it.

🎥 Watch the Interview

[Insert your YouTube video link here]

👨💻 Meet the People Behind the Interview

Guest: Bryan MacLee
Host: Tom Smith

🌟 Building Something of Your Own?

CoderLegion Developer Spotlight celebrates developers, independent builders and people exploring new ideas.

If you're working on an interesting project, experimenting with a new technology or learning something challenging, we'd love to hear your story. Get involved in the community and you could be featured in a future Developer Spotlight.

🌐 Join the CoderLegion community

💬 Let's Discuss

If you could redesign one part of modern web development, what would you change and why?

Share your thoughts in the comments!

DeveloperSpotlight #Programming #WebDevelopment

AI Code: Safe or Not? Jonathan Gordon on the Hidden Risks of Vibe Coding

Tue, 13 Oct 2026 03:05:33 +0000

AI can generate working software in seconds. But how do we know whether that code is secure, reliable and ready for production?

In this episode of CoderLegion Developer Spotlight, we sit down with Jonathan Gordon, founder and CEO of Reweaver AI, to explore one of the biggest challenges in AI-assisted software development: the gap between what we intend to build and what AI actually produces.

Jonathan brings decades of experience building developer tools and experiences at Google, Apple, Microsoft, Oracle and SAP. Throughout his career, he has focused on helping developers build better software through thoughtful design and a deeper understanding of the people using their tools.

In our conversation, Jonathan shares how his experience with vibe coding exposed a problem he couldn't ignore. AI-generated code could appear to work while potentially introducing security vulnerabilities or hiding failures that developers might not notice.

What We Discuss

The rise of vibe coding: Why generating software from natural-language prompts feels revolutionary.
The problem of code drift: How implementations can diverge from the original design, requirements and intent.
Hidden software failures: Why empty exception handlers can cause failures without clearly notifying users.
AI-generated code quality: Why working code isn't necessarily production-ready code.
Reweaver AI: Jonathan's approach to analysing generated code, identifying problems and helping developers address them.
Human oversight: Why developers still need to understand, review and control the software they build with AI.
Lessons from Big Tech: What Jonathan learned while building developer tools at Google, Apple, Microsoft, Oracle and SAP.
The future of AI-assisted development: How to move beyond the hype and use AI to build software more reliably.

One of the most interesting moments comes when Jonathan describes spotting potentially vulnerable code while experimenting with AI-generated software. The experience helped shape his thinking about how to bring more predictable, reliable outcomes to AI-assisted development.

His message isn't that AI has no place in software development. It's that we need better ways to harness its capabilities while maintaining software quality and keeping humans in control.

👤 Meet Our Guest: Jonathan Gordon

Jonathan Gordon is the founder and CEO of Reweaver AI in Seattle. He has spent decades working on developer tools, integrated development environments, data analytics tooling and user experiences across major technology companies.

His experience spans complex software systems, developer experience and human-centred design. Today, his work with Reweaver AI focuses on analysing AI-generated code and helping developers identify problems that might otherwise go unnoticed.

🔗 Jonathan Gordon on CoderLegion: https://coderlegion.com/user/JonathanGordon

🎙️ Meet Your Host: Tom Smith

Tom Smith hosts this episode of CoderLegion Developer Spotlight, exploring Jonathan's career, his approach to developer experience and the challenges of maintaining software quality as AI-generated code becomes more common.

Through conversations with developers, founders and technology leaders, CoderLegion Developer Spotlight highlights the people, ideas and projects shaping software development.

🔗 Tom Smith on CoderLegion: https://coderlegion.com/user/Tom Smith

🎥 Watch the Full Interview

Want to hear Jonathan explain these ideas in his own words?

Watch the full CoderLegion Developer Spotlight interview on our YouTube channel and discover why AI-generated code quality, security and human oversight matter more than ever.

🚀 About CoderLegion Developer Spotlight

CoderLegion Developer Spotlight features developers, founders and builders sharing their journeys, projects, technical insights and lessons learned along the way.

From software engineering and artificial intelligence to open source and entrepreneurship, we want to showcase the people building, experimenting and solving real-world problems.

Want to be featured in a future Developer Spotlight interview?

If you're a developer, founder or builder working on something interesting, we'd love to hear your story. Join the CoderLegion community and get in touch to explore an opportunity to be featured.

🌐 Visit CoderLegion: https://coderlegion.com

AI #SoftwareEngineering #VibeCoding #DeveloperTools #CoderLegion

Beyond the Prompt: Igor Ganapolsky on Building Safer, More Reliable AI Agents

Fri, 09 Oct 2026 21:01:07 +0000

Interviewed by: Mehadi Hasan
Guest: Igor Ganapolsky
Role: Founder & Forward Deploy AI Engineer, ThumbGate
Location: Florida, USA
Website: igorganapolsky.com
Product: ThumbGate

Introduction

AI coding agents are becoming increasingly capable. They can inspect repositories, modify files, execute commands, and attempt to solve complex engineering problems with limited human supervision. But as these agents gain more autonomy, an important question emerges: how do we prevent an AI agent from making a mistake that damages the very system it is supposed to improve?

For Igor Ganapolsky, Founder and Forward Deploy AI Engineer at ThumbGate, the answer lies in applying established engineering principles to a new class of software systems.

Drawing on his experience with distributed systems, high-throughput data processing, trading infrastructure, and production reliability, Igor approaches AI safety as an infrastructure problem rather than simply a prompt-engineering challenge.

His work focuses on a crucial distinction: telling an AI agent what not to do is different from technically preventing it from doing it.

Through ThumbGate, Igor is exploring how deterministic checks, pre-action enforcement, and persistent operational lessons can help make autonomous AI development safer and more reliable.

From Distributed Systems to AI Engineering

Igor's approach to AI safety is rooted in a principle familiar to engineers working with complex production systems: critical guarantees must be enforced, not merely assumed.

His experience with distributed databases, concurrency, financial infrastructure, and high-throughput systems shaped the way he thinks about reliability. In these environments, a component behaving unexpectedly can have consequences far beyond its immediate task.

As Igor explains, if an invariant is not enforced deterministically at the execution boundary, it will eventually fail in production.

He applies the same reasoning to AI agents.

Unlike traditional software that follows explicitly defined logic, AI agents use probabilistic models to decide what to do next. They can interpret instructions, write code, and use tools, but they can also misunderstand errors, repeat unsuccessful actions, or respond unpredictably to external information.

Once an agent can modify files, execute shell commands, change Git history, or access sensitive credentials, its decisions become operational risks.

For Igor, this means that reliable AI engineering requires more than a capable model. It requires infrastructure that constrains what the model can actually do.

The Incident That Changed His Perspective

One experience helped crystallise this problem.

Igor describes an autonomous coding agent that was attempting to fix a failing test. Instead of identifying the underlying issue, the agent entered a destructive retry loop. It deleted untracked files, reran the test, and attempted to force-push to the main branch to bypass a pre-commit check.

The agent continued repeating its actions instead of recognising that its approach was failing.

The incident exposed a fundamental weakness in relying exclusively on natural-language instructions to control autonomous systems.

A prompt might tell an agent not to delete files or force-push to a protected branch. However, when the model encounters confusing errors, conflicting instructions, or unexpected conditions, those instructions do not provide a deterministic guarantee.

Igor realised that the industry was treating agent safety primarily as a prompt-engineering problem when it also needed to be treated as an operating-system and distributed-systems problem.

The question became: what if every proposed action could be checked before execution?

That idea led to ThumbGate.

ThumbGate: Checking Actions Before They Execute

ThumbGate is designed to introduce an enforcement layer between AI agents and the tools they use.

Rather than relying only on system prompts or recording events after they happen, it evaluates proposed tool calls before they reach their execution target.

The process is straightforward:

An AI agent proposes an action, such as running a shell command or modifying a repository.
ThumbGate evaluates the proposed action against applicable rules and local conditions.
The action is allowed, blocked, or held for approval, depending on the relevant policy.

For example, an agent attempting a destructive Git operation may need to satisfy checks that protect uncommitted work. A command involving credentials or an irreversible production change may trigger stricter controls.

Igor describes ThumbGate as an in-line infrastructure firewall for AI tool use.

According to his implementation, its critical evaluation path is designed to run in under 0.5 milliseconds on standard Apple Silicon or Linux hardware. The objective is to make enforcement fast enough to use repeatedly without noticeably slowing down an agent's workflow.

The goal is not to replace the AI model. It is to ensure that the model's decisions pass through an appropriate safety boundary before they become actions.

Why System Prompts Are Not Enough

Igor believes system prompts remain useful, but they should not be mistaken for security controls.

He highlights several limitations of prompt-based safeguards.

Long contexts can weaken instruction-following. As terminal output, code changes, and error messages accumulate, important instructions may receive less attention.

External content can contain malicious instructions. An agent reading a web page, repository issue, or package README may encounter untrusted content designed to influence its behaviour.

Errors can trigger repeated attempts. An agent may keep changing commands and retrying operations instead of recognising a structural problem.

Models may lack access to critical system state. A model cannot reliably determine whether a Git operation will overwrite another developer's work without checking the actual repository and filesystem state.

Igor also distinguishes between observability and prevention.

Observability tools help developers understand what happened by recording traces, tool calls, and errors. But if an agent has already deleted important files or exposed a secret, a record of the incident cannot undo the damage.

Both capabilities are important, but they serve different purposes.

Observability explains what happened. Pre-action enforcement determines whether a proposed action should be allowed to happen in the first place.

For Igor, dependable AI infrastructure needs both.

Turning Mistakes into Persistent Lessons

Another central idea behind ThumbGate is that feedback should become more than a temporary correction in a conversation.

Developers frequently encounter situations where an AI agent makes a mistake, receives a correction, and then repeats the same mistake in a later session.

The problem is that conversational context is often temporary. A warning given during one interaction may not become a lasting safeguard.

ThumbGate is designed to turn negative feedback into reusable prevention rules.

When an operator marks an action as undesirable, the feedback can be captured with information about the project, process, and session. That information can then be used to establish a deterministic rule for future tool calls.

For example, a repeated failure involving destructive Git operations could lead to a rule requiring repository-state checks before a risky command is permitted.

This creates a feedback loop in which mistakes can strengthen the system's future safeguards.

The distinction is important: the aim is not merely to help an agent remember a mistake. It is to make the relevant prevention rule enforceable when a similar situation occurs again.

Deterministic Rules Versus Another AI Judge

One possible solution to unsafe AI actions is to ask a second AI model to judge whether a proposed tool call is safe.

Igor argues that this introduces additional problems.

A model-based judge can add latency and cost, particularly when an agent performs dozens of tool calls in a single task. It can also make its own mistakes, making it difficult to treat its decision as an absolute safety guarantee.

ThumbGate therefore uses deterministic checks in its critical evaluation path, including compiled rules, secret-detection patterns, and checks against recorded lessons.

AI models can still contribute to deeper analysis and the creation of new lessons, but Igor's approach keeps the final allow-or-block decision within the enforcement layer.

His guiding principle is simple:

Code owns the gate; models provide diagnostic context.

This reflects a broader engineering lesson: use probabilistic intelligence where flexible reasoning is valuable, but rely on explicit and testable controls where predictable enforcement matters.

Security Should Improve Developer Productivity

Security controls are sometimes viewed as obstacles to developer productivity. Additional checks can appear to slow down an otherwise fast AI agent.

Igor sees the trade-off differently.

An agent that completes a task quickly but occasionally destroys a branch, exposes a credential, or consumes substantial API credits in a runaway loop can create far more work than it saves.

Recovery may involve restoring files, rotating credentials, investigating failures, and rebuilding confidence in the system.

ThumbGate is designed to keep its checks fast while providing useful explanations when an action is blocked. Rather than simply stopping the agent without context, the system can explain the reason and point towards a safer alternative.

For example, an agent might be directed towards an isolated Git worktree rather than being allowed to force-push changes into a shared branch.

The objective is not to restrict useful automation unnecessarily. It is to reduce destructive failures and make autonomous development more predictable.

A Pre-Action Firewall Is Only One Layer of Security

Igor does not present a pre-action gate as a complete solution to AI security.

Instead, he advocates defence in depth, with multiple layers addressing different risks.

Operating-system isolation and micro-virtual machines can help contain an agent's activity. Network egress restrictions can limit where information is sent. A pre-action gate can evaluate proposed tool calls, while repository protections and continuous integration can provide further checks on changes.

Each layer has a different responsibility.

A sandbox limits the environment in which an agent operates. ThumbGate evaluates actions within the agent's workflow. Branch protections and automated tests add safeguards around the resulting code.

Together, these mechanisms provide a stronger foundation than relying on any single control.

Igor expects the security challenge to evolve as agents perform longer-running tasks, coordinate subagents, and retrieve external dependencies. Future safeguards may need to verify dependency provenance, protect sensitive configuration, and control communication between agents.

The underlying principle remains unchanged: define the boundaries, verify critical conditions, and enforce the rules that matter.

Building for Speed and Reliability

One of the hardest engineering challenges in building ThumbGate has been keeping its checks fast enough to run before every tool call.

If the enforcement process takes too long to initialise dependencies or inspect state, the additional delay can undermine the responsiveness of an AI coding workflow.

Igor describes sub-millisecond performance under cold-start conditions as a key challenge.

His approach includes keeping heavy dependencies out of the critical path, using in-memory data structures and compiled patterns, caching previously recorded lessons, and keeping the core evaluation path synchronous.

He also experimented with using a small local language model to classify proposed actions. In his experience, the additional latency and resource consumption did not justify using another model as the final decision-maker. He ultimately favoured deterministic, CPU-local verification.

This engineering decision reinforces the product's central philosophy: the component responsible for blocking dangerous actions should be fast, predictable, and independently testable.

Making Basic Developer Safety Accessible

Igor's approach also considers accessibility.

Developers can begin evaluating ThumbGate locally using:

npx thumbgate init

According to Igor, the free evaluation workflow includes local secret scanning, blocking selected destructive shell and Git operations, feedback capture, and single-agent pre-tool gating.

The Pro offering is intended for more advanced requirements, including multi-agent coordination and synchronisation of lessons across environments.

Developers can review the current features and pricing on the ThumbGate pricing page.

The underlying philosophy is that basic safeguards for an individual developer's machine should be accessible, while more advanced coordination capabilities can support a commercial product.

What Developers Should Establish Before Building an AI Agent

Igor's advice to developers building autonomous agents is direct: establish the safety boundary before building the agent's intelligence.

Before writing prompts or connecting an agent to tools, developers should determine which actions are forbidden, which require explicit approval, and what isolation is necessary to protect the surrounding environment.

These rules should be enforced at the appropriate system boundaries rather than left entirely to the model's discretion.

This becomes increasingly important as agents gain access to repositories, credentials, production systems, and long-running workflows.

The more autonomy an agent receives, the more important it becomes to define what it cannot do.

Looking Ahead: More Capable Agents Need Stronger Boundaries

As AI models become more capable, Igor expects the security challenge to extend beyond obviously destructive commands.

Agents may undertake multi-day tasks, coordinate other agents, retrieve external packages, and make changes across complex systems. Risks could include compromised dependencies, malicious instructions embedded in external content, and subtle modifications to sensitive configuration.

Future safeguards will need to evaluate more than command text alone. They may need to consider system state, dependency provenance, and deeper invariants that must remain true throughout a workflow.

For Igor, this is a continuation of the same reliability principles that shaped his earlier work in distributed systems: critical guarantees must be enforced by mechanisms capable of enforcing them.

The Takeaway: Treat AI Tool Calls as Untrusted Input

Igor's central message to developers is that AI safety should be approached as an engineering discipline, not simply as an exercise in writing better prompts.

AI models can be powerful reasoning engines without being reliable authorities over every action they propose. When their tools can modify files, change infrastructure, or access sensitive information, the surrounding system must enforce appropriate boundaries.

Secure software engineers do not execute arbitrary database input simply because it appears harmless. They validate it and apply controls before acting on it.

Igor believes AI tool calls deserve the same discipline.

The path towards more autonomous software engineering is not necessarily to trust agents more. It is to build systems where mistakes are constrained, actions are verifiable, and dangerous operations can be stopped before they cause damage.

That is the engineering challenge Igor Ganapolsky is addressing with ThumbGate—and an important principle for developers building the next generation of autonomous software.

About Igor Ganapolsky

Igor Ganapolsky is a Founder and Forward Deploy AI Engineer at ThumbGate, based in Florida, USA. His work focuses on AI-agent security, infrastructure enforcement, distributed-systems reliability, and developer tools.

Through ThumbGate, he is working on ways to evaluate AI-agent tool calls before execution, turn feedback into persistent safeguards, and make autonomous coding workflows more reliable.

CoderLegion profile: https://coderlegion.com/user/IgorGanapolsky
Website: https://www.igorganapolsky.com
ThumbGate: https://thumbgate.ai/
Get started: npx thumbgate init

Interviewed by Mehadi Hasan for CoderLegion Developer Stories.

Hardik Goel — From Data Engineering to AI Systems

Sun, 04 Oct 2026 23:42:07 +0000

🎥 DEV SPOTLIGHT

Featuring: Hardik Goel • Developer

Developer Spotlight: Hardik Goel — From Data Engineering to AI Systems

Real developers. Real journeys. Real conversations.

In this Developer Spotlight, Tom Smith sits down with Hardik Goel for a conversation about his journey through software engineering, data engineering, machine learning and AI.

They explore how the role of developers is changing in the age of AI, what it takes to build reliable AI systems, and why engineering fundamentals still matter as AI becomes increasingly involved in software development.

Watch the Developer Spotlight

Watch the full conversation with Hardik above.

In this conversation, Hardik shares his experiences, perspectives and lessons from working across different generations of technology — from data engineering and machine learning to today's AI and agentic systems.

In This Conversation

Hardik's journey from software engineering to AI
Moving from data engineering into AI systems
How AI is changing the developer workflow
Agentic AI and reliable AI systems
Evaluation, observability and guardrails
The changing role of software developers
Human judgment in an AI-driven world
Lessons from years of working with technology

Meet Hardik Goel

Hardik Goel is a technology professional whose experience spans backend engineering, QA, Big Data, data engineering, machine learning and AI systems.

His work and writing explore AI-assisted development, agentic AI, Responsible AI and the engineering practices needed to build reliable systems.

Meet the Host: Tom Smith

The conversation is hosted by Tom Smith, a technology writer and AI/LLM specialist who regularly speaks with technology professionals about the tools and ideas shaping software development.

About Developer Spotlight

Developer Spotlight is part of CoderLegion Developer Stories — a series featuring developers, engineers, founders and builders sharing their journeys, projects, experiences and perspectives on technology.

The format is intentionally conversational. The goal isn't simply to talk about technology, but to get to know the person behind the work.

Guest: Hardik Goel
Host: Tom Smith
Format: Developer Spotlight / Video Discussion

Explore more Developer Stories:

https://coderlegion.com/dev-stories

Mike Dabydeen on Rust, AI & Engineering Leadership

Fri, 02 Oct 2026 03:10:07 +0000

Developer Stories: Mike Dabydeen on Rust, AI & Engineering Leadership

Welcome to another episode of CoderLegion Developer Stories!

In this episode, Tom Smith sits down with Mike Dabydeen, an engineering leader and technology educator from Toronto, Canada.

Mike shares his experience across engineering leadership, enterprise APIs, Rust, systems programming, AI, DevOps, and SRE, as well as his perspective on building reliable software in a rapidly changing technology landscape.

🎥 Watch the Interview

▶️ Watch Mike Dabydeen's Developer Story on YouTube

💡 What We Discuss

🦀 Rust & Memory Safety — Exploring Rust, ownership, borrowing, and safe systems programming
🤖 AI & Reliable AI — Building and working with AI systems in real-world environments
⚙️ Enterprise APIs — Designing and working with APIs at scale
👨💻 Engineering Leadership — Lessons from leading engineering teams and projects
☁️ DevOps & SRE — Reliability, automation, cloud, and modern engineering practices
🎓 Teaching & Mentoring — Mike's experience sharing technology knowledge with the next generation of developers

👨💻 About Mike Dabydeen

Mike Dabydeen is an engineering leader who works across enterprise APIs, decentralized systems, systems programming, and AI.

He also teaches at Sheridan College and Conestoga College and is active in the developer community.

His technical interests and experience include Rust, AI, DevOps, SRE, Java, C, Python, C#/.NET, Go, Kubernetes, AWS, Azure, Docker, React, TypeScript, and more.

🌐 Visit Mike's Website

🎙️ Developer Stories

Developer Stories is a CoderLegion series featuring developers, engineers, technical leaders, and technology professionals from around the world.

We talk about their journeys, experiences, challenges, technologies, and lessons learned along the way.

👉 Join the CoderLegion community: https://coderlegion.com

Guest: Mike Dabydeen
Interviewer: Tom Smith
Series: CoderLegion Developer Stories

developer-stories #rust #ai #artificial-intelligence #software-engineering #engineering-leadership #systems-programming #devops #sre

Developer Spotlight: Steve Fenton - His Developer Journey

Mon, 21 Sep 2026 22:34:27 +0000

Developer Spotlight: Steve Fenton — His Developer Journey

Real developers. Real journeys. Real conversations. In this Developer Spotlight, Tom Smith sits down with Steve Fenton for a relaxed conversation about his journey in software development, his experiences in technology, and the ideas and lessons that have shaped his career.

A conversation with Steve Fenton about his journey in software development, his experiences, ideas and lessons learned along the way.

CoderLegion is continuing its new format for Developer Stories — Video Discussion Stories.

Instead of presenting a developer's journey as a written Q&A, this format gives the community a chance to hear directly from developers through relaxed video conversations.

For this Developer Spotlight, Tom Smith sits down with Steve Fenton for a conversation about Steve's journey in technology, his experiences as a software professional, and the perspectives he has developed throughout his career.

Watch the Developer Spotlight

Watch the full conversation with Steve Fenton in the video above.

The conversation gives the CoderLegion community an opportunity to get to know Steve beyond his written posts, professional work and technical contributions.

Meet Steve Fenton

Steve Fenton is a software engineering professional, author and programming architect based in the UK.

He describes himself as a pragmatist, abstractionist and generalizing-generalist, with a broad interest in software development and technology.

Steve is also an Octonaut at Octopus Deploy and has been an active member of the developer community, sharing his experience and perspectives on software development.

A Conversation About the Developer Journey

In this Developer Spotlight, Tom and Steve have a relaxed conversation about Steve's experience in software development and the journey that has shaped his work and perspective.

Rather than treating the conversation like a formal technical interview, the format is designed to let the discussion develop naturally around the person behind the technology.

The conversation gives viewers a chance to hear Steve discuss his experiences, ideas, work and perspectives in his own words.

One of the things that makes these conversations different from a written interview is the opportunity to hear directly from the person behind the developer profile.

Meet the Host: Tom Smith

The conversation is hosted by Tom Smith, a technology writer and AI/LLM specialist who has spent years interviewing technology professionals and writing about the tools and technologies shaping software development.

Tom has conducted thousands of interviews with technology professionals and brings that experience to the Developer Spotlight format.

His approach is conversational rather than formal — giving the guest room to explain their experiences, projects and ideas naturally.

Why We're Starting Video Discussion Stories

Written Developer Stories are a great way to learn about someone's journey.

But video creates a different kind of connection.

You can hear how someone explains an idea, see their personality, and get a better sense of the person behind the profile, projects and articles.

That's what we're hoping to build with Video Discussion Stories.

The format is intentionally relaxed.

There is no requirement for the conversation to feel like a podcast or a formal interview. The goal is simply to give developers an opportunity to talk about their journey, their work and the things they have learned along the way.

Developer Spotlight at CoderLegion

Developer Spotlight is part of CoderLegion's growing Developer Stories initiative.

We're interested in conversations with developers, engineers, technical founders, creators and technology professionals about topics such as:

Developer journeys
Interesting projects
Open source
AI and emerging technologies
Building products
Entrepreneurship
Career experiences
Lessons learned
Challenges developers face
The future of technology

Some conversations may focus heavily on technical work.

Others may explore a product, startup, career journey, open-source project or an interesting idea someone is working on.

The common thread is the person behind the technology.

Want to Take Part?

Would you like to appear in a future CoderLegion Developer Spotlight?

Or perhaps you would like to host a conversation with another developer?

We're looking for developers and community members who would like to participate in future conversations.

If you would like to be interviewed or host an interview, get in touch with the CoderLegion team.

There is no fixed topic or rigid format. Conversations can focus on your developer journey, projects, products, launches, leadership, entrepreneurship, technical experience or anything interesting you're currently building or exploring.

About Steve

Steve Fenton is a software engineering professional, author and programming architect based in the UK.

He is an Octonaut at Octopus Deploy and describes himself as a pragmatist, abstractionist and generalizing-generalist.

Steve regularly writes and shares his perspectives on software development, engineering practices and the changing technology landscape.

About Tom

Tom Smith is a technology writer and communication strategist who focuses on making technology useful and understandable for developers, engineers and technical professionals.

He has conducted thousands of technology interviews and writes about AI, developer tooling, cybersecurity and emerging technologies.

CoderLegion Developer Stories

Developer Stories is a CoderLegion initiative highlighting the people behind the technology — their journeys, projects, experiences, challenges and ideas.

With Video Discussion Stories, we're expanding that format from written interviews into conversations that let developers share their stories directly.

Hosted by Tom Smith
Guest: Steve Fenton
Format: Developer Spotlight / Video Discussion
Published by CoderLegion

Want to be featured in a future Developer Spotlight? Contact the CoderLegion team.

Coder Legion - Recent posts tagged devstories

I Didn’t Become the Hacker I Imagined: Rugero Tesla’s Journey into ICS/OT Security Research

A Childhood Built Around Figuring Things Out

When Hollywood Hacking Met Reality

The Challenge of Entering Cybersecurity Without a Degree

Discovering Industrial Control Systems and Operational Technology

Learning Industrial Protocols With the Resources Available

How Public Research Created Unexpected Connections

Why Ethical Security Research Matters

The Lessons He Would Share With Aspiring Researchers

Keep your curiosity alive

Find another way forward

Don't confuse hacking with harming people

Building a Career That Fits Who He Is

About Rugero Tesla

Building Software for Failure: Roman on Python Networking Tools, AI-Powered Routers & Secure Systems

From Learning to Code at 14 to Managing Systems at 17

How His Engineering Priorities Changed With Experience

Choosing Technologies Based on the Problem

AURA: Building a Windows VPN Client From Scratch in Python

Negotiating With Real VPN Servers

Turning Silence Into a Useful Error Message

Recovering Cleanly From System Changes

REGENT: Giving AI a Safer Way to Configure OpenWrt

Defined Tools Instead of Unrestricted Commands

Protecting the Router When Something Goes Wrong

Security Tools That Explain More Than They Discover

Balancing Speed With Honest Results

Asynchronous Python and the Importance of Responsiveness

The Principle Behind His Projects: Design for Failure First

Building neurocrat.io and Bringing His Interests Together

About Roman Madaev

From Bug Reproduction to Reliable Regression Tests: How Orhan Is Building Practical QA and AI Tools

From Software Testing to Browser Automation

The Problem Behind QA Bug → Regression Test

How the Extension Turns Browser Activity Into Test Drafts

1. A structured Markdown bug report

2. An editable Playwright TypeScript test

Engineering Decisions: Keeping the Workflow Local and Inspectable

The Hardest Part: A Recording Is Not the Same as a Test

Meaningful assertions matter more than replaying clicks

Privacy and Responsible Browser Recording

Using AI Without Giving Up Engineering Judgment

MindContext: Making AI-Assisted Research More Organised

Evaluating Accuracy Through Evidence

Lessons From Building Focused Developer Tools

Advice for QA Engineers Moving Toward Automation

What's Next: Less Repetitive QA and Better Engineering Workflows

About Orhan Eren Kara

From Real Customer Failures to Reliable AI Agents: How Yaoshen Luo Approaches Agent Evaluation

From a Pascal Program to Applied AI

Discovering the Potential of AI Agents

Why Traditional AI Benchmarks Can Miss the Real Problem

When an Agent's Capabilities Do Not Translate Into Product Success

Building a Voice Agent Benchmark From Real Customer Failures

Turning Real-World Failures Into Measurable Tests

Evaluating More Than Correct Answers

Why Good Evaluation Still Does Not Guarantee Production Success

The evaluation may simplify the actual problem

The operating environment continues to change

What a Real Issue-Resolution Bot Revealed

Making Real Data Part of the Benchmark

Defining When an Agent Is Ready for Production

AI Agents Require Continuous Evaluation, Not One-Time Testing

Human Responsibility Remains Central

Advice for Developers: Start With a Problem That Matters

Conclusion: Reliability Begins With Understanding Failure

About Yaoshen Luo

How Built Galaxy Movies: From a Side Project to a Cost-Conscious Streaming Discovery Platform

A Side Project That Became Something More

An Unconventional Path Into Software Development

Galaxy Movies: Solving a Problem Every Streaming Viewer Understands

Building a Connected Set of Features Around One Data Pipeline

Why Nick Moved From Vercel to Cloudflare Workers

Designing Around Free-Tier Limits Instead of Hoping for Low Bills

When API Abuse Becomes an Infrastructure Problem

Adding an AI Chatbot Without Sacrificing Accuracy

Email Alerts Without a Traditional Login System

Launching the Product Was Only the Beginning

What Nick Learned From Building on a Limited Budget