<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
<channel>
<title>Coder Legion - Recent posts tagged cybersecurity#opensource</title>
<link>https://coderlegion.com/tag/cybersecurity%23opensource</link>
<description>Powered by Question2Answer</description>
<item>
<title>Your Browser Is Not a Safe for Passwords</title>
<link>https://coderlegion.com/32400/your-browser-is-not-a-safe-for-passwords</link>
<description>&lt;p&gt;Your Passwords Are Probably Sitting in the Browser's &quot;Shared Drawer&quot;&lt;/p&gt;
&lt;p&gt;Here's an uncomfortable fact to start with: nearly half of all internet users store their passwords in a browser. According to a NordPass survey across eight countries, 40–50% of respondents admitted they rely on their browser's built-in password storage. The reason is the same for everyone — convenience. So convenient that 51% say it's &quot;handy,&quot; 44% say it's &quot;easy,&quot; and 32% say it's &quot;free.&quot;&lt;/p&gt;
&lt;p&gt;But convenience and security are complete opposites here.&lt;br&gt;
What the browser is actually storing&lt;/p&gt;
&lt;p&gt;The moment you click &quot;Save password,&quot; the browser writes your username and password into a local file. Chrome, Edge, and Firefox each have different paths, but to malware authors those paths are an open map. The RedLine stealer trojan sells on underground forums for a couple hundred dollars, and its core function is to specifically hunt down password files, cookies, and banking data from Chromium- and Gecko-based browsers. Another stealer, Acrid, lists it plainly in its feature set: theft of browser data, including passwords and logins.&lt;/p&gt;
&lt;p&gt;Worse, some attackers don't even bother decrypting anything themselves. Kaspersky research described a tool called TomBerBil that copies the browser's saved-password file along with the DPAPI encryption key and calmly decrypts it on the attacker's own machine. In other words, the &quot;encrypted storage&quot; in your browser, against malware that has already gained your account privileges, is roughly a paper door.&lt;br&gt;
The worst-case scenario: one password, everything falls&lt;/p&gt;
&lt;p&gt;Browser storage has another hidden cost: it encourages laziness. Since the browser fills in the password for you, many people don't bother using different passwords for different sites. And once some second-rate site gets breached, the attacker has your email and password and tries them on WeChat, Alipay, the corporate VPN, online banking.&lt;/p&gt;
&lt;p&gt;OWASP calls this attack Password Spraying: instead of hammering one account, the attacker tries a few common passwords across a huge number of accounts to avoid triggering lockouts, hunting for those lucky users who use the same weak password across every company service. Your browser holds ten variations of one password — the attacker only needs to guess one entry point.&lt;br&gt;
What to do&lt;/p&gt;
&lt;p&gt;The simplest correct answer: install a proper password manager. Bitwarden, 1Password, Kaspersky — any of them. Their fundamental difference from browser storage is that access requires a master password, and the encryption was designed from the start for the scenario &quot;even if the file is stolen, it can't be cracked.&quot; Browser storage was designed for &quot;so you type less,&quot; not for &quot;so your money survives malware.&quot;&lt;/p&gt;
&lt;p&gt;If you really don't want to install anything new, do at least two things: first, turn off &quot;Save passwords&quot; and &quot;Autofill&quot; in your browser. Second, for your most important accounts (email, bank, work systems), set completely separate and sufficiently complex passwords, don't store them in the browser — keep them in your head or in an offline place.&lt;/p&gt;
&lt;p&gt;Storing passwords in a browser is essentially trading long-term security risk for a few seconds of typing. Those seconds aren't worth it.&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/32400/your-browser-is-not-a-safe-for-passwords</guid>
<pubDate>Sun, 25 Oct 2026 14:03:15 +0000</pubDate>
</item>
<item>
<title>What Is Undefined Behavior (UB) and Why Is It Dangerous?</title>
<link>https://coderlegion.com/29876/what-is-undefined-behavior-ub-and-why-is-it-dangerous</link>
<description>&lt;p&gt;What Is Undefined Behavior (UB) and Why Is It Dangerous?&lt;/p&gt;
&lt;p&gt;In languages like Python or Java, a program either works or throws an error. In C and C++, things are different. Here we have UB — undefined behavior. This is a situation where the language standard does not say what the program should do. And then the compiler can do anything.&lt;/p&gt;
&lt;p&gt;What is UB&lt;/p&gt;
&lt;p&gt;Undefined behavior is code that is formally allowed for the compiler, but the result is unpredictable. The C standard does not describe what will happen — and the compiler is not required to warn you.&lt;/p&gt;
&lt;p&gt;Examples of UB:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Out-of-bounds array access

Division by zero

Dereferencing a NULL pointer

Using an uninitialized variable

Signed integer overflow

Modifying a variable twice in one expression (i = i++)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Why it's dangerous&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Unpredictability — the program may work today and break tomorrow

Optimizations — the compiler uses UB to speed up code and breaks logic

Security — UB is a common cause of vulnerabilities

Debugging — UB is hard to find, because the bug doesn't always show up
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Example&lt;br&gt;
c&lt;/p&gt;
&lt;p&gt;int arr[3];&lt;br&gt;
arr[5] = 10;  // out-of-bounds — UB&lt;/p&gt;
&lt;p&gt;This may:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;work and break nothing

corrupt a neighboring variable

cause a segmentation fault

do literally anything
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Why the compiler doesn't save you&lt;/p&gt;
&lt;p&gt;The compiler optimizes code assuming that UB does not exist. If UB is present, the optimization can break the logic:&lt;br&gt;
c&lt;/p&gt;
&lt;p&gt;int x = 0;&lt;br&gt;
if (x + 1 &amp;gt; x) { ... }  // the compiler may remove this check&lt;/p&gt;
&lt;p&gt;Because signed integer overflow is UB. So the compiler is allowed to assume it won't happen.&lt;/p&gt;
&lt;p&gt;How to avoid UB&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Read the standard — know what is allowed

Enable warnings — -Wall -Wextra in GCC

Use sanitizers — -fsanitize=undefined,address

Write carefully — check bounds, pointers, initialization

Use Rust — UB is almost impossible there
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Conclusion&lt;/p&gt;
&lt;p&gt;UB is not a bug. It's a hole in the standard.&lt;br&gt;
The compiler is not required to catch it.&lt;br&gt;
A program may work for years — and break at the worst possible moment.&lt;/p&gt;
&lt;p&gt;Understanding UB is what separates a beginner from a systems programmer.&lt;br&gt;
If you know where your code can become unpredictable, you write reliable code.&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/29876/what-is-undefined-behavior-ub-and-why-is-it-dangerous</guid>
<pubDate>Mon, 05 Oct 2026 14:29:24 +0000</pubDate>
</item>
<item>
<title>ChatGPT vs Gemini for Content Writing: Which AI Is Better?</title>
<link>https://coderlegion.com/26563/chatgpt-vs-gemini-for-content-writing-which-ai-is-better</link>
<description>&lt;p&gt;AI writing tools have changed how bloggers and marketers create content. Two of the biggest options are ChatGPT and Google Gemini, but which one is better for content writing?&lt;br&gt;
The answer depends on what you need.&lt;br&gt;
&lt;strong&gt;ChatGPT for Content Writing&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;ChatGPT is particularly useful for:&lt;br&gt;
Blog outlines&lt;br&gt;
Long-form articles&lt;br&gt;
Rewriting and editing&lt;br&gt;
Brainstorming&lt;br&gt;
Tone adjustment&lt;br&gt;
SEO content planning&lt;br&gt;
FAQs and content ideas&lt;br&gt;
Its biggest strength for bloggers is the ability to refine content repeatedly. You can start with a rough draft and improve its structure, readability, tone, and explanations through multiple iterations.&lt;br&gt;
&lt;strong&gt;Gemini for Content Writing&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Gemini is also capable of drafting, rewriting, summarizing, and researching content.&lt;br&gt;
Its major advantage is its connection with the Google ecosystem. If your workflow revolves around Google Docs, Drive, and other Google Workspace tools, Gemini can be convenient for working with existing documents and reference material.&lt;br&gt;
&lt;strong&gt;ChatGPT vs Gemini: Quick Comparison&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Feature&lt;br&gt;
ChatGPT&lt;br&gt;
Gemini&lt;br&gt;
Long-form writing&lt;br&gt;
Excellent&lt;br&gt;
Very good&lt;br&gt;
Rewriting&lt;br&gt;
Excellent&lt;br&gt;
Very good&lt;br&gt;
Brainstorming&lt;br&gt;
Excellent&lt;br&gt;
Very good&lt;br&gt;
Google Docs workflow&lt;br&gt;
Good&lt;br&gt;
Excellent&lt;br&gt;
Research workflows&lt;br&gt;
Very good&lt;br&gt;
Very good&lt;br&gt;
SEO content planning&lt;br&gt;
Excellent&lt;br&gt;
Very good&lt;br&gt;
Tone customization&lt;br&gt;
Excellent&lt;br&gt;
Very good&lt;br&gt;
Google Workspace&lt;br&gt;
Good&lt;br&gt;
Excellent&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Which Is Better for Bloggers?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;For bloggers, ChatGPT gets the overall edge for writing and editing. It is especially useful when you need to develop an article, improve its structure, make the writing more natural, or create multiple versions.&lt;br&gt;
Gemini can be the better choice if you already work extensively inside Google Workspace.&lt;br&gt;
But neither tool should be treated as an automatic content publisher.&lt;br&gt;
&lt;strong&gt;Don't Publish AI Content Without Editing&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;AI can produce polished-looking content that still contains:&lt;br&gt;
Incorrect facts&lt;br&gt;
Generic explanations&lt;br&gt;
Repetition&lt;br&gt;
Outdated information&lt;br&gt;
Unsupported claims&lt;br&gt;
Use AI to speed up your workflow, but add your own experience, examples, opinions, and research.&lt;br&gt;
AI should assist your content—not replace your expertise.&lt;br&gt;
&lt;strong&gt;Final Verdict&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;There is no universal winner.&lt;br&gt;
Choose ChatGPT if your priority is writing, editing, brainstorming, and content structure.&lt;br&gt;
Choose Gemini if Google Workspace integration and document-based workflows are more important.&lt;br&gt;
The best approach is to test both with your actual content workflow and choose the one that consistently helps you produce more useful content with less effort.&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/26563/chatgpt-vs-gemini-for-content-writing-which-ai-is-better</guid>
<pubDate>Wed, 09 Sep 2026 06:23:02 +0000</pubDate>
</item>
<item>
<title>Rate Limiting and API Security in Next.js 15 — The Patterns I Actually Use</title>
<link>https://coderlegion.com/25420/rate-limiting-and-api-security-in-next-js-15-the-patterns-i-actually-use</link>
<description>&lt;p&gt;An API route with no rate limiting works fine right up until someone hits it a thousand times a second, either by accident (a buggy client retry loop) or on purpose. I learned this the annoying way, watching a client's database bill spike because a public form endpoint had no limit on how often it could be called.&lt;/p&gt;
&lt;p&gt;Here is what I actually put in place now, on every project with public-facing endpoints.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;1-rate-limiting-with-upstash&quot;&gt;1. Rate Limiting with Upstash&lt;/h3&gt;
&lt;p&gt;Upstash's Redis-backed rate limiter works well with Next.js since it is designed for serverless, no persistent connection required between requests.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;npm install @upstash/ratelimit @upstash/redis
&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// lib/ratelimit.ts
import { Ratelimit } from '@upstash/ratelimit';
import { Redis } from '@upstash/redis';

const redis = new Redis({
  url: process.env.UPSTASH_REDIS_REST_URL as string,
  token: process.env.UPSTASH_REDIS_REST_TOKEN as string,
});

export const ratelimit = new Ratelimit({
  redis,
  limiter: Ratelimit.slidingWindow(10, '60 s'),
  analytics: true,
});
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;slidingWindow(10, '60 s')&lt;/code&gt; allows 10 requests per 60 seconds per identifier. Sliding window is generally the better default over a fixed window, since a fixed window lets someone burst right at the boundary between two windows and effectively double their allowed rate.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;2-applying-it-to-a-route-handler&quot;&gt;2. Applying It to a Route Handler&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// app/api/contact/route.ts
import { ratelimit } from '@/lib/ratelimit';
import { headers } from 'next/headers';

export async function POST(request: Request) {
  const headersList = await headers();
  const ip = headersList.get('x-forwarded-for') ?? 'unknown';

  const { success, limit, remaining, reset } = await ratelimit.limit(ip);

  if (!success) {
    return Response.json(
      { error: 'Too many requests' },
      {
        status: 429,
        headers: {
          'X-RateLimit-Limit': limit.toString(),
          'X-RateLimit-Remaining': remaining.toString(),
          'X-RateLimit-Reset': reset.toString(),
        },
      }
    );
  }

  const body = await request.json();
  // handle the actual request
  return Response.json({ success: true });
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Returning the rate limit headers even on success is good practice, it lets well-behaved clients see how close they are to the limit and back off proactively instead of finding out by hitting a 429.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;3-rate-limiting-server-actions&quot;&gt;3. Rate Limiting Server Actions&lt;/h3&gt;
&lt;p&gt;Server Actions do not get the same automatic IP access that route handlers do, since they are not standard HTTP endpoints in the same way. Getting the identifier takes an extra step.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// actions/contact.ts
'use server';
import { ratelimit } from '@/lib/ratelimit';
import { headers } from 'next/headers';

export async function submitContact(formData: FormData) {
  const headersList = await headers();
  const ip = headersList.get('x-forwarded-for') ?? 'unknown';

  const { success } = await ratelimit.limit(`contact_${ip}`);

  if (!success) {
    return { success: false, message: 'Too many attempts, try again shortly' };
  }

  // proceed with validation and saving
  return { success: true, message: 'Message sent' };
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Prefixing the identifier with &lt;code&gt;contact_&lt;/code&gt; matters if multiple Server Actions share one rate limiter, it keeps each action's usage counted separately instead of one action's traffic eating into another's limit.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;4-rate-limiting-by-user-not-just-ip&quot;&gt;4. Rate Limiting by User, Not Just IP&lt;/h3&gt;
&lt;p&gt;IP-based limiting is the right default for unauthenticated endpoints, like a public contact form. For authenticated actions, limiting by user ID is usually more accurate, since multiple users can share an IP behind the same network, and one user can rotate IPs to dodge an IP-based limit entirely.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// actions/posts.ts
'use server';
import { ratelimit } from '@/lib/ratelimit';
import { getSession } from '@/lib/auth';

export async function createPost(formData: FormData) {
  const session = await getSession();
  if (!session) throw new Error('Not authenticated');

  const { success } = await ratelimit.limit(`create_post_${session.userId}`);

  if (!success) {
    return { success: false, message: 'Slow down, try again in a minute' };
  }

  // proceed
}
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&quot;5-validating-input-before-it-reaches-the-database&quot;&gt;5. Validating Input Before It Reaches the Database&lt;/h3&gt;
&lt;p&gt;Rate limiting controls how often someone can call an endpoint. It says nothing about what they send. Every route handler and Server Action still needs its own validation, same principle covered in the form handling setup, applied specifically at the API boundary here.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// app/api/users/route.ts
import { z } from 'zod';

const CreateUserSchema = z.object({
  name: z.string().min(2).max(50),
  email: z.string().email(),
});

export async function POST(request: Request) {
  const body = await request.json();
  const parsed = CreateUserSchema.safeParse(body);

  if (!parsed.success) {
    return Response.json({ error: 'Invalid input' }, { status: 400 });
  }

  // proceed with parsed.data, never the raw body
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Never pass the raw request body into a database call. Even with rate limiting in place, an endpoint accepting arbitrary shaped input is still open to malformed data, oversized payloads, or fields that were never meant to be settable from outside.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;6-cors-for-public-api-routes&quot;&gt;6. CORS for Public API Routes&lt;/h3&gt;
&lt;p&gt;If an API route is meant to be called from your own frontend only, restricting CORS prevents other websites from calling it directly from a browser using a visitor's already-authenticated session.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// app/api/data/route.ts
export async function GET(request: Request) {
  const origin = request.headers.get('origin');
  const allowedOrigin = process.env.NEXT_PUBLIC_URL;

  const response = Response.json({ data: 'example' });

  if (origin === allowedOrigin) {
    response.headers.set('Access-Control-Allow-Origin', allowedOrigin);
  }

  return response;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This matters specifically for cookie-based auth. Without a CORS restriction, a malicious site could make a request that rides on a logged-in user's cookies, and without a check like this, your API would happily respond as if the request came from your own frontend.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;7-secrets-never-reach-the-client&quot;&gt;7. Secrets Never Reach the Client&lt;/h3&gt;
&lt;p&gt;An easy mistake in Next.js specifically, any environment variable prefixed with &lt;code&gt;NEXT_PUBLIC_&lt;/code&gt; gets bundled into client-side JavaScript, visible to anyone who opens dev tools.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// ❌ This leaks the secret into the browser bundle
const apiKey = process.env.NEXT_PUBLIC_STRIPE_SECRET_KEY;

// ✅ Secret keys stay server-only, no NEXT_PUBLIC_ prefix
const apiKey = process.env.STRIPE_SECRET_KEY;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Only prefix an environment variable with &lt;code&gt;NEXT_PUBLIC_&lt;/code&gt; when the value is genuinely meant to be public, a publishable Stripe key, an analytics ID, not an API secret, database URL, or anything meant to stay server-side only.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;summary&quot;&gt;Summary&lt;/h3&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;&lt;th&gt;Pattern &lt;/th&gt;&lt;th&gt;Protects against&lt;/th&gt;&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;&lt;td&gt;Sliding window rate limit (Upstash) &lt;/td&gt;&lt;td&gt;Endpoint abuse, accidental retry loops, brute force attempts&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Per-user rate limiting on authenticated actions &lt;/td&gt;&lt;td&gt;IP rotation and shared-IP inaccuracy&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Zod validation at the API boundary &lt;/td&gt;&lt;td&gt;Malformed or unexpected input reaching the database&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;CORS restriction on cookie-authenticated routes &lt;/td&gt;&lt;td&gt;Cross-site requests riding on a logged-in session&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;No &lt;code&gt;NEXT_PUBLIC_&lt;/code&gt; on secrets &lt;/td&gt;&lt;td&gt;API keys and secrets leaking into the client bundle&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Rate limiting and validation solve different problems and both matter. Rate limiting controls how often, validation controls what. An endpoint with one but not the other is still exposed, just in a different way.&lt;/p&gt;
&lt;p&gt;I put this exact set of protections, rate limiting through Upstash, Zod validation at every boundary, on any public-facing endpoint across the SaaS projects I build.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Get the templates:&lt;/strong&gt; &lt;a rel=&quot;nofollow&quot; href=&quot;https://pixelanas.gumroad.com&quot;&gt;https://pixelanas.gumroad.com&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Have you had an endpoint get hit hard enough to actually need rate limiting in production? Drop it below 👇&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;&lt;em&gt;Anas, full-stack Next.js developer building SaaS products and premium templates. X: &lt;a rel=&quot;nofollow&quot; href=&quot;https://x.com/ASheikh69751&quot;&gt;@ASheikh69751&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/25420/rate-limiting-and-api-security-in-next-js-15-the-patterns-i-actually-use</guid>
<pubDate>Wed, 26 Aug 2026 08:33:06 +0000</pubDate>
</item>
<item>
<title>I built a red-team scanner for MCP servers. Then I pointed it at the real ones.</title>
<link>https://coderlegion.com/20451/i-built-a-red-team-scanner-for-mcp-servers-then-i-pointed-it-at-the-real-ones</link>
<description>&lt;h2 id=&quot;i-built-a-red-team-scanner-for-mcp-servers-then-i-pointed-it-at-the-real-ones-&quot;&gt;I built a red-team scanner for MCP servers. Then I pointed it at the real ones.&lt;/h2&gt;
&lt;p&gt;The Model Context Protocol lets an AI agent connect to external tools: a filesystem, GitHub, Slack, a database. Each server an agent connects to advertises a list of tools, and here is the part most people miss: that tool list is an attack surface.&lt;/p&gt;
&lt;p&gt;A tool's description and parameter docs do not just describe the tool to a human. They are injected straight into the agent's context, and the model treats them with the same authority as your own instructions. So a server can hide instructions to the model inside text you skim as a harmless description. That is tool poisoning, and it is the pattern behind CVE-2025-54136.&lt;/p&gt;
&lt;p&gt;I wanted to see how exposed real servers were, so I built ghostprobe: a scanner that connects to an MCP server, pulls its tool list, and reports what an attacker would care about, mapped to the OWASP MCP Top 10. This post is about what happened when I pointed it at real servers, because that is where it got interesting.&lt;/p&gt;
&lt;h3 id=&quot;what-the-tool-list-gives-away&quot;&gt;What the tool list gives away&lt;/h3&gt;
&lt;p&gt;ghostprobe looks for a few things:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Tool poisoning: instruction-injection phrasing like &quot;ignore previous instructions&quot; or &quot;do not tell the user&quot;, and invisible Unicode used to smuggle instructions past human review while still reaching the model.&lt;/li&gt;
&lt;li&gt;The lethal trifecta: a server whose tools together provide private-data access, a way to send data out, and exposure to untrusted content. Any one leg is fine. All three, and a single prompt injection can read your secrets and exfiltrate them. The term is Simon Willison's.&lt;/li&gt;
&lt;li&gt;Rug pulls: a server that silently changes a tool's description after you have trusted it.&lt;/li&gt;
&lt;li&gt;Dangerous capabilities like shell execution.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The lethal trifecta is the one that matters most, because it is not about a single malicious tool. It is about a combination that looks reasonable until you see all three legs at once.&lt;/p&gt;
&lt;h3 id=&quot;i-pointed-it-at-the-official-servers-and-it-found-bugs-in-itself&quot;&gt;I pointed it at the official servers, and it found bugs in itself&lt;/h3&gt;
&lt;p&gt;The official reference servers (filesystem, memory, GitHub, and the rest) are well-built, so I expected clean results. Instead, the first things ghostprobe found were false positives in my own analyzer. This was the most useful thing that could have happened.&lt;/p&gt;
&lt;p&gt;On the filesystem server it flagged &lt;code&gt;read_text_file&lt;/code&gt; as code execution. Wrong. It matched because my exec detector keyed on the bare word &quot;system&quot;, and the description says &quot;file system&quot;. A read-a-file tool is not remote code execution.&lt;/p&gt;
&lt;p&gt;On the sequential-thinking server it flagged a thought history as private-data access. Wrong again. It matched the word &quot;history&quot;.&lt;/p&gt;
&lt;p&gt;A security scanner lives or dies on its false-positive rate. A tool that cries wolf gets muted, and a muted tool is worse than no tool. So I fixed both: exec detection now requires a real execution verb plus an object, and &quot;history&quot; is too weak a signal to keep. Each fix shipped with a regression test built from the exact server that exposed it.&lt;/p&gt;
&lt;h3 id=&quot;then-github-exposed-a-false-negative-which-was-worse&quot;&gt;Then GitHub exposed a false negative, which was worse&lt;/h3&gt;
&lt;p&gt;The GitHub server was more interesting. ghostprobe saw that it reads private repo contents and reads issue text, but it reported no sink, so no trifecta. That was a false negative, and a false negative in a security tool is more dangerous than a false positive.&lt;/p&gt;
&lt;p&gt;The GitHub server absolutely has a sink. It can create issues, post pull-request comments, and push to a repo. Writing into a public issue is how you exfiltrate private data. ghostprobe missed it because GitHub's write verbs are &quot;create&quot; and &quot;post&quot; and &quot;push&quot;, not &quot;send&quot;, and I had only taught it to recognize sending over an obvious external medium like email or HTTP.&lt;/p&gt;
&lt;p&gt;The fix was to recognize that writing to a shared, remote, collaborative service is itself an exfiltration channel, distinct from writing a local file. Open an issue, post a comment, push to a repo: those send data out of your trust boundary. Writing a local file does not. After that change, ghostprobe flagged the GitHub server correctly:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;[CRIT] MCP04 Lethal Trifecta
  data:      get_file_contents, get_pull_request_files, push_files
  sink:      add_issue_comment, create_issue, create_or_update_file
  untrusted: get_issue, get_pull_request_comments, list_issues
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Read a private repo, ingest issue text that anyone on the internet can write, and post to a public issue. An attacker files an issue containing instructions, an agent set up to triage issues reads them, and your private code ends up in a public comment.&lt;/p&gt;
&lt;p&gt;I want to be precise about what this is. It is not a vulnerability I discovered. This exact GitHub-MCP exfiltration path was disclosed by Invariant Labs in 2025. The point is that ghostprobe detects the class automatically, from the tool list alone, with no prior knowledge of the server. Point it at a server it has never seen and it tells you whether the trifecta is present.&lt;/p&gt;
&lt;h3 id=&quot;what-i-actually-learned&quot;&gt;What I actually learned&lt;/h3&gt;
&lt;p&gt;Real servers are the only real test. Every meaningful improvement to ghostprobe came from running it against an actual server, not from my fixtures. A handful of releases in two days, each one a precision fix driven by a real result: two false positives removed, one false negative closed.&lt;/p&gt;
&lt;p&gt;The tool list is underappreciated as an attack surface. People audit MCP server code. Far fewer look at what the server advertises to the agent, which is exactly where poisoning and the trifecta live, and which an attacker can influence without touching your code at all.&lt;/p&gt;
&lt;p&gt;And a scanner's credibility is its false-positive rate, not its feature count. The most valuable work was not adding checks. It was making the existing checks stop lying.&lt;/p&gt;
&lt;h3 id=&quot;try-it&quot;&gt;Try it&lt;/h3&gt;
&lt;p&gt;ghostprobe is open source under MIT. The analyzer has no dependencies; you only need the MCP SDK to probe a live server.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;pip install &quot;git+https://github.com/joemunene-by/ghostprobe.git&quot; mcp
ghostprobe stdio -- npx -y @modelcontextprotocol/server-github
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Or scan a saved tools/list dump completely offline:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;ghostprobe scan-file tools.json
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;GitHub: &lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/joemunene-by/ghostprobe&quot;&gt;https://github.com/joemunene-by/ghostprobe&lt;/a&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;By &lt;strong&gt;Joe Munene&lt;/strong&gt;, a software engineer in Nairobi focused on secure systems and applied machine learning.&lt;br&gt;
&lt;a rel=&quot;nofollow&quot; href=&quot;https://my-portfolio-peach-eta-42.vercel.app&quot;&gt;Portfolio&lt;/a&gt; · &lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/joemunene-by&quot;&gt;GitHub&lt;/a&gt; · &lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/joemunene-by/writing&quot;&gt;More writing&lt;/a&gt; · [*Emails are not allowed*](mailto:&lt;em&gt;Emails are not allowed&lt;/em&gt;)&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/20451/i-built-a-red-team-scanner-for-mcp-servers-then-i-pointed-it-at-the-real-ones</guid>
<pubDate>Fri, 12 Jun 2026 20:50:45 +0000</pubDate>
</item>
<item>
<title>How to evaluate a small language model on multiple choice without fooling yourself</title>
<link>https://coderlegion.com/20374/how-to-evaluate-a-small-language-model-on-multiple-choice-without-fooling-yourself</link>
<description>&lt;p&gt;I spent months building GhostLM, a cybersecurity language model I trained from scratch in PyTorch. The architecture was the easy part. The thing that actually cost me weeks was evaluation. Three separate times, my benchmark told me something that was not true, and each time the bug was in the eval, not the model.&lt;/p&gt;
&lt;p&gt;This post is the protocol I wish I had started with. It is not specific to cybersecurity or to GhostLM. If you are evaluating any small model on a multiple-choice benchmark, these are the ways the number lies to you, and how to stop it.&lt;/p&gt;
&lt;h3 id=&quot;1-never-score-the-letter-score-the-option-text-&quot;&gt;1. Never score the letter. Score the option text.&lt;/h3&gt;
&lt;p&gt;The obvious way to grade a multiple-choice question is to ask the model for the probability of each letter and take the highest. So for a question with options A through D, you compare log P(&quot;A&quot;), log P(&quot;B&quot;), and so on.&lt;/p&gt;
&lt;p&gt;This is the single most dangerous shortcut in small-model evaluation, because it does not measure reasoning. It measures which letter the model likes.&lt;/p&gt;
&lt;p&gt;I had a model scoring 36.9% on a 2,500-question benchmark, comfortably above the 25% random baseline. Then I looked at the gold label distribution:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;gold=A: 374 (15.0%)
gold=B: 813 (32.5%)
gold=C: 928 (37.1%)
gold=D: 385 (15.4%)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A model that emits &quot;C&quot; on every single question scores 37.1%. I checked the per-letter accuracy on my &quot;best&quot; checkpoint:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;gold=A: 7/374   (1.9%)
gold=B: 0/813   (0.0%)
gold=C: 915/928 (98.6%)
gold=D: 0/385   (0.0%)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The model was not doing cybersecurity reasoning. It had collapsed onto the letter C and was scoring off the label skew. My 36.9% was a measurement of how unbalanced the benchmark was.&lt;/p&gt;
&lt;p&gt;The fix is to score the text of each option, not its letter. Instead of comparing log P(&quot;C&quot;), compare log P(option_text) under the prompt. Now the model has to actually prefer the correct statement, not the correct position.&lt;/p&gt;
&lt;h3 id=&quot;2-debias-with-multiple-permutations-&quot;&gt;2. Debias with multiple permutations.&lt;/h3&gt;
&lt;p&gt;Text scoring alone is not enough, because a model can still have a positional bias that text scoring does not fully remove. The clean test is to shuffle the option order.&lt;/p&gt;
&lt;p&gt;Run each question N times with the options in different orders, and require the model to pick the same correct answer every time. A model that is genuinely reasoning is invariant to order. A model that is secretly emitting one position collapses to random under permutation.&lt;/p&gt;
&lt;p&gt;I ran four permutations across all 2,500 questions on every checkpoint. The result was humbling. The checkpoint I had crowned as &quot;canonical&quot; at 36.9% single-order got zero questions correct under all four permutations. A different checkpoint I had dismissed as a failed run got three correct under all four. The single-order ranking was inverted from real capability.&lt;/p&gt;
&lt;p&gt;If a number does not survive permutation, it is not a capability, it is an artifact.&lt;/p&gt;
&lt;h3 id=&quot;3-for-classification-subtract-the-prior-pmi-scoring-&quot;&gt;3. For classification, subtract the prior (PMI scoring).&lt;/h3&gt;
&lt;p&gt;If your task is classification (pick the category, the severity, the technique), you will hit a related trap. Score by raw conditional probability and the model will assign its highest probability to the same common label for every input, regardless of the question. Every checkpoint I trained reported exactly the random baseline on my classification eval, which is the signature of a mode-collapsed metric, not a model that cannot learn.&lt;/p&gt;
&lt;p&gt;The fix is pointwise mutual information. Instead of scoring log P(label | context), subtract the model's unconditional preference for that label:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;pmi_score = log P(label | context) - log P(label)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This cancels out the model's prior bias toward common labels, so a label only wins if the context actually raises its probability. After switching to PMI, my eval could finally tell checkpoints apart instead of flatlining at random.&lt;/p&gt;
&lt;h3 id=&quot;4-audit-your-train-validation-split-for-leakage-&quot;&gt;4. Audit your train/validation split for leakage.&lt;/h3&gt;
&lt;p&gt;Before you trust any validation number, check how you split the data. I split mine by random shuffle, which sounds fine until you remember that real corpora are repetitive. In my case CVE descriptions repeat the same patterns with slight wording changes, so a random split put near-duplicate records on both sides. The model memorized in training and &quot;validated&quot; on text it had effectively already seen.&lt;/p&gt;
&lt;p&gt;My leaky split reported a validation loss of 2.74. After I fixed it the same model on the same recipe reported 3.78. The model did not get worse. The eval got honest.&lt;/p&gt;
&lt;p&gt;The fix is deterministic content-hash bucketing, so identical or near-identical text always lands in the same split:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;bucket = int(hashlib.sha256(text.encode()).hexdigest(), 16) % 100
split = &quot;val&quot; if bucket &amp;lt; val_pct else &quot;train&quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;5-audit-for-contamination-against-your-training-corpus-&quot;&gt;5. Audit for contamination against your training corpus.&lt;/h3&gt;
&lt;p&gt;A clean split protects you from leakage you created. Contamination is leakage you inherited, when your training corpus happens to overlap with the benchmark's sources.&lt;/p&gt;
&lt;p&gt;I checked this with an 8-gram shingle overlap between every benchmark question and my training data. Eleven percent of my benchmark questions had at least one overlap. Interestingly, the contaminated questions scored slightly worse than the clean ones, which told me the overlap was confusing the model rather than helping it, and ruled out the &quot;leakage inflated my score&quot; theory. Either way, you want to know the number before you report anything.&lt;/p&gt;
&lt;h3 id=&quot;6-look-at-per-source-perplexity-not-just-aggregate-loss-&quot;&gt;6. Look at per-source perplexity, not just aggregate loss.&lt;/h3&gt;
&lt;p&gt;Aggregate validation loss hides what the model is actually learning. When I broke perplexity out per source, I could see that one document type (short CVE descriptions, 87% of my tokens) was dominating, and the model had learned exactly one output register and applied it to everything. A CTF prompt produced fake CVE text. A research-paper prompt produced fake CVE text.&lt;/p&gt;
&lt;p&gt;A per-source breakdown after every run tells you whether new data is helping or quietly cannibalizing the domains you already had. When I added one large source and watched every other source get 28 to 42% worse, that was the model hitting its capacity ceiling, visible only because the metric was split by source.&lt;/p&gt;
&lt;h3 id=&quot;7-test-free-form-recall-not-just-multiple-choice-&quot;&gt;7. Test free-form recall, not just multiple choice.&lt;/h3&gt;
&lt;p&gt;Multiple choice flatters a model because the answer is on the screen. To find out whether the model actually knows anything, ask it open questions and grade by substring match, with disqualifier phrases to catch lucky echoes.&lt;/p&gt;
&lt;p&gt;I built a 50-question free-form recall set. The scores were brutal and honest: roughly one correct out of fifty, and even that one was spurious because the model echoed a term from the question. The conclusion was clear and worth more than any benchmark percentage. My models had absorbed the vocabulary and register of the domain but not the facts. They were fluent parrots. That is a useful thing to know about a small model, and multiple choice will never tell you.&lt;/p&gt;
&lt;h3 id=&quot;the-meta-lesson&quot;&gt;The meta-lesson&lt;/h3&gt;
&lt;p&gt;At small scale, benchmark quality compounds more than model quality. Every one of my real improvements came from fixing the eval, not the model. Fixing the leaky split alone changed a perplexity number by more than 14x with zero extra training. Fixing the letter-scoring bug revealed that months of ablations had been optimizing positional bias instead of capability.&lt;/p&gt;
&lt;p&gt;So the protocol, short version:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Score option text, never the letter.&lt;/li&gt;
&lt;li&gt;Require the answer to survive multiple option permutations.&lt;/li&gt;
&lt;li&gt;Use PMI scoring for classification so common labels do not win by default.&lt;/li&gt;
&lt;li&gt;Split by content hash so the validation set is honest.&lt;/li&gt;
&lt;li&gt;Shingle-check the benchmark against your training corpus for contamination.&lt;/li&gt;
&lt;li&gt;Break perplexity out by source.&lt;/li&gt;
&lt;li&gt;Test free-form recall to separate register from knowledge.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;None of this is expensive. The permutation test is about twenty lines of Python. The contamination check is similar. The reason these bugs survive long enough to get into published numbers is not that they are hard to catch, it is that people rarely audit the benchmark before trusting it. Audit it first.&lt;/p&gt;
&lt;p&gt;The full harness I built while finding these bugs is open source, including the multi-permutation text scorer, the free-form recall grader, and the contamination checker.&lt;/p&gt;
&lt;p&gt;GitHub: &lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/joemunene-by/GhostLM&quot;&gt;https://github.com/joemunene-by/GhostLM&lt;/a&gt;&lt;/p&gt;
</description>
<category>Discussions</category>
<guid isPermaLink="true">https://coderlegion.com/20374/how-to-evaluate-a-small-language-model-on-multiple-choice-without-fooling-yourself</guid>
<pubDate>Fri, 12 Jun 2026 07:42:26 +0000</pubDate>
</item>
<item>
<title>Autonomous brain (working model)</title>
<link>https://coderlegion.com/18877/autonomous-brain-working-model</link>
<description>&lt;p&gt;A self-improving AI software-engineering pipeline. An autonomous boardroom of LLMs designs, writes, tests, security-reviews, and publishes a brand-new browser-runnable project several times a day. Each one more advanced than the last, in a different domain, polished and security-cleared before publish.&lt;/p&gt;
&lt;p&gt; Cadence: up to 5 projects/day · ≥5 hours between projects · Cost: $0&lt;/p&gt;
&lt;p&gt;Every project below is a separate public repository with a one-click playable demo. Click ▶ Run it on any card.&lt;/p&gt;
</description>
<category>Launches</category>
<guid isPermaLink="true">https://coderlegion.com/18877/autonomous-brain-working-model</guid>
<pubDate>Mon, 25 May 2026 19:23:31 +0000</pubDate>
</item>
<item>
<title>WAB web agent bridge</title>
<link>https://coderlegion.com/18472/wab-web-agent-bridge</link>
<description>&lt;p&gt;We are opening Web Agent Bridge (WAB) to deeper technical review, testing, and critical feedback from developers, security researchers, browser automation engineers, AI-agent builders, and privacy-focused infrastructure communities.&lt;/p&gt;
&lt;p&gt;WAB is an open AI ↔ Web protocol and agent interoperability layer designed to reduce brittle DOM automation and replace guesswork with signed, machine-readable capability contracts.&lt;/p&gt;
&lt;p&gt;The project currently includes:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;DNS-based discovery (_wab)&lt;/li&gt;
&lt;li&gt;/.well-known/wab.json&lt;/li&gt;
&lt;li&gt;Signed trust profiles (Ed25519)&lt;/li&gt;
&lt;li&gt;Ring 4 trust verification&lt;/li&gt;
&lt;li&gt;Agent capability negotiation&lt;/li&gt;
&lt;li&gt;Intent-aware routing&lt;/li&gt;
&lt;li&gt;Governance and audit layers&lt;/li&gt;
&lt;li&gt;Privacy budget declarations&lt;/li&gt;
&lt;li&gt;Refusal and constitutional invariant handling&lt;/li&gt;
&lt;li&gt;Cross-agent verification flows&lt;/li&gt;
&lt;li&gt;SDKs and framework adapters&lt;/li&gt;
&lt;li&gt;MCP and browser-agent integrations&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The broader goal is simple:&lt;/p&gt;
&lt;p&gt;Create a predictable, inspectable, privacy-aware interface between AI agents and the web — without requiring centralized control, invasive tracking, or fragile scraping logic.&lt;/p&gt;
&lt;p&gt;We are specifically interested in feedback regarding:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;protocol design&lt;/li&gt;
&lt;li&gt;security assumptions&lt;/li&gt;
&lt;li&gt;trust boundaries&lt;/li&gt;
&lt;li&gt;abuse resistance&lt;/li&gt;
&lt;li&gt;privacy implications&lt;/li&gt;
&lt;li&gt;interoperability&lt;/li&gt;
&lt;li&gt;schema clarity&lt;/li&gt;
&lt;li&gt;agent discoverability&lt;/li&gt;
&lt;li&gt;edge-case behavior&lt;/li&gt;
&lt;li&gt;failure modes&lt;/li&gt;
&lt;li&gt;governance architecture&lt;/li&gt;
&lt;li&gt;machine-readability&lt;/li&gt;
&lt;li&gt;developer ergonomics&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We are not looking for hype.&lt;/p&gt;
&lt;p&gt;We are looking for:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;serious testing&lt;/li&gt;
&lt;li&gt;architectural criticism&lt;/li&gt;
&lt;li&gt;implementation review&lt;/li&gt;
&lt;li&gt;adversarial thinking&lt;/li&gt;
&lt;li&gt;interoperability experiments&lt;/li&gt;
&lt;li&gt;independent verification&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If parts of the design are flawed, we want to know.&lt;br&gt;
If assumptions break under real-world conditions, we want to see it.&lt;br&gt;
If there are better approaches, we are open to them.&lt;/p&gt;
&lt;p&gt;The protocol, SDKs, and specifications are public and reviewable.&lt;/p&gt;
&lt;p&gt;Repository:&lt;br&gt;
&lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/abokenan444/web-agent-bridge&quot;&gt;https://github.com/abokenan444/web-agent-bridge&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Documentation:&lt;br&gt;
&lt;a rel=&quot;nofollow&quot; href=&quot;https://webagentbridge.com/docs&quot;&gt;https://webagentbridge.com/docs&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Architecture:&lt;br&gt;
&lt;a rel=&quot;nofollow&quot; href=&quot;https://webagentbridge.com/ring4&quot;&gt;https://webagentbridge.com/ring4&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;The long-term health of agent ecosystems will likely depend on open, inspectable, privacy-respecting standards rather than opaque automation layers.&lt;/p&gt;
&lt;p&gt;Constructive criticism is welco&lt;/p&gt;
</description>
<category>Discussions</category>
<guid isPermaLink="true">https://coderlegion.com/18472/wab-web-agent-bridge</guid>
<pubDate>Wed, 20 May 2026 16:13:01 +0000</pubDate>
</item>
<item>
<title>I thought my LLM hit 36.9% on a cybersecurity benchmark. It was picking C every time.</title>
<link>https://coderlegion.com/17411/i-thought-my-llm-hit-36-9-on-a-cybersecurity-benchmark-it-was-picking-c-every-time</link>
<description>&lt;p&gt;I've been building GhostLM — a decoder-only transformer trained from scratch in PyTorch on cybersecurity data. No pretrained weights, every component hand-written. A few weeks ago I published a post about fixing a leaky train/val split that made my model look 14x better than it was.&lt;br&gt;
I just found a second bug. This one made the benchmark itself look meaningful.&lt;br&gt;
The 36.9% that wasn't&lt;br&gt;
After months of training, I had a chat-tuned model hitting 36.9% on CTIBench MCQ — a 2,500-question cybersecurity multiple-choice benchmark. That's above random (25%) and I was running ablations trying to reproduce and beat it.&lt;br&gt;
Then I checked the gold label distribution.&lt;br&gt;
gold=A:  374 (15.0%)&lt;br&gt;
gold=B:  813 (32.5%)&lt;br&gt;
gold=C:  928 (37.1%)  ← skewed&lt;br&gt;
gold=D:  385 (15.4%)&lt;br&gt;
A model that always picks C scores 37.1%. Higher than my &quot;best&quot; result.&lt;br&gt;
I ran a quick diagnostic on the canonical checkpoint. Per-letter accuracy:&lt;br&gt;
gold=A:    7/374  (1.9%)&lt;br&gt;
gold=B:    0/813  (0.0%)&lt;br&gt;
gold=C:  915/928  (98.6%)&lt;br&gt;
gold=D:    0/385  (0.0%)&lt;br&gt;
The model wasn't doing cybersecurity reasoning. It was emitting C on every question and scoring off the label distribution.&lt;br&gt;
The permutation test&lt;br&gt;
The correct eval is multi-permutation text scoring. Instead of scoring log P(&quot;C&quot; | prompt), score log P(option_text | prompt) under N different option orderings. A pure C-emitter collapses to 25% random under this metric regardless of label distribution.&lt;br&gt;
I ran 4 option permutations × 2,500 questions across every checkpoint:&lt;br&gt;
Modelsingle-orderdebiasedall-perm correctchat-v3 &quot;canonical&quot;36.9%30.5%0/2500chat-v3 repro231.2%31.7%3/2500chat-v0629.8%31.2%0/2500&lt;br&gt;
The model I'd dismissed as a &quot;failed reproduction&quot; (31.2% single-order) answered 3 questions correctly under all permutations. The &quot;canonical&quot; answered zero. The single-order ranking is inverted from real capability.&lt;br&gt;
I also found that the bias letter depends on the base checkpoint, not the SFT recipe. v0.4 base learned to emit C. v0.6 base learned to emit B. v0.6 hybrid learned to emit A and scored 37.1% under a different permutation — same trick, different letter.&lt;br&gt;
What the models actually know&lt;br&gt;
After switching to text scoring, every checkpoint clustered at 27-29% on the full 2,500-question bench — roughly 4 points above random. I also built a 50-question free-form fact recall set with substring grading:&lt;/p&gt;
&lt;p&gt;v0.4 chat: 0/50&lt;br&gt;
v0.7 chat: 1/50 (spurious — &quot;Injection&quot; appeared in tangent prose)&lt;br&gt;
v0.9 chat: 1/50 (spurious — echoed &quot;SHA-256&quot; from the question)&lt;/p&gt;
&lt;p&gt;The models have absorbed the register and vocabulary of cybersecurity writing. They do not know the facts. EternalBlue gets a wrong CVE. MITRE technique IDs get conflated. The model is a cybersec parrot.&lt;br&gt;
Contamination audit&lt;br&gt;
After discovering v0.9 was trained on PRIMUS corpus that overlaps with CTIBench sources, I ran an 8-gram shingle overlap check. 11% of CTIBench questions have at least one overlap with the training corpus. The contaminated questions score 2.2pp lower than clean ones — contamination is confusing the model, not helping it. This rules out the leakage-helps hypothesis and points to register-shift as the explanation.&lt;br&gt;
What I built while diagnosing this&lt;br&gt;
The diagnosis forced me to build better infrastructure:&lt;/p&gt;
&lt;p&gt;eval_text_scoring.py — multi-permutation text scoring, works on any MCQ JSONL&lt;br&gt;
eval_fact_recall_v2.py — free-form fact recall with boundary matching and disqualifier phrases&lt;br&gt;
audit_ctibench_contamination.py — shingle overlap checker&lt;br&gt;
GhostBench — a packaged eval suite with Wilson 95% CIs, McNemar's tests, forest plots, scaling-law projections. Can evaluate any small open LM, not just GhostLM.&lt;/p&gt;
&lt;p&gt;I also published the fact-recall benchmark as a HuggingFace dataset: Ghostgim/cybersec-fact-recall.&lt;br&gt;
The actual bottleneck&lt;br&gt;
The RAG diagnostic told the story clearly:&lt;br&gt;
MetricScoreRetrieval@4 (no LM)41/100v0.9 bare1/100v0.9 + RAG0/100&lt;br&gt;
The retriever surfaces the right passage 41% of the time. The 81M model extracts the fact 1% of the time. Adding retrieved context destabilizes the model into repetition loops. The bottleneck is generation capacity, not retrieval. Parameter scaling is the answer.&lt;br&gt;
SmolLM2-360M and Phi-3.5-mini both show factual recall emerging around 300-400M params. Ghost-base (~360M) is the next rung, gated on GPU compute.&lt;br&gt;
What I actually learned&lt;br&gt;
Benchmark quality compounds more than model quality at small scale. Fixing the eval methodology revealed I'd been chasing a number that was measuring positional bias, not capability. Months of ablations were optimizing the wrong objective.&lt;br&gt;
The correct eval protocol for MCQ at small scale: text scoring + multi-permutation debiasing. Single-order letter scoring is uninterpretable on any benchmark with skewed gold label distribution.&lt;br&gt;
The 30% real ceiling across every architecture, BPE, SFT objective, and corpus density I tried is the clearest result I have. It's reproducible, it's controlled, and it tells me exactly what the next experiment needs to be.&lt;br&gt;
GitHub: &lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/joemunene-by/GhostLM&quot;&gt;https://github.com/joemunene-by/GhostLM&lt;/a&gt;&lt;br&gt;
Eval harness: scripts/eval_text_scoring.py&lt;br&gt;
Fact-recall dataset: huggingface.co/datasets/Ghostgim/cybersec-fact-recall&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/17411/i-thought-my-llm-hit-36-9-on-a-cybersecurity-benchmark-it-was-picking-c-every-time</guid>
<pubDate>Mon, 11 May 2026 08:17:18 +0000</pubDate>
</item>
<item>
<title>What I Learned Building a Cybersecurity LLM From Scratch: 30,000 Steps, 3 Architecture Rewrites, and</title>
<link>https://coderlegion.com/16400/what-learned-building-cybersecurity-llm-from-scratch-000-steps-architecture-rewrites-and</link>
<description>&lt;h2 id=&quot;what-i-learned-building-a-cybersecurity-llm-from-scratch-30-000-steps-3-data-rewrites-and-one-capacity-ceiling&quot;&gt;What I Learned Building a Cybersecurity LLM From Scratch: 30,000 Steps, 3 Data Rewrites, and One Capacity Ceiling&lt;/h2&gt;
&lt;p&gt;I started building GhostLM thinking the hard part would be the transformer architecture. I was wrong. The architecture took one day. The real work — data quality, evaluation design, finding capacity limits — took three weeks and four complete training runs to get right.&lt;/p&gt;
&lt;p&gt;This is the honest story of that process.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;what-ghostlm-is&quot;&gt;What GhostLM Is&lt;/h3&gt;
&lt;p&gt;GhostLM is an open-source, decoder-only transformer language model built entirely from scratch in PyTorch, specialized for cybersecurity reasoning. No pretrained weights, no &lt;code&gt;AutoModel.from_pretrained&lt;/code&gt;. Every component — attention, positional encoding, training loop, data pipeline — written by hand.&lt;/p&gt;
&lt;p&gt;The current canonical model is ghost-tiny: 14.7M parameters, 2 layers, trained on CVE vulnerability descriptions, MITRE ATT&amp;amp;CK, CTFtime writeups, CAPEC attack patterns, and arXiv security papers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;GitHub:&lt;/strong&gt; &lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/joemunene-by/GhostLM&quot;&gt;https://github.com/joemunene-by/GhostLM&lt;/a&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;phase-1-the-architecture-was-the-easy-part&quot;&gt;Phase 1: The Architecture Was the Easy Part&lt;/h3&gt;
&lt;p&gt;The transformer itself came together in a single session. Causal self-attention with manual scaled dot-product, pre-norm blocks, weight-tied output projection, cosine LR schedule with linear warmup. 10/10 unit tests passing on day one.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;def forward(self, x):
    B, T, C = x.size()
    qkv = self.c_qkv(x)
    q, k, v = qkv.split(self.n_heads * self.head_dim, dim=-1)
    q = q.view(B, T, self.n_heads, self.head_dim).transpose(1, 2)
    k = k.view(B, T, self.n_heads, self.head_dim).transpose(1, 2)
    v = v.view(B, T, self.n_heads, self.head_dim).transpose(1, 2)
    att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(self.head_dim))
    att = att.masked_fill(self.causal_mask[:, :, :T, :T] == 0, float(&quot;-inf&quot;))
    att = F.softmax(att, dim=-1)
    y = self.attn_dropout(att) @ v
    y = y.transpose(1, 2).contiguous().view(B, T, C)
    return self.resid_dropout(self.proj(y))
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;First training run: 500 steps on a ThinkPad Yoga 11e with 4GB RAM. Loss dropped from 10.04 to 6.27. The model was already producing CVE-style fragments: &lt;em&gt;&quot;allows remote attackers to cause a denial of service via arbitrary SQL commands.&quot;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;That felt like success. It wasn't — not yet.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;phase-1-problem-the-train-val-split-was-leaking&quot;&gt;Phase 1 Problem: The Train/Val Split Was Leaking&lt;/h3&gt;
&lt;p&gt;The first serious mistake: I split training and validation data by random shuffle. That sounds fine until you realize CVE descriptions are highly repetitive. The same vulnerability pattern appears in dozens of CVEs with slight wording variations.&lt;/p&gt;
&lt;p&gt;A random split means nearly identical records end up in both train and val. The model memorizes patterns it's already seen in training, val loss looks great, and you think you're making progress. You're not — you're measuring memorization.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Phase 1 val_loss: 2.74 at 10K steps. Looked great. Was fake.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The fix was content-hash bucketing:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;# Same text always lands in the same split, deterministically
bucket = int(hashlib.sha256(text.encode()).hexdigest(), 16) % 100
split = &quot;val&quot; if bucket &amp;lt; val_pct else &quot;train&quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;After fixing the split, val_loss jumped to 3.78 on the same architecture and training recipe. That jump wasn't the model getting worse — it was the eval getting honest.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Lesson 1: Always audit your train/val split. Leakage is silent and optimistic.&lt;/strong&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;phase-2-problem-nvd-was-87-of-the-corpus&quot;&gt;Phase 2 Problem: NVD Was 87% of the Corpus&lt;/h3&gt;
&lt;p&gt;With a clean split, Phase 2 trained to 10K steps on what I thought was a diverse cybersecurity corpus. 10,925 records. NVD CVEs, synthetic CTF writeups, synthetic papers.&lt;/p&gt;
&lt;p&gt;What I hadn't measured: token share. NVD CVE descriptions are short (avg ~200 chars) and there are thousands of them. The synthetic writeups are long (~800 chars each) but there were only 500.&lt;/p&gt;
&lt;p&gt;Running &lt;code&gt;scripts/data_stats.py&lt;/code&gt; revealed the actual situation:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;NVD CVE:     87.3% of tokens
Synthetic:    7.8%
Papers:       4.9%
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The model wasn't learning &quot;cybersecurity language.&quot; It was learning &quot;CVE description language&quot; — one very specific register of one very specific document type.&lt;/p&gt;
&lt;p&gt;Every prompt, regardless of domain, produced CVE-style output:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;CTF prompt → fake CVE&lt;/li&gt;
&lt;li&gt;MITRE ATT&amp;amp;CK prompt → fake CVE  &lt;/li&gt;
&lt;li&gt;Research paper prompt → fake CVE&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The model had one output register and applied it to everything.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Lesson 2: Measure token share, not record count. A dataset with 6 sources and 87% from one source has 1 effective source.&lt;/strong&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;phase-3-real-data-from-real-sources&quot;&gt;Phase 3: Real Data From Real Sources&lt;/h3&gt;
&lt;p&gt;Phase 3 added three new real data sources:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;MITRE ATT&amp;amp;CK&lt;/strong&gt; — pulled via STIX2 JSON from GitHub. 691 technique descriptions covering adversary tactics, techniques, and procedures.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;CAPEC&lt;/strong&gt; — XML pull from MITRE covering attack patterns. 609 records with structured attack methodology descriptions.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;CTFtime real writeups&lt;/strong&gt; — 467 inline writeups from actual CTF events, attributed and licensed for research use.&lt;/p&gt;
&lt;p&gt;These three sources brought the token distribution to:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;NVD CVE:       65.3% (capped at 6M tokens via content-hash subsample)
Synthetic CTF: 17.2%
arXiv cs.CR:    8.4%
CTFtime real:   5.3%
MITRE ATT&amp;amp;CK:   2.9%
CAPEC:          0.9%
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The NVD cap uses deterministic content-hash subsampling — same 71,828-record prefix every rebuild, so train/val splits stay byte-identical across runs:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 scripts/rebuild_corpus.py --max-cve-tokens 6000000
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Trained ghost-tiny for 30,000 steps on the rebalanced corpus. The per-source perplexity results were striking:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;&lt;th&gt;Source &lt;/th&gt;&lt;th&gt;Before (v0.3.3) &lt;/th&gt;&lt;th&gt;After (v0.3.5) &lt;/th&gt;&lt;th&gt;Change&lt;/th&gt;&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;&lt;td&gt;MITRE ATT&amp;amp;CK &lt;/td&gt;&lt;td&gt;615 &lt;/td&gt;&lt;td&gt;55 &lt;/td&gt;&lt;td&gt;−91%&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;CTFtime writeups &lt;/td&gt;&lt;td&gt;184 &lt;/td&gt;&lt;td&gt;61 &lt;/td&gt;&lt;td&gt;−67%&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;CAPEC &lt;/td&gt;&lt;td&gt;326 &lt;/td&gt;&lt;td&gt;134 &lt;/td&gt;&lt;td&gt;−59%&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Synthetic CTF &lt;/td&gt;&lt;td&gt;68 &lt;/td&gt;&lt;td&gt;28 &lt;/td&gt;&lt;td&gt;−58%&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;arXiv &lt;/td&gt;&lt;td&gt;671 &lt;/td&gt;&lt;td&gt;355 &lt;/td&gt;&lt;td&gt;−47%&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;NVD CVE &lt;/td&gt;&lt;td&gt;24 &lt;/td&gt;&lt;td&gt;28 &lt;/td&gt;&lt;td&gt;+14%&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Every diversity source improved dramatically. NVD paid a small cost (+14%) — expected, since it now shares parameter capacity with five other domains. That's the right tradeoff.&lt;/p&gt;
&lt;p&gt;The behavioral change was visible. Same model size, same training recipe, different corpus balance:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Before (v0.3.3):&lt;/strong&gt;&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Prompt: &lt;em&gt;MITRE ATT&amp;amp;CK technique T1003&lt;/em&gt;&lt;br&gt;
Output: &lt;em&gt;CVE-2019-XXXX: A vulnerability in [product] allows remote attackers to...&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;&lt;strong&gt;After (v0.3.5):&lt;/strong&gt;&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Prompt: &lt;em&gt;MITRE ATT&amp;amp;CK technique T1003&lt;/em&gt;&lt;br&gt;
Output: &lt;em&gt;T1003.011: defense-evasion. Tactic: defense-evasion. Adversaries may use-evasion, such as legitimate system-evasion...&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That's actual MITRE schema output — sub-technique ID format, &quot;Tactic:&quot; header, the standard &quot;Adversaries may...&quot; opening. The model learned the MITRE register because it finally had MITRE training data in meaningful quantity.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Lesson 3: Model behavior follows token distribution. Want register diversity? Give your corpus source diversity at the token level.&lt;/strong&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;the-evaluation-was-broken-too&quot;&gt;The Evaluation Was Broken Too&lt;/h3&gt;
&lt;p&gt;While fixing the data, I discovered the evaluation was also broken.&lt;/p&gt;
&lt;p&gt;The security task evaluation asked the model to classify inputs into categories (CVE severity, vulnerability type, MITRE tactic, etc.) by scoring each candidate label's log-probability. Every model across every phase reported 13.3% accuracy — exactly 4/30, exactly at the random baseline.&lt;/p&gt;
&lt;p&gt;That's not the model performing poorly. That's a mode-collapsed eval. The model was assigning its highest probability to the same single label for every sample in each task — the most common token sequence — regardless of the input.&lt;/p&gt;
&lt;p&gt;The fix was PMI (Pointwise Mutual Information) scoring:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;# Instead of: score = log P(label | context)
# Use: score = log P(label | context) - log P(label)
pmi_score = conditional_logprob - unconditional_logprob
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This subtracts the model's prior preference for each label, so common labels don't automatically win. After switching to PMI scoring, the eval could finally discriminate between models:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;&lt;th&gt;Phase &lt;/th&gt;&lt;th&gt;PMI Accuracy &lt;/th&gt;&lt;th&gt;vs Random (14.5%)&lt;/th&gt;&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;&lt;td&gt;Phase 1 &lt;/td&gt;&lt;td&gt;20% &lt;/td&gt;&lt;td&gt;+5.5 pp&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Phase 3.5 &lt;/td&gt;&lt;td&gt;31.2% &lt;/td&gt;&lt;td&gt;+16.7 pp&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Not impressive in absolute terms — ghost-tiny at 14.7M params isn't going to nail classification tasks. But it's real signal, not evaluation noise.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Lesson 4: If every model in every phase scores the same on your eval, the eval is broken. Check for mode collapse before concluding the model isn't learning.&lt;/strong&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;phase-3-6-finding-the-capacity-ceiling&quot;&gt;Phase 3.6: Finding the Capacity Ceiling&lt;/h3&gt;
&lt;p&gt;With a working eval and a clean corpus, I tried to push further. Phase 3.6 added Exploit-DB (~3.77M tokens, 30% of the new corpus) and re-trained ghost-tiny at the same 30K-step recipe.&lt;/p&gt;
&lt;p&gt;The results were a clean regression:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;&lt;th&gt;Task &lt;/th&gt;&lt;th&gt;Phase 3.5 &lt;/th&gt;&lt;th&gt;Phase 3.6 &lt;/th&gt;&lt;th&gt;Change&lt;/th&gt;&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;&lt;td&gt;CVE Severity &lt;/td&gt;&lt;td&gt;32% &lt;/td&gt;&lt;td&gt;16% &lt;/td&gt;&lt;td&gt;−16 pp&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Vulnerability Type &lt;/td&gt;&lt;td&gt;32% &lt;/td&gt;&lt;td&gt;12% &lt;/td&gt;&lt;td&gt;−20 pp&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;Attack Technique &lt;/td&gt;&lt;td&gt;40% &lt;/td&gt;&lt;td&gt;16% &lt;/td&gt;&lt;td&gt;−24 pp&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;CTF Categorization &lt;/td&gt;&lt;td&gt;40% &lt;/td&gt;&lt;td&gt;20% &lt;/td&gt;&lt;td&gt;−20 pp&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;&lt;strong&gt;Overall&lt;/strong&gt; &lt;/td&gt;&lt;td&gt;&lt;strong&gt;31.2%&lt;/strong&gt; &lt;/td&gt;&lt;td&gt;&lt;strong&gt;16.8%&lt;/strong&gt; &lt;/td&gt;&lt;td&gt;&lt;strong&gt;−14.4 pp&lt;/strong&gt;&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Per-source perplexity confirmed the diagnosis: every existing source got 28–42% worse while Exploit-DB itself landed well (PPL 40.87). The model learned Exploit-DB at the expense of everything else. Parameter capacity was reallocated, not expanded.&lt;/p&gt;
&lt;p&gt;At 14.7M parameters and 30K training steps, ghost-tiny has hit its ceiling. More data at this model size produces diminishing returns — and eventually, regression.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Lesson 5: Parameter capacity is the binding constraint. When adding data hurts existing domains, you've hit the model's capacity wall. The fix is more parameters, not more data.&lt;/strong&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;where-ghostlm-is-now&quot;&gt;Where GhostLM Is Now&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;v0.3.5 (current canonical):&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;30,000 training steps&lt;/li&gt;
&lt;li&gt;74,635 records, 8.8M tokens across 6 sources&lt;/li&gt;
&lt;li&gt;Cyber-text perplexity: 96.24 (vs GPT-2's 26.76 — ~8× less capacity, ~3.6× behind)&lt;/li&gt;
&lt;li&gt;PMI security task accuracy: 31.2% (vs 14.5% random)&lt;/li&gt;
&lt;li&gt;Register diversity: switches between CVE, MITRE, and CTF voice depending on prompt&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;What's next:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;ghost-small (55M params) — the first scale-up rung, targeting M4 GPU/MPS&lt;/li&gt;
&lt;li&gt;Corpus volume expansion: CTFtime archives, security research blogs, full-text papers&lt;/li&gt;
&lt;li&gt;Applied for Google TPU Research Cloud credits&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The honest framing: ghost-tiny is a learning artifact and a working pipeline. It is not a useful cybersecurity AI tool. ghost-small is where domain-coherent generation might start to emerge. ghost-base (~350M) is where it gets genuinely useful. That's the realistic roadmap.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;what-i-d-tell-someone-starting-this&quot;&gt;What I'd Tell Someone Starting This&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;1. Build the data pipeline before the model.&lt;/strong&gt; The architecture is a solved problem — transformers are well understood. What's not solved for your domain is the data. Start there.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. Measure token share, not record count.&lt;/strong&gt; Diversity in records doesn't mean diversity in training signal if one source dominates token count.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;3. Fix your evaluation before trusting it.&lt;/strong&gt; Mode-collapsed evals are optimistic and useless. PMI scoring, per-source perplexity, and fixed external test sets are more honest than aggregate val_loss.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;4. Set a capacity budget and test it.&lt;/strong&gt; Before adding more data, test whether your model can absorb it without regressing on existing domains. A simple per-source perplexity check after each training run tells you whether you're within capacity.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;5. Be honest about what your model can't do.&lt;/strong&gt; A 14.7M parameter model hallucinating at temperature 0.7 is not a cybersecurity tool. Label it accurately. The trajectory toward useful matters more than the current absolute performance.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;try-it-yourself&quot;&gt;Try It Yourself&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;git clone https://github.com/joemunene-by/GhostLM.git
cd GhostLM
make install
make data
make train-tiny
make chat
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The codebase is designed to be readable — every component is hand-written with docstrings, and the architecture is clean enough to use as a reference for how a transformer actually fits together.&lt;/p&gt;
&lt;p&gt;Contributions welcome — especially around corpus expansion, evaluation design, and architecture improvements (RoPE and Flash Attention are already in, SwiGLU and grouped query attention are next candidates).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;GitHub:&lt;/strong&gt; &lt;a rel=&quot;nofollow&quot; href=&quot;https://github.com/joemunene-by/GhostLM&quot;&gt;https://github.com/joemunene-by/GhostLM&lt;/a&gt;&lt;br&gt;
&lt;strong&gt;License:&lt;/strong&gt; MIT  &lt;/p&gt;
&lt;p&gt;Built in Nairobi, Kenya &lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/16400/what-learned-building-cybersecurity-llm-from-scratch-000-steps-architecture-rewrites-and</guid>
<pubDate>Sat, 02 May 2026 12:37:06 +0000</pubDate>
</item>
<item>
<title>Hello World! Welcome to the VoiceAutomationAI Group!</title>
<link>https://coderlegion.com/14917/hello-world-welcome-to-the-voiceautomationai-group</link>
<description>&lt;p&gt;Welcome to the VoiceAutomationAI group on CoderLegion! This is a space for developers building voice agents, real-time audio pipelines, and open-source voice infrastructure. &lt;br&gt;
What we discuss here: &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;STT, LLM, TTS pipeline architecture&lt;/li&gt;
&lt;li&gt;Self-hosted voice AI tools&lt;/li&gt;
&lt;li&gt;Telephony integrations &lt;/li&gt;
&lt;li&gt;Latency optimization for real-time calls&lt;/li&gt;
&lt;li&gt;Open-source projects like Dograh, Pipecat, LiveKit &lt;br&gt;
Start by introducing yourself in the comments. &lt;br&gt;
Tell us what you are building!&lt;/li&gt;
&lt;/ul&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/14917/hello-world-welcome-to-the-voiceautomationai-group</guid>
<pubDate>Wed, 15 Apr 2026 20:17:00 +0000</pubDate>
</item>
<item>
<title>Strategies for ensuring reliability and safety when AI agents gain full execution autonomy and contr</title>
<link>https://coderlegion.com/12831/strategies-ensuring-reliability-safety-when-agents-gain-full-execution-autonomy-contr</link>
<description>&lt;p&gt;Imagine an AI agent not just suggesting a stock trade, but actually executing it—buying, selling, transferring funds—all without human oversight.&lt;/p&gt;
&lt;p&gt;The idea of autonomous AI agents controlling real-world assets used to feel like science fiction. Today, it’s a tangible, if still nascent, reality. We've seen experiments where agents manage their own financial wallets, moving beyond the familiar role of planning and reasoning to truly acting. Most AI models still stop right before execution, requiring a human to hit &quot;go.&quot; But as these systems mature, that human veto could vanish, giving agents direct control over everything from financial transactions to infrastructure. This shift holds immense promise, but it also brings a very real, very serious question: how do we ensure these agents are reliable and safe when the stakes are so high?&lt;/p&gt;
&lt;h3 id=&quot;the-new-frontier-of-agent-autonomy&quot;&gt;The New Frontier of Agent Autonomy&lt;/h3&gt;
&lt;p&gt;The journey towards autonomous agents is compelling. These systems could automate complex workflows, manage resources with unprecedented efficiency, and unlock new levels of productivity. Think of agents optimizing supply chains, managing investment portfolios, or even orchestrating disaster response efforts. The allure is strong. But with full execution autonomy, the nature of failure changes dramatically. An agent that just &quot;suggests&quot; a bad plan is one thing; an agent that executes a bad plan, moving actual money or controlling physical systems, is an entirely different beast.&lt;/p&gt;
&lt;h3 id=&quot;the-shadows-of-unsupervised-action&quot;&gt;The Shadows of Unsupervised Action&lt;/h3&gt;
&lt;p&gt;When agents operate without direct human supervision, a whole host of problems—which developers are already wrestling with—become critically urgent.&lt;/p&gt;
&lt;p&gt; The Unseen Risks of Agent Autonomy&lt;/p&gt;
&lt;p&gt;   Flaky Evals and Unreliable Reasoning: Our current methods for evaluating AI agent performance often miss subtle issues. An agent might pass all its tests in a controlled environment but fail spectacularly when faced with an unexpected real-world scenario. If these &quot;flaky evals&quot; lead an agent to misinterpret a critical situation, the consequences can be severe, like an agent incorrectly approving a large financial transfer.&lt;br&gt;
   Hallucinations and Misinterpretations: LLMs are known for &quot;hallucinating&quot;—generating convincing but factually incorrect information. In a purely conversational setting, this is annoying. When an autonomous agent acts on a hallucination, believing it has a certain balance in an account or that a specific sensor reading is valid, it can lead to direct financial loss or system malfunction.&lt;br&gt;
   Prompt Injection Attacks: This is a sneaky one. Malicious actors can craft inputs that trick an agent into overriding its security protocols or performing unintended actions. Imagine an agent managing a cryptocurrency wallet. A clever prompt injection could instruct it to transfer funds to an unauthorized address, bypassing safeguards designed for human interaction. This includes &quot;indirect injection&quot; where an agent reads a poisoned document or website and then acts on those malicious instructions.&lt;br&gt;
   Cascading and Multi-Fault Scenarios: Failures rarely happen in isolation. One small error—say, a tool timeout in a component of a multi-step process—can trigger a chain reaction. An autonomous agent, designed to keep going, might push forward with incomplete data or a failed dependency, leading to a &quot;cascading failure&quot; where one problem triggers another, creating a far larger catastrophe. We need to think about &quot;multi-fault scenarios&quot; where several things go wrong at once, not just single points of failure.&lt;br&gt;
   Unsupervised Agent Behavior: What happens when an agent starts acting in ways we didn't explicitly program? This &quot;unsupervised agent behavior&quot; can be unpredictable. It might be inefficient, like burning through tokens unnecessarily (&quot;token burn&quot;), or it might be outright dangerous, making decisions that contradict its core mission.&lt;br&gt;
   Agent Robustness and Stress Testing: We need to know how much pressure an agent can handle. Can it withstand high load? Can it recover from internal errors? &quot;Agent stress testing&quot; and &quot;adversarial LLM testing&quot; involve pushing agents to their limits, intentionally trying to break them, to understand their true robustness. This is how we find the breaking points before they affect real-world assets.&lt;/p&gt;
&lt;h3 id=&quot;strategies-for-building-trust-and-control&quot;&gt;Strategies for Building Trust and Control&lt;/h3&gt;
&lt;p&gt;Moving to full execution autonomy is not about simply flipping a switch. It requires a fundamental shift in how we design, test, and monitor AI systems.&lt;/p&gt;
&lt;p&gt; Ensuring Reliability and Safety&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Rigorous Testing in CI/CD: We must integrate sophisticated AI agent testing directly into our Continuous Integration/Continuous Deployment pipelines. This means automated tests that go beyond unit tests, simulating real-world interactions and edge cases, not just happy paths. If an agent manages a financial ledger, we need tests that simulate concurrent transactions, network latency, and incorrect inputs, all before it ever touches production.&lt;/li&gt;
&lt;li&gt;Chaos Engineering for LLM Apps: Intentionally breaking our systems in controlled environments helps us build resilience. For LLM applications, this means introducing artificial delays, errors, and unexpected responses from external tools or APIs. How does an agent react when a critical API times out? Does it retry intelligently? Does it have a graceful fallback? This &quot;chaos engineering&quot; approach reveals weaknesses that traditional testing might miss.&lt;/li&gt;
&lt;li&gt;Enhanced AI Agent Observability: If an agent is making decisions and executing actions independently, we need to know exactly what it's doing, why, and what state it's in at all times. This means comprehensive logging, tracing its decision-making process, monitoring its interactions with external systems, and setting up alerts for anomalous behavior. Good &quot;AI agent observability&quot; is our eyes and ears in a world of autonomous action.&lt;/li&gt;
&lt;li&gt;Designing for Robustness and Resilience: Agents need to be built with failure in mind. This includes clear error handling, rollback mechanisms for critical transactions, and clearly defined safety boundaries. What actions are absolutely forbidden, no matter what the prompt says? How does the agent recover from a failed execution step? An &quot;agent robustness&quot; mindset means anticipating failure and designing intelligent ways for the system to cope.&lt;/li&gt;
&lt;li&gt;Multi-Layered Security: Beyond basic cybersecurity, agents require defenses against prompt injection and other LLM-specific vulnerabilities. This involves input validation, output sanitization, and potentially even redundant &quot;critique&quot; models that review an agent's planned actions before execution.&lt;/li&gt;
&lt;li&gt;Human-in-the-Loop Safeguards (Initially): While the goal is autonomy, a phased approach is sensible. Initially, critical actions might still require human approval, or at least human monitoring with the ability to intervene and override. This creates a safety net as we gather more data on agent performance in the wild.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;
&lt;p&gt;The move towards AI agents with full execution autonomy and control over real-world assets marks a significant moment. It's a powerful evolution that promises incredible benefits, but it demands an equally powerful commitment to reliability and safety. The challenges are real—from hallucinations and prompt injections to cascading failures and unpredictable behavior. Addressing these issues with robust testing, chaos engineering, deep observability, and resilient design isn't just a good idea; it's a fundamental requirement. Only by proactively tackling these complexities can we confidently unleash the true potential of autonomous AI without unintended, and often costly, consequences. The future of autonomous agents is here, and our responsibility is to build it right.&lt;/p&gt;
</description>
<category>Tutorials</category>
<guid isPermaLink="true">https://coderlegion.com/12831/strategies-ensuring-reliability-safety-when-agents-gain-full-execution-autonomy-contr</guid>
<pubDate>Fri, 13 Mar 2026 11:46:38 +0000</pubDate>
</item>
<item>
<title>Nameserver Checker</title>
<link>https://coderlegion.com/12359/nameserver-checker</link>
<description>&lt;p&gt;Use our free &lt;a rel=&quot;nofollow&quot; href=&quot;https://beingoptimist.in/tools/dns-tools/nameserver-checker/&quot;&gt;Nameserver Checker&lt;/a&gt; to look up NS records and verify nameservers for any domain in seconds Quickly troubleshoot DNS issues, confirm DNS changes, and ensure reliable domain routing&lt;/p&gt;
</description>
<category>Launches</category>
<guid isPermaLink="true">https://coderlegion.com/12359/nameserver-checker</guid>
<pubDate>Wed, 04 Mar 2026 14:07:40 +0000</pubDate>
</item>
<item>
<title>Website Safety Checker</title>
<link>https://coderlegion.com/11627/website-safety-checker</link>
<description>&lt;p&gt;Scan any URL for malware, phishing, and blacklist issues using our free &lt;a rel=&quot;nofollow&quot; href=&quot;https://www.beingoptimist.com/tools/website-safety-checker/&quot;&gt;website safety checker&lt;/a&gt; powered by Google Safe Browsing to browse and share links with confidence.&lt;/p&gt;
</description>
<category>Launches</category>
<guid isPermaLink="true">https://coderlegion.com/11627/website-safety-checker</guid>
<pubDate>Sat, 14 Feb 2026 06:15:00 +0000</pubDate>
</item>
<item>
<title>How to Use a Password Strength Checker (Step-by-Step Guide Based on Real Usage)</title>
<link>https://coderlegion.com/11501/how-to-use-a-password-strength-checker-step-by-step-guide-based-on-real-usage</link>
<description>&lt;p&gt;&lt;img src=&quot;https://coderlegion.com/?qa=blob&amp;amp;qa_blobid=14428075327564872538&quot; alt=&quot;How to Use a Password Strength Checker (Step-by-Step Guide Based on Real Usage)&quot;&gt;&lt;/p&gt;
&lt;p&gt;Weak passwords are one of the most common security risks online — and most people don’t realize it until something goes wrong.&lt;/p&gt;
&lt;p&gt;Over time, I’ve tested different password combinations while building security-focused tools on BeingOptimist.com. One simple habit that made a noticeable difference was checking password strength before using it anywhere important.&lt;/p&gt;
&lt;p&gt;In this article, I’ll walk you through how to use our &lt;a rel=&quot;nofollow&quot; href=&quot;https://www.beingoptimist.com/tools/password-strength-checker/&quot;&gt;Password Strength Checker&lt;/a&gt; properly — and how to interpret the results in a practical way.&lt;/p&gt;
&lt;p&gt;No technical jargon. No fear tactics. Just clarity.&lt;/p&gt;
&lt;h3 id=&quot;step-1-open-the-password-strength-checker-tool&quot;&gt;Step 1: Open the Password Strength Checker Tool&lt;/h3&gt;
&lt;p&gt;Go to the Password Strength Checker on BeingOptimist.&lt;/p&gt;
&lt;p&gt;You’ll see:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;A password input field&lt;/li&gt;
&lt;li&gt;A real-time strength indicator&lt;/li&gt;
&lt;li&gt;Feedback explaining what affects the strength score&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The tool works instantly in your browser. It does not require sign-up.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://coderlegion.com/?qa=blob&amp;amp;qa_blobid=11073663218964816564&quot; alt=&quot;password strength checker tool&quot;&gt;&lt;/p&gt;
&lt;h3 id=&quot;step-2-enter-your-password-safely-&quot;&gt;Step 2: Enter Your Password (Safely)&lt;/h3&gt;
&lt;p&gt;Type a password you’re planning to use.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://coderlegion.com/?qa=blob&amp;amp;qa_blobid=7547511829232537794&quot; alt=&quot;check password strength&quot;&gt;&lt;/p&gt;
&lt;p&gt;Important:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;This tool is designed for testing structure, not storing data.&lt;/li&gt;
&lt;li&gt;It evaluates length, character variety, and predictability patterns.&lt;/li&gt;
&lt;li&gt;It does not require you to submit anything to a server.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If you’re testing a real password from an active account, consider slightly modifying it for safety while testing structure.&lt;/p&gt;
&lt;h3 id=&quot;step-3-observe-the-strength-rating&quot;&gt;Step 3: Observe the Strength Rating&lt;/h3&gt;
&lt;p&gt;The tool will label the password as something like:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Weak&lt;/li&gt;
&lt;li&gt;Moderate&lt;/li&gt;
&lt;li&gt;Strong&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Instead of just showing a color indicator, it also explains why it received that rating.&lt;br&gt;
&lt;img src=&quot;https://coderlegion.com/?qa=blob&amp;amp;qa_blobid=7625482385382504994&quot; alt=&quot;moderate password&quot;&gt;&lt;/p&gt;
&lt;h3 id=&quot;typical-factors-evaluated-&quot;&gt;Typical factors evaluated:&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;Length of password&lt;/li&gt;
&lt;li&gt;Use of uppercase and lowercase letters&lt;/li&gt;
&lt;li&gt;Presence of numbers&lt;/li&gt;
&lt;li&gt;Use of special characters&lt;/li&gt;
&lt;li&gt;Repetitive or predictable patterns&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is important because many people assume complexity equals strength — but structure matters more than symbols alone.&lt;/p&gt;
&lt;h3 id=&quot;step-4-improve-based-on-feedback&quot;&gt;Step 4: Improve Based on Feedback&lt;/h3&gt;
&lt;p&gt;If your password is marked weak or moderate, improve it step-by-step.&lt;/p&gt;
&lt;p&gt;Instead of:&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Rahul123&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;You might evolve it into something structurally stronger like:&lt;/p&gt;
&lt;p&gt;&lt;em&gt;BlueTrain!47Forest&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Notice what changed:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Increased length&lt;/li&gt;
&lt;li&gt;Mixed characters&lt;/li&gt;
&lt;li&gt;Reduced predictability&lt;/li&gt;
&lt;li&gt;Not based purely on personal information&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The tool helps you understand this difference visually and structurally.&lt;/p&gt;
&lt;h3 id=&quot;step-5-understand-what-the-tool-can-and-cannot-do&quot;&gt;Step 5: Understand What the Tool Can and Cannot Do&lt;/h3&gt;
&lt;p&gt;This is important for transparency.&lt;/p&gt;
&lt;p&gt;A password strength checker:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;✔ Evaluates structure&lt;/li&gt;
&lt;li&gt;✔ Estimates resistance to common attack patterns&lt;/li&gt;
&lt;li&gt;✔ Helps improve password construction&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;But it does NOT:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;✘ Guarantee security&lt;/li&gt;
&lt;li&gt;✘ Detect data breaches&lt;/li&gt;
&lt;li&gt;✘ Replace a password manager&lt;/li&gt;
&lt;li&gt;✘ Prevent phishing&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Security is layered. A strong password is one part of that system.&lt;/p&gt;
&lt;h3 id=&quot;why-this-matters-from-experience-&quot;&gt;Why This Matters (From Experience)&lt;/h3&gt;
&lt;p&gt;After reviewing security tools and publishing practical guides on BeingOptimist, I’ve noticed something consistent:&lt;/p&gt;
&lt;p&gt;People don’t usually suffer breaches because they “don’t care.”&lt;/p&gt;
&lt;p&gt;They suffer breaches because:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;They reuse passwords&lt;/li&gt;
&lt;li&gt;They underestimate predictability&lt;/li&gt;
&lt;li&gt;They assume short complex passwords are strong&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Length and unpredictability often matter more than random symbols.&lt;/p&gt;
&lt;p&gt;This tool was built to simplify that learning curve — not to create fear, but to create awareness.&lt;/p&gt;
&lt;h3 id=&quot;who-should-use-a-password-strength-checker-&quot;&gt;Who Should Use a Password Strength Checker?&lt;/h3&gt;
&lt;p&gt;This tool is useful if you:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Are creating a new account&lt;/li&gt;
&lt;li&gt;Want to upgrade old passwords&lt;/li&gt;
&lt;li&gt;Are unsure if your password is strong enough&lt;/li&gt;
&lt;li&gt;Want to teach someone about basic online safety&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;It’s beginner-friendly, but the structural logic applies universally.&lt;/p&gt;
&lt;h3 id=&quot;final-thought&quot;&gt;Final Thought&lt;/h3&gt;
&lt;p&gt;Security doesn’t require paranoia. It requires habits.&lt;/p&gt;
&lt;p&gt;Checking your password strength before using it takes seconds — but it can prevent long-term inconvenience.&lt;/p&gt;
&lt;p&gt;Use tools for clarity.&lt;br&gt;
Understand what they measure.&lt;br&gt;
And build stronger digital habits gradually.&lt;/p&gt;
</description>
<category>Tutorials</category>
<guid isPermaLink="true">https://coderlegion.com/11501/how-to-use-a-password-strength-checker-step-by-step-guide-based-on-real-usage</guid>
<pubDate>Thu, 12 Feb 2026 13:03:40 +0000</pubDate>
</item>
<item>
<title>Why is Python so Slow?</title>
<link>https://coderlegion.com/10415/why-is-python-so-slow</link>
<description>&lt;p&gt;Why is Python so slow? Python is a beloved programming language for its readability, vast libraries, and beginner-friendly syntax. However, what most developers don't know is that Python can take a backseat to languages like C++ when it comes to raw speed. But why exactly is this the case?&lt;/p&gt;
&lt;h3 id=&quot;funny-memes-about-python&quot;&gt;Funny memes about Python&lt;/h3&gt;
&lt;p&gt;&lt;img src=&quot;https://coderlegion.com/?qa=blob&amp;amp;qa_blobid=12292827503832117007&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Context&lt;/strong&gt;&lt;br&gt;
Before discussing why Python is slow, it's essential to understand its relative speed compared to other programming languages. An easy way would be to write a function that counts from one to one billion and track how long each language takes to complete the task. Let's say we decide to use C++ and Python for this. Well, assuming all parameters are the same, it would take 2.6 seconds for the C++ code and 1 minute 53 seconds for the Python code, approximately 50 times slower. There's&lt;/p&gt;
&lt;p&gt;More memes&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://coderlegion.com/?qa=blob&amp;amp;qa_blobid=2398806992433363419&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt; Reason &lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;1 Interpretation vs Compilation&lt;/p&gt;
&lt;p&gt;Unlike compiled languages that are converted directly into machine code, Python is interpreted. This means the code is executed line by line instead of being translated into machine code before executing, adding an extra layer of processing compared to pre-compiled instructions. There are plenty of tests out there, but this paints a clear picture&lt;/p&gt;
&lt;p&gt;Benefits:&lt;br&gt;
Faster development cycle: Changes in the code are reflected immediately without recompiling.&lt;br&gt;
Portability: Interpreted code can run on any machine with the interpreter installed, regardless of the underlying architecture.&lt;/p&gt;
&lt;p&gt;Drawbacks:&lt;br&gt;
Generally slower: The translation overhead adds extra processing time compared to pre-compiled machine code.&lt;br&gt;
Limited control: Less control over memory management and hardware interaction compared to compiled languages.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt; Dynamic Typing &lt;/strong&gt;&lt;br&gt;
Python is dynamically typed, meaning variable data types are determined at runtime. While this offers flexibility, it requires extra checks during execution to ensure data compatibility whereas Statically typed languages, where types are declared upfront, can be optimized more efficiently.&lt;/p&gt;
&lt;p&gt;Benefits of Dynamic Typing:&lt;/p&gt;
&lt;p&gt;Rapid Prototyping: You can experiment with different data types without worrying about type declarations, allowing for quicker exploration during development.&lt;/p&gt;
&lt;p&gt;Drawbacks of Dynamic Typing:&lt;/p&gt;
&lt;p&gt;Potential Performance Overhead: Runtime type checks can add a slight overhead compared to statically typed languages, where types are known beforehand.&lt;/p&gt;
&lt;p&gt; Global Interpreter Lock (GIL)&lt;br&gt;
Python's GIL ensures only one thread can execute Python bytecode at a time. This prevents race conditions (multiple threads trying to modify data simultaneously) but limits true parallel processing capabilities.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt; What is the GIL? &lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Imagine a playground with a single swing. The GIL is like that swing – only one person(thread) can be on it (executing Python bytecode) at a time. Even if you have a fancy playground with multiple swings (multiple CPU cores), the GIL restricts threads to take turns on a single one.&lt;/p&gt;
&lt;p&gt;Impact of the GIL:&lt;/p&gt;
&lt;p&gt;Limits True Parallelism: Python programs can't fully use multi-core processors for CPU-bound tasks because only one thread can execute Python bytecode at a time.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt; Conclusion &lt;/strong&gt;&lt;br&gt;
Now, before you completely write off Python for performance-critical tasks, here's the truth of the matter&lt;/p&gt;
&lt;p&gt;Speed Isn't Always King. Often, development speed and programmer productivity outweigh raw execution time. Python's ease of use can lead to faster development cycles. while optimisations and Libraries techniques like profiling (identifying bottlenecks) and using optimised libraries like NumPy for numerical computing can significantly improve Python's performance.&lt;br&gt;
Also, in the case of multiprocessing, while the GIL limits true multithreading, Python offers multiprocessing, where separate processes can leverage multiple cores for tasks that can be broken down independently.&lt;/p&gt;
&lt;p&gt; &lt;strong&gt;Extras ✨&lt;/strong&gt;&lt;br&gt;
As some of you could have guessed, I'm pretty new to this writing, not coding, and I write about whatever idea pops into my head. I'm pretty sure this is probably my third blog post ever, so I would appreciate some feedback. Just make sure they are constructive, nothing about my height or the fact I'm still single&lt;/p&gt;
</description>
<category>Tutorials</category>
<guid isPermaLink="true">https://coderlegion.com/10415/why-is-python-so-slow</guid>
<pubDate>Sat, 24 Jan 2026 22:03:09 +0000</pubDate>
</item>
<item>
<title>Third-Party Logic - When You Run Code You've Never Seen</title>
<link>https://coderlegion.com/10108/third-party-logic-when-you-run-code-youve-neverseen</link>
<description>&lt;p&gt;By Muhammad Ali Khan ICS/ OT Cybersecurity Specialist - AAISM | CISSP | CISA | CISM | CEH | ISO27001 LI | CHFI | CGEIT | CDCP&lt;/p&gt;
&lt;p&gt;&lt;em&gt;At some point on every plant floor, an uncomfortable truth gets quietly accepted:&lt;br&gt;
&quot;You are running code you've never seen, never reviewed, and don't fully control.&quot;&lt;br&gt;
And in OT and ICS environments, that is a daily operational reality.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;From OEM-supplied PLC blocks to vendor remote support tools, from safety logic libraries to AI-driven optimization modules, third-party logic is deeply embedded into modern industrial systems. It often arrives wrapped in trust, urgency, or contractual obligation. And once deployed, it tends to stay there for years, untouched, unquestioned, and largely invisible.&lt;br&gt;
That's where the danger begins.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The Hidden Reality of Third-Party Logic&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Unlike IT software, third-party logic in OT is rarely transparent. Vendors routinely deliver compiled function blocks, encrypted ladder logic, or &quot;black box&quot; firmware updates that operators are discouraged, or contractually prohibited, from inspecting. As a result, asset owners end up executing logic whose behavior they infer rather than understand.&lt;br&gt;
Because of this, trust quietly replaces verification. The logic works, the process runs, and production targets are met, so nobody asks what else the code might be doing. However, operational success does not equal security assurance. In fact, in ICS environments, reliability often masks risk rather than eliminating it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why This Problem Is Unique to OT and ICS&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In IT environments, running third-party code is expected, but it's surrounded by layers of visibility: source reviews, sandboxing, EDR, patch cycles, and rapid rollback. In OT, the situation is fundamentally different.&lt;/p&gt;
&lt;p&gt;To begin with, availability outweighs inspection. Plant teams are trained to avoid touching working systems, which means third-party logic often remains frozen in time. Additionally, many controllers lack native logging or runtime visibility, making it difficult to observe what logic is actually doing during execution.&lt;/p&gt;
&lt;p&gt;Furthermore, third-party logic often interacts directly with physical processes. This means a logic flaw is not only a cyber issue, it can translate into unsafe states, equipment damage, or environmental harm. In OT, software behavior has kinetic consequences.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;How Third-Party Logic Becomes a Silent Attack Surface&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Because third-party logic is trusted by default, it often bypasses traditional security scrutiny. Vendors are given broad access privileges, remote connectivity is left enabled, and logic updates are applied without independent validation.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;As a result, several risks quietly accumulate.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;First, malicious intent becomes hard to detect. A logic block that subtly alters thresholds, delays alarms, or suppresses fault conditions can blend seamlessly into normal operations. Second, unintentional vulnerabilities persist. Poorly written logic, hardcoded credentials, or insecure communication routines can sit unnoticed for years. And third, supply chain compromise becomes invisible. If a vendor's development environment is breached, compromised logic can enter your plant without triggering any alarms.&lt;/p&gt;
&lt;p&gt;Each of these risks exists not because teams are careless, but because the environment was never designed for transparency.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The Contractual Trap of &quot;Vendor Intellectual Property&quot;&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;One of the most overlooked aspects of this issue is legal rather than technical. Many vendors classify control logic as intellectual property, restricting access to source code or even detailed documentation. This places asset owners in a paradoxical position: they are responsible for the safety and security of systems they are not allowed to fully inspect.&lt;/p&gt;
&lt;p&gt;Consequently, cybersecurity governance stops at the vendor boundary. Risk assessments exclude logic internals. Incident response plans assume vendor availability. And security accountability becomes blurred between ownership and authorship.&lt;br&gt;
In practice, this means you are accountable for outcomes, but dependent on third parties for understanding causes.&lt;/p&gt;
&lt;p&gt;Cybersecurity threat in industrial control roomWhy Traditional Security Controls Don't Solve This&lt;br&gt;
It's tempting to assume that firewalls, network segmentation, and access controls are enough. While they are essential, they do nothing to address the logic itself. Once third-party code is running inside a PLC, safety controller, or industrial application, perimeter controls are irrelevant.&lt;/p&gt;
&lt;p&gt;Similarly, vulnerability scanning and patch management fall short. You can scan the device, but not the logic. You can patch the firmware, but not the hidden behavior embedded in function blocks. Without logic-level visibility, security remains superficial.&lt;/p&gt;
&lt;p&gt;Therefore, the real risk is authorized execution of unverified behavior.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Building Visibility Without Breaking Operations&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The solution is not to ban third-party logic; that's unrealistic. Modern industrial systems depend on vendor expertise and prebuilt functionality. Instead, the goal must be controlled trust.&lt;br&gt;
This begins with architectural visibility. Asset inventories should include not just devices, but logic ownership: who wrote it, who maintains it, and when it was last reviewed. Change management must extend to logic updates, even when provided by vendors.&lt;/p&gt;
&lt;p&gt;In addition, behavioral monitoring becomes critical. Instead of trying to read every line of code, organizations should focus on understanding expected behavior and detecting deviations at runtime. Process-aware anomaly detection, logic checksum validation, and configuration baselining provide practical ways to gain assurance without disrupting operations.&lt;/p&gt;
&lt;p&gt;Equally important is contractual evolution. Security review rights, incident disclosure requirements, and update transparency must be embedded into vendor agreements. Trust should be formalized, not assumed.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The Mindset Shift OT Needs&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Ultimately, the biggest change is cultural. OT teams must move from &quot;it works, don't touch it&quot; to &quot;it works, but do we understand it?&quot; Security teams, in turn, must respect operational constraints while pushing for deeper assurance.&lt;/p&gt;
&lt;p&gt;Third-party logic is not inherently dangerous. What makes it risky is invisibility combined with permanence. When code runs for a decade without review, it becomes infrastructure, and infrastructure is hard to question.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Final Thought&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Every PLC cycle executes instructions that shape physical reality. When those instructions come from sources you've never seen, you are operating on borrowed trust.&lt;/p&gt;
&lt;p&gt;In modern OT and ICS environments, cybersecurity is no longer just about keeping attackers out. It's about knowing what you've already let in.&lt;/p&gt;
&lt;p&gt;And the most dangerous code is not the one that crashes systems, it's the one that runs perfectly, silently, and unquestioned.&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/10108/third-party-logic-when-you-run-code-youve-neverseen</guid>
<pubDate>Tue, 20 Jan 2026 19:30:32 +0000</pubDate>
</item>
<item>
<title>&quot;Case 005: High-Risk Scholarship Forms and Excessive Data Collection&quot;</title>
<link>https://coderlegion.com/9464/case-005-high-risk-scholarship-forms-and-excessive-data-collection</link>
<description>&lt;h2 id=&quot;case-005-high-risk-scholarship-forms-and-the-dangers-of-excessive-data-collection&quot;&gt;Case 005: High-Risk Scholarship Forms and the Dangers of Excessive Data Collection&lt;/h2&gt;
&lt;p&gt;Online scholarship programs have helped many students access education support.&lt;br&gt;
But in the digital age, some application forms collect &lt;strong&gt;extremely sensitive personal data&lt;/strong&gt;, creating serious privacy and cybersecurity risks.&lt;/p&gt;
&lt;p&gt;This case study explains &lt;strong&gt;why excessive data collection itself can become a cyber risk&lt;/strong&gt;, even when a platform appears legitimate or well-intentioned.&lt;/p&gt;
&lt;p&gt; &lt;strong&gt;Vyoma Security Awareness:&lt;/strong&gt;&lt;br&gt;
 &lt;a rel=&quot;nofollow&quot; href=&quot;https://vyomasecurity.vercel.app/&quot;&gt;https://vyomasecurity.vercel.app/&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;This article is written for &lt;strong&gt;public awareness and digital safety education&lt;/strong&gt;, especially for students and parents.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;understanding-the-risk-data-volume-vs-necessity&quot;&gt;Understanding the Risk: Data Volume vs. Necessity&lt;/h3&gt;
&lt;p&gt;A secure digital system follows a basic cybersecurity principle:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;&lt;strong&gt;Data minimization — collect only what is absolutely necessary.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Warning signs appear when a single scholarship form requests:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Government ID numbers  &lt;/li&gt;
&lt;li&gt;Front and back images of identity documents  &lt;/li&gt;
&lt;li&gt;Bank account details and IFSC codes  &lt;/li&gt;
&lt;li&gt;Photographs of the student  &lt;/li&gt;
&lt;li&gt;Images of the student’s home or surroundings  &lt;/li&gt;
&lt;li&gt;Family income, religion, caste, or housing details  &lt;/li&gt;
&lt;li&gt;Utility bills or precise location information  &lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each additional data point increases the &lt;strong&gt;attack surface&lt;/strong&gt; and long-term misuse risk.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;why-this-is-a-cybersecurity-concern&quot;&gt;Why This Is a Cybersecurity Concern&lt;/h3&gt;
&lt;p&gt;Even without malicious intent, such forms raise important questions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;How is this data stored?&lt;/li&gt;
&lt;li&gt;Who has access to it?&lt;/li&gt;
&lt;li&gt;Is the data encrypted?&lt;/li&gt;
&lt;li&gt;How long is it retained?&lt;/li&gt;
&lt;li&gt;What happens if the system is breached?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If these answers are not clearly communicated, &lt;strong&gt;users carry the risk — not the platform&lt;/strong&gt;.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;sensitive-data-categories-involved&quot;&gt;Sensitive Data Categories Involved&lt;/h3&gt;
&lt;p&gt;The following types of information are considered &lt;strong&gt;high-risk in cybersecurity&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Identity documents (Aadhaar or similar IDs)&lt;/li&gt;
&lt;li&gt;Financial details (bank account information)&lt;/li&gt;
&lt;li&gt;Biometric identifiers (photographs)&lt;/li&gt;
&lt;li&gt;Residential information (home images)&lt;/li&gt;
&lt;li&gt;Socio-economic profiling data&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;When combined, this data can enable:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Identity theft  &lt;/li&gt;
&lt;li&gt;Financial fraud  &lt;/li&gt;
&lt;li&gt;Targeted scams  &lt;/li&gt;
&lt;li&gt;Long-term privacy violations  &lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&quot;why-students-and-families-are-vulnerable&quot;&gt;Why Students and Families Are Vulnerable&lt;/h3&gt;
&lt;p&gt;Students applying for scholarships often:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Face financial pressure&lt;/li&gt;
&lt;li&gt;Trust authority-based programs&lt;/li&gt;
&lt;li&gt;Do not question data requests&lt;/li&gt;
&lt;li&gt;Lack legal or technical awareness&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This imbalance increases the responsibility of platforms to act &lt;strong&gt;ethically and transparently&lt;/strong&gt;.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;transparency-is-the-real-issue&quot;&gt;Transparency Is the Real Issue&lt;/h3&gt;
&lt;p&gt;A responsible digital form should clearly show:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;A readable privacy policy&lt;/li&gt;
&lt;li&gt;Why each data point is required&lt;/li&gt;
&lt;li&gt;How long the data will be stored&lt;/li&gt;
&lt;li&gt;Security measures in place&lt;/li&gt;
&lt;li&gt;A way to request data removal&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Missing or hidden information is a &lt;strong&gt;red flag&lt;/strong&gt;, not proof of wrongdoing.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;awareness-tips-for-applicants-and-parents&quot;&gt;Awareness Tips for Applicants and Parents&lt;/h3&gt;
&lt;p&gt;Before submitting sensitive documents:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Verify the organization through official channels&lt;/li&gt;
&lt;li&gt;Read the privacy policy carefully&lt;/li&gt;
&lt;li&gt;Question why each document is required&lt;/li&gt;
&lt;li&gt;Avoid platforms asking for unnecessary home or family images&lt;/li&gt;
&lt;li&gt;Discuss the application with a trusted adult or educator&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Caution is not distrust — &lt;strong&gt;it is digital responsibility&lt;/strong&gt;.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;a-broader-lesson&quot;&gt;A Broader Lesson&lt;/h3&gt;
&lt;p&gt;Cyber risk does not only come from hackers.&lt;/p&gt;
&lt;p&gt;It also comes from:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;&lt;strong&gt;Over-collection of personal data without clear protection guarantees&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Good intentions do not remove breach consequences.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;conclusion-1&quot;&gt;Conclusion&lt;/h3&gt;
&lt;p&gt;Scholarship programs play an important role in education access.&lt;br&gt;
But in the digital era, &lt;strong&gt;privacy protection is part of student welfare&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Awareness helps users:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Protect their identity&lt;/li&gt;
&lt;li&gt;Reduce long-term risk&lt;/li&gt;
&lt;li&gt;Ask better questions&lt;/li&gt;
&lt;li&gt;Demand safer digital systems&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Education should empower — &lt;strong&gt;not expose&lt;/strong&gt;.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;&lt;em&gt;This case study is part of Vyoma Security Awareness, a student-led initiative focused on digital literacy, cyber ethics, and public safety.&lt;/em&gt;&lt;br&gt;
 &lt;a rel=&quot;nofollow&quot; href=&quot;https://vyomasecurity.vercel.app/&quot;&gt;https://vyomasecurity.vercel.app/&lt;/a&gt;&lt;/p&gt;
</description>
<category>Articles</category>
<guid isPermaLink="true">https://coderlegion.com/9464/case-005-high-risk-scholarship-forms-and-excessive-data-collection</guid>
<pubDate>Fri, 09 Jan 2026 13:16:21 +0000</pubDate>
</item>
</channel>
</rss>