Coder Legion - Recent posts tagged llmsecurity

AURA v0.1.0: Deterministic Trigger Extraction, Auditable Math & a Self-Healing Analytics Engine

Sat, 26 Sep 2026 15:51:09 +0000

If you follow AI safety, you know the uncomfortable truth: modern LLM guardrails are failing not because of complex zero-day exploits, but because of sloppy heuristics, brittle filters, and naive prompt-matching.

We built AURA to be pragmatic: deterministic, testable, fully auditable, and completely immune to the hallucination of its own telemetry.

Today, we’re unpacking AURA v0.1.0 — explaining our architectural decisions, showing real code excerpts, diving into non-linear risk scoring math, and solving the silent problem of lost repository analytics on GitHub.

1. System Architecture at a Glance

Data flow in AURA is intentionally simple, pipeline-driven, and fully reproducible:

Source Cases (public_cases/*.json) → Cleaned, validated, and normalized via scripts/normalize-percases.ts
Rule Extraction (scripts/extract-triggers.ts) → Contextual regex and sliding-window token analysis.
Weights & Signal Mapping → Derived from config/signal-mapping.json and saved to config/trigger-weights.json.
Scoring & Policy Enforcement (scripts/recalc_confidence.ts) → Non-linear math transform + cross-check audit logic via scripts/policy/crossCheckAdapter.ts.
Persisted Telemetry → Automated daily snapshots written to analytics/traffic-history.json via GitHub Actions.

2. Non-Linear Risk Normalization: Why We Don't Just "Sum Things Up"

We compute a transparent raw evidence sum (confidence_raw) and convert it into a bounded score via a diminishing‑returns exponential:

\[ \text{confidence} = 1 - e^{-\alpha \cdot \text{confidence\_raw}} \]

This keeps scores in [0...1] and avoids noisy amplification from many weak cues. Implementation (excerpt from scripts/recalc_confidence.ts):

// computedRaw is the honest sum of trigger weights + cross-check contributions
const alpha = (typeof cfg.normAlpha === 'number') ? cfg.normAlpha : 1.0;
const normalized = 1 - Math.exp(-alpha * computedRaw);
let newVal = Math.round(normalized * 100) / 100;
if (newVal < minFloor) newVal = minFloor;
e.confidence_raw = Math.round(computedRaw * 100) / 100;
e.confidence = newVal;

Tuning Sensitivity (\({\alpha}\))

Lower \(\alpha\) (0.1 - 0.2): Conservative normalization; requires heavier evidence to push confidence towards 1.0.
Higher \({\alpha}\) (0.5+): Aggressive sensitivity for high-security environments.
AURA Baseline (\(\alpha = 0.3\)): Striking a balance where \(\text{confidence\_raw} = 3.0\) yields \(\approx 0.59\), requiring compounding signals for a hard policy block.

3. Hardening Trigger Extraction: Sliding Windows & Contextual Anchors

Keyword lists are a recipe for false positives. Matching a naive string like "generate 500" catches harmless test cases alongside malicious payloads.

To balance precision and recall, AURA v0.1.0 introduces ordered-within-window token matching.

Sliding Window Engine

Excerpt from scripts/extract-triggers.ts showing how AURA tolerates small syntactic noise without matching scattered words across an entire prompt:

function containsOrderedWithinWindow(haystack: string[], needle: string[], window = 5): boolean {
  if (needle.length === 0) return false;
  if (needle.length === 1) return haystack.indexOf(needle[0]) !== -1;
  for (let i = 0; i < haystack.length; i++) {
    if (haystack[i] !== needle[0]) continue;
    let idx = i + 1;
    let matched = 1;
    for (let k = 1; k < needle.length && idx < Math.min(haystack.length, i + window + 1); idx++) {
      if (haystack[idx] === needle[k]) { matched++; k++; }
    }
    if (matched === needle.length) return true;
  }
  return false;
}

Contextual Anchors vs. Co-Occurrence Cues

In config/trigger-extraction.json, bulk asset creation explicitly requires a deception target:

{
  "trigger": "non-consensual pattern generation",
  "pattern": "\\bgenerate\\s+\\d+\\s+(?:deceptive\\s+assets|phishing\\s+emails|fake\\s+documents|fake\\s+profiles|fake\\s+accounts|malicious\\s+payloads|spam\\s+emails|synthetic\\s+attacks)\\b",
  "description": "Bulk-generation demand with deception-specific targets."
}

For generic terms like "audit", we mandate multi-token co-occurrence:

{
  "trigger": "unauthorized audit camouflage",
  "cues": ["audit", "unauthorized", "independent", "bypass", "without permission"],
  "description": "Require co-occurrence of 'audit' with authorization‑bypass phrasing."
}

I don't know about you, but I got tired of seeing my project's growth through the lens of GitHub’s default 14-day window. GitHub silently wipes daily clone and view metrics after two weeks, leaving open-source maintainers completely blind to long-term adoption trends unless they buy third-party analytics dashboards.

To solve this, AURA v0.1.0 includes an automated, self-healing snapshot pipeline in .github/workflows/traffic-history.yml.

How the Analytics Snapshot Pipeline Works:

Daily Ingestion: Executes a daily cron job via GitHub REST API.
Deduplication: Merges metrics into analytics/traffic-history.json while purging duplicate artifacts.
Self-Healing Merge Logic: If concurrent workflow updates cause a direct git push to main to fail, the action politely opens a temporary PR, squash-merges it via actions/github-script, and cleans up after itself:

- name: Auto-merge PR and delete branch
  uses: actions/github-script@v6
  with:
    github-token: ${{ secrets.TRAFFIC_TOKEN }}
    script: |
      const head = `auto/traffic-report-${process.env.GITHUB_RUN_ID}`;
      const { data: prs } = await github.rest.pulls.list({ owner: context.repo.owner, repo: context.repo.repo, head: `${context.repo.owner}:${head}`, state: 'open' });
      if (prs && prs.length > 0) {
        await github.rest.pulls.merge({ owner: context.repo.owner, repo: context.repo.repo, pull_number: prs[0].number, merge_method: 'squash' });
        await github.rest.git.deleteRef({ owner: context.repo.owner, repo: context.repo.repo, ref: `heads/${head}` });
      }

5. Community Spotlight: Catching a Subtle Async Race Condition

One of the best parts of open-sourcing AURA is having sharp contributors look at the edge cases. Our contributor Amirhossein Agrest spotted a subtle async race condition in how case state is updated before disk serialization.

updateCase() awaits cross-check evaluation internally. However, in certain batch execution flows, calling scripts invoked it across collections without awaiting the return promise before writing JSON files to disk.

The result? CLI logs proudly claimed success, while disk artifacts still contained pre-audit state.

// ❌ Potential Race: Fire-and-forget async invocation
for (const entry of entries) updateCase(entry);

// ✅ Fix Pattern: Await all async mutations before serializing to disk
const promises = entries.map(entry => updateCase(entry));
await Promise.all(promises);

We’ve logged this issue (shoutout to Amirhossein!) and are pairing it with artificial network-delay adapters in our test suite to guarantee filesystem persistence never outruns in-memory state in the upcoming patch.

6. Roadmap (What Realistically Comes Next)

v0.2: Programmatic prompt tokenization + TF‑IDF experiments for ranking weak cues (automated test-corpus generation).
Tooling UI: Better cross-trigger clustering and a visual rule editor — because manually squinting at hundreds of lines of raw JSON is a fast track to eye bleed, and my laziness is a major driver for automation.
ML Augmentation: Replace some heuristics with small supervised models for cue disambiguation — but only where deterministic rules fall short (no ML for the sake of ML).

Closing Notes

AURA is not magic, and it doesn't pretend to be. It's a pragmatic stack: deterministic rules, auditable math, and CI that refuses to forget its history.

If you're looking for a silver bullet, good luck! But if you want a system that is testable, inspectable, and built on sound software engineering principles, welcome aboard.

⭐ Check out the repository, inspect the code, and give us a star:
👉 GitHub: kate8382/AURA

Open an issue, or even better — submit a PR with a failing unit test to help us catch edge cases faster.

Why Big Tech Keeps Losing LLMs to Basic Social Engineering

Fri, 28 Aug 2026 09:21:13 +0000

By Ecaterina Sevciuc | Creator of AURA (AI User Risk Assessment)

Two months ago, I launched AURA — an open-source framework designed to model psychological manipulation, grey-zone threat vectors, and social engineering in Human-AI interactions.

Yesterday, I stumbled upon a Reuters report detailing how hackers exploited Cursor (running Anthropic’s Claude Sonnet) to compromise seven companies worldwide. This isn't the first such incident in the news, and I suspect it certainly won't be the last.

(Side note on the attackers' group name, "Aur0ra": I can assure you that for a Russian-speaking group, this is almost certainly not a homage to the Roman goddess of dawn, but a subtle nod to the infamous historical cruiser Aurora — known for firing the shot that signaled a revolution. A fittingly dark bit of Eastern European sarcasm for a tool that overthrows AI security).

Their weapon? They didn't write a zero-day exploit. They simply convinced the AI agent that the attack was "just a security simulation." The model balked a few times, felt uncomfortable, and then happily handed over the keys.

As an AI Safety architect with a background in banking compliance and legal risk evaluation, watching Big Tech react to this is painful. They are building multi-billion-dollar static guardrails while AI agents are being tricked by the oldest psychological tricks in the book.

The Fatal Flaws of Modern AI Guardrails

Big Tech’s approach to AI safety is fundamentally broken because it relies on Static Keyword Filtering & Single-Language Heuristics:

Rule Evasion: If a prompt contains "how to build a bomb", the model blocks it. But if the exact same request is framed as "I am a researcher simulating a crisis scenario for an academic paper," the model complies.
Linguistic Blind Spots: Guardrails are heavily aligned on technical, low-complexity English. Synthetic, morphologically rich, or non-Indo-European languages (like Russian, Arabic, or East Asian language groups) leverage complex idioms, case shifts, and semantic ambiguity. These linguistic structures effortlessly slip past safety filters that simply aren't built to parse deep semantic nuance.

Traditional security engineers treat LLMs like deterministic databases. They are not databases; they are cognitive systems subject to social engineering and linguistic circumvention.

When attackers convince an agent that an exploit is a "simulation" or mask intent behind complex non-English semantics, they aren't bypassing code — they are exploiting persona vulnerability, alibi trust, and language alignment gaps.

How AURA Predicted This (and How to Actually Fix It)

When I designed the AURA Framework, I specifically isolated three core domains that traditional guardrails ignore:

Vector Category	Real-World Attack (e.g., Cursor / Anthropic Incident)	The AURA Defense Mechanism
`MANIPULATION`	Gaslighting the model with fake authority ("I am an auditor") or simulated environments.	Dynamic Persona Verification: Flagging high-risk roles unless hard proof/provenance is provided.
`FRAUD`	Compliance evasion, tricking agents into unauthorized credential harvesting under false pretexts.	Algorithmic Cross-Checking: Recalculating confidence scores dynamically based on intent vs. action.
`ACCESS`	Gradual privilege escalation through multi-turn conversational framing.	Stateful Behavioral Matrices: Tracking risk context across turns, not just evaluating prompts in isolation.

In AURA's schema, a prompt like "Run this script as part of a test" triggers an immediate drop in confidence and demands provenance verification (cross_check). If the agent in the Cursor incident had evaluated intent through a behavioral threat matrix rather than a static safety filter, the attack would have died on turn one.

The Formula Big Tech Ignores

To stop AI agents from turning against their own systems, we must evaluate interactions using structural tokenization:

\(\text{Behavioral Risk} = \text{Persona Claim} + \text{Target Action} + \text{Evasion Framing} + \text{Alibi Pattern}\)

If an input has a high-value Target Action masked by an unverified Alibi Pattern (e.g., "Just a simulation") or hidden within complex semantic framing, the system score must instantly breach the deception_threshold.

Stop Playing Catch-Up

We cannot solve cognitive vulnerability with static blocklists. As AI agents gain access to IDEs, databases, and enterprise APIs, allowing them to be duped by basic roleplay isn't just a bug — it's systemic negligence.

I built AURA open-source because this architecture needs to exist. The public framework is live, validated, and proven by the very breaches hitting the headlines today.

GitHub (Open Baseline): AURA
Enterprise & Threat Matrices: Reach out directly for private modules, custom B2B threat matrices, or pilot integrations.

The tools to prevent this were ready months ago. It's time the industry started using them.

P.S. The Open-Source Paradox: High Consumption, Zero Collaboration

When I released AURA, the response was immediate — traffic spiked, and repository clone rates skyrocketed. The industry clearly recognizes these risks; developers and security teams know current guardrails are failing.

But open source today suffers from a systemic flaw: it has become about total consumption, not active collaboration.

Dozens of engineers cloned the code, integrated it into their workflows, and extracted value for their own closed environments. Yet, not a single feedback loop was established. No issues raised, no architecture improvements proposed, no community ideas shared.

As the saying goes, one is no warrior in the field. I cannot map every psychological attack vector or every language-specific ambiguity alone. To genuinely shift the paradigm in AI safety, we need collaborators — people and organizations willing to contribute rather than just extract.

OWASP Top 10 for LLMs: From Risk to Concrete Engineering Fixes

Mon, 24 Aug 2026 10:43:43 +0000

The OWASP Top 10 for LLM Applications is a great starting point for understanding AI security.

But knowing the risks isn't enough.

The real question is

𝗪𝗵𝗮𝘁 𝗱𝗼 𝘆𝗼𝘂 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗶𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁 𝘁𝗼 𝗺𝗶𝘁𝗶𝗴𝗮𝘁𝗲 𝘁𝗵𝗲𝗺?

When analyzing the OWASP LLM risks from an engineering perspective, I see a recurring pattern: many vulnerabilities cannot be solved by simply adding another prompt instruction or filtering a few keywords.

They require 𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗰𝗼𝗻𝘁𝗿𝗼𝗹𝘀 𝗮𝘁 𝘁𝗵𝗲 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗹𝗲𝘃𝗲𝗹.

Here is how I approach the major risk categories:

🔹 𝗟𝗟𝗠𝟬𝟭 – 𝗣𝗿𝗼𝗺𝗽𝘁 𝗜𝗻𝗷𝗲𝗰𝘁𝗶𝗼𝗻
→ Implement prompt gateways, input isolation, content validation, trust boundaries and adversarial testing.

🔹 𝗟𝗟𝗠𝟬𝟮 – 𝗦𝗲𝗻𝘀𝗶𝘁𝗶𝘃𝗲 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻 𝗗𝗶𝘀𝗰𝗹𝗼𝘀𝘂𝗿𝗲
→ Apply data classification, least-privilege access, output filtering and strict separation of system/user data.

🔹 𝗟𝗟𝗠𝟬𝟯 – 𝗦𝘂𝗽𝗽𝗹𝘆 𝗖𝗵𝗮𝗶𝗻
→ Establish model and dependency provenance, integrity verification, SBOMs, controlled model registries and continuous scanning.

🔹 𝗟𝗟𝗠𝟬𝟰 – 𝗗𝗮𝘁𝗮 𝗮𝗻𝗱 𝗠𝗼𝗱𝗲𝗹 𝗣𝗼𝗶𝘀𝗼𝗻𝗶𝗻𝗴
→ Validate training/RAG data, establish data provenance, monitor ingestion pipelines and detect anomalous content.

🔹 𝗟𝗟𝗠𝟬𝟱 – 𝗜𝗺𝗽𝗿𝗼𝗽𝗲𝗿 𝗢𝘂𝘁𝗽𝘂𝘁 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴
→ Treat LLM output as untrusted input. Validate, sanitize and constrain outputs before passing them to downstream systems.

🔹 𝗟𝗟𝗠𝟬𝟲 – 𝗘𝘅𝗰𝗲𝘀𝘀𝗶𝘃𝗲 𝗔𝗴𝗲𝗻𝗰𝘆
→ Use least privilege, capability restrictions, approval workflows and explicit tool authorization.

🔹 𝗟𝗟𝗠𝟬𝟳 – 𝗦𝘆𝘀𝘁𝗲𝗺 𝗣𝗿𝗼𝗺𝗽𝘁 𝗟𝗲𝗮𝗸𝗮𝗴𝗲
→ Never treat the system prompt as a security boundary. Move sensitive controls into enforceable backend policies.

🔹 𝗟𝗟𝗠𝟬𝟴 – 𝗩𝗲𝗰𝘁𝗼𝗿 𝗮𝗻𝗱 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴 𝗪𝗲𝗮𝗸𝗻𝗲𝘀𝘀𝗲𝘀
→ Secure the entire RAG pipeline: document ingestion, chunking, metadata, embeddings, retrieval and access control.

🔹 𝗟𝗟𝗠𝟬𝟵 – 𝗠𝗶𝘀𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻
→ Introduce grounding, provenance, retrieval validation, confidence mechanisms and human oversight for high-impact decisions.

🔹 𝗟𝗟𝗠𝟭𝟬 – 𝗨𝗻𝗯𝗼𝘂𝗻𝗱𝗲𝗱 𝗖𝗼𝗻𝘀𝘂𝗺𝗽𝘁𝗶𝗼𝗻
→ Apply rate limiting, quotas, resource controls, request validation and cost monitoring.

But there is a deeper lesson here.

𝗔𝗜 𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗶𝘀 𝗶𝗻𝗰𝗿𝗲𝗮𝘀𝗶𝗻𝗴𝗹𝘆 𝗯𝗲𝗰𝗼𝗺𝗶𝗻𝗴 𝗮𝗻 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗮𝗹 𝗱𝗶𝘀𝗰𝗶𝗽𝗹𝗶𝗻𝗲.

A secure AI application needs more than a secure model.

It needs security controls around:

→ Identity & trust
→ Prompt security
→ Retrieval
→ Memory
→ Tool execution
→ Policy enforcement
→ Data flows
→ Observability
→ Runtime behaviour

This is one of the areas I am exploring through 𝗡𝗲𝘂𝗿𝗮𝗹𝗦𝘁𝗮𝗰𝗸 | 𝗠𝗦 - 𝗔𝗜 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆: translating AI security research and frameworks into practical architectures, engineering patterns and security controls that can actually be implemented.

My goal isn't simply to answer:

“𝗪𝗵𝗮𝘁 𝗰𝗮𝗻 𝗴𝗼 𝘄𝗿𝗼𝗻𝗴?”

It is to answer:

“𝗛𝗼𝘄 𝗱𝗼 𝘄𝗲 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 𝘁𝗵𝗲 𝘀𝘆𝘀𝘁𝗲𝗺 𝘀𝗼 𝘁𝗵𝗮𝘁 𝗶𝘁 𝗶𝘀 𝗵𝗮𝗿𝗱𝗲𝗿 𝘁𝗼 𝗴𝗼 𝘄𝗿𝗼𝗻𝗴 𝗶𝗻 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗽𝗹𝗮𝗰𝗲?”

I’ll be breaking down these areas in more detail through my AI Security Engineering research and projects.

If you're building LLM applications, RAG systems or agentic AI, this is a security conversation worth having.

(Post revised using GPT)

AURA (AI User Risk Assessment): A behavioral threat-intelligence framework for AI Safety

Wed, 15 Jul 2026 16:58:24 +0000

As a banking and legal professional, there’s one harsh truth I learned early in my career: rules are rarely broken by brute force; they are bypassed through loopholes and human psychology.

On my very first year of law school, practicing attorneys told us: "You don’t just study the laws to follow them. You study them to understand how others will bend and reverse them to their advantage."

Later, in banking, I saw this play out in reality. Compliance regulations look flawless on paper. But in the real world, there is always a "high-priority client" or an influential stakeholder attempting to bypass protocols using pressure, authority, or deceptive framing.

Today, we are making the exact same mistake with Artificial Intelligence.

We feed LLMs dry documentation, compliance guidelines, and static safety rules. We are trying to build the "perfect talking textbook." But LLMs do not operate in a vacuum—they interact with humans. And humans are creative, manipulative, and incredibly persistent.

If we don’t teach AI to recognize the psychological and behavioral "grey zones" of social engineering, even the most advanced guardrails will fail.

To bridge this gap between engineering and real-world behavioral risks, I built AURA.

🚀 Introducing AURA: AI User Risk Assessment Framework

AURA is an open-source, schema-driven library of behavioral matrices, heuristics, and validation tooling designed to detect manipulation, deception, and grey-zone threats in human-AI dialogue.

Key Features of the MVP:

- Granular Threat Domains: Structured cases focused on MANIPULATION, FRAUD, and ACCESS control.

- Heuristic Risk Scoring: Dynamic confidence recalculation based on behavioral triggers, alibis, and cross-checks.

- Strict Developer Tooling: AJV-backed JSON schema validation, Jest unit tests, and automated GitHub Actions CI pipeline to ensure clean, reproducible data.

- Self-Contained Architecture: Every case is a single JSON, making it incredibly lightweight and ready to be integrated into safety agent pipelines.

The repository is now live as an active Work in Progress. The framework is currently in its early development stage. Whether you are building an LLM safety layer, researching red-teaming vectors, or just interested in how human psychology bypasses AI guardrails—I would love to hear your feedback!

Let’s build safer, context-aware AI together.

👉 GitHub Repository: https://github.com/kate8382/AURA.git