In an earlier blog post we've described, at a high level, how we use AI models to read threat intelligence reports, surface detection opportunities, and match them against our existing detection catalogue. In this follow-up post, we're exploring that work in detail.

TL;DR

Detection engineering is a repetitive, manual process that struggles to keep pace with the volume of modern threat intelligence. Every report must be analysed, mapped to the right taxonomy, translated into platform-specific detection logic and validated before it can be used.

We've built an agentic workflow that uses LLMs, MCP and reusable skills to automate much of that work. Analysts receive structured detection opportunities and draft detections that are grounded in threat intelligence and aligned with our existing detection model.

The key is not replacing analysts, but augmenting them. Skills provide the domain knowledge and guardrails that make the workflow reliable, while MCP gives the agent access to the tools and data it needs to reason about coverage and existing detections.

The result is a faster, more consistent detection engineering process that reduces repetitive work while keeping expert validation and decision-making with human analysts.

Detection engineering is hard to scale

According to SOC Prime’s definition of detection engineering, it is:

... about developing an environment inside an organisation where several teams collaborate to address risks and target potential threats better. Detection engineering involves dealing with designing, developing, testing, and maintaining threat detection logic. This threat detection logic can be a rule, a pattern, or even a textual description.

At mnemonic, this work spans several platforms, including our security platform Argus and the Argus Network Analyser, Microsoft Sentinel, YARA, and the tooling our threat hunters rely on. A single detection is more than its query. Our detection model carries common metadata (an identifier, a title, a description, status, authorship, MITRE ATT&CK tags, a TLP marking) and then platform-specific logic underneath, such as the event-selection query that actually decides what triggers.

That detection model is what makes detection engineering powerful, and also what makes it time-consuming. Every new piece of attacker knowledge has to be read, understood, placed in the right taxonomy, expressed in the right query language, tested and checked against what already exists. Multiply that across platforms and across the steady stream of threat intelligence the industry produces, and it is no wonder that the human time spent on this repetitive translation work becomes a bottleneck.

Our first attempt in 2023 was not very promising

Back in 2023, we experimented with using language models to do the job of a threat intelligence analyst. The goal was to take unstructured reports and extract structured information from them: the threat actor, the targeted countries and sectors, the techniques in play.

It did not work well enough to implement in production. Three problems stood out:

• The models hallucinate and invent MITRE techniques that did not exist

• There were limits on the size of the reports that could be analysed

• The response was not consistent

For detection engineering, an inconsistent and occasionally fabricated answer is worse than no answer at all.

Things have changed in 2026

Today the models have improved, and new ways to instruct them make it possible to achieve consistent outcome:

• Prompt: Natural-language instructions and examples that guide the model for this task.
• Output schema/constraints: Explicit rules for structure/format (e.g. JSON schema) to make outputs predictable and machine-readable.
• MCP server: A standardised interface the model uses to discover and call external services/tools consistently.
• Skills: Reusable, packaged workflows that combine prompts + tools + checks to solve recurring tasks reliably.

Just as important, the models became agentic: instead of answering a single prompt in isolation, they can now plan a task, call tools and work through several steps on their own. This is what lets them carry a report all the way to a detection.

Let’s see an example of the workflow we follow to convert unstructured reports:

into actionable detection rules:

A threat report arrives. It is scheduled from a vendor feed such as CrowdStrike's or handed in manually. An agent reads it, using tools to consult MITRE ATT&CK and our data-source catalogue, and produces a set of detection opportunities: specific attacker behaviours worth detecting, anchored in what the report actually says.

A second step maps each opportunity against our existing detections by querying our detection inventory through a dedicated MCP server, so we can see whether we already cover it. The output is structured, not a paragraph of text.

In our example, an actor used an existing local administrator account to authenticate to an internet-exposed FortiGate appliance, then enabled the on-demand packet sniffer on a domain controller to capture LDAP traffic. The agent produces a detection opportunity describing the behaviour, the relevant MITRE techniques, the data sources involved, the detection logic, expected false positives and tuning notes, and prerequisites.

That opportunity then becomes a detection recommendation, and finally a draft detection in our common model: metadata plus the platform-specific selection logic that filters for Fortinet FortiGate system events whose message indicates a packet sniffer command targeting LDAP/LDAPS ports. The analyst reviews the draft, checks the logic and decides whether it is ready to merge.

Skills: the building block that makes this reliable

A general-purpose model brings knowledge, broad language ability and reasoning across many topics. What it lacks is our domain knowledge: how an Argus detection is structured, which event properties matter, how our taxonomy works, what a good rule looks like.

Skills (originally developed by Anthropic, but now an open specification) are how we hand the model that domain knowledge. A skill is a reusable workflow for a recurring task rather than a one-off prompt. It combines instruction, the actions the model is allowed to take, and the validation and guardrails that keep it in check. Skills are designed to be reliable and testable, they can be scoped to a specific repository for domain-specific work, and they can be composed so one skill can delegate to another to tackle a larger task.

In our setup, implementation of a detection is handled by an orchestrator skill that delegates to specialist companion skills: one for detection patterns and templates, one for placing the detection correctly in our taxonomy, one for event selection, one for scaffolding the rule, and one for verification and linting, plus a separate skill for generating tests. Guardrails apply throughout. The model is following a workflow we wrote, with checks at each step.

See our skill example here:

What we have learned from using AI models in our detection engineering

Automation pays off where the work is repetitive. Reading reports, placing detections in a taxonomy, and translating logic between query languages are necessary steps, but not the best use of an analyst's expertise. Handing them to an agent shortens the detection engineering cycle.

Much of that speed comes from reuse. Detection engineering is full of recurring patterns, and when they are captured in skills the agent starts from what we already know works rather than a blank page. That is what shortens the time to a credible first draft and keeps the output consistent.

But everything the agent produces is a draft. A generated detection can misread a report, over-generalise a behaviour, or look plausible while flooding analysts with false positives. Treating these as drafts requiring validation lets us move fast without shipping mistakes.

Validation is the decisive step. Before anything is merged and deployed, an analyst confirms the detection makes sense in context, that its logic is correct, and that the operational risk is understood. This is where human judgement matters most, and the part we are careful not to automate away.

Equally important is giving models a structured view of what we already detect. By exposing our detection inventory through an MCP server, an agent can answer coverage questions directly (what we cover for a vendor, where the gaps are, how a new opportunity maps to existing rules) in seconds rather than hours of manual collation.

Taken together, these lessons point to one important principle: this approach augments expert judgement rather than replacing it. The agent absorbs the repetitive work and surfaces options quickly, but decisions with real operational consequence stay with the people who understand the systems, threats and business context. The analyst still owns the outcome; what changes is how much of their time is freed for the work that genuinely needs a human.