Backdated, written in November 2026 from git and CI records. Part 3 was the restart.
The installer test
I wanted a nightly test that installs Patronus into a real VM, with real ZFS, and reboots into it. On October 4 it went in as a workflow. It ran on a Mac build host through UTM, so an x86 guest was emulated on an ARM machine.
It never worked. From the Forgejo record, October 1 to 15, nightly-realistic-installer-test.yml: 57 failures, 6 cancelled, 0 successes. The overall repo had 249 runs those two weeks: 87 success, 116 failure, 36 cancelled, 10 skipped.
The log, roughly:
Oct 4 and 5: file handoff between the VM and the host. First assumed instant, then polled for 60 seconds, then 180.
Oct 5: a serial console capture finally showed where boot stuck. GRUB 2.12 showed its menu, printed “Booting `Linux lts’” and then nothing. I switched the CD-ROM from USB to IDE. The commit message calls it the most likely cause, not a proven one.
Oct 5: “retry the live-boot stage – confirmed non-deterministic”.
Oct 10: a failing screenshot step turned out to be macOS denying Screen Recording permission (TCC), not a product bug.
Oct 13: boot budget pushed to 2700 seconds and the job ceiling to 120 minutes.
Oct 14: “pause nightly installer test schedule (TCC emulation can’t finish in time)”, where TCC is a typo for TCG, QEMU’s software emulation.
Part 5 is what replaced it.
Bugs that mattered more
Meanwhile, in the same stretch:
The firewall validated nothing before building nft scripts, so a rule field could inject ruleset text (Oct 12). Fields are now validated first.
The web UI’s NAT rules changed an in-memory list and never reached nftables (Oct 12).
A literal interface name “any” or “all” was spliced into nft syntax (Oct 13).
Captive portal MAC addresses weren’t validated before use in nft commands (Oct 11).
Dashboard threat and alert counts were constants (42, 38 and 3) whatever the real state. A commit on the 14th derived them from real data and listed what was still a placeholder: model accuracy, health score and packet rate, “no real ML classifier or packet-rate counter exists yet to back them honestly”.
On Oct 13, the container image couldn’t run nft as a non-root user until it got CAP_NET_ADMIN and CAP_NET_RAW on the binary.
Design decisions
The commit that replaced the UTM setup says x86 under TCG never booted there.
State on October 15
CI on main in the fortnight: 66 successes and 43 failures.
Detection logic that lives only inside a SIEM’s web console has the same problems as server config edited by hand over SSH. Nobody reviews the change, nobody knows why a condition was added, and when a rule silently stops matching after a log format change, nobody notices for months. Treating detections as code fixes most of that, and Sigma is the most practical way I know to do it without tying yourself to one vendor.
In part 1 we covered which events to collect. This part is about turning them into rules you can review, version and, most importantly, test.
What Sigma actually is
Sigma is an open, YAML-based format for describing log detections, maintained by the SigmaHQ project along with a large public rule repository. A rule describes the log source in abstract terms (process creation on Windows, say) and the matching logic as field conditions. A converter then translates it into your backend’s query language: Splunk SPL, Elasticsearch, Microsoft Sentinel’s KQL and many others through pySigma backends.
The abstraction is the point. The same rule can run against Sysmon data in one environment and against an EDR’s process events in another, as long as a processing pipeline maps the field names.
A rule worth having
Here’s a rule for a pattern that turns up in a lot of web server compromises: the IIS worker process starting a command shell. Web applications rarely have a legitimate reason to do that, and a web shell almost always does.
title: Command shell spawned by IIS worker process
id: 6b1f3c52-8f0e-4b8a-9d3e-2c7a5e1f9b40
status: experimental
description: >
Detects cmd.exe or PowerShell started by w3wp.exe, which usually
indicates a web shell or command injection in a hosted application.
references:
- https://attack.mitre.org/techniques/T1505/003/
author: canutethegreat
date: 2026-10-14
tags:
- attack.persistence
- attack.t1505.003
logsource:
category: process_creation
product: windows
detection:
selection:
ParentImage|endswith: '\w3wp.exe'
Image|endswith:
- '\cmd.exe'
- '\powershell.exe'
- '\pwsh.exe'
filter_health_check:
CommandLine|contains: '\inetpub\scripts\healthcheck.cmd'
condition: selection and not filter_health_check
falsepositives:
- Application pools that deliberately shell out (document them in a filter)
level: high
Notice the filter. Real environments always have the one legacy app that shells out for a reason, and the right place to record that exception is in the rule, in version control, with a commit message saying who asked for it. Part 3 goes deeper on how to keep exceptions from eating your coverage.
Converting and linting in CI
The sigma-cli tool wraps pySigma. A minimal CI step that validates and converts every rule looks like this:
pip install sigma-cli
sigma plugin install splunk
sigma plugin install sysmon
# Fail the build on structural problems
sigma check rules/
# Produce backend queries as a build artifact
sigma convert -t splunk -p sysmon rules/ -o build/splunk.conf
sigma check catches malformed YAML, missing fields and duplicate IDs. Conversion failures matter too, because a modifier or field that your backend can’t express means the rule never runs where you think it does. I’ve seen rule sets where a quarter of the rules failed conversion and the failures were simply ignored.
Testing against real logs
Linting proves a rule is well formed. It says nothing about whether the rule fires on the activity you care about, or whether it fires on a Tuesday-morning patch run. For that you need recorded events.
Keep two fixture directories next to each rule set. The positive fixtures contain events that must match, ideally captured from a lab machine where you reproduced the behaviour, or taken from public datasets such as the OTRF Security-Datasets project or the EVTX-ATTACK-SAMPLES collection. The negative fixtures contain normal activity from your own estate that must not match: a day of process events from a busy web server is a good start.
To evaluate Sigma directly against EVTX files without a SIEM, WithSecure’s open source Chainsaw works well:
Then a short script turns those results into pass or fail. The exact JSON layout varies between Chainsaw versions, so treat this as a sketch you adapt to the output you get:
import json, sys
def hits(path):
with open(path) as f:
return {d["name"] for doc in json.load(f) for d in doc.get("detections", [])}
expected = {"Command shell spawned by IIS worker process"}
pos, neg = hits("out/positive.json"), hits("out/negative.json")
missing = expected - pos
noisy = expected & neg
if missing or noisy:
print("did not fire:", sorted(missing))
print("fired on benign data:", sorted(noisy))
sys.exit(1)
Now a rule change that breaks detection fails the build, and so does a change that would page someone on ordinary traffic. Most teams skip that second check, and it’s the one that stops analysts from muting your rules.
Keep production honest too
Fixtures drift away from reality. Log schemas change when an agent is upgraded, a field gets renamed, or Sysmon’s config stops collecting the event you depended on. Fixture tests won’t catch any of that, because the fixtures still have the old shape.
The fix is a scheduled end-to-end check. On a dedicated test host, run a harmless action that a specific rule should catch (for the rule above, a test IIS site whose handler starts cmd.exe /c echo detection-test), then query the SIEM a few minutes later and confirm the alert exists. If it doesn’t, that is itself an alert, routed to whoever owns the pipeline. Frameworks like Atomic Red Team provide catalogued, benign test procedures mapped to ATT&CK techniques, which saves you writing your own for common cases.
Review like you mean it
A pull request for a detection should answer the same questions as any code review, plus a few of its own. What behaviour does it catch, and what’s the evidence? Which fixtures prove it? What’s the expected alert volume, measured by running the converted query over the last 30 days of real data? Who triages it, and what does the runbook say they should do first?
That last question gets skipped constantly. A rule without a response plan just moves the problem from “we didn’t see it” to “we saw it and didn’t know what to do”.
Once you have dozens of tested rules, the next problem shows up fast: too many alerts. That’s the subject of part 3.
This series started with storing passwords, moved through MFA, then OAuth and OpenID Connect. All of that work produces one thing at the end: a session. From then on, every request is authenticated by whatever token the browser sends, and the password and the security key are no longer involved. If the session is weak, everything before it was ceremony.
What a session token needs to be
Whatever form it takes, a session token has to be hard to guess, tied to one login, and revocable. For a classic server side session that means a random identifier with at least 128 bits from a CSPRNG, stored in a server side table or cache, with all the state kept on the server. OWASP’s floor is 64 bits of entropy. I don’t see a reason to stop there when 128 costs nothing.
Two lifecycle rules catch a lot of bugs. Issue a new session identifier on every privilege change, especially login, or you’re open to session fixation, where an attacker plants a known session ID in the victim’s browser before they log in and then shares the authenticated session. And enforce both an idle timeout and an absolute lifetime, so a session that’s been stolen doesn’t stay useful for months just because something keeps it busy.
Cookie attributes that matter
The browser is where sessions get stolen, and cookie attributes are how you tell the browser to protect them. A good default looks like this:
Secure keeps the cookie off plaintext HTTP. HttpOnly stops JavaScript from reading it, which won’t prevent an XSS payload from making requests as the user but does stop it from shipping the token off to be used from elsewhere, hours later. The __Host- prefix is underused. The browser only accepts a cookie with that name if it has Secure, Path=/ and no Domain attribute, which means a sibling subdomain can’t overwrite or shadow it. That closes off a whole family of cookie tossing attacks for free.
What SameSite does and doesn’t do
SameSite controls whether the browser attaches a cookie to requests that start on another site. Strict never sends it cross site, including when a user follows a link to your app from an email, which makes them look logged out. Lax sends it on top level GET navigations but not on cross site POSTs, iframes or subresource requests. None always sends it and requires Secure. Chrome began treating cookies with no SameSite attribute as Lax in 2020, and other browsers have moved in similar directions, though not identically.
That made classic CSRF much harder, but I wouldn’t remove CSRF tokens because of it. Three reasons. Lax still sends cookies on top level GETs, so any state changing GET endpoint is still exposed. Chrome’s default-Lax behaviour includes a two minute window in which a freshly set cookie without an explicit attribute is still sent on cross site top level POSTs, to avoid breaking some login flows. And “site” means registrable domain, not origin. evil.example.com and app.example.com are the same site, so a subdomain takeover or an XSS on a marketing subdomain bypasses SameSite completely. Keep the anti-CSRF token or check the Origin header on state changing requests, and treat SameSite as defence in depth.
Revocation is the real requirement
Logging out has to end the session on the server. A logout that only deletes the cookie leaves the token valid for whoever copied it. Users also need “sign out of all devices”, support staff need to kill sessions for a compromised account, and a password change or MFA reset should invalidate every other session. With server side sessions all of that is a delete query.
This is where JWTs get awkward. A signed, self contained token is valid until its exp unless the server checks something on every request. The usual answer is short lived access tokens (5 to 15 minutes) plus a refresh token that the server does track. RFC 9700 describes refresh token rotation: each refresh returns a new refresh token and invalidates the old one, and if an old one is ever presented again you assume theft and revoke the whole family.
def refresh(presented_token):
rec = db.refresh_tokens.find(hash(presented_token))
if rec is None:
raise Unauthorized()
if rec.used:
# a rotated token came back: someone has a copy
db.refresh_tokens.revoke_family(rec.family_id)
alert("refresh token reuse", rec.user_id)
raise Unauthorized()
rec.used = True
new_rt = issue_refresh_token(rec.user_id, family_id=rec.family_id)
return issue_access_token(rec.user_id, ttl=600), new_rt
Notice that this is a database lookup on every refresh. Once you need revocation, and you will, you have server side state again. That’s fine. Just be honest that a JWT session isn’t really stateless.
Where JWTs go wrong
Most JWT vulnerabilities come from verification code trusting the token’s own header. The alg field says how the token claims to be signed, and early libraries used it to choose the verification method. That allowed alg: none tokens with no signature at all, and algorithm confusion, where a token signed with HS256 using the server’s RSA public key as the HMAC secret passed verification. CVE-2015-9235 in Node’s jsonwebtoken is the well known instance of the latter. The fix is to pin the expected algorithms on the server and never read them from the token:
Library bugs happen even when you do this. CVE-2022-21449, nicknamed Psychic Signatures, made Java 15 through 18 accept ECDSA signatures with both values set to zero as valid for any message, which broke ES256 JWT verification on those runtimes until the April 2022 patch. Keep the JWT library and the runtime patched, and test that a garbage signature is rejected.
RFC 8725, the JWT Best Current Practices, covers the rest: validate iss and aud, use explicit typing so one kind of token can’t be replayed as another, and don’t put secrets in the payload, since a signed JWT is encoded, not encrypted.
Storage is the last trap. Single page apps often put tokens in localStorage, where any XSS can read them and send them elsewhere. A backend-for-frontend that holds the tokens server side and gives the browser an HttpOnly, __Host- cookie avoids that, and it fits the OAuth guidance from the previous part of this series.
What I’d use
For a typical web app with its own backend, I’d use opaque server side sessions in a hardened cookie. They’re simpler, revocation is trivial, and nothing about them is clever. I reach for JWTs when a token really has to be verified by services that can’t share a session store, and then I keep them short lived, pin the algorithm, and pair them with rotating refresh tokens. Choosing JWTs because they sound modern is how teams end up rebuilding server side sessions badly, one denylist at a time.