CanuteTheGreat's BLOG

Tag: rust

  • Patronus dev log 3: back in September, with 43 red CI runs and no green ones

    Patronus dev log 3: back in September, with 43 red CI runs and no green ones

    Part 3 in the Patronus dev log series.

    Backdated, written in November 2026 from git and CI records. Part 2 ended with nine months of silence.

    I picked Patronus back up the same day as Horcrux, September 24, 2026. The first job was the same: make CI run. Patronus got a Rust CI workflow that day (fmt, clippy, tests, an eBPF check, a full-workspace build). It failed. Over September 24 to 30 the Forgejo record shows 57 runs: 43 failures, 7 cancelled, 7 skipped, 0 successes. Of those, rust-ci.yml had 42 failures.

    The first failures were the dull kind: a missing libmnl-dev, a missing libelf, no linker on the runner image, and a real bug. An eBPF map-create call took its name as i8 and needed u8, which was blocking the detection and CNI crates from building at all.

    The Dockerfile again

    Same mistake as in Horcrux, found the same day. The Dockerfile had become a generic FROM rust build on a Debian slim image. The repo notes now say so, in short:

    A generic FROM rust -> cargo build -> debian slim Dockerfile is wrong.
It crept in once and was fixed on 2026-09-24.

    Patronus builds through real Portage against its own overlay. The ebuild kept breaking, in familiar ways: virtual/rust doesn’t exist (same bug as Horcrux, the commit says so), acct-user and acct-group variable names were wrong, a USE flag was called carp in the code and ucarp in IUSE, the meta USE flags defaulted on with no sub-backend, and the ebuild forwarded cargo features that don’t exist (aliases, tc, prometheus). Building Rust from source inside the Docker build took 2 hours 45 minutes, so I switched to dev-lang/rust-bin. And the Gentoo Docker build got capped at 60 minutes and made non-blocking while I sorted it out.

    On the 29th the installer picked up root-on-ZFS, preferred when the ZFS tools are present.

    Design decisions

    Gentoo and USE flags: Gentoo is the core design point. My reason is freedom to choose. USE flags come first because they let you pick which features (web UI, eBPF/XDP, SD-WAN, detection, cloud integrations) get compiled in, and the repo notes call that the main advantage over pfSense and OPNsense. Hardware specific compiles are the second reason. A container build must pass USE flags through to emerge rather than compile a fixed set.

    rust-bin over source Rust: the from-source build took 2h45m (the commit’s number).

    Architecture: the repo notes have one rule: one SQLite store as the single source of truth, one “Applier” per subsystem, one apply step at boot, and no extra in-memory list, fixture or config file that only one process knows about.

    State on September 30

    57 runs, none green, and about 60 commits. The Gentoo build was non-blocking at the end of the week and still no run was green.

    Other parts: Part 1, Part 2, Part 4, Part 5.

  • Horcrux dev log 2: my own Forgejo, a Gentoo ISO builder, and an ISO that never quite booted

    Horcrux dev log 2: my own Forgejo, a Gentoo ISO builder, and an ISO that never quite booted

    Part 2 in the Horcrux dev log series.

    Backdated, written in November 2026 from the repo history. Part 1 was the first week in October.

    Nothing happened in the repo for two months after the October burst. Then on December 12 there were three commits: “Major platform enhancements and infrastructure cleanup”, “storage, ha, and clustering improvements” and “added forgejo workflows”. The last one matters because I’d decided to host the code on my own Forgejo and run CI on my own runners.

    It didn’t work. Between December 12 and 31 Forgejo recorded nine workflow runs for Horcrux, all of them skipped. The workflows were copied from GitHub Actions and asked for ubuntu-latest, which none of my runners had. A job with no matching runner is skipped, not failed, so nothing showed up red. I’ll come back to that in part 3.

    The big commit

    On the 19th: “Refactor Web UI into modular pages and add Gentoo build system”, 129 files, 47,358 insertions. The UI had been a few enormous page files. Now it’s modules for audit, auth, backup, dashboard, HA, Kubernetes, metrics, monitoring, storage and system. The same commit added a Gentoo/Catalyst ISO build system.

    On the 29th there was a batch of NAS features (SMB and NFS among them) and a CLI output cleanup.

    The ISO commits

    The last commits of the year are all about booting. Five of them:

    c01076e4 Fix ISO build to include kernel and bootloader
  "The ISO builder was creating incomplete ISOs missing critical boot components."
b5531304 Improve ISO build with initramfs library support and robustness
385e3e3b Add serial console support to ISO boot configuration
  "Enables boot output visibility when testing in QEMU"
(Dec 31) Fix ISO boot with kernel modules and proper init system
(Dec 31) Fix library copying for horcrux binaries in ISO

    So the first version of the builder made ISOs with no kernel and no bootloader. Each commit fixes whatever the last one exposed: kernel and GRUB, then initramfs libraries, then a serial console so I could watch QEMU, then modules and an init system, then the shared libraries for the Horcrux binaries. I can’t find a commit where it reaches a login prompt.

    State

    The codebase got bigger. As far as I can tell, the ISO script still didn’t produce a bootable image, and none of the nine CI runs executed. After December 31 the repo is silent until September 24, 2026.

    Design decisions

    The installer is a Gentoo ISO because Horcrux is Gentoo first. For me that comes down to freedom to choose: USE flags are the main reason, and compiling for specific hardware is the second. I’ve used Gentoo since 2006. The image is built with Gentoo’s Catalyst.

    The other change in December was hosting. The code and CI moved from GitHub to my own Forgejo, which is now the primary. GitHub likes to block things, so I’d rather not depend on it. The repos are still mirrored there. The copied workflows asked for a runner label that mine didn’t have, which is why every run was skipped.

    Other parts: Part 1. The series continues with the latest Horcrux dev log post.

  • Patronus dev log 2: Kubernetes, a React UI, and a Rust workspace that wouldn’t build

    Patronus dev log 2: Kubernetes, a React UI, and a Rust workspace that wouldn’t build

    Part 2 in the Patronus dev log series.

    Backdated, written in November 2026 from the git log and CI records. Part 1 covered October 8 to 11.

    October 12 to 14

    The run continues. A Kubernetes operator with a Helm chart and CRD generation (the log calls it Sprint 36 and 37), “Integration Testing & CI/CD Pipeline” as Sprint 39 (October 12), a project status document as Sprint 40, then a topology visualization, LZ4 compression for WAN links, a React dashboard hooked up to the Rust backend, and OpenTelemetry tracing on the 13th. One commit on the 14th, then nothing until December.

    In total, 98 commits in October 2025, and all 15 CI runs that month were skipped.

    December

    December has commits on four days. On the 12th: “Remove GitHub CI/CD configuration”. I moved off GitHub Actions to my own Forgejo, the same week I did it for Horcrux (part 2 of that series). Same reason as there: GitHub likes to block things, and the repo is still mirrored to GitHub.

    December 16: “fix: Complete Rust workspace compilation and dependency resolution”, 54 files. Its message lists missing workspace dependencies (uuid, sysinfo, reqwest, chrono), API breaks from base64, axum and sysinfo upgrades, netlink imports, missing nix features, an async recursion that needed Box::pin, “syntax errors in derive macros and missing brackets” and missing feature flags. The workspace claimed 19 crates and 45,000 lines on day one, and two months later it hadn’t compiled cleanly. That contradicts the “Complete Implementation” commit from part 1.

    December 17, “Comprehensive code review fixes and feature completions”: unwrap() calls replaced in backup and validation code, NaN-safe float comparisons in the detection modules, and “wire web handlers to actual sysinfo metrics (CPU, memory, disk, network)”. So until then the dashboard metrics weren’t coming from the machine. Also: alert conditions now checked against system metrics, CIDR matching for NetworkPolicy completed, and a hardcoded address removed from HA election.

    December 19 added flow tracking, an installer crate and eBPF/XDP enhancements. The repo then goes quiet until September 24, 2026, nine months.

    What I think of this period

    In those weeks the repo’s documents described a finished product, while the code still needed a compile fix, and then a review pass before the dashboard showed real numbers. The 15 skipped runs in October and 11 more in December mean CI never tested any of it.

    The commits from this period say what was done and never why, so I have no design decisions to report.

    Other parts: Part 1.