Crate planarian_backend_netwatch

Expand description

netwatch (SPEC §12d “Which units”; owner decision, Nov 2): which network tools are protecting traffic, from changes the kernel attributes to the unit that made them.

The kernel half (crates/backends/netwatch-ebpf, built by nix/netwatch-ebpf.nix) watches the netlink receive paths and reports, for every message, the sender’s cgroup and what it touched. This crate keeps each unit’s current footprint from those reports (Tracker), decides whether it is protecting traffic, and writes and reads the files the hold reads (§12d), as specified below. The fake backend scripts the same Tracker, so the fake and the real machine share one meaning.

§The file contract (the single source)

Agreed with lane-nix (the hold’s script, planarian-hold condition in nix/posture.nix), lane-daemon and the coordinator, Nov 2.

Directory /run/planarian/footprint/ (DIR). Every file is written to a temporary name and renamed into place, so the hold never reads a torn file. /run is cleared at boot, so nothing here crosses a boot.

Clock. Every timestamp <t> is CLOCK_BOOTTIME whole seconds (CLOCK): the integer part of /proc/uptime’s first field, or clock_gettime(CLOCK_BOOTTIME) truncated. Not Unix time, so a stepped wall clock cannot make a stamp fresh. A stamp is fresh when now - <t> is between -AHEAD_SECS and FRESH_SECS (−5 to 30 s).

.watch, one line:

attached <t> while every probe is attached and events flow, rewritten at least every TICK_SECS (5 s);
or failed <reason> when an attach failed or the watch was lost.

Anything else, a missing file, or a stale attached means the watch is not known to be healthy.

When the files are written: whenever a drain of the ring changed anything, and whenever a unit’s invocation state changed (below), not only on the tick; the tick (every TICK_SECS) refreshes every at, a file whose invocation has ended included (coordinator, Nov 3).

.unattributed: present while any change that steers traffic stands that netwatch could not charge to a unit, or after a lost event (ring-buffer overflow, dropped receive) it could not place. Its content is one reason per line; readers test only for presence. It is removed only when no such change stands.

<unit>: one file per network tool, named as the systemd unit without .service (an instance keeps its @: gym-tpl@a; file_name). Lines, in this order:

protecting yes, protecting no, or protecting unknown: the unit’s latest invocation has a cgroup whose InvocationID netwatch has not read yet (systemd sets it just after making the cgroup, before the unit’s first process). unknown reads as protecting. yes is a latch per invocation (reviewer and coordinator, Nov 3): an invocation found protecting at any point stays protecting yes, with the iface and endpoint lines that made it so, until a new invocation replaces it; neither ended nor its interface or routes going clear it. A tun-based VPN’s interface leaves with its last process, before its cgroup goes and with no netlink request, so what stands at ended is not what it did. Steering seen while a unit has no live invocation (none known yet, or the last one ended) is the next one’s: its cgroup was made between netwatch’s cgroups read and its ring read. One replaced before it ended passes its latch on (fail closed);
at <t>: when netwatch last confirmed it, refreshed at least with every heartbeat;
invocation <32 lowercase hex>: the unit’s latest systemd InvocationID, from its cgroup’s trusted.invocation_id (or user.invocation_id), kept after the cgroup goes. A file whose invocation is not 32 lowercase hex is not trusted;
ended: that invocation’s cgroup is gone (IN_DELETE seen, or found absent at the start of a drain: no directory at its path, or one with another inode, which on cgroupfs is the cgroup id, never reused in a boot) and the ring has been drained after. A probe runs inside the syscall, and a cgroup can be removed only once its last process has exited, so everything the invocation did is in the file. Written only for an invocation whose id this netwatch read while its cgroup stood and whose removal it then saw: after a restart of netwatch, one that went meanwhile never reads ended. If the ring lost events in the drain that ended it, the unit is unattributed;
iface <name>: one per interface the unit created that steers traffic (zero or more): what the hold passes;
endpoint <udp|tcp> <address> <port>: one per peer endpoint it needs (zero or more; IPv4 or IPv6 without brackets);
unattributed: a change charged here that the kernel could not fully attribute, or a lost event that touched it. A file with this line always says protecting yes too (coordinator, Nov 3): fail closed, and the file says what the hold reads.

On every attach (boot or restart of the watch; coordinator, Nov 2, from the reviewer’s #38 finding 3c), in this order, so a restart never downgrades a footprint and a change made while nobody watched fails closed (Tracker::attach):

Load the claims kept before: .state (netwatch’s own record, a dot-file the hold does not read), or, if it is gone, the footprint files’ iface lines. A claim whose object (interface, route, rule, table) still stands is kept, so a footprint that was protecting yes still is; a claim whose object is gone is dropped.
Snapshot what stands now through netlink dumps: links, routes and rules (IPv4 and IPv6), nftables tables, and legacy tables (/proc/net/*_tables_names).
Unclaimed steering found standing is recorded in .unattributed as pre-attach <object>: it was made while nobody watched. It clears only when its object is gone (an interface’s routes go with it). A default route over a physical interface no unit created (the machine’s uplink: no link kind) is not “an interface it created” and is not recorded; one over a virtual link (dummy, wireguard, tun, …) or over no interface is. At boot the watch starts before network-pre.target with /run fresh, so nothing is unclaimed.
Only then, protecting no (and at) for every name in /etc/planarian/export.json’s top-level networkTools with no claim. A name ending in @ is a template: it is written per instance as instances start. A file is kept across the unit’s stop, and replaced by a fresh footprint when the unit comes back.

The probes attach before the snapshot is taken, so a change racing it is seen as an event too (applying it twice changes nothing).

Interfaces. “An interface it created” is seen at the return of register_netdevice, on success only (on entry the name can still be a template, tun%d, which it resolves), which runs in the creator’s context for every kind: a tun or tap made by ioctl(TUNSETIFF) (classic OpenVPN, wireguard-go, tailscaled), which no netlink message announces, as well as netlink-made links, whose RTM_NEWLINK is a second witness. Only a virtual link can be one a tool created; virtual is read live each tick (/sys/class/net/<if> under /sys/devices/virtual/), so a tun made by ioctl or a link moved in from another netns is never taken for a physical NIC. And (coordinator, Nov 2 and 3; the spec reads “…over a virtual interface it did not create…”) a default or all-covering route the unit added over a virtual link it did not create makes it protecting too: one another unit made (OpenVPN on a persistent tun a setup unit made: both hold), one moved in from another netns, one made before the watch attached. Routes over a physical NIC stay exempt.

Protecting means the footprint includes, now: an interface the unit created that carries a default or all-covering route (a /0 or /1, in any table, whoever added the route); a policy rule that catches all traffic (no selectors, to a table other than the kernel’s local, main and default); DNS servers on its interface; the system proxy; or firewall rules that drop or redirect traffic (Tracker). Fail closed where the kernel’s messages are not read in full: any nftables batch or legacy table change (nf_setsockopt) from a unit counts as firewall rules that steer. On a kernel without legacy xtables (CONFIG_NETFILTER_XTABLES_LEGACY unset: no /proc/net/*_tables_names), nothing can show such a claim gone, so a unit that ever called nf_setsockopt stays protecting until reboot, across netwatch’s restarts and the unit’s own stop (claims outlive the unit, as a route it left behind still steers). Deliberate: it fails closed (coordinator, Nov 2). A route by itself makes nobody protecting: only through an interface its creator made. A default route over a physical NIC no unit created (NetworkManager’s DHCP route over wlan0) steers nothing, watched or found at attach; one over a virtual link, or over none, that nobody can be charged with is unattributed (fail closed).

Attribution. The unit is the first <name>.service under system.slice, past any slices (a template’s instances sit in the template’s slice). The probe reads the sender’s cgroup path from kernfs at the request (task->cgroups->dfl_cgrp->kn, up to four names toward the root, at offsets the loader takes from BTF), so a unit is known even if its cgroup is gone before netwatch reads the event. The path is used only when whole (the walk reached the root, no name cut short); else the cgroup id is looked up. systemd makes a unit’s cgroup before its first process starts, so netwatch records ids as cgroups are made (inotify on system.slice and the slices under it, waking the loop at once) and keeps them after the cgroup goes: a oneshot that changes something and exits at once is still charged with it. Keeping old ids is safe because cgroup ids are 64-bit and not reused within a boot. An id is forgotten only once its cgroup has been gone for two walks (a tick apart, so none of its events is still unread) and its unit claims nothing in the footprint, so the map stays bounded on a long-lived host. A cgroup made and gone before netwatch saw it at all leaves the change unattributed (.unattributed: fail closed).

Namespaces (reviewer, #42). How traffic leaves the machine is the host netns’s business, so only requests that act in netwatch’s own netns (/proc/self/ns/net; the unit runs in the host’s) count. Each probe reports the netns the request acts on: the kernel’s own sock_net(skb-> sk) for netlink, the socket’s for setsockopt (the netns whose tables it changes). That is the target, not the sender’s netns (nsproxy): a process can open a socket in one netns and then setns to another, and what changes is the socket’s. The tracker’s keys (names, and indices resolved through the host’s /sys/class/net) are therefore implicitly the host’s; a debug assertion in parse::take keeps it so. Rust eBPF has no CO-RE, so the loader reads where sk_buff.sk, sock.__sk_common.skc_net.net and net.ns.inum are from the running kernel’s BTF and sets them in the object before load (an object without those globals is unusable). The BTF reader refuses a kind of type it does not know (kind 0, or past ENUM64): it cannot skip data it cannot size without misreading every type after it. So a future kernel that adds a kind makes netwatch fail closed (failed BTF: unknown kind N, the hold holds), and the reader in btf.rs needs updating for it (coordinator, Nov 2). A container’s or a PrivateNetwork= service’s own interfaces, routes and firewall are ignored, so a container’s wg0 going away leaves the host VPN’s wg0 claim standing. Fail closed: a netns the probe could not read, or a link message elsewhere that may reach into the host (moved by IFLA_NET_NS_PID/_FD, or aimed by IFLA_TARGET_NETNSID), is a loss nothing can place (.unattributed, until the next attach).

Starting (coordinator, Nov 2). Whatever happens, .watch is written before systemd hears READY=1, and systemd hears it once, so boot never waits on the start timeout and the hold already reads the watch as attached or not.

An unusable object or input (the object missing, not ELF, a program absent or of the wrong type, a map missing; the export unreadable or without networkTools): write failed <reason>, send READY=1, exit
1. No retry can fix it.
The kernel refuses a load or an attach, or the snapshot fails: write failed <reason>, send READY=1, stay up and try again every loader::RETRY_SECS (30 s), rewriting the reason each time. The first attach that succeeds reconciles (above) and writes attached.

Probes: rtnetlink_rcv_msg, nf_setsockopt, genl_rcv_msg and register_netdevice are built in, so fentry, or fexit for register_netdevice (they need vmlinux’s BTF); nfnetlink_rcv_msg and nfnetlink_rcv_batch are in the nfnetlink module (=m on NixOS kernels), so kprobes by symbol, which need the module loaded: the initrd loads it (boot.kernelModules), since netwatch starts early in sysinit.target, before every network tool and ordered after nothing (lane-nix, Nov 2). If it is not loaded yet, the attach is a kernel refusal and is retried, so the watch recovers once it is (loader::PROBES).

Not yet observed (docs/bypasses.md, temporary): DNS servers set through resolved (D-Bus) or /etc/resolv.conf, the system proxy, and WireGuard peer endpoints (generic netlink is not read).

The hold’s order. When a network tool fails, it is held unless all of these pass: (1) no unexpired release; (a) the unit’s file says invocation equal to the unit’s InvocationID (else it describes another run) and ended (else what the run did may be still to come), waited for by polling, at most about 3 s; (2) .watch is attached and fresh; (3) no .unattributed; (4) the unit’s file exists; (5) its at is fresh; (6) it says protecting no, with no unattributed line (coordinator, Nov 3: a VPN that steered and died before the next write read as the last run’s protecting no). A unit whose cgroup never empties (a process left behind with KillMode=none) never reads ended, so it is held: fail closed. FootprintView::protecting_now is (2) to (6); hold_may_release is (a) and (2) to (6), the reference the property tests check; the release is the hold’s own.

Modules§

btf: Just enough of the kernel’s BTF (/sys/kernel/btf/vmlinux) to find where a struct member lives, so the probes can read the netns of a request on any kernel: Rust eBPF has no CO-RE, so the loader resolves the offsets here and hands them to the object as globals (crate docs, “Namespaces”). Trusted code in the loader, so: read-only, every read bounds-checked, recursion bounded, and an answer only when each member found lies inside its struct; anything else is an error (the loader writes failed), never a panic.
contract 🔒: The files the hold reads (see the crate docs, the single source).
ebpf: The event the eBPF program reports (crates/backends/netwatch-ebpf, Event): #[repr(C)] { cgroup: u64, tgid: u32, hook: u32, len: u32, captured: u32, netns: u32, cgflags: u32, cgpath: [[u8; NAME]; LEVELS], data: [u8; CAPTURE] }, native-endian. Parsed by hand: no unsafe, and a short buffer is an error rather than a misread.
host: What the watch needs from the running machine, as pure functions where it can be (tested), reading /etc, /sys and /proc only in read_*.
model 🔒: What the kernel attributed to whom, and what it means (§12d).
parse: From a captured netlink message to the changes it makes (§12d). Read-only requests (RTM_GET*, dumps) change nothing. A message this module cannot fully read but that may change how traffic leaves is not guessed at: it is reported as Parsed::Unreadable, which marks the sender’s footprint unattributed (fail closed).

Structs§

Change: One attributed change: object names what was touched (an interface, a route prefix, a rule, an nft table or chain), so a later removal of the same object undoes it. steers is whether it currently steers traffic (§12d: a default or all-covering route, a catch-all rule, DNS on the unit’s interface, the system proxy, a dropping or redirecting firewall rule); an interface by itself steers nothing.
Claims: The tracker’s claims, as .state keeps them across a restart of the watch (a dot-file the hold does not read).
Endpoint
Footprint: A unit’s footprint as the files carry it.
FootprintView: Everything the hold needs, read back from the files (or scripted by the fake): the watch, why .unattributed stands (empty: it is absent), and the footprint of each network tool, keyed by its file name (crate::file_name).
Invocation: What netwatch knows of a unit’s latest invocation, from its cgroup.
Tracker: Each unit’s current footprint, kept from the kernel’s reports (or the fake’s script). apply takes one attributed change; lost records an event the ring buffer dropped; view renders what the files carry.

Enums§

ChangeKind: The kind of object a netlink message touched.
Owner: Who made a change: a unit (its cgroup), or nobody the kernel could name.
Proto
WatchState: The watch’s state, as .watch carries it.

Constants§

AHEAD_SECS: A stamp further ahead than this is not trusted either (the hold’s script).
CLOCK: The clock every <t> is on: seconds since boot, suspend included (the first field of /proc/uptime), so a wall-clock step cannot make a stale file look fresh.
DIR: Where the footprints live.
FRESH_SECS: A file confirmed longer ago than this is stale.
TICK_SECS: How often netwatch rewrites every file.

Functions§

file_name: The file name of a unit’s footprint: systemd’s %N (the unit name with any @instance, without .service). Network tools are system services.
hold_may_release: The hold’s condition on a failure, as lane-nix’s script implements it (crate docs, “The hold’s order”): it may release only when the unit’s footprint describes the invocation that failed (invocation, systemd’s InvocationID), that invocation has ended (its cgroup is gone and the ring drained after, so nothing it did is still to come), and the footprint reads protecting no at now, with nothing unattributed. Anything else holds. The reference the property tests check.
read: Read what dir holds. Fail closed: a footprint that does not parse is left out (so it reads as missing, which counts as protecting), and an unreadable or unknown .watch is WatchState::Unknown.
read_claims: The claims kept before a restart: .state if it reads, else the footprint files’ iface lines (Tracker::claims_from_files).
write: Write view to dir: .watch, .unattributed (present or removed), and one file per unit; a unit’s file that is no longer in the view is removed, so a reader never trusts a footprint netwatch no longer keeps.
write_state: Keep the tracker’s claims in dir/.state, atomically.