Documentation
Overview
Kanea runs containers on one machine and gives that machine the things a platform normally needs a cluster for: service discovery, load balancing, network policy, TLS, autoscaling, GitOps, backups. It is one static binary, and the concepts it asks you to learn number about six.
If you have not installed it yet, the install section on the front page is four steps. This page assumes you have a node running and want to know what you are looking at.
Concepts
Kanea borrows from Nomad and Kubernetes and keeps the smallest set of concepts that covers the work. If you know either system, the middle columns tell you what maps to what.
| Kanea | Nomad | Kubernetes | What it is |
|---|---|---|---|
| Node | Client + server agent | Node + control plane | One machine running kanea agent. In v1 there is exactly one. |
| Project | Namespace | Namespace | A named group of services, and the isolation boundary: network policy, secrets, containerd namespace and DNS all key off it. |
| Service | Job + group | Deployment + Service | A declarative long-running workload with count replicas. |
| Task | Task | Container | The container inside a service. Exactly one per service in v1 — sidecars are a v1.1 concept. |
| Alloc | Allocation | Pod | One running instance of a service. count = 3 means three allocs. |
| Storage | CSI volume | PV / PVC | A named volume backend — local, host, nfs, smb or s3 — that services mount by name. |
| Pipeline | — | Tekton / CI job | A build run producing an image, from a build block or a git push. |
The one that surprises people is project. It is not a label — it is a boundary
that four subsystems enforce independently. A service in project shop
cannot reach a service in analytics unless somebody wrote that down, it
cannot read analytics' secrets even by naming them, its containers live in
a separate containerd namespace, and its DNS names sit under a separate suffix.
Naming
Project and service names must be DNS-1123 labels: lowercase alphanumeric and
-, starting and ending alphanumeric, at most 63 characters. This is
checked when the spec is parsed, not when something fails later.
The reason is that names compose into DNS without an escaping step. A service
web in project shop is web.shop.kanea internally
and web.shop.<base_domain> publicly. A name that needed encoding to
become a hostname would be a name that means two different things in two places.
Every project and service takes a description — free text, up to 512
characters, shown in the dashboard. That is where the human-readable detail belongs;
the name stays a label.
Lifecycle
Nothing in Kanea is applied directly to the runtime. A spec becomes desired state in the Store, and a reconciler converges the world toward it — continuously, not once.
Job spec (HCL) ──parse/validate──▶ Desired state (Store)
│
Reconciler loop
│
┌───────────────┼────────────────┐
▼ ▼ ▼
containerd eBPF datapath Edge proxy
(tasks/images) (policy/LB) (routes/TLS)
│ │ │
└───────────────┴────────────────┘
▼
Actual state / events / metrics
Consequences worth knowing up front:
- Drift is repaired. Delete a container by hand and it comes back. The reconciler is comparing, not remembering what it did.
- Restart policies are
always(default),on-failurewith backoff, andnever. - Updates are rolling by default, health-gated, bounded by
max_parallel. - Dependencies start first. A
depends_onedge — or any${service.…}reference, which creates one implicitly — means the dependent does not start until its dependency is healthy. If the dependency degrades later, dependents keep running: no cascading stops, just events. - Storms are capped. Per-service restart rate limits plus a node-wide circuit breaker that pauses rollouts and scale actions when failure rates spike. A trip emits an event and a notification.
What counts as a deploy
A deploy is a spec-hash mismatch. The reconciler hashes the parts of a service
that are baked into a container at creation time; if an alloc's recorded hash differs
from the current one, that alloc is replaced under the update policy. If it matches,
nothing happens, however many times you run kanea run.
This is why kanea restart is not a separate path into the runtime: it
bumps a generation counter that participates in the hash, and the ordinary rolling
update does the rest.
max_parallel bounds allocs that are down, not replacements in
flight. Anything already unavailable spends the budget first — so a deploy that
starts going wrong stops, instead of walking through every replica.
min_healthy applies only to allocs the current deploy has already
replaced, and health means a probe said so. A service with no
health_check block never reports healthy for any alloc — which is fine,
and is why the update logic asks whether a check is configured before it asks whether
it passed.
Secrets
Secrets are never written in a spec. They are referenced as
secret:<path> and resolved when an alloc starts:
env = {
DATABASE_URL = "secret:shop/database-url"
}
Two rules make that reference safe rather than merely tidy:
- References are project-scoped at validation time. A service in
shopmay namesecret:shop/…orsecret:shared/…and nothing else. A spec reaching for another project's secret fails to parse — it does not fail at runtime, where the failure would be a log line nobody reads. - The default injection is a tmpfs file, at
/run/kanea/secrets/<alloc>/<name>. Environment variables work and are documented as the weaker option, because they are visible in/proc/<pid>/environ, in runtime inspect APIs, and to every child process.
The API and the MCP server are write-only for secrets. There is no get —
not for an operator, not for an agent, at any permission tier.
Your first service
The smallest useful thing needs no file at all:
kanea run --image nginx:1.27-alpine --name web --project demo kanea ps -p demo kanea logs -f demo/web
When you want it written down, the same deployment as a spec is three lines plus a
wrapper — and from there every block in the
job spec reference is additive. Run
kanea plan first; it prints the create/change/destroy diff, and it is
where every validation rule fires.
Architecture
One binary produces two long-running processes and a CLI. The datapath that networks
your services is not one of them — it is a set of eBPF programs compiled into the binary
and loaded into the kernel, not a daemon Kanea drives. Everything else on the node
— containerd, buildkitd — is software Kanea drives rather than
software it contains, but it is no longer software you have to find: kanea
init installs each of them at a version pinned by SHA-256 in the binary, under
Kanea's own prefix and on Kanea's own sockets, so nothing already on the node changes.
The shape
┌──────────────── kanead (control plane) ────────────────┐
│ │
Browser ──HTTPS──▶ │ ┌────────────┐ ┌───────────┐ ┌──────────┐ ┌─────────┐ │
CLI ──────HTTPS──▶ │ │ API server │ │ Dashboard │ │Reconciler│ │Autoscale│ │
Webhooks ────────▶ │ │ REST + WS │ │ (embedded)│ │ │ │ (eBPF) │ │
│ └─────┬──────┘ └───────────┘ └────┬─────┘ └────┬────┘ │
│ └─────────────┬─────────────┴────────────┘ │
│ ┌────────┴─────────┐ │
│ │ Store (bbolt) │ │
│ └────────┬─────────┘ │
│ ┌─────────┬───────┼────────┬──────────┐ │
│ │ Runtime │Network│ GitOps │ Notifier │ │
│ │containerd│ eBPF │BuildKit│ │ │
│ └────┬────┴───┬───┴────┬───┴──────────┘ │
└────────┼────────┼────────┼───────────────┬─────────────┘
│ │ │ ▼
│ │ │ State replicator ──▶ S3
▼ ▼ ▼
┌── kanea-edge (separate process; reads a projection, never the Store) ──┐
│ L7 routing · TLS termination · middleware · request metrics │ ◀── :80/:443
└────────────────────────────────────────────────────────────────────────┘
External: containerd · buildkitd · Linux kernel (eBPF, cgroups v2, netfilter, bpffs)
Two processes, one binary
kanead is the control plane: API, dashboard, reconciler, autoscaler,
GitOps, notifications, backups. kanea-edge is the ingress proxy, and it
is a separate systemd unit running as a separate unprivileged user.
That split is the single most consequential design decision in the system, and it buys two things:
- Restarting, upgrading or crashing the control plane does not interrupt public
traffic. The edge unit deliberately has no
After=kanead.service. - The process that terminates untrusted public traffic cannot mutate the platform. It has no Store access and no write path at all.
bbolt takes a lock on the whole database file, so a second process opening it —
even read-only — would block until kanead exits. So the edge does not
open it. kanead projects what the edge needs into
/run/kanea-edge/: routes.json (0644, host → service
frontend; nothing secret, the domains are in public DNS) and
certs.json (0640, private keys). Two files, two permission sets,
so neither has to compromise for the other.
Both are written temp-then-rename(2), so a half-written file is never
observable. The projection carries the Store index it was built from, which is how a
stale snapshot can be recognised rather than merely suspected.
A missing or stale snapshot is not an outage: the edge keeps serving the last table
it loaded for as long as kanead is away, and starts with an empty table
rather than refusing to start. "The control plane is down" must never become "the
site is down".
The Store
One embedded bbolt database holds every mutation, behind a Store interface
with monotonic indexes. Buckets: projects, services, allocs, events, certs, secrets,
pipelines, audit, kv.
- Single writer. All mutations serialise. Reads are bounded and paginated, because a long read transaction blocks the writer.
- Raft-shaped on purpose. The interface and its index semantics are what a Raft FSM would need, so a clustered implementation can replace it without touching call sites.
- Metrics and logs never touch it. Time series live in a bounded in-memory ring; logs go to file pipelines with non-blocking drains. This is a hard constraint, not a performance preference — a metrics write that contends with the reconciler's writer would make observability and convergence share a failure mode.
- Migrations are explicit. The Store does not migrate itself at open. A
migration rewrites state in place, and the copy that makes a bad migration survivable
needs the database open and the migration not yet started — exactly one window,
between
OpenandMigrate.
Runtime driver — containerd
- Driven over its socket with the official Go client. One containerd namespace per
project (
kanea-<project>), which makes image and container isolation free rather than enforced. - Responsibilities: image pull (credentials from the secrets store, digest pinning supported), task lifecycle, per-alloc netns setup, cgroup metrics sampling, and stdout/stderr capture.
- Hardening is not opt-in. Every alloc starts from a baseline capability set
— the uid-switching grants PUID-style images need to chown a volume, drop to their
configured user and bind a low port, and nothing more (
CAP_NET_RAWis deliberately excluded) — plusno-new-privileges, the default seccomp profile, and its own PID and IPC namespaces.capabilities = ["none"]drops to nothing; a capability a workload genuinely needs beyond the baseline is named in the spec and drawn from a permitted set; the privilege-equivalent ones are rejected when the spec is parsed. No job spec can lift any of it on its own — there is noprivilegedfield. The one way past these defaults is a host device or socket the operator granted on the node (R17, R18), which a spec requests by name and can never define. - Disk hygiene is part of the driver. Image GC, build-cache caps across both content stores — containerd's and the rootless buildkitd user's — per-service log caps, and watermark alerts at 80% and 90%. One disk holds images, logs, state and volumes; pressure must never surprise the control plane.
Network driver — eBPF datapath
The datapath is Kanea's own: three small eBPF programs, a handful of pinned maps and
plain netlink plumbing, all loaded and written by kanead from one object
compiled into the binary. There is no network agent, no kvstore and no CNI. The loader
is the standalone github.com/cilium/ebpf library — importing
github.com/cilium/cilium would pull the Kubernetes client graph, which is
the one dependency the project does not have. The programs are compiled ahead of time
and committed, so go build needs no clang and the node needs no BTF: they
read only UAPI context types, so there is no CO-RE and no vmlinux.h.
IP is identity
kanead allocates every alloc address from the node CIDR and every service
VIP from the service CIDR, durably in the Store. Because the platform that hands out
addresses is the same one that writes the kernel's maps, the identity map — alloc
IP → {project, service} — is written by the allocator itself. There is no
identity-allocation protocol, no label race and no settle window. The numeric project
and service ids behind the maps are Store-allocated, monotonic and never reused,
which is what keeps a pinned map meaningful across a kanead restart.
Attach is deny-closed by construction
Per alloc, in order: netns → identity map write → veth created with the host side
down → policy programs attached at tc → addresses and static neighbours → link up → the
host /32 route last. The first moment a packet can reach or leave the
alloc, policy is already enforcing and identity is already resolved; a skipped step
fails closed, because an identity miss is a drop. The deny-by-default guarantee is
structural, not temporal — there is no unlabelled window to hold shut with
retries, the way the previous design held its reserved:init state closed.
Attach has no wait loop and completes in milliseconds.
Load balancing is connect-time
One cgroup connect4 program at the root cgroup, held by a pinned
bpf_link, rewrites VIP:port → backend at
connect(2) — for host processes and containers alike, which is the one
property the edge depends on: it dials a VIP with a plain dialer. There is no per-packet
NAT and no conntrack entry per flow, and an established connection never consults a
map: kanead can restart, or recreate every map, without touching live
traffic. Backends update by generation flip — the new set is written under the
next generation and one atomic map update commits it, so a concurrent
connect() sees a complete old set or a complete new one, never a torn one.
A VIP with no backends refuses at connect() rather than black-holing into a
timeout. Service ports are TCP-only in v1, refused at plan otherwise.
Policy is SYN-gated map entries
The tc program on each alloc's host-side veth admits a source carrying the host identity
(the edge's upstream dials, kanead's DNS replies and probes), a source in
the same project, or a source named by an allow_from edge (R14) — and drops
every cluster-internal source the identity map does not know. A source outside
the cluster CIDR carries no identity by construction — it is the internet answering a
connection the alloc opened, un-NATed by conntrack on the way back in — and passes
(v1.65); nothing unsolicited arrives that way, because the pod CIDR is unroutable from
off-node and published ports terminate at the edge. The mirror-image rule guards the
other direction: a packet leaving an alloc must carry a cluster source, so a
forged external address cannot ride that pass. There is no policy
file, no selector language and no translation step that could make a rule silently match
nothing; policy is map entries the datapath enforces directly, and rules only ever
union, so allow_from can never weaken the project default-deny.
Enforcement is per connection attempt: TCP that is not a SYN passes, which is what lets cross-project replies flow without a conntrack. That is deliberately weaker than stateful tracking — an in-node ACK probe traverses the filter and is stopped only by the receiving stack's RST — and it is stated in the threat model rather than hidden. The upgrade to an LRU conntrack map is additive.
A second, small egress program is load-bearing for A10: it drops the cloud-metadata
range 169.254.0.0/16 in the kernel with a per-alloc drop counter — not
asserted in a policy file — along with any service-CIDR destination that escaped
connect-time rewrite, and it counts per-endpoint traffic. Masquerade for routed pod
traffic is one nftables rule in an owned kanea table, and since v1.65 the
rule and net.ipv4.ip_forward are re-asserted every thirty seconds —
a firewall reload that flushes the ruleset costs seconds, not a daemon restart. A
FORWARD-drop policy another tool installs can still eat that traffic, and
kanea doctor detects and names it, along with a missing
kanea table and forwarding turned off.
Internal DNS
Allocs resolve through kanead's own resolver, bound to the datapath's host
anchor (the node CIDR's .1) and never a wildcard. It answers
<service>.<project>.kanea names authoritatively and forwards
everything else upstream. The upstream list is, in order of precedence: the
--dns-upstream flag; the server config's dns stanza
(v1.66); the host's own /etc/resolv.conf, read once at startup. The
stanza is how you pin resolvers on a node whose resolv.conf is DHCP's to rewrite:
# /etc/kanea/kanea.hcl
dns {
upstreams = ["1.1.1.1", "10.0.0.53:5353"] # a bare address gets :53
}
Entries are validated when the file parses; an empty list is refused by name — a
stanza that meant "no upstreams" would silently turn external resolution into
SERVFAIL. An explicit --dns-upstream wins over the stanza and the daemon
says so in its log, the same precedence every half of the server config follows.
Datapath state is derived state. Programs, maps and the cgroup link are pinned
under /sys/fs/bpf/kanea with a schema stamp; a kanead restart
leaves the dataplane untouched, and a stamp mismatch recreates and repopulates the maps
inside the first reconcile pass — safe precisely because established flows bypass them.
Nothing under the pin root is ever backed up.
The edge
A Go reverse proxy: host-based L7 routing to service frontends, TLS termination with Let's Encrypt certificates, WebSocket and gRPC support, HTTP→HTTPS redirects and security headers.
The middleware chain runs in a fixed order, per service, from the spec's expose block:
Host match → IP allow/deny → rate limit → header transforms → upstream proxy
All of it is validated at kanea plan time and fails closed. An ingress
control that silently does nothing at runtime is worse than one that is absent.
Hardening is mandatory rather than tunable: read/header/idle timeouts and header size
caps against slowloris, per-route upstream timeouts, bounded connection pools,
client-supplied X-Forwarded-* stripped, and an unknown Host
answered with 404 — which is also the DNS-rebinding defence for the co-hosted API.
Because it already sits in the request path, the edge is the primary source of L7 metrics for exposed services: requests per second and latency percentiles at no extra data-plane cost. The datapath's own map counters cover east-west.
ACME runs in kanead, never in the edge — obtaining a certificate means
writing one, and the edge does not write.
Resource isolation
The control plane must survive anything a workload does. Enforcement is cgroups v2, arranged as two sibling slices:
/sys/fs/cgroup
├── kanea.slice # kanead, kanea-edge (+ containerd and buildkitd, via their own units)
│ memory.min = system_reserve_memory # kernel-protected floor (default 256 MiB; build nodes raise it)
│ memory.swap.max = 0 # the floor is RAM, not swap
│ cpu.weight = 10000
│ OOMScoreAdjust = -900 # the global OOM killer picks workloads first
└── kanea-workloads.slice # every alloc lives under this one parent
memory.max = total RAM − system_reserve_memory
memory.swap.max = 0
cpu.weight = 100
└── per-alloc: memory.max · cpu.max · pids.max from the spec's resources block
Calling mlockall on a Go control plane is rejected outright: the GC grows
the heap unpredictably and RLIMIT_MEMLOCK turns pin overflow into hard
allocation failure — the lock itself could crash kanead. The guarantee
comes from memory.min (the kernel refuses to reclaim the floor under
pressure), the OOM score, and no swap in the slice.
Per-alloc limits are enforced where declared. Omit resources
(or a field of it) and the alloc is unbounded: all cores, all allocatable memory,
capped only by the workload parent's collective ceiling — which, with the
control-plane floor, is the isolation that actually protects the platform. A
default pids.max still applies to every alloc. Declared resources are
also the admission units: kanea plan renders the workload budget and
apply refuses a total above it unless the operator has explicitly enabled
oversubscription.
Metrics and autoscaling
Three scrapers feed one bounded in-memory time series: containerd cgroup metrics (CPU, memory), the edge (requests per second, latency percentiles for exposed services), and the datapath's own per-CPU map counters (east-west flows and drops, on by default). Roughly 27 MiB at the 2 000-alloc target, and there is a test that says so.
A missing metric and an idle service lead to opposite decisions, so the distinction is preserved end to end — in the time series, in the evaluator, in the Prometheus exporter and in the dashboard. Each layer has a test asserting it.
The evaluator applies the scaling block with guardrails and a circuit
breaker, and the budget is 20 seconds from a sustained breach to a decision: a
15-second averaging window — three samples at the 5-second scrape resolution — plus one
evaluation tick. A large spike decides sooner.
The autoscaler is not a second scheduler. It writes one number — the desired
count — and the reconciler converges. kanea scale uses the same route,
which is why manual and automatic scaling cannot disagree about mechanism.
State replication and restore
- Change data capture. Store mutations produce change segments; snapshots and segments are replicated to any S3-compatible bucket. The client is hand-written against the REST API — SigV4 and four verbs — and CI exercises it against MinIO.
- Archives are chunked AEAD, with keys HKDF-derived from the master key rather than being it. The last chunk is sealed under different additional data, which is the only thing standing between a truncated snapshot and a restore that decrypts cleanly into half a platform.
- Manifests are unencrypted and hashes are over the ciphertext, so someone holding the bucket and no key can still see what is there and whether it is intact — which is the question you ask before going to find the escrowed key.
- Restore is staged, never performed in place. It can be requested over the API, and the daemon performs it at the next start, before anything opens the Store. The API has no method that restores — that is the interface, not a check.
- The replication cursor is derived from the sink, never stored: writing it to the Store would emit a change that needs shipping, which would write it again.
- The destination can change at runtime — the dashboard's Settings page, or
PUT /v1/settings/backup. A new destination is probed with a real test write before anything commits, so a typo is a refusal with the old replication untouched; the old destination receives one final segment ship, the new one an immediate full snapshot. The unit's flags remain the seed: a settings record, once written, wins, and deleting it reverts.
The master key is generated by kanea init, shown once, and must be typed
back — it is discarded if that fails. Without it, every archive is unreadable. The
DR runbook is
worth reading before you need it.
API, dashboard and MCP
The API is REST plus one multiplexed WebSocket, and every route is deny-by-default
— including the WebSocket and every MCP route. The exceptions are enumerable:
/login, the ACME challenge path, and /healthz.
There is exactly one authentication mechanism that is not the §13 one: the git webhook route, which uses a per-project HMAC over the raw body, rejects replays, and is audited. It never deploys from the request — it marks the project, and the sync loop re-reads the source over Kanea's own credential.
The dashboard is a React SPA embedded in the binary with go:embed and served
by the daemon. A service page charts CPU, memory, request rate and p95 over a real
time axis, streams logs live (virtualized, filterable, copy and download), shows a
restart or deploy as rollout progress — the same spec-hash rule the planner uses,
served on the wire since v1.64 — and gives an admin a shell into any running
alloc, over the same exec websocket the CLI uses: the CSRF token rides the
handshake as a subprotocol, since a browser cannot set the header there. Its Settings
page shows the node's configuration and lets an admin change
what changes at runtime — the backup destination and the notification channels
(node-wide defaults and per-project, each with a test button) — plus accounts, API
tokens and the audit log, one tab each; what belongs to the unit (listen address,
subnets, DNS, the
port policy) is shown read-only with a note saying so. The MCP server exposes 20 tools
in three tiers (read, mutate, destructive) over stdio and streamable HTTP.
Every tool reaches the platform by making an HTTP request against the API's own handler. A tool's only verb is "send this request", so it can never be more privileged than the credential its caller presented — nothing in the MCP package may hold a Store, a secrets store, or an auth store. That is what makes "no side channels" structural rather than a rule somebody has to remember.
There are also no secret tools at any tier. The safety requirement is that no tool returns a secret value; the implementation goes further and gives an agent no secrets verb whatsoever, and a test fails if one appears.
Exposing the API and dashboard
The API and the dashboard are one listener: the dashboard is served by
kanead on the same address the REST API, the WebSocket and the MCP
HTTP transport bind, in front of kanea-edge entirely — neither is a
service, and neither takes an expose block. Exposing them means
setting where that listener binds, and there are two ways: the
--listen/--listen-cert/--listen-key flags
(asked by kanea init and rendered into the
unit), or — since v1.61 — a bind stanza in the server config, which
survives re-runs and makes moving the listener later an edit plus
systemctl restart kanead, never a re-init:
# /etc/kanea/kanea.hcl
bind {
api_addr = "192.168.1.10:8600"
api_tls = "self-signed" # acme | self-signed | provided | plaintext
# api_domain = "kanea.home.example" # required by acme; names a self-signed cert
# api_cert = "/etc/kanea/api.pem" # provided only — always with api_key
# api_key = "/etc/kanea/api.key"
}
| Field | Type | Notes |
|---|---|---|
api_addr | string | The host:port the API and dashboard bind. Required by every other field in the stanza — TLS with nothing to serve it on is a parse error. |
api_tls | string | acme, self-signed, provided or plaintext — the same mode vocabulary services use. Unset resolves at the daemon: a declared pair means provided, loopback means plaintext, and anything beyond loopback refuses there. |
api_domain | string | Required by acme (an IP cannot hold an ACME certificate). For self-signed it names the certificate, and it is required when api_addr binds every interface (":8600", 0.0.0.0) — otherwise the certificate would have no name at all. |
api_cert, api_key | string | Your own PEM pair, for provided only. Always together, and refused beside a managed mode or plaintext — a control that cannot act is refused, never silently dropped. |
What each mode gives you:
self-signed— a certificate from the node's own CA, the onekanea ca showinstalls on your devices, with a real IP SAN when the address is bare. Renewed automatically. The right default on a home network.acme— a Let's Encrypt certificate forapi_domain, issued and renewed by the same account and pass that serve your services. Needs--acme-emailconfigured on the daemon, refused at startup without it.provided— your ownapi_cert/api_keypair, for a certificate something else manages.plaintext— explicit HTTP. Allowed beyond loopback because you typed it, logged loudly, and it implies the insecure-cookie posture — a Secure cookie over plain HTTP is a login that silently fails.
Precedence: an explicit --listen always beats the stanza, and
--listen none keeps the node socket-only regardless of the file. When
the stanza is declared and --listen was not passed,
kanea init skips the listen question and renders no listen flags into
the unit — the file owns the listener. On the flags path, a non-loopback
--listen requires the --listen-cert/--listen-key
pair and is refused up front without it.
One consequence worth knowing when adopting the stanza on an existing node: an init
run from before it existed rendered --listen 127.0.0.1:8600 into the
kanead unit, and that flag shadows the file — the stanza appears to do nothing.
Remove the flag from the unit's ExecStart, then
systemctl daemon-reload and restart; the
troubleshooting page has the steps.
Job spec reference
Specs are HCL v2. If you have written Nomad job files the shape will be familiar,
though the vocabulary is Kanea's. Every rule below is enforced when the file is parsed
— errors carry file, line and column — and kanea plan is where you see
them.
A service needs an image and nothing else. This is a complete, valid spec:
spec_version = 1
project "demo" {}
service "web" {
project = "demo"
task "app" {
image = "nginx:1.27-alpine"
}
}
Everything on this page is additive to that.
Top level
A spec file contains spec_version and any number of
project, service and storage blocks, plus an
optional variables block of shared values. Multiple
files applied together are parsed as one set, and file order is irrelevant — a
service may reference a project declared in another file.
| Field | Type | Notes |
|---|---|---|
spec_version | number | Currently 1. Future spec revisions are gated on this field, which is how an upgrade can tell an old file from a new one. |
project "<name>" | block | Zero or more. Name is a DNS-1123 label. |
service "<name>" | block | Zero or more. |
storage "<name>" | block | Zero or more. May also be declared in the server config. |
variables | block | Optional. Shared values the rest of the spec references as ${name} (R30). |
variables
Declare a value once, reference it anywhere as ${name} — or as a bare
identifier where HCL takes an expression, like count. The node may
supply defaults from a variables stanza in its own
/etc/kanea/kanea.hcl; the spec's block wins on a collision, and
pipeline-supplied values (${GIT_SHA_SHORT} and friends) sit above both.
variables {
domain = "shop.example.com"
replicas = 3
}
service "web" {
project = "shop"
count = replicas
expose {
domains = ["${domain}", "www.${domain}"]
}
}
Values are strings, numbers or bools — a list or object is refused by name, and so
is redeclaring a name or shadowing a built-in. A variable's value may reference node
variables and built-ins, never another spec variable. Variables are not
secrets: the node's stanza is served to any signed-in caller over
GET /v1/vars, so a secret stays a secret: reference — a
variable whose value contains one is fine.
project
A named group of services, and the isolation boundary for network policy, secrets, containerd namespaces and DNS.
| Field | Type | Notes |
|---|---|---|
description | string | Free text, ≤ 512 chars. Shown in the dashboard. |
git | block | Optional GitOps source for this project. |
notifications | block | Optional channels and filters. |
git
| Field | Type | Notes |
|---|---|---|
url | string | Required. Clone URL. |
branch | string | Defaults to the repository's default branch. |
path | string | Directory within the repository holding specs, e.g. .kanea/. |
auth_ref | secret ref | Deploy key or token. Project-scoped like every other reference. |
webhook_secret_ref | secret ref | HMAC key for the push webhook. |
poll_interval | duration | How often to poll when no webhook arrives. |
require_approval | bool | Sync marks the project; a human promotes. |
A synced spec that declares a different project is refused. It is the same boundary that scopes secrets, and it is the only thing between "can push to one repo" and "owns every service on the node".
The webhook never deploys from the request either. It marks the project as dirty and the sync loop re-reads the source over Kanea's own credential — so a forged body can at most cause a legitimate sync to happen sooner.
notifications
Up to five channel blocks — telegram, slack (Discord accepts
the same shape on its /slack endpoint), ntfy,
smtp, webhook — plus filters:
| Field | Type | Notes |
|---|---|---|
on | list(string) | Event globs, e.g. ["deploy.failed", "scale.*"]. Validated against the known event vocabulary at parse time — a filter that could never match is a spec error, not a silence. |
severity | string | Floor: info, warning, error. Composes with on as an AND. |
telegram | block | chat_id, token_ref |
slack | block | url_ref — an incoming-webhook URL is a credential in path form, so it is referenced, never inlined |
ntfy | block | url, token_ref |
smtp | block | host, port, from, to, username, password_ref |
webhook | block | url, secret_ref (HMAC signature over the body) |
Egress is checked at dial time against every resolved address, and redirects are refused. A hostname is not a destination.
storage
A named volume backend. Declared here or in the server config; services mount it by
name with a volume block.
type | Fields | Notes |
|---|---|---|
local | — | A directory on the node, managed by Kanea. |
host | path | A directory the operator already owns. See R15 — it does nothing until an operator allowlists its parent in /etc/kanea/kanea.hcl's storage { allowed_host_paths = […] } stanza (or --allowed-host-paths, which wins). |
nfs | server, export, options | |
smb | server, share, auth_ref, options | |
s3 | bucket, endpoint, auth_ref, mode | mode = "ro" selects mountpoint-s3, "rw" selects s3fs. |
Every file operation costs an object-store round trip — around 30 ms. Listing or
creating a 200-file directory takes tens of seconds, no driver implements
truncate (s3fs silently no-ops it), and a FUSE call against a dead
backend blocks uninterruptibly for tens of seconds. Use them for bulk, read-mostly
data. Never for a hot path or many small files.
service
| Field | Type | Default | Notes |
|---|---|---|---|
project | string | — | The owning project. |
description | string | — | Free text, ≤ 512 chars. |
count | number | 1 | Replicas. Must sit inside scaling's min/max if that block is present. |
depends_on | list(string) | — | Start ordering. See R10. |
task | block | — | Exactly one in v1. |
build | block | — | Build from source instead of pulling. |
network | block | — | Ports and ingress policy. |
expose | block | — | Public exposure and middleware. |
health_check | block | — | Zero or more, each labelled. |
volume | block | — | Zero or more mounts. |
scaling | block | — | Autoscaling policy. |
update | block | rolling | Rollout strategy. |
restart | block | always | Restart policy. |
task
| Field | Type | Notes |
|---|---|---|
image | string | Optional only if a build block is present (R8). Digests are supported and preferred. |
command | list(string) | Overrides the image entrypoint. An argument array, never a shell string (R12). |
capabilities | list(string) | Grants added to the baseline set; "none" starts from nothing instead (R13). |
env | map | Values may be secret: references or ${service.…} interpolations. |
resources | block | cpu in MHz (1000 = one core), memory in MiB. An omitted limit is unbounded: all cores / all allocatable memory. |
registry_auth_ref | string | secret: reference to a docker config.json used to pull the image. Project-scoped like every other reference (R5). |
device | block | Requests a host device the operator granted, by name (R17). See below. |
socket | block | Requests a host unix socket the operator granted, by name (R18). See below. |
device and socket
A GPU for transcoding, a USB dongle, or the container runtime's socket for
a watchtower-style updater. Both blocks name a grant and never a
path: the operator defines grants on the node, and the spec asks for one
by name. There is no field to write a device path into, which is the
property the whole model rests on — a spec cannot request
/dev/mem because there is nowhere to say it.
service "jellyfin" {
task "app" {
image = "jellyfin:10.9"
device "dri" {
grant = "gpu" # defined by the operator, not here
}
}
}
service "watchtower" {
task "app" {
image = "watchtower:1.7"
socket "runtime" {
grant = "containerd"
mount_path = "/var/run/docker.sock"
}
}
}
| Field | Type | Notes |
|---|---|---|
grant | string | Required, on both blocks. Names a device/socket grant in the node's server config, /etc/kanea/kanea.hcl. |
mount_path | string | socket only, required. Where the socket appears in the container. A device appears at its host path. |
read_only | bool | socket only. Restricts the filesystem entry, not the protocol spoken over it. |
The operator side lives in the node's server config,
/etc/kanea/kanea.hcl, probed automatically at daemon start —
no flag, no unit editing, and a kanea init re-run never
touches it. The default is that the file does not exist — a spec asking
for a grant on a node with none fails its alloc rather than starting
without it. Grants name the projects that may claim them:
# /etc/kanea/kanea.hcl — the node's, never the repository's
bind { # the API/dashboard listener (v1.61)
api_addr = "192.168.1.10:8600"
api_tls = "self-signed" # acme | self-signed | provided | plaintext
}
variables { # node-wide spec-variable defaults (v1.63) — never secrets
domain = "home.lan"
}
device "gpu" {
nodes = ["/dev/dri/renderD128"]
allow = ["media"]
}
socket "containerd" {
path = "/run/kanea/containerd.sock"
allow = ["ops"]
}
Setting it up, end to end:
- Create the file (init already made the directory):
sudo install -o root -g root -m 0644 /dev/null /etc/kanea/kanea.hcl, then write the grant blocks above — and, if you use host volumes, thestorageallowlist stanza in the same file. It must stay root-owned and writable only by its owner; kanead refuses a group- or world-writable policy file. sudo systemctl restart kanead— the file is read once, at startup, never polled.- Verify: the startup log carries
server config loadedwith the path, and awarnline naming the passthrough consequence when grants are present.kanea psshows the alloc converging.
A socket grant is root on this node for whoever holds it. A container with the runtime socket can start other containers without the hardening defaults. There is no containment story and none is claimed: the control is that granting it happens on the machine, in a file no spec author can write, and names one project.
network
network {
port "http" { container = 8096 }
publish "http" {
host = 8096 # http://<node>:8096
mode = "http" # "http" (default) | "tcp"
ip_restriction { allow = ["192.168.0.0/16"] }
}
policy { allow_from = ["analytics/collector"] }
}
port "<name>"—containeris the port inside the container. The name is whatexposeandhealth_checkrefer to, and what${service.x.port.<name>}resolves.publish "<port name>"— a node port the edge binds for this service, with or without a domain. The label names theportabove; there is deliberately no field for a container port number, so a published port can never forward somewhere the service did not declare. See R21.mode = "tcp"relays bytes for Postgres, a game server, or anything that is not HTTP. It keepsip_restrictionand nothing else — arate_limitorheadersblock on one is a plan error rather than a control that is silently dropped. On a tcp listener the upstream sees the edge's address, not the client's, soip_restrictionis the whole mitigation and it is enforced at accept time.- Which node ports a spec may claim belongs to the node, not the spec
(
--publish-ports, unprivileged by default). A repository anyone can push to must not be able to take :22. See R22. policy.allow_from— fully-qualified"<project>/<service>"peers permitted to reach this service. See R14.
expose
North-south exposure: the edge proxy, TLS, and the middleware chain. The block may
repeat: each expose is one complete route with its own domains, port,
TLS and middleware, so one service can serve its UI and its API on different names
and ports. Only the first block may omit domains, and blocks that
declare auth must declare the same auth.
| Field | Type | Notes |
|---|---|---|
domains | list(string) | Omitted → <service>.<project>.<base_domain>. No two services may claim the same domain, counting generated ones. |
port | string | Which declared network { port } the route proxies to. Omitted → the port named http, or the sole declared port. Explicit beats every convention; naming an undeclared or udp port is a plan error. |
tls | block | mode — acme, self-signed, provided or plaintext; name selects one of the node's provided grants. Omit the block and the node's --tls-default decides. A mode names a source, never a path. |
ip_restriction | block | allow, deny — lists of CIDRs. Empty allow means the world; deny wins. |
rate_limit | block | requests, window, per, burst. per is ip, service, or header:<name>. |
headers | block | request_set, request_remove, response_set, response_remove. |
The chain evaluates in a fixed order regardless of declaration order:
Host match → IP allow/deny → rate limit → header transforms → upstream
The headers block rejects any attempt to set or remove the
X-Forwarded-* set, or the hop-by-hop headers. Those carry the client
identity that IP restriction, rate limiting and the audit log are all keyed on — a
spec able to rewrite X-Forwarded-For would be forging the thing every
other control trusts.
health_check
health_check "http" {
type = "http" # http | tcp | exec
path = "/healthz" # http only
port = "http" # port name, required for http and tcp
interval = "10s"
timeout = "2s"
failures = 3
}
An exec check runs inside the task's container and takes
command as an argument array, never a shell string — the same rule
as task.command, for the same reason.
Alloc health is only ever written by a probe. A service without a
health_check block has every alloc unhealthy forever, and that is
correct — Kanea asks whether a check is configured before it asks whether one passed.
It matters because min_healthy in the update block is
meaningless without one.
volume
volume "data" {
storage = "local-ssd" # a declared storage block
mount_path = "/var/lib/data"
read_only = false
}
Mount failures fail the alloc loudly. A volume that silently is not there is worse than one that stops the deploy.
scaling
scaling {
min = 2
max = 10
metric "cpu" { target = 70 } # percent of resources.cpu
metric "rps" { target = 500 } # requests per second, per alloc
metric "p95_latency_ms" { target = 800 }
cooldown = "2m"
}
cpucomes from containerd cgroup metrics and is a percentage of the alloc's declared limit — the number the workload can actually use.rpsandp95_latency_mscome from the edge for exposed services, and from the datapath's own map counters for east-west traffic.countmust fall inside[min, max], otherwise the autoscaler would immediately contradict the spec.
update and restart
update {
strategy = "rolling" # rolling | replace
max_parallel = 1
min_healthy = "30s"
auto = true # follow task.image's tag (R19); off by default
interval = "6h" # how often to re-resolve it; minimum 5m
deadline = "10m" # how long a new digest has to prove itself
}
restart {
attempts = 5
backoff = "10s,30s,1m,5m"
}
max_parallel bounds allocs that are down, not replacements in
flight — anything already unavailable spends the budget first, so a failing deploy
halts rather than marching through every replica. min_healthy applies only
to allocs this deploy has already replaced.
auto is the watchtower feature, as policy rather than as a container
holding your runtime socket. Kanea re-resolves the tag the service already
declares, and when the digest behind it moves it pins the new one — the tag stays in
the spec, the digest is what runs. That is an ordinary deploy nobody typed: it rolls
with max_parallel, waits min_healthy and is gated by the
service's health check, because it goes through the same machinery every other deploy
does. If the new digest does not converge within deadline, the previous
one is re-pinned and the service goes back to what worked.
It is refused on a service whose image is already a digest (nothing to
follow) and on one with a build block (the pipeline owns that image).
Private registries need task.registry_auth_ref. Both outcomes emit
events: image.updated and image.update_failed.
build
build {
context = "./web"
dockerfile = "Containerfile" # optional; auto-detected when omitted
target = "registry.example.com/shop/web"
tag = "${GIT_SHA_SHORT}"
cache_repo = "registry.example.com/shop/web-cache"
registry_auth_ref = "secret:shop/registry"
}
- Builds run on rootless BuildKit, unprivileged end to end.
ContainerfileandDockerfileboth work, andContainerfilewins when both exist.- The push credential is materialised as a
config.jsonfor the build and never enters the build context. - A service with a
buildblock and notask.imageis legitimate: the reconciler skips it until the first build pins a digest. - Builds are serialised, and refused when full rather than queued forever.
Isolation is collective, so a second concurrent build would share the first's budget.
queuedis a real state and shutdown cancels what is still waiting.
Interpolation
Three kinds of ${…} appear in a spec, and they resolve at different times.
| Form | Resolved | Notes |
|---|---|---|
${VAR} | Parse time | From the spec's variables block, the node's variables stanza in /etc/kanea/kanea.hcl, and built-ins such as KANEA_PROJECT. Node under spec under caller (R30). |
${GIT_SHA_SHORT} | Checkout time | Survives parsing as a literal reference — its value does not exist until a commit is checked out. |
${service.<n>.host} | Alloc start | Becomes <n>.<project>.kanea. Validated at plan time; resolved as a DNS name, never an IP, so load-balancer reprogramming cannot break it. |
${service.<n>.port.<p>} | Alloc start | The named port's frontend port. |
secret:<path> | Alloc start | Not interpolation — a reference the reconciler resolves. Project-scoped (R5). |
Validation rules
Sixteen rules, all enforced at parse or plan time with file/line/column diagnostics. They are numbered because the error messages cite them.
Names are DNS-1123 labels. Project and service names: lowercase alphanumeric and -, alphanumeric at both ends, ≤ 63 characters. Parse errors abort the run.
Variables. Flat ${VAR} interpolation in any attribute, from the spec's variables block, the node's variables stanza and built-ins — node under spec under caller. Reserved names (the built-ins, service) cannot be declared, values are primitives, and a definition never references a sibling.
Secrets are referenced, never inlined. Resolved at alloc start. Primary injection is a tmpfs file at /run/kanea/secrets/<alloc>/<name>; env vars are supported and documented as weaker.
kanea plan is a real dry run and shows the create/change/destroy diff before anything is applied.
Secret references are project-scoped. A service may name secret:<own-project>/… or secret:shared/… and nothing else. Cross-project references are rejected. Git, registry, storage and notification credentials follow the same scoping.
spec_version = 1 is declared by every file; future revisions are gated on it.
Health check types are http, tcp and exec. exec runs inside the container and takes an argument array — never a shell string.
The minimal service is image-only. At least one of task.image or build must be present. When both are, the pipeline-built digest wins and task.image is the pre-first-build value.
Service references are same-project in v1, validated at plan against the whole applied set — the referenced service and port must exist, and file order is irrelevant. Resolved as DNS names, never IPs. Cycles are rejected, with the cycle shown in the diagnostic.
Dependencies gate starts, not stops. depends_on and every implicit reference edge mean a dependent will not start until its dependencies are healthy. If a dependency degrades afterwards, dependents keep running and events are emitted — no cascading stops.
A declared limit is enforced; an omitted one is the node's capacity. Omit resources and the alloc gets all cores and all allocatable memory, bounded by the workload parent cgroup — not by a per-alloc number nobody typed. A default pids.max applies regardless. A declared memory.max breach OOM-kills the alloc, emits an event, and the restart policy applies. Scaling on cpu or memory (percent-of-limit) requires the corresponding limit and is refused at plan without it. Declared values are the admission units counted against the node's workload budget. Functions keep small defaults (cpu = 100, memory = 64) — the wasm sandbox's caps are promises.
task.command is an argument array. The first element must be non-empty; later ones may be empty, because some programs use that meaningfully — redis-server --save "" is how you disable snapshots.
task.capabilities adds to a baseline, and "none" takes the baseline away. Every runc alloc starts from the baseline set — CHOWN, DAC_OVERRIDE, FOWNER, FSETID, KILL, NET_BIND_SERVICE, SETGID, SETUID — so PUID-style images start without a capability line. The effective grant is the union of baseline and declared list; ["none"] is the full drop-ALL posture, ["none", "CAP_NET_RAW"] exactly one grant. Declarable beyond the baseline: SETPCAP, SETFCAP, NET_RAW (never baseline — the datapath's identity is the IP), SYS_CHROOT, MKNOD, AUDIT_WRITE. Privilege-equivalent ones — SYS_ADMIN, SYS_MODULE, SYS_PTRACE, BPF, PERFMON and friends — are rejected at parse time: granting them would be the privileged escape hatch v1 deliberately does not have. Effective capabilities go into the bounding, effective and permitted sets, never inheritable or ambient.
allow_from only ever adds reachability. Each entry is a fully-qualified "<project>/<service>"; the datapath's ingress rules only ever union, so an entry can never weaken the project default-deny. There is no wildcard — "analytics/*" is a parse error, because naming the peer is the point. Same-project entries are accepted and redundant.
host volumes are operator-gated. The path is validated as absolute, clean and ..-free at parse time — but whether it may be mounted is not the spec's decision. kanead refuses any path outside storage.allowed_host_paths in the server config /etc/kanea/kanea.hcl (or --allowed-host-paths, which wins when set), whose default is empty. The check is applied after symlink resolution, and the directory must already exist.
expose fails closed. A service may only be exposed if it declares a port, and the upstream port must be unambiguous — declared with port = "<name>", or named http, or the only one declared. The block may repeat, each one a complete route validated independently; only the first may omit domains, and blocks that declare auth must agree. Every domain is validated as a hostname, and no two services may claim the same one, counting generated FQDNs. Middleware is checked here too: CIDRs must parse, rate_limit needs a positive requests and a valid window, and headers may not touch the hop-by-hop or X-Forwarded-* sets.
task.device names a grant, not a device. There is no field for a device path, so a spec cannot ask for one. Parse time checks only that the grant name is a DNS-1123 label. The node refuses a grant it does not have, a grant whose allow list does not name the requesting project, and a path that is no longer a character or block device — checked after symlink resolution, at every alloc start. The device appears at its host path, and the grant carries the cgroup permissions (rw by default, never m unless written). A failed grant fails the alloc; it never starts without what it asked for.
task.socket is R17 for unix sockets, and is privilege delegation. mount_path is validated as absolute, clean and ..-free, may not sit under /dev, /proc or /sys, and may not collide with another socket or a volume. The bind carries nosuid, noexec and nodev. None of that makes it safe and none of it is meant to: a container holding the container runtime's socket can create containers without the hardening defaults, so the grant is equivalent to root on the node. The server config is the only control over it, which is why it is project-scoped and empty by default.
update.auto follows the tag the service declares. Off by default. Kanea re-resolves task.image's tag every interval (default 6h, minimum 5m) and pins the digest behind it when it moves; the declared tag is never overwritten, because it is what the next poll re-reads. The pinned digest is server-owned and survives kanea apply — except when you edit image or turn auto off, which both hand authority back to the spec. A failed update reverts to the digest that was running if the new one has not converged within deadline: converged means healthy where a check block exists, and running without crash-looping where it does not. Refused on a digest-pinned image and on a service with a build block.
Full example
Everything above, in one file. This parses and validates against Kanea's own parser.
# shop.hcl — everything for one project spec_version = 1 project "shop" { description = "E-commerce storefront stack" git { url = "https://github.com/example/shop-deploy.git" branch = "main" path = ".kanea/" auth_ref = "secret:shop/github-deploy-key" } notifications { slack { url_ref = "secret:shop/slack-webhook" } on = ["deploy.failed", "service.unhealthy", "scale.*"] severity = "warning" } } storage "local-ssd" { type = "local" } service "postgres" { project = "shop" task "db" { image = "postgres:16-alpine" env = { POSTGRES_PASSWORD = "secret:shop/postgres-password" } resources { cpu = 1000 memory = 1024 } } network { port "pg" { container = 5432 } } volume "data" { storage = "local-ssd" mount_path = "/var/lib/postgresql/data" } health_check "tcp" { type = "tcp" port = "pg" interval = "10s" } } service "web" { project = "shop" description = "Storefront frontend" count = 3 depends_on = ["postgres"] build { context = "./web" target = "registry.example.com/shop/web" tag = "${GIT_SHA_SHORT}" } task "app" { image = "registry.example.com/shop/web:latest" env = { NODE_ENV = "production" DATABASE_URL = "secret:shop/database-url" DATABASE_HOST = "${service.postgres.host}" DATABASE_PORT = "${service.postgres.port.pg}" } resources { cpu = 500 memory = 512 } } network { port "http" { container = 3000 } } expose { domains = ["shop.example.com", "www.shop.example.com"] tls { mode = "acme" } ip_restriction { deny = ["198.51.100.7/32"] } rate_limit { requests = 100 window = "1m" per = "ip" burst = 20 } headers { response_set = { Strict-Transport-Security = "max-age=63072000; includeSubDomains" } response_remove = ["Server", "X-Powered-By"] } } health_check "http" { type = "http" path = "/healthz" port = "http" interval = "10s" timeout = "2s" failures = 3 } scaling { min = 2 max = 10 metric "cpu" { target = 70 } metric "rps" { target = 500 } cooldown = "2m" } update { strategy = "rolling" max_parallel = 1 min_healthy = "30s" } restart { attempts = 5 backoff = "10s,30s,1m,5m" } }
Sample stacks
Five complete stacks, from a static site to a Kafka cluster. Every one parses and
validates against Kanea's own parser — copy the file, change the names and domains,
and start with kanea plan, which will tell you about anything the copy
broke before anything runs.
A static site
The smallest production-shaped thing: two replicas behind Let's Encrypt, with a
health check. The check is not decoration — it is what a rolling deploy waits on
(min_healthy has nothing to measure without one), what anything
declaring depends_on this service waits for, and the difference
between a replica that stops answering showing up in kanea status
and staying a mystery.
spec_version = 1 project "web" {} service "site" { project = "web" count = 2 task "nginx" { image = "nginx:1.27-alpine" resources { cpu = 200 memory = 128 } } network { port "http" { container = 80 } } expose { domains = ["example.com", "www.example.com"] tls { mode = "acme" } } health_check "http" { type = "http" path = "/" port = "http" interval = "10s" timeout = "2s" failures = 3 } }
This works the moment example.com resolves to the node and ports 80/443
reach it — the ACME HTTP-01 flow needs nothing else. From here the natural next step
is replacing the stock image with your own: add a build
block and the pipeline builds and pins a digest on every push.
kanea plan site.hcl kanea run site.hcl --wait=60s kanea status web/site
Two apps and a database
A frontend and an API sharing PostgreSQL and Redis. This is the shape most
multi-service deployments take, and it exercises the machinery that matters:
depends_on gates the apps until their backends are healthy —
which is why postgres and redis have checks — and the
${service.…} references resolve to internal DNS names at alloc start,
so nothing here hard-codes an address. One secret, referenced from two services,
never written in the file.
spec_version = 1 project "paste" { description = "Pastebin: frontend, API, database, cache" } storage "db-data" { type = "local" } service "postgres" { project = "paste" task "db" { image = "postgres:16-alpine" env = { POSTGRES_DB = "paste" POSTGRES_USER = "paste" POSTGRES_PASSWORD = "secret:paste/db-password" } resources { cpu = 1000 memory = 1024 } } network { port "pg" { container = 5432 } } volume "data" { storage = "db-data" mount_path = "/var/lib/postgresql/data" } health_check "up" { type = "tcp" port = "pg" interval = "10s" } } service "redis" { project = "paste" task "cache" { image = "redis:7-alpine" command = ["redis-server", "--save", ""] resources { cpu = 250 memory = 256 } } network { port "redis" { container = 6379 } } health_check "up" { type = "tcp" port = "redis" interval = "10s" } } service "api" { project = "paste" count = 2 depends_on = ["postgres", "redis"] task "app" { image = "registry.example.com/paste/api:1.4.2" env = { DB_HOST = "${service.postgres.host}" DB_PORT = "${service.postgres.port.pg}" DB_USER = "paste" DB_PASSWORD = "secret:paste/db-password" REDIS_ADDR = "${service.redis.host}:${service.redis.port.redis}" } resources { cpu = 500 memory = 512 } } network { port "http" { container = 8080 } } expose { domains = ["api.paste.example.com"] tls { mode = "acme" } rate_limit { requests = 300 window = "1m" per = "ip" burst = 50 } } health_check "http" { type = "http" path = "/healthz" port = "http" interval = "10s" timeout = "2s" failures = 3 } update { strategy = "rolling" max_parallel = 1 min_healthy = "30s" } } service "web" { project = "paste" count = 2 depends_on = ["api"] task "app" { image = "registry.example.com/paste/web:1.4.2" env = { API_URL = "http://${service.api.host}:${service.api.port.http}" } resources { cpu = 250 memory = 256 } } network { port "http" { container = 3000 } } expose { domains = ["paste.example.com"] tls { mode = "acme" } } health_check "http" { type = "http" path = "/" port = "http" interval = "10s" timeout = "2s" failures = 3 } }
Create the secret first, then deploy — order within the file never matters:
kanea secret put paste/db-password # value on stdin kanea run paste.hcl --wait=90s
Details worth stealing: redis-server --save "" is a command as an
argument array with a meaningfully empty argument (R12 — a shell string could not
say that); the rate limit lives only on the API route, because the frontend serving
assets at API rates would be rate-limiting your own pages; and postgres and redis
declare no expose and no publish, so they are reachable
from this project's services and from nothing else on the network.
Jellyfin with local media
A media server is the stack where the node itself gets a say: the library is a
directory the operator already owns, and hardware transcoding needs a GPU device.
Both cross the line a job spec cannot cross alone — a host volume does
nothing until its path is allowlisted on the node (R15), and the device
block names a grant, never a path (R17).
spec_version = 1 project "media" {} storage "config" { type = "local" } storage "library" { type = "host" path = "/srv/media" } service "jellyfin" { project = "media" task "app" { image = "jellyfin/jellyfin:10.9.11" device "dri" { grant = "gpu" # hardware transcoding; the grant is defined on the node } resources { cpu = 4000 memory = 4096 } } network { port "http" { container = 8096 } publish "http" { host = 8096 # http://<node>:8096, LAN only ip_restriction { allow = ["192.168.0.0/16"] } } } volume "config" { storage = "config" mount_path = "/config" } volume "media" { storage = "library" mount_path = "/media" read_only = true } health_check "http" { type = "http" path = "/health" port = "http" interval = "15s" timeout = "5s" failures = 3 } }
The node's half, in the server config — without it the spec is valid and the alloc fails, loudly:
# /etc/kanea/kanea.hcl — the node's, never the repository's
storage {
allowed_host_paths = ["/srv/media"]
}
device "gpu" {
nodes = ["/dev/dri/renderD128"]
allow = ["media"]
}
A failed grant fails the alloc rather than starting without the GPU, because a
transcoder silently falling back to software looks healthy and does the wrong thing.
The library is mounted read_only — a media server has no business
writing to it — while /config is an ordinary local volume Kanea
manages. The publish block binds :8096 on the node for the LAN, with
the edge enforcing the CIDR allowlist; for access from outside, add an
expose block with a domain instead of
widening the CIDR.
SMB and S3 volumes
The same volume block, backed by things that are not on the node: a
file browser over a NAS share and a read-only bucket. The credential for either
driver is one secret whose value is <user>:<secret> —
username and password for SMB, access key and secret key for S3 — resolved at mount
time into a 0600 file, never onto a command line. Omit
auth_ref entirely for a public bucket or an open share.
spec_version = 1 project "files" {} # A share on the NAS. The secret's value is "<username>:<password>". storage "nas" { type = "smb" server = "192.168.1.20" share = "documents" auth_ref = "secret:files/nas" } # A bucket, read-only. The secret's value is "<access-key>:<secret-key>". storage "archive" { type = "s3" bucket = "household-archive" endpoint = "https://minio.internal:9000" # omit for AWS S3 auth_ref = "secret:files/archive" mode = "ro" # mountpoint-s3; "rw" selects s3fs } service "filebrowser" { project = "files" task "app" { image = "filebrowser/filebrowser:v2.32.0" user { uid = 1000 gid = 1000 } resources { cpu = 500 memory = 256 } } network { port "http" { container = 80 } } expose { domains = ["files.example.com"] tls { mode = "acme" } } volume "documents" { storage = "nas" mount_path = "/srv/documents" } volume "archive" { storage = "archive" mount_path = "/srv/archive" read_only = true } health_check "http" { type = "http" path = "/health" port = "http" interval = "15s" timeout = "5s" failures = 3 } }
Create both secrets first:
kanea secret put files/nas # username:password on stdin kanea secret put files/archive # access-key:secret-key kanea run files.hcl
The user block does double duty here (R23, R24): the process runs as
1000:1000, and the volumes inherit that ownership — no
uid/gid on the volume needed. On a mounted filesystem
there is nothing to chown, so the ownership travels in the mount
options instead, which is exactly why it works on a share the NAS controls. The
inheritance is resolved at parse time, not on the node, so a spec means the same
thing everywhere. (host and nfs volumes are the
exception: those drivers cannot carry ownership, so declaring it on one is a plan
error, and inheritance skips them.)
mode on the S3 storage selects the driver — "ro" is
mountpoint-s3 and the default, "rw" is s3fs — and the
storage reference's warning applies in full: an object
store is not a filesystem, so keep it for bulk, read-mostly data. A custom
endpoint points at MinIO or any S3-compatible store; omit it for AWS.
An nfs export is the same shape with server and
export. For all of them, a mount that cannot be established fails the
alloc loudly rather than starting the service beside an empty directory, and a
mount that dies later is supervised and remounted.
A Kafka cluster (KRaft)
Three brokers, no ZooKeeper. Each broker is its own count = 1
service rather than one service with count = 3, and that is
the load-bearing decision: a Kafka broker has an identity — a node id and an
advertised name that must be stable across restarts — and replicas of one service
are deliberately interchangeable. Three services give each broker its own DNS name,
its own data volume, and its own line in the quorum.
spec_version = 1 project "kafka" { description = "Three-broker KRaft cluster, no ZooKeeper" } storage "kafka-data" { type = "local" } service "kafka-1" { project = "kafka" task "broker" { image = "apache/kafka:4.0.0" env = { KAFKA_NODE_ID = "1" KAFKA_PROCESS_ROLES = "broker,controller" KAFKA_LISTENERS = "PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093" KAFKA_ADVERTISED_LISTENERS = "PLAINTEXT://kafka-1.kafka.kanea:9092" KAFKA_CONTROLLER_LISTENER_NAMES = "CONTROLLER" KAFKA_LISTENER_SECURITY_PROTOCOL_MAP = "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT" KAFKA_CONTROLLER_QUORUM_VOTERS = "1@kafka-1.kafka.kanea:9093,2@kafka-2.kafka.kanea:9093,3@kafka-3.kafka.kanea:9093" KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR = "3" KAFKA_LOG_DIRS = "/var/lib/kafka/data" CLUSTER_ID = "MkU3OEVBNTcwNTJENDM2Qg" } resources { cpu = 1000 memory = 2048 } } network { port "client" { container = 9092 } port "controller" { container = 9093 } } volume "data" { storage = "kafka-data" mount_path = "/var/lib/kafka/data" } health_check "up" { type = "tcp" port = "client" interval = "15s" timeout = "5s" failures = 3 } } service "kafka-2" { project = "kafka" task "broker" { image = "apache/kafka:4.0.0" env = { KAFKA_NODE_ID = "2" KAFKA_PROCESS_ROLES = "broker,controller" KAFKA_LISTENERS = "PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093" KAFKA_ADVERTISED_LISTENERS = "PLAINTEXT://kafka-2.kafka.kanea:9092" KAFKA_CONTROLLER_LISTENER_NAMES = "CONTROLLER" KAFKA_LISTENER_SECURITY_PROTOCOL_MAP = "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT" KAFKA_CONTROLLER_QUORUM_VOTERS = "1@kafka-1.kafka.kanea:9093,2@kafka-2.kafka.kanea:9093,3@kafka-3.kafka.kanea:9093" KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR = "3" KAFKA_LOG_DIRS = "/var/lib/kafka/data" CLUSTER_ID = "MkU3OEVBNTcwNTJENDM2Qg" } resources { cpu = 1000 memory = 2048 } } network { port "client" { container = 9092 } port "controller" { container = 9093 } } volume "data" { storage = "kafka-data" mount_path = "/var/lib/kafka/data" } health_check "up" { type = "tcp" port = "client" interval = "15s" timeout = "5s" failures = 3 } } service "kafka-3" { project = "kafka" task "broker" { image = "apache/kafka:4.0.0" env = { KAFKA_NODE_ID = "3" KAFKA_PROCESS_ROLES = "broker,controller" KAFKA_LISTENERS = "PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093" KAFKA_ADVERTISED_LISTENERS = "PLAINTEXT://kafka-3.kafka.kanea:9092" KAFKA_CONTROLLER_LISTENER_NAMES = "CONTROLLER" KAFKA_LISTENER_SECURITY_PROTOCOL_MAP = "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT" KAFKA_CONTROLLER_QUORUM_VOTERS = "1@kafka-1.kafka.kanea:9093,2@kafka-2.kafka.kanea:9093,3@kafka-3.kafka.kanea:9093" KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR = "3" KAFKA_LOG_DIRS = "/var/lib/kafka/data" CLUSTER_ID = "MkU3OEVBNTcwNTJENDM2Qg" } resources { cpu = 1000 memory = 2048 } } network { port "client" { container = 9092 } port "controller" { container = 9093 } } volume "data" { storage = "kafka-data" mount_path = "/var/lib/kafka/data" } health_check "up" { type = "tcp" port = "client" interval = "15s" timeout = "5s" failures = 3 } }
The quorum voters and advertised listeners are written as literal internal DNS
names (<service>.<project>.kanea), not
${service.…} references — deliberately. A reference is also a start
dependency (R10), and three brokers referencing each other is a cycle
kanea plan rejects (R9). Kafka's quorum is designed to form as
the peers come up, so the ordering edge is not wanted; the literal names are exactly
the strings the interpolation would have produced, minus the edge. Note the shared
storage "kafka-data" block is safe: each service's volume gets its own
directory beneath it, so the brokers never share a log dir.
A client in the same project bootstraps with all three names —
kafka-1.kafka.kanea:9092,kafka-2.kafka.kanea:9092,kafka-3.kafka.kanea:9092;
a service in another project also needs allow_from naming it on each
broker (R14). The cluster is deliberately not exposed north-south: Kafka's protocol
hands clients the advertised names to dial, and those resolve only inside the node —
publishing :9092 would let an external client connect once and then fail on
redirect. Generate your own CLUSTER_ID
(kafka-storage.sh random-uuid) rather than shipping the example's.
kanea run kafka.hcl --wait=120s kanea ps --project=kafka kanea logs kafka/kafka-1 --tail=50 # watch the quorum form
CLI reference
One binary, one command tree. The CLI talks to kanead over a unix socket
— --socket overrides it on every client command — and the daemon commands
are the ones systemd runs for you.
The socket is root-owned, so client commands run under sudo — or without
it, after joining the kanea group init creates and logging in
again: sudo usermod -aG kanea <user>. Membership is root-equivalent,
exactly like docker's group, and is never granted by Kanea itself.
Services are addressed as project/service throughout, or as a bare service
name with --project.
The CLI also installs through Homebrew, on Linux and on
macOS. A Mac gets the authoring half: kanea plan parses and validates a
job spec with file-and-line diagnostics before anything dials a socket, so no daemon
is needed — the platform itself, and everything above that needs the socket, runs on
Linux.
Setup
kanea init
Interactive first install: preflight checks, configuration, the master-key ceremony,
the systemd units — and, since v0.5, the rest of the way to a working platform: it
asks for the dashboard's listen address (loopback by default; none keeps
the API socket-only), starts kanead, creates the first admin account over
the local socket, and ends with a summary of what it built — the dashboard URL, your
account, the internal DNS address and the subnet layout. Run it once, as root.
sudo kanea init [--data-dir=/var/lib/kanea] [--log-dir=…] [--unit-dir=/etc/systemd/system]
[--network=ebpf|netns] [--containerd=…] [--node-cidr=…] [--cluster-cidr=…]
[--listen=127.0.0.1:8600] [--listen-cert=…] [--listen-key=…]
[--admin-user=…] [--no-start] [--skip-checks] [--skip-units]
A non-loopback --listen requires the TLS pair and is refused up front
otherwise — the same refusal the daemon would make, moved in front of you.
--admin-user plus a piped password makes it scriptable;
--no-start writes the files and stops, which is the pre-v0.5 behaviour.
Re-running is safe: an existing master key and an existing account are left alone.
Since v1.61 the listener can live in the server config instead: a
bind { api_addr = … api_tls = … } stanza in
/etc/kanea/kanea.hcl, where api_tls is
acme, self-signed, provided or
plaintext — the same modes services use (the full field set is in
Exposing the API and dashboard). When it is
declared and --listen was not passed, init skips the listen question
and renders no listen flags into the unit — the file owns the listener, and moving
the API and dashboard later is an edit to the file plus
systemctl restart kanead, never a re-init.
The ceremony prints the master key and requires you to type it back; if that fails it is discarded and nothing is written. Without that key every backup this node ever makes is unreadable. Have somewhere to record it before you start.
kanea doctor
Verifies the node: dependencies and their versions, the containerd socket, bpffs and the cgroup2 mount, kernel version, cgroups v2 and slice placement, the effective memory floor, the build socket, disk headroom and clock synchronisation. Safe to run any time.
kanea upgrade
kanea upgrade [--check] [--version vX.Y.Z] [--no-fetch] [--skip-backup] [--dry-run] [--timeout=2m]
One command, both halves: it fetches the latest release (or --version),
verifies it — sha256 against the release's checksums.txt always, the
cosign keyless signature over that file when cosign is installed, a loud note when
it is not — installs it atomically over its own path, then takes a pre-upgrade
backup, restarts kanea-edge and then kanead in that order,
runs any state migrations, and waits for health. Already at the target version means
nothing to download, so running it twice is safe by construction.
--no-fetch restarts onto whatever binary is already installed —
the orchestration half alone — and --check only reports the running,
installed and latest versions. Air-gapped nodes get their binary from the
offline bundle flow; the fetch refuses with that pointer rather than hanging.
One thing it deliberately never does: rewrite systemd units. When release notes
say the units changed, re-run sudo kanea init after upgrading —
idempotent: the master key, accounts and settings are kept — then
sudo systemctl daemon-reload.
Deploying
kanea plan
kanea plan app.hcl [more.hcl …] [selector …] kanea plan --image=nginx:1.27-alpine --name=web --project=demo [--count=1]
A real dry run: the create/change/destroy diff, the resulting workload budget, and every validation rule. Multiple files are parsed as one set, so file order does not matter. This is where you find out about a cycle, a cross-project secret, a duplicate domain or a rate limit that would fail open.
kanea run
kanea run app.hcl [more.hcl …] [selector …] [--wait=60s] kanea run --image=nginx:1.27-alpine --name=web --project=demo [--count=1]
Applies the spec. kanea apply is an alias — same flags, same
behaviour. --wait is how long to wait for allocs to reach running
before returning; 0 returns immediately. Running it twice with an
unchanged spec does nothing — a deploy is a spec-hash mismatch, not an invocation.
A selector scopes both commands to part of the file: kanea run
app.hcl shop/web applies one service, shop alone a whole
project, and several selectors union. An argument that exists on disk is a spec
file; only a non-existent one is read as a selector, and one that is neither is
refused by name. The whole file is still parsed and validated — a selector never
changes what a spec means, only how much of it is sent — and every selector must
match at least one service in the file. An apply is additive either way: services
not in the request are never touched, so a scoped run cannot delete anything.
kanea stop
kanea stop [--project=p] <[project/]service> [--rm]
Scales to zero. --rm also deletes the service declaration, so the
reconciler stops holding desired state for it.
kanea start
kanea start [--project=p] <[project/]service> [count]
stop's counterpart: scales a stopped service back up. The daemon does
not remember the pre-stop count — a stopped record says zero — so it starts one
replica unless a count is given, or an autoscaled service's own floor (the scale
route refuses a count outside the declared bounds). A service already running is
left exactly as it is: start is idempotent, never a second spelling
of scale.
kanea restart
kanea restart [--project=p] <[project/]service>
Rolls the service's allocs through its update policy — a generation bump, the same route the dashboard uses, not a second path into the runtime. It is also the way out of an exhausted crash-restart budget: the bump is a new spec hash, and the restart count belongs to the hash that spent it.
Inspecting
kanea ps
kanea ps [--project=p] [--service=s] [-a]
The alloc table: id, service, state, health, restarts, age, address.
A removed alloc leaves no record (only failed-and-still-declared ones
persist to explain themselves), so a stopped service is invisible here —
-a adds what is declared but not running: services scaled to
zero (stopped) and slots the reconciler has not created yet
(pending).
kanea describe
kanea describe [--project=p] <[project/]service>
One service in full: the declared spec beside what is actually true —
image and its pinned digest under auto-update, routes from every
expose block and published port, volumes and grants, the
alloc table with health verdicts, a stats snapshot, and the service's
recent events. Stats and health render absent as absent (-),
never as zero: a missing metric and an idle service are different facts.
kanea status
kanea status [--project=p] [[project/]service]
Health, recent events, current and desired counts, and the scaling picture.
kanea logs
kanea logs [--project=p] <[project/]service> [-f] [--tail=N] [--alloc=ID]
Merged across allocs by default; --alloc narrows to one.
--tail shows the last N lines before following. Log drains are
non-blocking with drop counters — a slow reader can never stall a workload's
write().
kanea exec
kanea exec [--project=p] [--alloc=ID] [--user=UID] [-it] <[project/]service> -- <command…>
A debug shell inside an alloc. Admin-only, and audited whether or not the session establishes — "someone tried to open a shell on production" is worth keeping either way, so the attempt and the requested command are both recorded.
- The
--is required. The command crosses the wire as separate arguments rather than one joined string, because every rule for splitting a string back into arguments is wrong for something somebody will eventually pass. -itallocates a terminal and forwards stdin. A shell needs it.--usertakes a numeric uid only. Resolving a name would mean reading the container's own/etc/passwd, and a container-controlled file deciding which uid the control plane runs a process as is not a thing to build.
kanea ui
kanea ui [--addr=…] [--open]
Prints the dashboard URL; --open launches a browser.
Scaling and builds
kanea scale
kanea scale [--project=p] <[project/]service> <count>
Writes the desired count and returns; the reconciler converges. This is the same route the autoscaler uses, which is why manual and automatic scaling can never disagree about mechanism.
kanea build
kanea build [--project=p] <[project/]service> [--deploy=true] [--follow=true]
Triggers the service's build pipeline. --deploy rolls the built digest out
on success; --follow streams the build log. Builds are serialised — a
second one is queued, and refused rather than blocked when the queue is
full.
kanea project
kanea project sync <project> # re-read the git source now kanea project builds <project> [--service=s] [--limit=N]
Secrets and accounts
kanea secret
kanea secret put [--from-file=path] <project>/<name> # value on stdin kanea secret ls [<project>] kanea secret rm <project>/<name>
Not for an operator, not over the API, and not for an AI agent at any tool tier.
ls lists names. The value goes in and is only ever resolved into a
running alloc.
kanea user and kanea token
kanea user add [--role=admin|viewer] <name> kanea user ls kanea user rm <name> kanea token create [--role=viewer] [--expires-in=720h] <name> kanea token ls kanea token rm <id>
Accounts live in the Store, not in a config file. Tokens default to never expiring;
--expires-in takes a Go duration. The first admin is created by
kanea init itself; OIDC and LDAP identities never appear in
user ls — they are ephemeral, a session and nothing else.
Backup and restore
kanea backup create [--reason="on-demand"]
kanea backup list
kanea backup verify <archive-id>
kanea restore --from s3://bucket/prefix [--snapshot=ID] [--target=path]
[--s3-endpoint=…] [--s3-region=…] [--s3-access-key=…] [--s3-path-style]
[--master-key=path] [--data-dir=…]
verify reads the archive and checks its hashes and authentication tags
without restoring anything — an archive that cannot be verified is one you find out
about now rather than during an outage.
The command stages the restore; it is performed at the next daemon start, before anything opens the Store. That is the interface rather than a safety check — the API has no method that restores at all, and there is no restore button in the dashboard, because a restore replaces everything on the node and belongs at a terminal.
Daemons
Normally systemd runs these. The flags are here because systemctl cat will show them to you.
kanea agent
kanea agent [--config=/etc/kanea/kanea.hcl] [--data-dir=…] [--log-dir=…]
[--volume-dir=…] [--socket=…]
[--containerd=…] [--network=ebpf|netns] [--node-cidr=…] [--cluster-cidr=…]
[--service-cidr=…] [--bpf-dir=…] [--dns-listen=…]
[--allowed-host-paths=…] [--passthrough-config=…]
[--listen=…] [--listen-cert=…] [--listen-key=…] [--dashboard=true]
[--autoscale=true] [--log-level=info]
[--backup-s3-endpoint=…] [--backup-s3-region=…] [--backup-s3-access-key=…]
[--oidc-client-id=…] [--ldap-url=…] [--acme-dns-tsig-key=…]
--listen beyond loopback requires --listen-cert and
--listen-key. Unset, the listener comes from the server config's
bind stanza when one is declared (below); --listen none
forces socket-only regardless of the file. Credential-shaped options such as
--oidc-client-secret, --ldap-bind-password and
--acme-dns-tsig-secret take secret: references, never
literals.
--ldap-url enables directory logins beside local accounts and OIDC:
ldaps:// (or ldap:// with StartTLS forced — there is no
insecure option), a user search under --ldap-user-base-dn with
--ldap-user-filter, and group-to-role mapping through
--ldap-admin-groups/--ldap-viewer-groups, deny-by-default.
A local account with the same name always wins, and the login rate limit runs before
any bind reaches the directory.
The operator-owned settings live in the server config
/etc/kanea/kanea.hcl: which directories host volumes may
come from (R15, a storage { allowed_host_paths = […] } stanza), which
devices and sockets are granted to projects (R17, R18,
device/socket blocks), and — since v1.61 — where the
API and dashboard listen (a bind stanza: api_addr, with
api_tls choosing among acme — a Let's Encrypt certificate
for api_domain, issued and renewed by the same pass that serves your
services; self-signed — the node CA, with a real IP SAN when the
address is bare; provided — your own
api_cert/api_key pair; and plaintext —
explicit HTTP, allowed beyond loopback because it was typed and logged loudly;
the full field set is in Exposing the API and
dashboard).
The file is probed once at
startup when it exists; absent means every setting's zero value. It must be a
regular file, owned by root (or the daemon's uid), and writable
only by its owner — anything else refuses startup, as does a file that does not
parse. Stanzas from the fuller PRD §15.1 sketch that this version does not read are
warned by name at startup, never silently ignored.
--allowed-host-paths, --passthrough-config and
--listen remain as
explicit overrides: setting one means the corresponding half of the file is not
consulted (--passthrough-config names a separate grants file with the
same trust check), off on the first two — or --listen none —
disables that half regardless of the
file, and --config moves or (--config off) disables the
file itself. Either way this is deliberately node configuration — a job spec can
reference what it permits and can never add to it.
kanea edge
kanea edge [--routes=/run/kanea-edge/routes.json] [--certs=…] [--http=:80]
[--poll=…] [--drain=…] [--memory-limit=128MiB] [--log-level=info]
Its own process and its own systemd unit, with no After=kanead.service —
north-south traffic surviving a control-plane restart is the entire reason it is
separate. --drain is how long in-flight requests get on shutdown.
MCP
kanea mcp [--verbose]
A stdio MCP server for a local AI agent — 20 tools in read, mutate and destructive tiers. The same server is available over streamable HTTP on the API listener for remote agents.
Tools reach the platform only by making requests against the API's own handler, so an agent is never more privileged than the credential it was given. Tiers are advertised as well as enforced, and the advertisement fails closed. A refusal comes back as a tool result rather than a protocol error, because the model is what has to react to it.
kanea version
Prints the version stamped in at build time. kanea upgrade compares it against what the running daemon reports.
Troubleshooting
Where to look when something is wrong, in the order that usually finds it: the workload first, then the daemons underneath it, then the node. Everything in this section is read-only and safe to run on a live node.
Checking services
Start with what Kanea believes is true:
kanea ps -a # every alloc — including stopped services and pending slots kanea status shop/web # health, recent events, current vs desired counts kanea describe shop/web # the full picture: spec, routes, volumes, allocs, stats, events
Three columns in ps carry most of the signal. State:
pending means the reconciler has not created the slot yet — usually an
image still pulling, or a dependency that is not healthy. Health: a
- means the service declares no health check, which is a different fact
than failing one — a check-free service is never reported unhealthy, only running or
not. Restarts: a climbing count is a crash loop; when it stops climbing the
restart budget is exhausted and the alloc has been failed and left alone
(see below for the way out).
Then the processes underneath. A standard install runs four units:
systemctl status kanead kanea-edge kanea-containerd kanea-buildkit
kanead— the control plane. Workloads and north-south traffic both survive it being down; what stops is change: deploys, scaling, certificate renewal, the API and dashboard.kanea-edge— the ingress proxy. Deliberately independent ofkanead(noAfter=in either direction): it serves the last route snapshot it read from disk whether or not the control plane is up.kanea-containerd— Kanea's own containerd, socket at/run/kanea/containerd.sock. Restarting it does not stop running containers (KillMode=process— shims outlive it), but nothing can be created or probed while it is down. Absent when the node adopted an existing daemon with--containerd external.kanea-buildkit— the rootless build daemon. Only builds need it.
Finally the node itself: kanea doctor verifies dependencies and their
pinned versions, the containerd socket, bpffs and the cgroup2 mount, slice placement
and the effective memory floor, the build socket, disk headroom and clock sync — and
it names known interference, like a firewall FORWARD-drop policy (docker, ufw) eating
east-west traffic. Safe to run any time.
Viewing logs
Workload logs stream through the CLI:
kanea logs shop/web -f # merged across allocs, follow kanea logs shop/web --tail=200 # the last 200 lines first kanea logs shop/web --alloc=<id> # one alloc only (ids from kanea ps)
On disk they are one file per alloc under /var/log/kanea/allocs/
(<alloc-id>.log); a replaced alloc starts a fresh file under its new
id. Drains are non-blocking with drop counters, so a burst of logging can be dropped
but can never stall the workload's write() — if lines are missing under
load, that is the drop counter doing its job, not a lost file.
Daemon logs go to stderr, which under systemd means the journal:
journalctl -u kanead -e # control plane: reconciler, deploys, certificates, backups journalctl -u kanea-edge -e # ingress: TLS, routing, published ports journalctl -u kanea-containerd -e # runtime: image pulls, task create failures journalctl -u kanea-buildkit -e # the build daemon journalctl -u kanead -f --since "15 min ago"
A deploy that goes wrong is usually legible in kanead's journal; a task
that will not create at all (a missing shim, a device the node did not grant) often
explains itself one level down in kanea-containerd's.
Build logs are their own stream: kanea build shop/web --follow
live, kanea project builds shop for history, and one file per run under
/var/lib/kanea/builds/.
Slices and resources
Everything Kanea runs sits in one of two cgroup slices, and that split is the
resource-isolation story (architecture):
kanea.slice holds the control plane — all four units above — with a
kernel-guaranteed memory floor (MemoryMin, default 256 MiB —
raise it with kanea init --reserve on a node that runs builds), and
kanea-workloads.slice holds every alloc under a collective
ceiling of total RAM minus that reserve.
systemctl status kanea.slice # the control plane, with its live memory number systemctl status kanea-workloads.slice # every alloc as a child cgroup systemd-cgls kanea-workloads.slice # the tree, one cgroup per alloc systemd-cgtop # live CPU/memory per slice
The ceiling is computed and applied by kanead at startup — it depends on
how much memory the node has, which a unit file cannot know — so read it from the
cgroup filesystem, which is the ground truth either way:
cat /sys/fs/cgroup/kanea.slice/memory.min # the floor cat /sys/fs/cgroup/kanea-workloads.slice/memory.max # the ceiling cat /sys/fs/cgroup/kanea-workloads.slice/memory.current
Per-alloc cgroups live one level down
(/sys/fs/cgroup/kanea-workloads.slice/<alloc>/) with their own
memory.max, memory.current and cpu.max. A
declared resources limit is enforced exactly there; an omitted one reads
as max — unbounded within the collective ceiling, by design, never a
filled-in default.
The floor and the OOM score adjustments live in the unit files, not in the Go code
— a Kanea started outside its units runs without the guarantee, and the first time
the node is under memory pressure the kernel picks whatever is largest, which is
usually kanead. kanea doctor checks slice placement and
the effective floor for exactly this reason; journalctl -k | grep -i
oom shows what the kernel actually chose.
Common situations
-
A service is crash-looping.
kanea logsfor why. Once therestartbudget is exhausted the alloc is failed and left alone on purpose —kanea restart shop/webclears it, because the restart count belongs to the spec hash that spent it and a restart is a new one. So does deploying a fix. -
A deploy did nothing. A deploy is a spec-hash mismatch, not an invocation:
re-running an unchanged spec is a no-op by design.
kanea planshows the diff the daemon would see — an empty one means the spec really is what is running. -
The site is unreachable but the service is healthy. Work outward:
kanea describeshows the routes the service declares, thensystemctl status kanea-edge. The edge reads its world from two files —/run/kanea-edge/routes.jsonand its certificate bundle — so a route missing from that snapshot is a publishing problem inkanead's journal, and a route present in it is a proxying problem in the edge's. -
Browsers show a certificate error. Deliberate fail-closed behaviour, not
breakage: a domain whose certificate is not issued yet refuses the handshake, and a
providedcertificate that stops resolving serves plaintext — neither ever silently falls back to a weaker certificate.kanead's journal has the ACME or resolution failure. -
Containers cannot reach each other.
kanea doctorfirst: a foreign FORWARD-drop firewall policy (docker, ufw) is a finding it names. Then remember policy is deny-by-default — a cross-project call needsallow_fromon the callee. -
The
bindstanza is ignored — the dashboard stays on localhost. An explicit--listenalways beats the file, and an init run from before the stanza existed rendered--listen 127.0.0.1:8600into the kanead unit.systemctl cat kanead | grep -- --listenconfirms it; remove the flag (and--listen-cert/--listen-keyif present) fromExecStart, thensystemctl daemon-reloadand restart. A re-run ofkanea initwill not put it back — withbind.api_addrdeclared it renders no listen flags at all. -
The autoscaler stopped scaling. Its circuit breaker trips on purpose after
repeated failed actions, and says so in the journal, on the dashboard, and as
kanea_circuit_breaker_openin/v1/metrics. The trip survives a daemon restart by design — restartingkaneadis not a way around it; fixing the cause is.kanea scalestill works meanwhile: the breaker pauses the automatic decisions, not the route they travel.