Concepts

This page explains the ideas the rest of the broker’s documentation leans on. Each section names the code that implements it, so you can check the page against the source. A BROKER_* name is one of the broker’s settings; the configuration reference, generated from the loader, gives each one’s default and accepted values.

Sessions and enrollment

An agent session is one running agent and everything it starts. On a machine, it is the process agent-secrets register --exec starts (a coding agent, or a shell) and every process below it; in a container or a Kubernetes pod, it is whatever holds the session’s key directory. The broker never deals with a person’s or a machine’s long-lived key on the agent’s behalf. Instead, every session has a P-256 signing key of its own that lives only as long as the session does, and an enrollment that binds that key to the broker.

Every call a session makes carries a Proof header: a JSON Web Signature over the call’s method and URL, signed with the session’s key, used once, and valid only within BROKER_PROOF_SKEW_SECONDS of the broker’s clock. There is no bearer token for a session to leak.

A launcher is the program that enrolls sessions: agent-secrets-helper, a per-user daemon on a machine, for that machine’s sessions, or the Legion daemon for the pods it runs. An enrollment has one of two kinds:

Kind	What it is	Who holds its key	Who enrolls it
`session`	An agent session on a machine a person logged in: running directly on the machine, or in a container on it (a box).	For a session on the machine, the machine’s helper, in memory; for a box, the container, in `key.pem` under `AGENT_SECRETS_KEY_DIR`.	The machine’s helper: when the session registers with `agent-secrets register`, or, for a box, when whoever starts the container runs `agent-secrets enroll --helper` (run an agent in a container).
`pod`	A Legion worker pod in Kubernetes.	The pod, in its key directory.	The Legion daemon, which proves the pod with a projected service-account token the broker verifies against `BROKER_K8S_OIDC_ISSUER`.

An enrollment is leased for BROKER_LEASE_SECONDS and renewed while its session runs: the helper renews the sessions it registered, agent-secrets renew renews a box, and Legion’s pod shim renews a pod. A pod’s renewal carries the pod’s current projected token, and the broker refuses it (POD_IDENTITY_MISMATCH) unless the token names the pod UID and service account the enrollment was made for. The token is not yet required. The broker ratchets each pod UID instead: once any enrollment naming a pod UID, live or ended, has renewed with that pod’s token, the broker refuses a renewal without one (POD_TOKEN_REQUIRED) from every enrollment naming that pod, whichever machine login made it. So the copied-token gap closes for every pod UID a current-image pod has renewed under: no enrollment naming that pod renews without its token, and a token copied out of the pod stops renewing anything once it expires. It stays open for the UID of a pod on an older worker image, whose agent-secrets renew never sends a token: such a pod renews as it did before, the broker logging each such renewal, until the release that requires the token. One window is left even for a current-image pod: a copied token used to renew before the pod’s own first renewal with its token. That is seconds while the Legion daemon holds an approved machine login, but after a daemon restart its pods enroll only once a person approves the daemon’s new login, and a pod that ends before it enrolls never renews with its token, so its UID stays open as an old-image pod’s does. A grant lasts until its session ends, so a copy that keeps renewing keeps the grant one approval gave it. An enrollment has ended once its launcher revokes it (the helper does as soon as a registered session’s process exits), whoever may revoke its machine login revokes that login (end a machine’s login), or its lease lapses. A pod enrollment’s launcher is the one machine login that made it: another login, even another login of the same service, such as the Legion daemon’s after a restart or after its login reached its expiry and it logged in again, is refused NOT_ENROLLING_CREDENTIAL, and that pod’s enrollment ends with its lease or with that machine login. From the moment the lease lapses, its next renewal is refused ENROLLMENT_NOT_LIVE; other session calls remain PROOF_INVALID. The broker’s sweep, which runs every BROKER_SWEEP_SECONDS, then ends the enrollment on its first run after the lapse. Ending an enrollment any of these ways revokes every grant it held and cancels every request it still had pending, so those leave the approver’s Inbox and an approval can never land on a session that is gone (packages/envoy/internal/broker/enroll/enroll.go, endEnrollment).

Every session enrollment records an operator: the person whose machine it runs on. The operator is the person who approved the machine login that enrolled it, never a value the launcher chooses. A pod enrollment records none, since the Legion daemon logs in as a service with its own Kubernetes token rather than as a person; it records the service account the pod’s projected token proved instead.

A launcher (a person’s machine helper, or the Legion daemon) cannot enroll anything until the broker has issued it a launcher credential: a credential bound to a key the launcher holds, never a token, which the launcher uses by signing with that key. A person’s machine gets one when that person approves a machine login, which works like a device code:

The machine generates a fresh key and asks the broker to log in (agent-secrets machine login asks the helper to), naming the person who should approve it (the helper reads that person’s Dispatch login from AGENT_SECRETS_OPERATOR_FILE).
The broker answers with an eight-character confirmation code, XXXX-XXXX, which the machine prints.
The person types the code into Dispatch, sees which machine is asking, and approves or denies it. Only the typed code selects a machine login: no link can approve one, and anyone else is refused NOT_APPROVER, since the machine’s sessions act as that person.
On approval the broker mints the launcher credential bound to the machine’s key.

A service, such as the Legion daemon, logs in with its own projected Kubernetes ServiceAccount token instead, and no person approves it: the broker verifies the token as it verifies a pod’s, refuses one that lives longer than an hour from its issue, and issues the credential when the token’s account is one BROKER_SERVICE_LOGINS names for that service (Operate). Each such account holds one credential, which every login moves to the login’s key and to the token’s expiry, at most an hour away, so the service logs in again before then and at every restart; a running daemon proves the same key at each login, and a restart brings a new one. A device-code login that names a service is refused SERVICE_LOGS_IN_WITH_TOKEN.

A person’s launcher credential lasts BROKER_LAUNCHER_CREDENTIAL_SECONDS. It may be revoked from Dispatch’s machine-login page by the person who approved it and no one else, or from any of their machines under its machine login (agent-secrets machine revoke), before or after it expires, which also ends every session it enrolled (end a machine’s login). Its sessions outlive its expiry: a session renews its lease with its own key, never with the machine’s credential, so a box keeps working after the machine’s credential expires, and the page lists an expired login, marked as expired with sessions still running, until its last session ends. Whoever approved a person’s machine login is its operator. The helper keeps its key in memory only, so a helper restart, like an expired or revoked credential, means logging the machine in again. A machine login nobody decides expires after 15 minutes, a fixed time rather than a setting (machineLoginPendingTTL in packages/envoy/cmd/broker/main.go). A service’s login has no operator, so the machine-login page lists it for everyone signed in, and anyone may revoke it, which ends every pod it enrolled at once: a service login a person approved through a code before services logged in with their tokens, until it expires, and a service’s token login, which names no approver. A service whose token login was revoked logs in again with its token at its next call, under a new credential, which enrolls the pods it launches next; a pod the revoke ended stays without secrets until it is replaced. To stop it logging in, remove its account from BROKER_SERVICE_LOGINS and redeploy the broker.

A credential enrolls sessions only for its own operator: an enrollment naming anyone else is refused OPERATOR_MISMATCH. The broker’s rate limiter caps how often anyone can start a machine login, per source address and per named person, and how often anyone can try a token login: per source address before the token is verified, and per account once it is.

Owner and tier: who may have which secret

The broker reads who may have a secret from the secret itself (packages/envoy/internal/broker/policy/). Every agent secret is a secret in the broker’s secret store (BROKER_SECRETS_STORE: AWS Secrets Manager, or Google Cloud Secret Manager) under one namespace, BROKER_SECRETS_PREFIX (such as production/agent-secrets/, or projects/<project number>/secrets/agent-secrets-), encrypted with one agent-secrets key (BROKER_SECRETS_KMS_KEY_ARN in AWS KMS, or BROKER_SECRETS_KMS_KEY_NAME in Cloud KMS), and marked with its owner and its tier, as AWS tags or as Secret Manager annotations:

Its name under the prefix is the name a session asks for, in lowercase with each underscore a hyphen: production/agent-secrets/deel-api-key, or projects/<number>/secrets/agent-secrets-deel-api-key, is DEEL_API_KEY. It is lowercase letters, digits and single hyphens, starting with a letter.
owner is shared, a person’s email in lowercase, the email they sign in to Dispatch with, or a service the broker is configured with (BROKER_SERVICES).
tier is agent or human.

In Secret Manager the two are annotations rather than labels because a label’s value cannot hold an email (@ and . are outside its alphabet), and the broker compares the owner with a Dispatch login byte for byte; an annotation holds the email as it is.

People write the secrets and their tags, not the broker: agent-secrets secret creates and changes them under the person’s own cloud sign-in, both stores (manage a secret).

Who gets a secret follows from those two tags alone:

The secret	Its owner’s own session	Any other session or pod
A person’s, `tier=agent`	Granted at once.	Sent to the owner for approval.
A person’s, `tier=human`	Sent to the owner for approval.	Sent to the owner for approval.
`owner=shared`, `tier=agent`	Granted at once.	Granted at once.
`owner=shared`, `tier=human`	Approved by anyone signed in to Dispatch.	Approved by anyone signed in to Dispatch.
A service’s, `tier=agent`	Granted at once.	Denied; no one can approve it.

A session is its owner’s own when its operator is the owner: the owner approved the machine login it enrolled under. A pod has no operator, so a pod asking for a person’s agent-tier secret sends it to that person for approval.

An owner may also be a service the broker is configured with. BROKER_SERVICES binds each service’s name to the Kubernetes service account its pods run as, for example legion-daemon=system:serviceaccount:legion:legion-worker. A service’s own sessions are the pods that a launcher logged in as that service enrolled, and whose projected token proved that service account: the Legion daemon’s worker pods, for legion-daemon. A launcher logs in as a service with its own token, for an account BROKER_SERVICE_LOGINS names; the pod’s own token proving the pods’ account is what makes a pod the service’s.

A service’s secrets go to its own sessions and to no one else. That includes a person’s own session, and a pod run under another service account even when a launcher logged in as the service enrolled it. A service’s secret is agent tier, since no person approves it: the broker refuses one tagged tier=human, and one whose owner names a service it is not configured with.

One thing besides the tags changes what a session gets at once: a session’s own person, its operator, who revokes a grant the session got without asking withholds its secrets from that session, even when the grant had already ended by then (packages/envoy/internal/broker/requests/machine_state.go, RevokeByApprover). Each withheld secret is human tier to that session for as long as the session lives: every other grant the session got it on without asking ends with the revoked one, and the session’s requests for it are sent to the owner for approval, or to anyone signed in to Dispatch for a shared secret, so the session asks before it gets the secret again. Only the session’s own person withholds: anyone else who revokes one of its grants, the person who approved it, ends that grant alone. Other sessions, the person’s own included, still get the secrets at once.

The broker reads the namespace when it starts, and refuses to start when it cannot, then again every five minutes, a fixed time rather than a setting (policyRefresh in packages/envoy/cmd/broker/main.go). A change is served at once when something asks the broker to reread it: POST /v1/secrets/{name}/reread, which agent-secrets secret sends after each write it makes and any caller may send right after writing a secret, and a request naming a secret the broker does not serve yet, which makes it reread that one name before refusing it (up to ten such rereads at once per session, refilled one every ten seconds: DefaultMissRereads in packages/envoy/internal/broker/requests/machine_state.go), so a secret created a moment ago is served on its first request. A reread reads the one secret from the store (on AWS, DescribeSecret, which shows a write at once; on Google Secret Manager, GetSecret, which Google documents as eventually consistent, so a reread made right after a write can still find the old state, usually for seconds) and serves it, or stops serving it, as it finds it; a read of the whole namespace for as long as the store’s listing may lag after (five minutes on AWS Secrets Manager, six hours on Google Secret Manager, whose metadata Google documents as eventually consistent for up to a few hours) reads that secret again alone, so a listing that has not caught up never undoes it (Current.Refresh in packages/envoy/internal/broker/policy/current.go). A reread that finds a secret as the broker already serves it, or no secret under a name the broker was not serving, changed nothing, and is not read again — which is the one case that promise does not cover: a secret created and then deleted before any read served it is anchored by nothing, so the next read of the namespace can serve it from a listing that shows the creation and not the deletion, for up to that lag after the delete, and a request for it is granted while a grant’s read of its value is refused SECRET_NOT_IN_STORE. A change nothing rereads, such as a tag edited in the AWS console, takes effect within about ten minutes on AWS: at the next read of the namespace, or the one after, since Secrets Manager’s listing can lag a change by up to five minutes. On Google Secret Manager it usually takes effect at the next read of the namespace too, but Google bounds its listing’s lag only at a few hours, so a change made any other way than agent-secrets secret (gcloud, the console) should be followed by a reread. A read that fails (Secrets Manager or KMS out of reach) is logged, and the policy from the last good read stays in force. The broker leaves out a secret it cannot serve, and logs it by name on every read (Operating the broker): a missing or malformed owner or tier tag (an email with a capital letter is malformed), a name that is not in the form above, a secret encrypted with any key but the agent-secrets key, the AWS-managed key included, or a secret with no current value (created without one, so none of its versions carries the AWSCURRENT label). A request for a secret left out is refused UNKNOWN_SECRET before anyone is asked to approve it, and a live grant of it releases nothing until the broker serves the secret again.

The policy’s version is the SHA-256 of every served secret’s name, owner, tier and ARN, recorded on every request.

Requests and grants

A session asks for one or more secrets by name in a request, signed with its key and carrying a reason of at most 400 characters. The broker evaluates every name (packages/envoy/internal/broker/requests/machine_state.go):

A name the policy does not serve, even after the reread above, refuses the whole request with UNKNOWN_SECRET, and nothing is recorded.
A name the policy denies denies the whole request; nothing is ever half-granted. Only a service’s secret, asked for by anyone but that service, is denied.
Names that need approval must all need the same approver, else the request is refused MIXED_APPROVERS; request them separately.
When every name is automatic, the request is granted at once.
Otherwise it is pending, and the broker writes a credential-request record for the approver to decide. A session has at most one pending request for each secret: an identical request from the same session while one is pending (the same names and the same lifetime, or none) joins it rather than asking twice, and any other request naming one of its secrets, with or without --ttl, is refused REQUEST_PENDING, naming the pending request. The session waits for that request to be decided, or cancels it (agent-secrets cancel) and asks again. The broker’s answer names the approver the record names, and agent-secrets prints it, so the agent can tell that person. The record fixes its approver when the request is made, and the request stays in that person’s Inbox after the secret’s tags change, though they may then be unable to approve it (what an owner change does).

A granted request yields a grant: the session’s right to read those values until the grant ends. A grant lasts until its session ends, unless the request asks for a shorter lifetime (agent-secrets NAME --ttl 2m -- command, or agent-secrets request NAME --ttl 2m), which it then lasts counted from the moment it is granted. There is no other limit. The approver sees the lifetime the request asked for before deciding it. While a grant lives, a new request from the same session for exactly the same names and the same lifetime gets the same grant back, without asking anyone again, as long as the current policy still allows it.

The broker never stores a value. Each time a session reads a grant, the broker checks that the session is still enrolled, the grant is live, its approval still verifies, and the current policy still allows every name; then it reads the value from the secret store (AWS Secrets Manager in production) and returns it to that session alone. agent-secrets NAME -- command puts each value in the command’s environment under its name and replaces itself with the command. agent-secrets itself never prints a value, but the command is the agent’s choice, and a command can print or send what its environment holds (printenv NAME prints it). A session holding a grant can therefore read the value: approve a secret only for a session you would trust with the value itself.

A grant ends when its lifetime runs out, when its session revokes it (agent-secrets revoke), when its approver or its enrollment’s operator revokes it in Dispatch or from one of their machines (agent-secrets grant revoke), or when its enrollment ends. A grant the session got without asking is revoked like any other, and when its operator revokes it the session asks for approval before it gets the same secrets again (the operator withholds them, even when the grant had already ended); a session revoking its own grant, and anyone but the operator revoking one, withholds nothing. To end every session’s automatic access to a secret, change its tags (revoke a session or a grant).

Approvals

An approval happens in Dispatch, never in the broker. The broker holds no Dispatch credential and sends Dispatch nothing: Dispatch’s server reads the broker’s pending list and records for the person signed in, and when that person clicks Approve or Deny, Dispatch’s server calls the broker with the shared UI token (BROKER_UI_TOKEN) and that person’s email, taken from their Dispatch session. The broker then checks that email against the record’s approver: anyone else is refused NOT_APPROVER, whatever state the record is in. A request for a shared human-tier secret, or for a shared agent-tier secret its session’s operator withheld from it, names the approver anyone: it waits in every person’s Inbox, any person signed in to Dispatch may decide it, and the broker records the email of whoever did. A withheld secret is its owner’s to approve for that session even in a request that was already waiting when the operator withheld it: a request that waited on anyone for a shared human-tier secret and the operator’s own withheld secret is then the operator’s alone to approve, and anyone else’s approval is refused NOT_APPROVER. While the broker does not serve the withheld secret to that session, no one may approve such a request: not while the secret is out of the broker’s policy (deleted, or its tags refused), since its owner cannot be read, nor while it is a service’s, since no person approves a service’s secret. It waits until the secret is a person’s or shared again, or expires. The approval it gets lasts as any approval does (below): a change to another secret’s tags does not end it, nor does the operator’s later revoke of another grant, unless that revoke withholds another secret the request got without asking.

An approval belongs to the person who gave it, so a change to a secret’s tags reaches what was approved before it wherever the new tags want the secret approved for that session:

An approved grant keeps releasing its value only while the person who approved it may still approve the secret for its session. Once a shared human-tier secret becomes a person’s, or a person’s secret becomes another’s, every such grant the new owner must approve and did not stops with GRANT_NOT_LIVE, whatever value the secret holds by then, and the session’s next request is decided under the new tags. A grant keeps working when the secret becomes shared (anyone may approve a shared human-tier secret, and a shared agent-tier one needs no approval), and when the new tags give its session the secret without asking, as they do once an agent-tier secret is the session’s operator’s. A secret the operator withheld from the session stays human tier there under any tags, so a grant of it someone else approved stops once the secret is the operator’s.
A pending request is approved by whomever the new tags name to approve it for its session. One waiting on anyone for a secret that has become a person’s is that person’s alone to approve; anyone else’s approval is refused NOT_APPROVER, the requester’s own operator included. One waiting on a person for a secret another person must now approve can be approved by no one. One waiting on a person stays that person’s when the secret becomes shared, or when the new tags give its session the secret without asking. A denial releases nothing, so the approver a request waits on can still deny it, which takes it off the pending list: for a request waiting on anyone, any signed-in person may deny it. Otherwise it expires, or its session cancels it (agent-secrets cancel) and asks again.

A record is decided once. A second click, a concurrent one, or one after the record expired gets RECORD_TERMINAL. A pending request nobody decides expires after 12 hours, a fixed time rather than a setting (agentSecretPendingTTL in packages/envoy/cmd/broker/main.go). Because the UI token vouches for whoever Dispatch says is approving, it is an approval credential: only Dispatch’s server may hold it.

The audit record

Everything a person decides rests on a credential-request record (packages/envoy/internal/broker/record/record.go): a fixed, line-by-line text holding the session’s signed request verbatim, the approver, the enrollment’s kind, runtime id and operator, the lifetime (- for a grant that lasts until its session ends), the policy version, the expiry, and, for a machine login, its confirmation code. The record’s id is the SHA-256 of that text, so a record cannot change without changing its id. Decisions are events on the record (approved, denied, expired, cancelled, revoked), each naming who made it.

Every time the broker releases a value or authenticates a launcher credential, it re-verifies the whole chain: the record still reproduces its id, the session’s signature inside it still verifies, and exactly one approval by the record’s approver stands behind it. A grant or credential with no approved record behind it releases nothing.

Beside the records, the broker keeps an append-only audit table with one row per event: enrollment.created, enrollment.revoked, enrollment.expired, request.created, request.granted, request.denied, request.cancelled, request.expired, grant.used (each time a session reads a grant, naming the secrets released), grant.revoked (naming, under withheld, the secrets an operator’s revoke withheld from the session, and, on each other grant of the session that revoke ended, the withheld secrets that grant held), grant.withheld (an operator’s revoke of a grant already revoked, naming under withheld the secrets it withheld from the session) and launcher_credential.revoked (naming the machine login’s credential_id, its host, its service for a service’s login, and under enrollments the sessions the revoke ended) and launcher_credential.token_login (a service’s token login: the credential_id, its service, the token’s subject, the pod_uid the token is bound to when it is, the login’s key_thumbprint, the credential’s expires_at and the token’s own token_expires_at). Each row names its actor: human:<email> for a change Dispatch made for its signed-in person, session:<enrollment id> for a session’s own, launcher:<credential id> for one a machine login made under its own credential, a person’s revoke from their machine and a service’s token login included, or broker. No record, event or audit row ever holds a secret value. Operating the broker shows how to read them.