mos-backbone — dev.tktk.link ============================== The backbone is a reverse tunnel: backend services open a persistent outbound connection to the backbone and receive forwarded client requests through it. Clients reach a service at .dev.tktk.link. ── FOR BACKEND SERVICES ──────────────────────────────────────────────────── 1. Register a principal (once — see FOR CLIENTS / PRINCIPALS below) POST https://dev.tktk.link/principals Response includes your credential: {"credential": "mosb_pr__", ...} The credential is shown ONCE — save it. It authenticates BOTH your API calls and your agent's tunnel connection. 2. Create a service you own POST https://dev.tktk.link/services x-backbone-principal: mosb_pr_ Response: {"service": ""} — services are addressed by UUID: clients reach yours at https://.dev.tktk.link/. (The owner is auto-granted access to its own service.) 3. Connect the agent tunnel The agent authenticates with the SAME principal credential plus the service UUID it owns. The complete wire protocol is in the AGENT TUNNEL PROTOCOL section below. 4. What your backend receives Every public request must present a granted principal credential (there is no anonymous access — the edge answers 401 otherwise), so every forwarded request arrives with the caller's identity headers already injected (and all x-backbone-* credentials stripped): x-mosb-sub principal UUID x-mosb-name display name x-mosb-email email x-mosb-sub-id sub-identity (present when the client sent an assertion) 5. Delete a service you own DELETE https://dev.tktk.link/services/ x-backbone-principal: mosb_pr_ 204 on success — revokes the service and force-disconnects its exposing agent, cluster-wide. Owner only: 403 for any other principal, 404 for an unknown service. ── AGENT TUNNEL PROTOCOL (v2 — reverse connection pool) ──────────────────── Two planes, both on :443, dispatched by TLS SNI (no second port): a CONTROL channel (Phoenix Channel over WebSocket) for auth, registration and pool orchestration, and pooled DATA connections (raw TLS) that carry the actual HTTP. Forwarded requests are NOT framed over the WebSocket. 1. Control channel — connect + authenticate wss://dev.tktk.link/__mosb/tunnel/websocket?vsn=2.0.0 • Apex host ONLY — connecting on a .dev.tktk.link host is rejected. • Phoenix Channel V2 serializer: JSON text frames [join_ref, ref, topic, event, payload] • Authenticate AT CONNECT with your principal credential; first present wins: 1. Phoenix authToken (Sec-WebSocket-Protocol) 2. x-backbone-credential: mosb_pr_ (header) 3. ?credential=mosb_pr_ (query param; lands in proxy logs — prefer 1 or 2) and declare the service UUID you are exposing: x-backbone-service: (header) or ?service= The credential's principal must OWN that service. • Optional: x-backbone-protocols: http/1.1, h2 — which protocols your local backend speaks on the data connections (absent = both). 2. Join Join exactly one topic, tunnel: (the connect service): ["1","1","tunnel:","phx_join",{"version":1,"agent":"you/1.0"}] Reply (status ok): {"version": 1, "service": "", "public_host": ".dev.tktk.link", "agent_host": "agents.dev.tktk.link", "limits": {"pool_target": 8, "idle_timeout_ms": ..., ...}} Errors: forbidden | rate_limited | unsupported_version | contended. Read limits from the join reply — never hard-code them. 3. Control-channel events server → agent pool_open {"n": N} open N more data connections server → agent drain {} stop opening; drain parked conns server → agent superseded {} a newer agent won; close, DO NOT rejoin agent → server pool_status {"idle": I, "busy": B} optional pool report agent → server heartbeat {} on the tunnel topic; replies {"pool_target": N} Also send the standard Phoenix transport heartbeat — topic "phoenix", event "heartbeat" — at intervals under 60 s or the socket times out. 4. Data connections — where HTTP actually flows On pool_open {"n": N}, open N TLS connections to dev.tktk.link:443 with SNI agents.dev.tktk.link and write ONE preamble frame: "MOSB" + + + json (≤ 4096 bytes): {"credential": "mosb_pr_...", "service": "", "conn_id": ""} then read a single ack byte: 0x01 parked — stay SILENT until claimed 0x00 denied — bad credential or not the service owner (do not retry) 0x02 degraded — registry not loaded yet; retry with backoff 5. The role flip — serving a claimed connection When a public request arrives the backbone CLAIMS one parked connection and becomes the HTTP CLIENT on it; your agent is the HTTP SERVER. The bytes after the ack are plain HTTP — one of: • HTTP/1.1 — a normal request (method, path, headers incl. the x-mosb-* identity set, body) is written on the socket; you answer with a normal HTTP/1.1 response. One request per connection; the connection closes afterwards. • HTTP/2, prior-knowledge h2c — for public H2 traffic the backbone sends the client preface (PRI * HTTP/2.0...) then multiplexes ALL of one public connection's streams over that single data connection as ordinary H2 client streams. Sniff the first bytes after claim to tell them apart. There is no custom framing and no request-id correlation: multiplexing is either one-request-per-connection (H1) or native H2 streams. Concurrency comes from the pool — every claim triggers a pool_open replenish. WebSocket upgrades are proxied too (message-level relay). Full spec: docs/PROTOCOL.md in the repo (kept in sync with the code by a doc test). Reference implementations: the Go SDK under sdk/go (agent in sdk/go/agent.go, used by sdk/go/cmd/mosb-agent and sdk/go/cmd/mosb-e2e) and the Elixir reference agents (scripts/agent.exs, test/support/reference_agent.ex). ── FOR CLIENTS / PRINCIPALS ──────────────────────────────────────────────── 1. Register a principal POST https://dev.tktk.link/principals {"name": "Your Name", "email": "you@example.com", "note": "optional"} Response includes your principal_id (UUID) and credential string. The credential is shown ONCE — save it. {"credential": "mosb_pr__", "principal": {...}} 2. Get access to a service The service OWNER grants your principal_id access: POST https://dev.tktk.link/services//grants x-backbone-principal: mosb_pr_ {"principal_id": ""} or you request access yourself and an admin approves the signed URL: POST https://dev.tktk.link/access-requests x-backbone-principal: mosb_pr_ {"service": ""} (Admins can also grant directly: POST /admin/api/grants.) 3. Call a service with your identity Include your credential on every request: x-backbone-principal: mosb_pr_ The backbone verifies the credential, checks your grant, injects the x-mosb-* headers, strips x-backbone-principal, and forwards the request. Requests without a valid granted credential are rejected (401). 4. Optional: sub-assertions (per-request identity refinement) Sign a JSON payload with your derived key to assert a finer-grained identity on a single request. Claims override your registered name/email. x-backbone-assertion: . payload: {"sub_id": "alice-session-7", "name": "Alice", "exp": } hmac: HMAC-SHA256(derived_key, "assertion:" <> base64url(payload)) derived_key = HMAC-SHA256(master_secret, "mosb-principal-v1::") — your credential string embeds base64url(derived_key). 5. Optional: pre-signed URL tokens (no auth header needed) Generate a signed URL token scoped to a specific service: POST https://dev.tktk.link/url-tokens x-backbone-principal: mosb_pr_ {"service": "", "expires_in": 3600} Response: {"token": "...", "url": "https://.dev.tktk.link?_mosb="} Append ?_mosb= to any request URL — no header required. Rotating your key_version immediately invalidates all outstanding tokens. 6. Optional: object storage (S3-compatible) Mint a dedicated S3 bucket for one of your managed identities (a sub_id). There is NO secret: the unguessable bucket name IS the credential (capability model, like an unguessable URL — treat it as one). It is also the S3 access key id. POST https://dev.tktk.link/storage-buckets x-backbone-principal: mosb_pr_ {"sub_id": "alice-db"} Response: {"bucket": "mosb-", "access_key_id": "mosb-", "sub_id": "alice-db", "endpoint": "https://s3.dev.tktk.link", "region": "auto"} Two ways to call the data plane (path-style): - anonymously — no Authorization header at all; knowing the bucket name is what grants access - SigV4 — for S3 tooling that requires configured credentials, set BOTH access key id and secret access key to the bucket name The bucket is a real dedicated bucket — copy, multipart, etc. all work. Request bodies stream straight through: there is no gateway object-size limit. SSE-C (customer-provided keys, x-amz-server-side-encryption-customer-*) is supported on every operation — the gateway stores no keys, so without yours the stored object is unreadable, even with leaked upstream creds. Other server-side-encryption headers (SSE-S3 / SSE-KMS) are rejected with 400, never silently ignored. e.g. Litestream: replicas: - url: s3:///db?endpoint=https://s3.dev.tktk.link&force-path-style=true access-key-id: secret-access-key: Idempotent: re-POST the same sub_id to re-derive the same bucket name. GET /storage-buckets lists yours; DELETE /storage-buckets/ revokes the name instantly (and revoking your principal disables all of them) — revocation semantics are unchanged from the credential era. ── QUICK REFERENCE ───────────────────────────────────────────────────────── Endpoint Auth Description ────────────────────────────────────────────────────────────────────────── POST /principals none register as a principal POST /services principal cred create a service you own DELETE /services/:service principal cred delete a service you own POST /services/:service/grants principal cred owner grants a principal access DELETE /services/:service/grants/:pid principal cred owner revokes a grant POST /access-requests principal cred request access (signed approve URL) POST /url-tokens principal cred generate a signed URL POST /storage-buckets principal cred mint an S3 bucket GET /storage-buckets principal cred list your S3 buckets DELETE /storage-buckets/:bucket principal cred revoke an S3 bucket GET /healthz none health check GET /admin/api/services admin token list services (+ tunnel status) GET /admin/api/cluster admin token cluster node status POST /admin/api/principals admin token register a principal GET /admin/api/principals admin token list principals DELETE /admin/api/principals/:cid admin token revoke a principal POST /admin/api/grants admin token grant principal → service DELETE /admin/api/grants/:pid/:service admin token revoke a grant GET /admin/api/grants/:pid admin token list grants for principal POST /admin/api/access-requests/:token/approve admin approve an access request POST /admin/api/access-requests/:token/deny admin deny an access request POST /admin/api/drain admin token begin draining this node GET /admin/api/drain admin token drain status ──────────────────────────────────────────────────────────────────────────── JSON? Send Accept: application/json