Backend Architecture

Layering & dependency rule

                  ┌──────────┐
                  │  FastAPI │  api/v1/{reads,commands,attachments,events,dev}
                  └────┬─────┘
                       │ deps: AppContext
                  ┌────▼─────┐
                  │ services │  charge_service, attachment_service, event_bus
                  └────┬─────┘
                       │
              ┌────────▼────────┐
              │  domain / core  │  charge_state, deadline, canonical_json,
              │   (pure)        │  graphemes, clock, ids, config, errors
              └────────┬────────┘
                       │
                  ┌────▼─────┐
                  │  schemas │  StrictModel wire types (Pydantic v2 strict)
                  └────┬─────┘
                       │
                  ┌────▼─────┐
                  │    db    │  JsonStore + models (snapshot shapes)
                  └──────────┘

Allowed imports: api → services → domain/core → schemas → db. domain imports nothing from api/schemas/db/clock — that single inversion is what lets Python export vectors that Dart replays.

graph LR
  domain -. no FastAPI/Pydantic/IO .-> core
  api --> services --> domain
  api --> deps --> core
  services --> db
  services --> schemas

Application factory

app/main.py exposes two entry points:

• create_app() -> FastAPI — Uvicorn factory; reads Settings() once, picks SystemClock vs anchored demo clock (KX_DEMO_NOW), builds JsonStore(settings, clock), ChargeService, EventBroker, ChaosController, UuidGenerator.
• build_app(settings, store, clock, scheduler, id_generator) — explicit DI used by every test; no module-level mutable app state.

lifespan opens the store (acquires runtime/runtime.lock, seeds or loads state.json, verifies referenced attachment IDs, clears staging), installs store.set_publisher(broker.publish), schedules the next deadline, then yields.

Middleware stack (inside _install_middleware):

1. MaxBodySizeMiddleware — ASGI byte counter before Starlette's multipart parser; Content-Length + streamed body both checked; 413 with envelope detail limitBytes.
2. Request context — request.state.request_id = id_generator.next_uuid(), chaos.apply(path), shared barrier lease unless /_dev/reset or /events, deadline reconciliation via service.
3. CORSMiddleware — allowed headers Content-Type, Idempotency-Key, Last-Event-ID, X-Dev-Token; * origin rejected at config validation; allowed methods GET, POST, OPTIONS.

Exception handlers render every ApiError into {error: {code,message,details}, meta: {schemaVersion,requestId,serverTime,stateEpoch,storeRevision}}.

Request lifecycle

sequenceDiagram
  participant Client
  participant MW as Middleware
  participant API as Endpoint
  participant Svc as ChargeService
  participant Store as JsonStore
  Client->>MW: HTTP + headers (Idempotency-Key?, X-Dev-Token?)
  MW->>MW: MaxBodySize check, chaos injection, barrier shared lease
  MW->>API: call_next
  API->>API: validate schemas (StrictModel), stage attachments (Contest)
  API->>Svc: apply_charge_command / bookings / hub / ...
  Svc->>Store: commit(mutator) — single lock + os.replace
  Store-->>Svc: CommitResult + events
  Svc-->>API: Envelope payload
  API-->>Client: {data/meta} or {error/meta}
  Store->>Bus: publisher(events) -> SSE fans out

Commands prepare everything outside the store lock: key validation, body parsing, attachment streaming/hashing/digesting. Inside JsonStore.commit:

1. replay idempotency ledger lookup,
2. expectedStateEpoch check under lock,
3. deadline reconciliation,
4. transition(status, event) pure check,
5. attachment finalization,
6. version bump + updatedAt,
7. event ring append (charge.created etc., capped at EVENT_RING_CAPACITY=100),
8. idempotency ledger write (only for 2xx),
9. os.replace(tmp, state.json) + fsync + directory fsync.

No code path persists half a mutation.

Concurrency primitives

Primitive	Scope	Purpose
_RuntimeFileLock (fcntl LOCK_EX|LOCK_NB)	process	second backend fails fast with RuntimeLockedError
asyncio.Lock in JsonStore	event loop	serializes commit() callers
OperationBarrier	lifecycle	shared for reads/uploads/downloads, exclusive for _dev/reset — reset waits for in-flight ops

SSE (EventBroker) holds per-client asyncio.Queue (capacity sse_queue_capacity=64). Publish fans out after commit; overflow drops oldest per-contract, never blocks commit.

Clock & scheduler abstraction

class Clock(Protocol):
  def now(self) -> datetime: ...
class Scheduler(Protocol):
  async def schedule_at(self, at: datetime, fn: Callable[[], Awaitable[None]]): ...

• SystemClock — datetime.now(timezone.utc); AsyncioScheduler.
• ManualClock — tests control now, advance(), and scheduler firing synchronously.
• AnchoredDemoClock — production demo mode (KX_DEMO_NOW) replays a fixed instant through the same interface.

Every call site uses the injected clock.now(), never datetime.now() directly — greppable invariant.

Errors & observability

app/core/errors.py defines ApiError(code, status_code, message, details) subclasses (ValidationError, StoreEpochMismatch, VersionConflict, IdempotencyPayloadMismatch, etc.). ApiError never leaks user text in details — _redact_errors in commands.py keeps only field + rule.

Logging: kxinspect logger emits backend ready schema=… contract=… seed=… dev_routes=… at startup; chaos injection logs at debug. No request body is logged.

Extending

• New entity → app/db/models.py (ENTITY_KEYS, SEED_FILES), app/schemas/*, app/services/*, seed + fixture export + openapi-v1.json.
• New command → new route template constant (*_ROUTE), new ChargeEvent if lifecycle, new scope_key branch for idempotency.
• New dev action → app/api/v1/endpoints/dev.py, must check X-Dev-Token constant-time and take barrier.exclusive() when destructive.