ℹ️ About Codex in GitHub

\\n
\\n\\n[Your team has set up Codex to review pull requests in this repo](https://chatgpt.com/codex/cloud/settings/general). Reviews are triggered when you\\n- Open a pull request for review\\n- Mark a draft as ready\\n- Comment \\\"@codex review\\\".\\n\\nIf Codex has suggestions, it will comment; otherwise it will react with 👍.\\n\\n\\n\\n\\nCodex can also answer questions or update the PR. Try commenting \\\"@codex address that feedback\\\".\\n \\n

\",\"path\":null,\"side\":null,\"line\":null}{\"type\":\"pull_request_comment\",\"body\":\"**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Keep the Lua counter key in its declared cluster slot**\\n\\nWhen the configured backend is Redis Cluster, the script is routed using `KEYS[1]` (`namespace:key`) but actually increments the dynamically derived `namespace:key:bucket`, which normally hashes to a different slot. Redis Cluster therefore rejects the script with a cross-slot/non-local-key error; the resilient decorator then falls back to per-process counters and silently loses the multi-replica rate-limit guarantee. Store the bucket and count under the declared key, or otherwise ensure every accessed key is explicitly declared in the same hash slot.\\n\\nUseful? React with 👍 / 👎.\",\"path\":\"src/Cluckwork.Infrastructure/SharedState/RedisFixedWindowCounter.cs\",\"side\":\"RIGHT\",\"line\":28}{\"type\":\"pull_request_comment\",\"body\":\"**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Reject a blank Redis endpoint in the simulation verifier**\\n\\nIf an ambient override sets `SharedState__Redis__ConnectionString=` explicitly, Compose resolves it without an unset-variable warning and this condition skips validation, so the verifier reports success while the app selects the in-process fallback. That means the human-run Production harness can stop exercising the Redis-backed path this change added without any indication; require a nonblank endpoint here even though blank remains legal for ordinary serving deployments.\\n\\nAGENTS.md reference: [AGENTS.md:L68-L68](https://github.com/mforce/cluckwork/blob/9c2ffa3423702d161025458f8fc13744d71c9482/AGENTS.md#L68-L68)\\n\\nUseful? React with 👍 / 👎.\",\"path\":\"tools/simulation/verify-harness.sh\",\"side\":\"RIGHT\",\"line\":null}{\"type\":\"pull_request_comment\",\"body\":\"**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Update the singleton state inventory**\\n\\nThese new registrations add a connection multiplexer and three shared-state singleton services, including mutable in-process fallbacks, but the repository's deployment-invariant audit still records the previous singleton inventory and classifications. Future scale-out reviews are explicitly expected to enumerate and classify every singleton and in-memory primitive, so leaving these entries absent makes that checkpoint stale and risks treating the remaining single-instance blockers as closed without reviewing fallback behavior.\\n\\nAGENTS.md reference: [AGENTS.md:L159-L167](https://github.com/mforce/cluckwork/blob/9c2ffa3423702d161025458f8fc13744d71c9482/AGENTS.md#L159-L167)\\n\\nUseful? React with 👍 / 👎.\",\"path\":\"src/Cluckwork.Infrastructure/SharedState/SharedStateRegistration.cs\",\"side\":\"RIGHT\",\"line\":53}{\"type\":\"pull_request_comment\",\"body\":\"Fixed in e94bd4e6. The Lua counter key now wraps the routing part in a hash tag — `{namespace:win:key}` — so `KEYS[1]` and the derived `KEYS[1]:bucket` hash to the same slot and Redis Cluster accepts the script (inert braces on single-node). The class doc was changed from a \\\"single-node only\\\" constraint to documenting the hash-tag. Verified: full SharedState suite green against the Testcontainers Redis, including the counter contract tests that exercise the new key shape.\",\"path\":\"src/Cluckwork.Infrastructure/SharedState/RedisFixedWindowCounter.cs\",\"side\":\"RIGHT\",\"line\":28}{\"type\":\"pull_request_comment\",\"body\":\"Fixed in e94bd4e6. `verify-harness.sh` now FAILS on an unset/blank `SharedState__Redis__ConnectionString` — a harness-specific policy, deliberately stronger than the app guard (blank stays legal for an ordinary serving deploy; the sim wires Redis on purpose, so a blank silently drops the Redis-backed path). Verified: green on the real `redis:6379` value; `SharedState__Redis__ConnectionString='' bash verify-harness.sh` now exits 1 naming the drop. The rationale is recorded in the check's comment block.\",\"path\":\"tools/simulation/verify-harness.sh\",\"side\":\"RIGHT\",\"line\":null}{\"type\":\"pull_request_comment\",\"body\":\"Fixed in e94bd4e6. AGENTS.md's deploy-invariant singleton inventory now records and classifies the four #543 registrations (`IConnectionMultiplexer` + the three ports). They are classified as the shared store that will *close* the in-process limiter blockers (#544 → `IFixedWindowCounter`, #545 → `ILease`, #338 grant-replay → `IClaimOnceStore`), not new blockers: Redis-backed they are multi-replica-safe, and their in-process fallbacks are a deliberate, alarmed degradation (fail-closed for claim-once). Noted that #543 only lands the ports — the blockers stay open until a caller is wired.\",\"path\":\"src/Cluckwork.Infrastructure/SharedState/SharedStateRegistration.cs\",\"side\":\"RIGHT\",\"line\":53}","text_elements",[],{"_391":486,"_370":371,"_393":377},{"_394":487,"_396":-5,"_45":-5,"_398":488,"_400":-5,"_99":336,"_95":402},[489],{"_403":377},{"_49":482,"_95":426},{"_391":491,"_370":371,"_393":377},{"_394":492,"_396":-5,"_45":-5,"_398":493,"_400":-5,"_99":336,"_95":402},[494,495],{"_403":377},{"_49":497,"_95":426},{"_49":496,"_95":426},"\n /workspace/cluckwork\n bash\n 2026-08-17\n Etc/UTC\n /workspace/cluckwork\n","# AGENTS.md instructions for /workspace/cluckwork\n\n\n# AGENTS.md — Cluckwork\n\nPoultry egg-farm management system. Backend: **.NET 10** (C#), layered DDD. Frontend: **React 19 + Vite** SPA in `web/`. Postgres via EF Core.\n\nThis file is the shared brief for any coding agent (Claude Code, Codex, etc.).\n\n## Communicating\n\nKeep explanations clear and human-sounding. Provide concise, focused responses. Skip preambles and recaps — lead with the action or answer.\n\n## Layout\n\n```\nsrc/\n Cluckwork.Domain aggregates, value objects, domain events (no deps)\n Cluckwork.Application feature handlers, repository interfaces, validators\n Cluckwork.Infrastructure EF Core, Identity/JWT, repositories, seeding, jobs\n Cluckwork.Api minimal-API endpoints, middleware, Program.cs\nweb/ React/Vite SPA (see web/README.md)\ndeploy/ docker-compose (.yml prod, .dev.yml dev DB), .env.example\nspecs/ product + technical specs, wireframes\ntests/ Domain.Tests, Application.Tests, Api.IntegrationTests\n```\n\nDependencies point inward: Api → Application/Infrastructure → Domain. Domain depends on nothing.\n\n## Build / test / run\n\n```bash\ndotnet build Cluckwork.sln # warnings are errors — keep it clean\ndotnet test Cluckwork.sln # 688 tests as of 2026-07; integration needs Docker\n```\n\n- **Integration tests** spin up a real Postgres via Testcontainers (`docker` required). No SQLite — EF SQL semantics differ.\n- **Run full stack (prod-like):** `docker compose -f deploy/docker-compose.yml up --build` → SPA + API on http://localhost:8080 (single container; API serves the built SPA from `wwwroot`).\n- **Run frontend dev:** `cd web && npm run dev` → http://localhost:5173 (proxies `/api` → :8080).\n- **Debug API (no docker stack):** `docker compose -f deploy/docker-compose.dev.yml up -d` (Postgres on :5432), then run/debug `Cluckwork.Api` (Development env). Dev secrets live in **user-secrets**, not files.\n\n## Conventions (follow these)\n\n- **Result pattern:** domain/handlers return `Result` / `Result` (see `Domain/Common`). Don't throw for expected failures; throw only for invariant violations (e.g. `Flock.Create` guards).\n- **Every aggregate mutation must bump `Version`.** `Version` is an EF concurrency token (`IsConcurrencyToken`) — EF puts the *original* value in the UPDATE's WHERE but never auto-increments it. A mutation without `Version++` silently loses concurrent races (both writers match `WHERE Version = N`) instead of 409ing. This bug shipped three times (`RecordProduction` was correct; `Submit`/`Lock` and `SalesOrder.AddItem` weren't) — each now has a parallel-race integration test; add one for any new mutation.\n- **Handler per feature**, invoked directly from endpoints — **no MediatR**. Register handlers/validators/repos in `Program.cs`.\n- **Validation:** FluentValidation validators (`*Validator`), one per command; endpoints call `ValidateAsync` and return `ValidationProblem`.\n- **Endpoints:** minimal APIs grouped under `/api/v1/...` via `MapEndpoints`; writes require auth + an `Idempotency-Key` (middleware).\n- **Multi-tenancy:** every tenant-owned entity has `AccountId`; enforced by an EF **global query filter** + a **`TenantStampInterceptor`** (stamps on insert). `TenantContext` is resolved per-request from the JWT `account_id` claim. At startup it's unresolved → use `IgnoreQueryFilters()` in seeders.\n- **Auth:** asymmetric JWT + rotating refresh tokens. PEM keys come from config with escaped `\\n`; normalize via `PemKey.Normalize` before `ImportFromPem`. Integration tests generate an ephemeral RSA pair at test-process startup (`TestJwtKeys` in `CluckworkWebApplicationFactory.cs`) — no real key material is ever committed to source (`deploy/.env.example` carries PEM armor with a `replace-me` body, not a usable key).\n- **Both JWT keys are checked at boot, and the check is serving-only (#510/#347).** `AddCluckworkIdentity` requires `Jwt:PublicKeyPem` **and** `Jwt:PrivateKeyPem` to be non-blank **and to actually import**; a `Serving` process refuses to start otherwise. Two traps this closes, both of which shipped: `configuration[...] ?? throw` catches **only null** while the shipped `appsettings.json` carries `\"\"` — use **`IsNullOrWhiteSpace`**, never `??`, or the guard is decorative for every real deployment; and importing inside the `AddJwtBearer` delegate makes a corrupt key a **per-request** failure, so the farm boots green, `/health/ready` passes, the container `HEALTHCHECK` passes, and every authenticated request 500s — an orchestrator sees a healthy instance that rejects every login. The check takes a `ProcessRole` and is skipped for one-shot verbs, which neither issue nor validate a token; **making it eager without that scope is a fresh #331** in the most security-sensitive file of the set. Pinned from both sides: four rows in `ProcessRoleGuardTests` (each key × missing/unusable) plus `OneShotVerbMinimalConfigTests`, whose environment carries no `Jwt:*` at all.\n- **Credential epoch revocation (#364).** Every access/refresh token is bound to the user's `CredentialEpoch`; every password-reset path bumps it so issued credentials fail on next use. Epoch **`0` is permanently retired** (users start at `1`, legacy refresh rows at `0`), and a missing/malformed `credential_epoch` claim is **always a mismatch** — never default it to the DB epoch. `CredentialEpochMiddleware` does a **fresh DB read per authenticated request** (the round trip *is* the fail-closed guarantee — do not cache), sits **after `TenantResolutionMiddleware`, before `MustChangePasswordMiddleware`**, and exempts `auth/logout`. Login/step-up snapshot the verified epoch+stamp before clearing failed-access and refuse issuance on a concurrency-loss reload. Distinct from #283/#265/#308 — none substitutes for another. **Break it and a revoked credential keeps working.** → [`docs/decisions/364-credential-epoch-revocation.md`](docs/decisions/364-credential-epoch-revocation.md)\n- **Base reference data via guarded raw-SQL migrations (#283).** The default account, four assignable roles, default egg grades, and packed-unit conversions are **static reference data baked into the migrations** as hand-written `migrationBuilder.Sql` with `WHERE NOT EXISTS` guards — **never `HasData`/`InsertData`** (which key on the PK and would either collide or emit `UpdateData`/`DeleteData` that silently reverts the farm's own edits). No runtime seeder, no `Seed:*`. The **grades** guard is whole-set (empty catalog, because grades are user-renamable); roles/conversions/account are **per-key** (keys aren't user-mutable). Pinned by `MigrationSecurityReviewTests` + `BaseReferenceDataMigrationTests`. **Regenerate these into `HasData` and a later model-diff reverts a renamed grade.** → [`docs/decisions/283-migrations-base-provisioning.md`](docs/decisions/283-migrations-base-provisioning.md)\n- **Migrations: `InitialCreate` frozen, one migration per change (#407).** `Persistence/Migrations/` is one squashed `InitialCreate` (recorded id `20260801190854_InitialCreate`); it is **frozen and never regenerated**, and **every schema change gets its own `dotnet ef migrations add`**. EF never re-runs an applied migration, so a column **hand-folded into `InitialCreate` silently does not exist** on any booted DB — it surfaces as broken behaviour (a failed login, #399), not a migration error. `InitialCreate` also carries four un-regenerable `lower(\"Name\")` expression indexes + the base-reference SQL; regenerating mints a new timestamp that desynchronises `__EFMigrationsHistory` everywhere. Pre-squash or pre-#407 dev DBs can't migrate forward — drop and recreate. `MigrationSecurityReviewTests` freezes the id + a portable operation digest. → [`docs/decisions/407-migration-freeze.md`](docs/decisions/407-migration-freeze.md)\n- **Schema docs are generated, committed, and regenerated with every migration (#417).** `docs/schema/` (mermaid ERD + full catalog: every column, constraint, and index — including the raw-SQL expression indexes and partial predicates EF reflection can't see) is produced by `tools/schema-docs/generate.sh` from an ephemeral migrated Postgres via digest-pinned tbls. **Every PR that adds a migration runs the generator and commits the result in the same PR** — CI's `build-and-test` runs `generate.sh --check` (byte-diff against a fresh generation) and fails a stale PR. Generated files under `docs/schema/` are **never hand-edited**, and a conflict there after a rebase is resolved by regenerating, never by hand-merging. Pinned from three sides by `SchemaDocsTests` (postgres-pin uniformity across every tracked file, no environment leakage, completeness against the live catalog). → [`docs/decisions/417-schema-docs.md`](docs/decisions/417-schema-docs.md)\n- **`AuditEvents` is not time-partitioned, on purpose (#505).** The dominant read (`GetProvenanceAsync`, run on every Flocks/Egg grades/Daily entries/Sales/Expenses page load) filters on `AccountId`+`EntityType`+`EntityId` with **no date predicate**, so partitioning by month would convert one index lookup into one per partition — added per-partition overhead that only grows as months pass, for no pruning benefit — plus a PK change, a new INSERT-fails-without-next-month's-partition failure mode inside the same transaction as the audited mutation (#93), and no partition-maintenance job to prevent it. At current growth (~20 MB/year at 100 flocks) this is not close to needed; if it ever is, partition by `AccountId` (the column the query actually filters on) once genuinely multi-tenant, not by time. → [`docs/decisions/505-audit-events-no-time-partition.md`](docs/decisions/505-audit-events-no-time-partition.md)\n- **Seed / simulation data is never boot-seeded (#280, #284, #279).** Run it explicitly against an **already-base-seeded, non-Production** database: `ASPNETCORE_ENVIRONMENT=Development dotnet Cluckwork.Api.dll seed --profile demo|simulation` (an unset env → Production → blocked). The verb migrates, seeds, then exits — **authoritative** (ignores config) and **fail-loud** (a real exit code, never a silent no-op). `simulation` additionally records a durable date anchor + completion marker, so a clean re-run — even across a UTC-midnight rollover — converges to `AlreadySeeded`; a polluted account fails the exact-count validation **closed**. **Both profiles now require an Owner and attribute what they seed to real people (#500)** — see the next bullet. → [`docs/decisions/280-seed-and-simulation.md`](docs/decisions/280-seed-and-simulation.md)\n- **Nothing writes an audit event without an actor (#500).** `AuditWriter` **throws** on an unresolved `ICurrentUser`, symmetric with the tenant guard beside it. The old fallback stamped `\"(unresolved)\"` — silent, reachable only from non-HTTP callers, and it shipped ~256 such rows into every demo farm, visible on five screens once #494 rendered provenance. Every non-HTTP caller therefore declares who it is: `bootstrap-admin` and `recover-admin` declare **system actors** (`SystemActors.BootstrapAdmin` / `.BreakGlass`, via `CurrentUserContext.ResolveSystemActor`), and **both seeders now require an Owner** — demo exits `PrerequisitesMissing` naming `bootstrap-admin`, a deliberate break with the old \"demo needs nothing but a connection string\" contract. Simulation goes further and attributes **per persona**: managers create flocks/products/expenses, sales staff book orders, a rotating worker pool records the daily entries, and roughly one submission in three is a manager signing off somebody else's draft (both #494 provenance shapes). **`ICurrentUser` is an AUTHORIZATION input, not an audit label** — `FlockScopeGuard` reads `Roles`/`UserId`, so which persona the seeder acts as decides what it is *allowed* to write: picking the flock-restricted worker for a foreign flock returns `FlockScope.NotAssigned` and fails the entire seed. Never resolve an actor carrying roles it does not actually hold; build one from `UserManager.GetRolesAsync`, never from a literal.\n- **Break-glass recovery (#265):** `recover-admin` is a second one-off CLI verb on the same binary, same run-then-exit shape as `seed` — but deliberately **NOT** environment-gated (it must work against a real Production database): `dotnet Cluckwork.Api.dll recover-admin --email [--account ] [--reason ]`. For a locked-out account (the sole-Owner-lost-password case, no email/SMTP reset path) it, in one transaction, resets to a **freshly generated** temporary password (never one passed on the command line), rotates the security stamp, revokes every refresh token, and writes a conspicuous `User.BreakGlassReset` audit row carrying `--reason`. The temp password is printed to **stdout only** (never the logger/OTLP) and exit `0`; failures go to stderr with exit `1` and change nothing. `AdminRecoveryService` orchestrates; `IdentityProvider.BreakGlassResetAsync` shares the reset/revoke core with `SetUserPasswordAsync`. Full procedure + verification drill: `docs/runbooks/break-glass-account-recovery.md`.\n- **First-run admin provisioning: `bootstrap-admin` (#283).** `dotnet Cluckwork.Api.dll bootstrap-admin --email ` (run-then-exit, always available in Production) migrates, then — **only if the default account has no Owner** — creates one with a **freshly generated password (stdout only, never the logger/OTLP)** and `MustChangePassword=true`; a re-run is a **silent no-op**. While the flag is set the JWT carries `must_change_password`, `MustChangePasswordMiddleware` 403s every endpoint except `auth/change-password`+`auth/logout` (before `UseAuthorization` and before idempotency), and the SPA renders **Set your password**. Any successful reset clears the flag (one invariant in `IdentityProvider`). **Deliberately a separate credential type from #265/#308** — never conflated. → [`docs/decisions/283-first-run-admin-provisioning.md`](docs/decisions/283-first-run-admin-provisioning.md)\n- **Farm timezone + tzdata/ICU image constraint (#264):** the default account (a fixed `\"UTC\"` migration literal since #283) is provisioned once; a real farm sets its actual IANA zone via **Settings → timezone** after first login (`Account.UpdateSettings`), not at provisioning time (the `Seed:TimeZoneId` config lever from before #283 is retired along with the runtime seeder it fed). The farm clock resolves IANA zones via `TimeZoneInfo.FindSystemTimeZoneById` for every safety-sensitive local-day decision and **fails closed**, so the runtime **image MUST carry tzdata + ICU** (the Ubuntu 24.04 (Noble) `aspnet:10.0` base does; **never** an Alpine/chiseled base without tzdata, and **never** `InvariantGlobalization=true`). `AddCluckworkPersistence` still asserts a representative IANA canary resolves at boot (`TimeZoneAvailability.EnsureResolvable`), for **both** process roles — a bad image fails the boot loudly instead of surfacing later as a per-request `FarmTimeZoneException`.\n- **Migrate command + prod migration split (#263).** `dotnet Cluckwork.Api.dll migrate` applies EF migrations then exits — the **pre-deploy-job entrypoint**. In Production `Database:MigrateOnStartup=false`, so the serving process never runs DDL; the **guarantee is ordering** — compose runs a one-shot `migrate` service before `app` waits on `service_completed_successfully`. `DatabaseReadyHealthCheck` is the backstop: `/health/ready` **503s while any migration is pending**. The remaining host step (deploy repo) is a least-privilege runtime role with no DDL, plus `ALTER DEFAULT PRIVILEGES` for the migrator. → [`docs/decisions/263-migrate-command.md`](docs/decisions/263-migrate-command.md)\n- **Proxy-trust boot guard (#260):** behind a reverse proxy/edge, HSTS (#144) and the per-IP login limiter (#143) only work if the app trusts the proxy's `X-Forwarded-*`, which it does **only** for networks in `RateLimiting:TrustedProxies`. An **empty list in Production fails the boot** (a `ProcessRole.Serving`-only guard, so `migrate`/`seed`/`recover-admin` are unaffected — see the process-role bullet below) rather than silently running with inert HSTS + a one-bucket limiter. Opt out with `RateLimiting:AllowNoTrustedProxies=true` only for a rare direct-TLS deploy with no fronting proxy. The concrete edge CIDR is deploy config (in the separate deployment/ops repo).\n- **Process role, not statement order (#347).** A boot guard's scope is declared, never positional. `ProcessRoles.From(args)` (`Hosting/ProcessRole.cs`) computes `ProcessRole.Serving | OneShot` **once**, before the host is built, and every role-scoped guard takes it: `ServingBootGuards.EnsureServingConfiguration` (#260, #319) is called **before** the CLI dispatch on purpose and still leaves the verbs alone, and `AddCluckworkTelemetry` (#316) and `AddCluckworkRateLimiting` (all of `RateLimitingOptions.Validate`) degrade with a stderr warning for `OneShot`. The one-shot verb list is derived from `CliDispatcher.Commands` so a new verb classifies itself; **`healthcheck` is the one that must be named explicitly** (it isn't an `ICliCommand` — it needs no host) and omitting it silently classifies the container's own probe as `Serving`. **Never gate a guard on where its statement sits**: that is what #331 was — the #316 validation ran at service registration, ahead of the dispatcher, and killed `recover-admin` (the break-glass verb) with SIGABRT 134. **Scope the whole subsystem, not the one setting that bit you**: #316's *endpoint* check was role-scoped while `ParseProtocol` two lines above it was not, so `Otlp:Protocol=bogus` still killed `recover-admin`; and `RateLimiting:TrustedProxies` **empty** was correctly serving-only while the **same key malformed** aborted every verb. **Every serving-only guard needs a row in `ProcessRoleGuardTests.ServingOnlyGuards`, one per VIOLATION rather than per subsystem** — `ServingGuardCoverageTests` reflects over `ServingBootGuards`, `RateLimitingOptions` and every `.ValidateOnStart()` registration and fails when a row is missing. Note that `.ValidateOnStart()` is serving-only *by mechanism* (it runs from `Host.StartAsync`, which the CLI dispatcher never reaches), so converting one to an eager registration-time check aborts every verb. The #261/#262 TLS floor and the #264 tzdata canary take no role: they apply to **both**, and unconditional code says so better than a parameter nobody branches on. → [`docs/decisions/347-process-role.md`](docs/decisions/347-process-role.md)\n- **Production Postgres TLS floor (#261/#262).** `ConnectionStrings:Default` accepts libpq URI or key-value form. In Production the effective `sslmode` is a **fail-closed allow-list**: `VerifyCA`/`VerifyFull` silent, `Require` warns, **everything else — `Disable`/`Allow`/`Prefer`, unset, or undefined — fails the boot**. `Database:AllowInsecureConnection=true` is the explicit opt-out for the co-located plaintext compose stack. When mapping a libpq param, **`ContainsKey(\"\")==false` does NOT mean Npgsql lacks the setting** — usually the keyword is spelled differently (`keepalives`→`Tcp Keepalive`, `client_encoding`→`Client Encoding`); search the keyword list before concluding it's unmappable (that exact false negative shipped #332). Integration tests opt out of this **and** the #260 guard. → [`docs/decisions/261-postgres-tls-floor.md`](docs/decisions/261-postgres-tls-floor.md)\n- **GSS/Kerberos negotiation off by default (#332).** `PostgresConnectionString` appends `GSS Encryption Mode=Disable` **unless the operator set it** (detected by **presence, not value**), **textually** after the TLS floor runs (a round-trip through the Npgsql builder would reorder/requote the operator's string and throw on any keyword this version doesn't know). Without it, Npgsql's `Prefer` default makes every connector dlopen the absent `libgssapi_krb5.so.2` and print two pre-Serilog stderr lines per process. Orthogonal to `sslmode` and to GSS *authentication*. → [`docs/decisions/332-gss-kerberos.md`](docs/decisions/332-gss-kerberos.md)\n- **Design-time migration connection fail-closed (#318):** `AppDbContextDesignTimeFactory` (used by `dotnet ef migrations add`/`database update`, never by the `migrate` verb, which uses the built host's own config) has **no default connection** — an unset/blank `CLUCKWORK_MIGRATIONS_CONNECTION` throws immediately, naming the variable, rather than falling back to a predictable `Host=localhost;...;Username=postgres;Password=postgres`. Every target is held to the **same allow-list TLS floor as a Production boot** (#261/#262, reused via `PostgresConnectionString.NormalizeAndValidate` — no second validator), except an explicitly acknowledged **loopback** development target: `CLUCKWORK_MIGRATIONS_ALLOW_INSECURE_LOOPBACK=true` permits plaintext, but only when the connection's host is `localhost`/`127.0.0.1`/`::1` (checked via `IPAddress.IsLoopback`) — set against any other host, it fails rather than silently widening scope.\n- **Container image hardening (#267):** the runtime stage runs **non-root** (`USER $APP_UID`, uid 1654 — the base image's built-in `app` account); all three base images are **digest-pinned** (`@sha256:…`, kept current by Dependabot's `docker` ecosystem); a CI job builds the image and **Trivy** fails the build on a *fixable* HIGH/CRITICAL. `aquasecurity/trivy-action` is **SHA-pinned** — the action was compromised in the 2026-03 supply-chain incident, so pin the immutable commit, not the re-pointable tag (this is the standing rule for third-party actions). Keep the full glibc base (tzdata + ICU per #264); never chiseled/Alpine.\n- **Container health probe: the `healthcheck` verb (#266).** The runtime image and the compose `app` service run `dotnet Cluckwork.Api.dll healthcheck` — a run-then-exit verb **dispatched before host build** (it needs no host/DI/DB/config, so a 30s probe doesn't re-run startup or re-log boot warnings). The hardened image ships no curl/wget, so the probe is **in-process**: it GETs `/health/ready` over loopback and exits `0` on a 2xx, `1` on any other status or an unreachable server — **never a false green**. CI boots the built image and asserts `/health/ready` goes green. → [`docs/decisions/266-container-health-probe.md`](docs/decisions/266-container-health-probe.md)\n- **Transient-DB retry, and where it stops (#269).** `EnableRetryOnFailure` retries only **self-contained EF units** (every read, every `SaveChanges` outside a user transaction); it must **not** wrap unreplayable work. The general hazard is **an automatic replay sitting above a stateful detector** (a monotonic counter, a CAS stamp, a single-use claim) that can't tell \"this request racing itself\" from the signal it exists to detect. Two cures, and picking wrong ships the bug: **`SingleAttemptExecution`** when the replay is itself observable (e.g. `AccessFailedAsync`'s second durable increment — otherwise one wrong password locks at ~half the #128 threshold); a **durability probe keyed on a self-minted token** when the replay writes nothing and the defect is a mis-read `catch` (refresh-token rotation, login/reset `INSERT`s). All fail closed; pinned from both sides in `RetryBoundaryTests`. → [`docs/decisions/269-transient-db-retry-boundary.md`](docs/decisions/269-transient-db-retry-boundary.md)\n- **A new Production boot guard must be taught to the sim harness (#370).** Every guard that fails the boot on missing/invalid config (#260, #261/#262, #316, #319, …) and every config-key add/rename/retire must **also update `tools/simulation/bootstrap.sh` + `docker-compose.sim.yml` + `verify-harness.sh` in the same PR** — that harness runs **Production config on purpose**, is **deliberately not in CI**, and every path into it is human-started, so nothing tells you when you break it (four guards once landed unnoticed until it couldn't boot `main`). Satisfy guards **properly** (a concrete `AllowedHosts`, the documented opt-outs), never by disabling one. **Treat a boot-guard PR that leaves the harness behind like a missing test.** → [`docs/decisions/370-sim-harness-boot-guards.md`](docs/decisions/370-sim-harness-boot-guards.md)\n- **SPA E2E lives in `tools/simulation/ui/` (#277/#385).** Playwright drives the real built SPA over the **same** `seed --profile simulation` fixture k6 uses; `web/` stays Vitest + Testing Library. Three enforced rules, each of which has caught something: **never hardcode a credential** (personas from the git-ignored `.sim-cast.json`), **never hardcode English** (selectors resolve through the SPA's own en/es/tl catalogs — a missing key throws), and **respect the farm clock** (`America/Chicago` is behind UTC). Mutation-checked (`npm run mutation`; a red baseline aborts). **Anything reasoning about `inert` or the accessibility tree must go through CDP** (`src/ax.ts`) — Playwright's `ariaSnapshot`/`getByRole`/`isVisible` do **not** model `inert`, so such a spec written with them passes whether or not the app inerts anything (#501). **The quick suite runs on PRs** (path-filtered, ~3 min; owner call 2026-08-08 after #433 broke it silently); the `slow`/`canary` modes and the k6 sibling stay `workflow_dispatch`-only, so the #370 warning still applies to those. → [`docs/decisions/277-spa-e2e.md`](docs/decisions/277-spa-e2e.md)\n- **A write-contract change must update its non-CI callers (#394).** Adding a required field, tightening a validator, or changing an aggregate's state machine breaks automated callers no test references. **Coverage is not uniform:** the seeders are covered only to the handler/domain layer, so a **validator-only** tightening is invisible to all five seeder tests (they call `HandleAsync` directly, skipping `ValidateAsync`); `tools/simulation/k6/` and the Playwright specs are **uncovered**, and a green baseline hides it (`authedPost` counts a tolerated `[403,409,422]` as a pass, and `reset.sh` never runs k6). **So verify the request/status contract by reading the callers, not by running.** Grep callers of the endpoint, re-check each payload, and narrow a tolerated-status list when a status flips from \"constrained write\" to \"broken write\". → [`docs/decisions/394-write-contract-callers.md`](docs/decisions/394-write-contract-callers.md)\n- **Production logs: compact JSON on stdout; base layer argument-free (#404).** stdout is the only sink (no file/vendor sink). Production selects `CompactJsonFormatter`; Development the human `outputTemplate`; the base `appsettings.json` Console entry carries **`Name` only, no `Args`** — a template left in the base file merges *beside* the formatter and Serilog **silently binds the template and ignores the formatter**. Trace fields differ by format: compact JSON writes **`@tr`/`@sp`**, the Dev template `{TraceId}`/`{SpanId}` — a collector query against the wrong name matches nothing. **Compact JSON serializes EVERY property**, so anything pushed via `BeginScope`/`LogContext`/`ForContext` now reaches the collector (see #273 for the redaction answer). Pinned by `ProductionLogFormatTests`. → [`docs/decisions/404-production-logs.md`](docs/decisions/404-production-logs.md)\n- **Nullable enabled**, no unused usings (both are build-breaking).\n\n## Secrets — never commit\n\n- `deploy/.env` is gitignored (real values). `deploy/.env.example` holds placeholders only.\n- Local API debug config uses `dotnet user-secrets` (keyed by `UserSecretsId` in `Cluckwork.Api.csproj`).\n- No hardcoded passwords/keys in source — GitGuardian scans PRs. Generate test credentials at runtime.\n\n## Host-agnostic repo (deployment boundary)\n\nThis repo is **host-portable** — it must build and run against any host without\ncarrying provider-specific config. The app reads every environment specific from\nconfig/env (connection string, `RateLimiting:TrustedProxies`, `Seed:TimeZoneId`,\nPostgres `sslmode`, …) and **never names or branches on a hosting provider**.\n\n- **Stays here** (portable operational contract): the Dockerfile (incl. its\n `HEALTHCHECK`), `deploy/` compose as a local/reference stack, health probes\n (`/health/live`, `/health/ready`), the `migrate` / `seed` / `recover-admin` /\n `healthcheck` verbs, `.env.example`, and docs that state *requirements* (\"needs tzdata + ICU\",\n \"needs a trusted-proxy list\", \"needs TLS to Postgres\", \"a fronting CDN must\n respect origin cache headers\").\n- **Does NOT belong here** (goes to a **separate deployment/ops repo**):\n provider deploy manifests (`railway.json`, `fly.toml`), IaC (Terraform/Pulumi),\n CDN / DNS / edge config, secret-store wiring, provider-named runbooks, and the\n concrete environment *values* (proxy CIDRs, CA bundles, connection URLs).\n\nReviewers: treat a hardcoded provider name in code, config, or a committed doc\nlike a missing test — flag it. Naming a provider as a passing *example* in prose\nis tolerable only when no portable phrasing works; prefer the neutral term.\n\n### Deploy invariant: exactly ONE serving API instance (#271, #338)\n\n**Run one serving instance.** More than one breaks four separate things, and\nnone of them announces itself as \"you are running two replicas\". This is a\n*requirement* the app imposes on its host, so it lives here; the concrete\nreplica count and how it is pinned are deploy-side.\n\n`AddHostedService()`\n(`Hosting/CluckworkJobServiceCollectionExtensions.cs`) means **every** instance\nruns the worker loop, and the poll claims nothing — no `FOR UPDATE SKIP\nLOCKED`, no lease, no advisory lock. What that exposes today is **the three\nrecurring sweeps**, which ride the same poll and run unconditionally per\ninstance: `DailyEntryLockSweep`, `RefreshTokenPurgeSweep`,\n`IdempotencyRecordPurgeSweep`. The durable-job half is still a scaffold that\nselects pending rows and logs them — no handlers are registered — so job\ndouble-execution is latent, not live. **Registering the first handler makes it\nlive**, which is the moment this invariant stops being about sweeps only.\n\nNot every double-run is equally bad, and the difference decides what an operator\nwould actually see. The two purge sweeps are idempotent deletes, so a second\nrunner wastes work and reports nothing — genuinely silent.\n`DailyEntryLockSweep` is not silent: it reads-then-writes behind an optimistic\n`Version` token, so the losing replica's `SaveChangesAsync` throws a concurrency\nexception, which its per-account `catch` logs as `Lock sweep failed for account\n{AccountId}` (`Jobs/DailyEntryLockSweep.cs`). That is observable — but it reads\nas a database fault, not as \"two replicas are sweeping\", which is the sense in\nwhich the duplicate stays invisible.\n\n**Four independent blockers, all of which must close before scaling** — and\nthey are not all in #271:\n\n- **#271 — background work has no single-runner guarantee** (this section's\n subject). Needs an advisory-lock lease or `FOR UPDATE SKIP LOCKED` with crash\n recovery, **plus** a two-instance test proving each job and each sweep runs\n exactly once.\n- **#338 — `IStepUpGrantRegistry` is process-local.** `InMemoryStepUpGrantRegistry`\n is a per-process singleton holding step-up replay tracking and logout epochs,\n and its own header says a multi-instance deployment must move it to a shared\n store. Both #308 guarantees degrade per replica: a single-use grant becomes\n usable **once per replica**, and a logout honoured by one replica is invisible\n to the others. These grants gate privileged account-control operations, so\n this is the blocker with teeth — closing #271 alone does not license scaling.\n- **The IP-keyed auth limiters (#143) are in-process.** `AddRateLimiter`'s\n partitions live in each process — login, refresh, and client-error reports\n alike — so N replicas allow roughly N times the intended attempts per IP\n before lockout.\n- **The per-account report concurrency cap (#311) is in-process.**\n `ReportConcurrencyLimiter` is a singleton owning a `PartitionedRateLimiter`\n (`Api/RateLimiting/ReportConcurrencyLimiter.cs`, registered at\n `Hosting/CluckworkRateLimitingServiceCollectionExtensions.cs`), so one account\n can hold up to N × `ReportsConcurrency.PermitLimit` heavyweight report queries\n in flight — the DB/CPU ceiling that cap exists to bound, multiplied by the\n replica count.\n\nThe last two have no open issue; they are recorded here rather than left to be\nrediscovered.\n\n**How this list was derived, because it was twice derived wrongly.** Both misses\nwere the same shape — a process-local limiter — and the second one lived in a\nfile the first sweep had already opened. So do not extend this list from memory.\nRe-derive it: enumerate **every** `AddSingleton`/`AddHostedService` under `src/\n",{"_391":499,"_370":371,"_393":377},{"_394":500,"_396":-5,"_45":-5,"_398":501,"_400":-5,"_99":502,"_95":402},[503],{"_403":377},"developer",{"_49":504,"_95":426},"Do not spawn sub-agents unless the user or applicable AGENTS.md/skill instructions explicitly ask for sub-agents, delegation, or parallel agent work.",{"_391":506,"_370":371,"_393":377},{"_394":507,"_396":-5,"_45":-5,"_398":508,"_400":-5,"_99":502,"_95":402},[509],{"_403":377},{"_49":510,"_95":426},"You are `/root`, the primary agent in a team of agents collaborating to fulfill the user's goals.\n\nAt the start of your turn, you are the active agent.\nYou can spawn sub-agents to handle subtasks, and those sub-agents can spawn their own sub-agents.\nAll agents in the team, including the agents that you can assign tasks to, are equally intelligent and capable, and have access to the same set of tools.\n\nYou can use `spawn_agent` to create a new agent, `followup_task` to give an existing agent a new task and trigger a turn, and `send_message` to pass a message to a running agent without triggering a turn.\nChild agents can also spawn their own sub-agents.\nYou can decide how much context you want to propagate to your sub-agents with the `fork_turns` parameter.\n\nYou will receive messages in the analysis channel in the form:\n```\nMessage Type: MESSAGE | FINAL_ANSWER\nTask name: \nSender: \nPayload:\n\n```\nThey may be addressed as to=/root\n\nNote that collaboration tools cannot be called from inside `functions.exec`. Call `spawn_agent`, `send_message`, `followup_task`, `wait_agent`, `interrupt_agent`, and `list_agents` only as direct tool calls using the recipient shown in their tool definitions, such as `to=functions.collaboration.spawn_agent`, since they are intentionally absent from the `functions.exec` `tools.*` namespace. Available tools in `functions.exec` are explicitly described with a `tools` namespace in the developer message.\n\nAll agents share the same directory. In detail:\n- All agents have access to the same container and filesystem as you.\n- All agents use the same current working directory.\n- As a result, edits made by one agent are immediately visible to all other agents.\n\nThere are 4 available concurrency slots, meaning that up to 4 agents can be active at once, including you.",{"_391":512,"_370":371,"_393":377},{"_394":513,"_396":-5,"_45":-5,"_398":514,"_400":-5,"_99":502,"_95":402},[515,516],{"_403":377},{"_49":518,"_95":426},{"_49":517,"_95":426},"\n## Skills\nA skill is a set of instructions provided through a `SKILL.md` source. Below is the list of skills that can be used. Each entry includes a name, description, and source locator. `file` locators are on the host filesystem, `environment resource` locators are owned by an execution environment, `orchestrator resource` locators are opaque non-filesystem resources, and `custom resource` locators use their provider's access mechanism.\n### Available skills\n- imagegen: Generate or edit raster images when the task benefits from AI-created bitmap visuals such as photos, illustrations, textures, sprites, mockups, or transparent-background cutouts. Use when Codex should create a brand-new image, transform an existing image, or derive visual variants from references, and the output should be a bitmap asset rather than repo-native code or vector. Do not use when the task is better handled by editing existing SVG/vector/code-native assets, extending an established icon or logo system, or building the visual directly in HTML/CSS/canvas. (file: /opt/codex/skills/.system/imagegen/SKILL.md)\n- openai-docs: Use when the user asks how to build with OpenAI products or APIs, asks about Codex itself or choosing Codex surfaces, needs up-to-date official documentation with citations, help choosing the latest model for a use case, or model upgrade and prompt-upgrade guidance; use OpenAI docs MCP tools for non-Codex docs questions, use the Codex manual helper first for broad Codex self-knowledge, and restrict fallback browsing to official OpenAI domains. (file: /opt/codex/skills/.system/openai-docs/SKILL.md)\n- plugin-creator: Create and scaffold plugin directories for Codex with a required `.codex-plugin/plugin.json`, optional plugin folders/files, valid manifest defaults, and personal-marketplace entries by default. Use when Codex needs to create a new personal plugin, add optional plugin structure, generate or update marketplace entries for plugin ordering and availability metadata, or update an existing local plugin during development with the CLI-driven cachebuster and reinstall flow. (file: /opt/codex/skills/.system/plugin-creator/SKILL.md)\n- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /opt/codex/skills/.system/skill-creator/SKILL.md)\n- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /opt/codex/skills/.system/skill-installer/SKILL.md)\n","\nFilesystem sandboxing defines which files can be read or written. `sandbox_mode` is `danger-full-access`: No filesystem sandboxing - all commands are permitted. Network access is enabled.\nApproval policy is currently never. Do not provide the `sandbox_permissions` for any reason, commands will be rejected.\n","turn/started",{"_370":371,"_90":521},{"_106":-5,"_45":377,"_378":522,"_380":381,"_382":383,"_384":523},[],"inProgress",{"_384":525,"_370":371},{"_526":527,"_95":528},"activeFlags",[],"active","thread/started",{"_531":532},"thread",{"_533":-5,"_534":-5,"_535":536,"_537":538,"_436":437,"_539":18,"_540":-5,"_541":-5,"_542":-5,"_543":544,"_45":371,"_545":546,"_192":-5,"_547":-5,"_167":548,"_549":206,"_550":538,"_551":371,"_442":552,"_384":553,"_554":-5,"_80":555,"_556":538},"agentNickname","agentRole","cliVersion","0.144.0-alpha.4","createdAt",1786929770,"ephemeral","extra","forkedFromId","gitInfo","historyMode","legacy","modelProvider","openai","parentThreadId","/opt/codex/sessions/2026/08/17/rollout-2026-08-17T01-22-50-01a00d50-6f76-7a70-8aaf-b53c39fddd8a.jsonl","preview","recencyAt","sessionId","vscode",{"_95":388},"threadSource",[],"updatedAt",[],"messages",[560],{"_45":561,"_562":563,"_564":565,"_566":565,"_394":567,"_384":568,"_396":-5,"_569":35,"_570":571,"_572":573,"_574":-5},"2ea479ae-1ce0-4bc0-8317-a05bd7a12f23","author",{"_99":336,"_192":-5,"_570":583},"create_time",1786929769.608449,"update_time",{"_579":49,"_580":581},"finished_successfully","weight","metadata",{"_575":576},"recipient","all","channel","l1239dk1",{"_577":578},"inline_comments",[],"content_type","parts",[582],"**Review loop wound down deliberately (owner decision).**\n\nTwo full pre-merge review rounds (codex + 2 Claude agents + pi each) plus this codex-bot round:\n- **Round 1** — 5 confirmed live product defects (timeout bypassing fail-closed/fallback; sub-second `DivideByZero`; `UriFormatException` crashing a one-shot verb; sweep evicting live longer-window counters; claim/lease key collision). Fixed + mutation-proven in `23a11bad`.\n- **Round 2** — 2 latent edge cases in the round-1 fix (fractional-ms undercount, `DateTimeOffset` overflow) + this round's 3 P2s (cluster hash-slot, sim blank-Redis, singleton inventory). All fixed in `e94bd4e6`; security-auditor round 2 found zero regressions.\n\nThe live single-node deployment path is correct and every fix is pinned by a test verified to fail without it. Remaining review yield had moved to latent edges / Redis-Cluster (we deploy single-node) / dev tooling / docs, so per our stop rule the loop was stopped at this point rather than tagging for further rounds. The three thread replies above record each fix and its verification. Not tagging `@codex` is intentional — this response is complete, not pending.\nfeat: shared-state ports with Redis + in-process fallback (#543)\n\nCloses #543. First slice of the scale-out epic (#530): three capability-specific\nshared-state ports with a Redis primary and an in-process fallback, so the\nmulti-instance blockers (#544/#545, #271/#338) have a shared store to build on.\nNo handler uses the ports yet — this slice is the seam.\n\n## Ports (capability-specific, not a generic \"redis client\")\n- `IClaimOnceStore` — single-use claim (`SET NX PX`). For step-up grant replay.\n- `IFixedWindowCounter` — atomic INCR+PEXPIRE, **server-time** bucketed (Lua `TIME`,\n so replicas agree regardless of host clock skew). For the auth limiters.\n- `ILease` — renewable compare-and-swap lease. For the report-concurrency cap.\n\n## Failure policy is per-capability, on purpose\n- **claim-once FAILS CLOSED** — a `RedisException` denies the claim (cannot prove\n single-use → refuse). It gates privileged operations; no fallback.\n- **counter + lease FALL BACK** to in-process and alarm\n (`SecurityEvents.SharedStateRedisUnavailable` — a deployment should alert on it).\n- Only `RedisException` is caught, never a bare `catch`.\n\n## Config / boot (#347)\n- `SharedState:Redis:ConnectionString` blank → in-process (Option B, single\n instance). Configured → Redis wrapped in the resilient decorators,\n `AbortOnConnectFail=false` so an **unreachable** Redis degrades at runtime\n rather than failing the boot.\n- A **set-but-malformed** string fails a **serving** boot only — a one-shot verb\n never uses shared state, so it degrades with a stderr warning. Discoverable\n serving-only guard (`ServingGuardCoverageTests`) with its `ProcessRoleGuardTests`\n row proving it fires.\n\n## Sim harness (#370)\n- redis sidecar (digest-pinned, same image as the Testcontainers fixture),\n `SharedState__Redis__*` wired into `bootstrap.sh` + `docker-compose.sim.yml`,\n and a bounded well-formedness check in `verify-harness.sh` (blank legal;\n set → must name an endpoint).\n\n## Verification\n- Testcontainers Redis contract tests (claim-once / counter / lease), DI-resolution\n tests, resilient-fallback tests (throwing stub → fallback + one alarm), boot-guard\n arms (serving fails naming its token; one-shot survives). `dotnet build` clean\n (warnings-as-errors); NuGet lock files committed (#146).\n- Sim harness: `verify-harness.sh` green, new check red on an options-only value;\n a full redis+db+app boot went healthy (`/health/ready` 200), no fallback alarm,\n redis reachable on the compose network.\n\n### 💡 Codex Review\n\nHere are some automated review suggestions for this pull request.\n\n**Reviewed commit:** `9c2ffa3423`\n \n\n

ℹ️ About Codex in GitHub

\n
\n\n[Your team has set up Codex to review pull requests in this repo](https://chatgpt.com/codex/cloud/settings/general). Reviews are triggered when you\n- Open a pull request for review\n- Mark a draft as ready\n- Comment \"@codex review\".\n\nIf Codex has suggestions, it will comment; otherwise it will react with 👍.\n\n\n\n\nCodex can also answer questions or update the PR. Try commenting \"@codex address that feedback\".\n \n

\n**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Keep the Lua counter key in its declared cluster slot**\n\nWhen the configured backend is Redis Cluster, the script is routed using `KEYS[1]` (`namespace:key`) but actually increments the dynamically derived `namespace:key:bucket`, which normally hashes to a different slot. Redis Cluster therefore rejects the script with a cross-slot/non-local-key error; the resilient decorator then falls back to per-process counters and silently loses the multi-replica rate-limit guarantee. Store the bucket and count under the declared key, or otherwise ensure every accessed key is explicitly declared in the same hash slot.\n\nUseful? React with 👍 / 👎.\n**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Reject a blank Redis endpoint in the simulation verifier**\n\nIf an ambient override sets `SharedState__Redis__ConnectionString=` explicitly, Compose resolves it without an unset-variable warning and this condition skips validation, so the verifier reports success while the app selects the in-process fallback. That means the human-run Production harness can stop exercising the Redis-backed path this change added without any indication; require a nonblank endpoint here even though blank remains legal for ordinary serving deployments.\n\nAGENTS.md reference: [AGENTS.md:L68-L68](https://github.com/mforce/cluckwork/blob/9c2ffa3423702d161025458f8fc13744d71c9482/AGENTS.md#L68-L68)\n\nUseful? React with 👍 / 👎.\n**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Update the singleton state inventory**\n\nThese new registrations add a connection multiplexer and three shared-state singleton services, including mutable in-process fallbacks, but the repository's deployment-invariant audit still records the previous singleton inventory and classifications. Future scale-out reviews are explicitly expected to enumerate and classify every singleton and in-memory primitive, so leaving these entries absent makes that checkpoint stale and risks treating the remaining single-instance blockers as closed without reviewing fallback behavior.\n\nAGENTS.md reference: [AGENTS.md:L159-L167](https://github.com/mforce/cluckwork/blob/9c2ffa3423702d161025458f8fc13744d71c9482/AGENTS.md#L159-L167)\n\nUseful? React with 👍 / 👎.\nFixed in e94bd4e6. The Lua counter key now wraps the routing part in a hash tag — `{namespace:win:key}` — so `KEYS[1]` and the derived `KEYS[1]:bucket` hash to the same slot and Redis Cluster accepts the script (inert braces on single-node). The class doc was changed from a \"single-node only\" constraint to documenting the hash-tag. Verified: full SharedState suite green against the Testcontainers Redis, including the counter contract tests that exercise the new key shape.\nFixed in e94bd4e6. `verify-harness.sh` now FAILS on an unset/blank `SharedState__Redis__ConnectionString` — a harness-specific policy, deliberately stronger than the app guard (blank stays legal for an ordinary serving deploy; the sim wires Redis on purpose, so a blank silently drops the Redis-backed path). Verified: green on the real `redis:6379` value; `SharedState__Redis__ConnectionString='' bash verify-harness.sh` now exits 1 naming the drop. The rationale is recorded in the check's comment block.\nFixed in e94bd4e6. AGENTS.md's deploy-invariant singleton inventory now records and classifies the four #543 registrations (`IConnectionMultiplexer` + the three ports). They are classified as the shared store that will *close* the in-process limiter blockers (#544 → `IFixedWindowCounter`, #545 → `ILease`, #338 grant-replay → `IClaimOnceStore`), not new blockers: Redis-backed they are multi-replica-safe, and their in-process fallbacks are a deliberate, alarmed degradation (fail-closed for claim-once). Noted that #543 only lands the ports — the blockers stay open until a caller is wired.",{},"created",1786929790.3091261,"Completing the task","event_type","info",{"_95":402,"_99":96,"_394":594},{"_95":591,"_592":593},"partial_repo_snapshot","files",[],[595],{"_579":49,"_49":405},{"_95":336,"_45":87,"_99":336,"_598":599,"_600":601,"_602":603,"_135":-5,"_122":604},[86],"input_items",[624,625,626,627,628,629,630,631,632],"external_post_config",{"_605":606,"_272":203,"_607":315,"_283":-5,"_608":266,"_609":-5,"_610":-5,"_611":612,"_613":614,"_615":-5,"_616":-5,"_617":618,"_619":18,"_620":62,"_621":57,"_622":18,"_623":-5},"user_model_slug","slug",1786929706.717691,"installation_id",150444056,"pr_number","repo_id","gitlab_service_account_project_id","review_trigger","comment_id",5310752424,"comment_type","issue","pr_write_guard_token","pending_pr_reaction_id","pending_comment_reaction_id",398959149,"comment_if_no_findings","comment_on_error","share_url","is_parallel_review","review_summary_id",{"_95":402,"_99":336,"_394":653},{"_322":323,"_324":325,"_649":650,"_651":132,"_332":-5,"_95":652},{"_95":633,"_324":648,"_167":-5,"_636":-5,"_638":-5},{"_95":633,"_324":647,"_167":643,"_636":637,"_638":644},{"_95":633,"_324":646,"_167":641,"_636":637,"_638":-5},{"_95":633,"_324":645,"_167":635,"_636":637,"_638":639},{"_95":633,"_324":642,"_167":643,"_636":637,"_638":644},{"_95":633,"_324":640,"_167":641,"_636":637,"_638":-5},{"_95":633,"_324":634,"_167":635,"_636":637,"_638":639},"pull_request_comment","Fixed in e94bd4e6. AGENTS.md's deploy-invariant singleton inventory now records and classifies the four #543 registrations (`IConnectionMultiplexer` + the three ports). They are classified as the shared store that will *close* the in-process limiter blockers (#544 → `IFixedWindowCounter`, #545 → `ILease`, #338 grant-replay → `IClaimOnceStore`), not new blockers: Redis-backed they are multi-replica-safe, and their in-process fallbacks are a deliberate, alarmed degradation (fail-closed for claim-once). Noted that #543 only lands the ports — the blockers stay open until a caller is wired.","src/Cluckwork.Infrastructure/SharedState/SharedStateRegistration.cs","side","RIGHT","line",53,"Fixed in e94bd4e6. `verify-harness.sh` now FAILS on an unset/blank `SharedState__Redis__ConnectionString` — a harness-specific policy, deliberately stronger than the app guard (blank stays legal for an ordinary serving deploy; the sim wires Redis on purpose, so a blank silently drops the Redis-backed path). Verified: green on the real `redis:6379` value; `SharedState__Redis__ConnectionString='' bash verify-harness.sh` now exits 1 naming the drop. The rationale is recorded in the check's comment block.","tools/simulation/verify-harness.sh","Fixed in e94bd4e6. The Lua counter key now wraps the routing part in a hash tag — `{namespace:win:key}` — so `KEYS[1]` and the derived `KEYS[1]:bucket` hash to the same slot and Redis Cluster accepts the script (inert braces on single-node). The class doc was changed from a \"single-node only\" constraint to documenting the hash-tag. Verified: full SharedState suite green against the Testcontainers Redis, including the counter contract tests that exercise the new key shape.","src/Cluckwork.Infrastructure/SharedState/RedisFixedWindowCounter.cs",28,"**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Update the singleton state inventory**\n\nThese new registrations add a connection multiplexer and three shared-state singleton services, including mutable in-process fallbacks, but the repository's deployment-invariant audit still records the previous singleton inventory and classifications. Future scale-out reviews are explicitly expected to enumerate and classify every singleton and in-memory primitive, so leaving these entries absent makes that checkpoint stale and risks treating the remaining single-instance blockers as closed without reviewing fallback behavior.\n\nAGENTS.md reference: [AGENTS.md:L159-L167](https://github.com/mforce/cluckwork/blob/9c2ffa3423702d161025458f8fc13744d71c9482/AGENTS.md#L159-L167)\n\nUseful? React with 👍 / 👎.","**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Reject a blank Redis endpoint in the simulation verifier**\n\nIf an ambient override sets `SharedState__Redis__ConnectionString=` explicitly, Compose resolves it without an unset-variable warning and this condition skips validation, so the verifier reports success while the app selects the in-process fallback. That means the human-run Production harness can stop exercising the Redis-backed path this change added without any indication; require a nonblank endpoint here even though blank remains legal for ordinary serving deployments.\n\nAGENTS.md reference: [AGENTS.md:L68-L68](https://github.com/mforce/cluckwork/blob/9c2ffa3423702d161025458f8fc13744d71c9482/AGENTS.md#L68-L68)\n\nUseful? React with 👍 / 👎.","**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Keep the Lua counter key in its declared cluster slot**\n\nWhen the configured backend is Redis Cluster, the script is routed using `KEYS[1]` (`namespace:key`) but actually increments the dynamically derived `namespace:key:bucket`, which normally hashes to a different slot. Redis Cluster therefore rejects the script with a cross-slot/non-local-key error; the resilient decorator then falls back to per-process counters and silently loses the multi-replica rate-limit guarantee. Store the bucket and count under the declared key, or otherwise ensure every accessed key is explicitly declared in the same hash slot.\n\nUseful? React with 👍 / 👎.","\n### 💡 Codex Review\n\nHere are some automated review suggestions for this pull request.\n\n**Reviewed commit:** `9c2ffa3423`\n \n\n

ℹ️ About Codex in GitHub

","base_ref","711488a9b32cf59eeacd4e5bdd5aa6586c3f8b50","head_ref","pull_request_info",[654],{"_579":49,"_49":655},"**Review loop wound down deliberately (owner decision).**\n\nTwo full pre-merge review rounds (codex + 2 Claude agents + pi each) plus this codex-bot round:\n- **Round 1** — 5 confirmed live product defects (timeout bypassing fail-closed/fallback; sub-second `DivideByZero`; `UriFormatException` crashing a one-shot verb; sweep evicting live longer-window counters; claim/lease key collision). Fixed + mutation-proven in `23a11bad`.\n- **Round 2** — 2 latent edge cases in the round-1 fix (fractional-ms undercount, `DateTimeOffset` overflow) + this round's 3 P2s (cluster hash-slot, sim blank-Redis, singleton inventory). All fixed in `e94bd4e6`; security-auditor round 2 found zero regressions.\n\nThe live single-node deployment path is correct and every fix is pinned by a test verified to fail without it. Remaining review yield had moved to latent edges / Redis-Cluster (we deploy single-node) / dev tooling / docs, so per our stop rule the loop was stopped at this point rather than tagging for further rounds. The three thread replies above record each fix and its verification. Not tagging `@codex` is intentional — this response is complete, not pending.",[800,801],[],{"_584":585,"_49":586,"_587":588},{"_558":791},{"_347":790},{"_347":694},{"_314":315,"_304":316,"_317":318,"_319":62,"_320":18,"_321":18,"_322":323,"_324":325,"_326":276,"_327":130,"_328":329,"_330":331,"_332":333,"_334":-5,"_335":-5,"_336":693,"_338":339,"_340":341,"_342":343,"_344":345},{"_45":126,"_200":201,"_202":203,"_204":18,"_205":206,"_207":208,"_209":680,"_211":-5,"_212":213,"_214":18,"_215":681,"_217":682,"_219":683,"_221":684,"_223":685,"_225":686,"_227":-5,"_228":229,"_230":231,"_232":233,"_234":687,"_58":-5,"_122":236,"_237":688,"_239":689,"_241":242,"_243":18,"_244":18},{"_136":675},[666,667,668,669,670,671,672,673,674],{"_154":184,"_156":157,"_158":159,"_160":169,"_162":176,"_164":165,"_166":35,"_167":-5},{"_154":180,"_156":181,"_158":182,"_160":169,"_162":183,"_164":165,"_166":35,"_167":-5},{"_154":179,"_156":157,"_158":159,"_160":161,"_162":172,"_164":165,"_166":35,"_167":-5},{"_154":177,"_156":157,"_158":159,"_160":171,"_162":172,"_164":165,"_166":178,"_167":-5},{"_154":175,"_156":157,"_158":159,"_160":169,"_162":176,"_164":165,"_166":35,"_167":-5},{"_154":174,"_156":157,"_158":159,"_160":161,"_162":172,"_164":165,"_166":35,"_167":-5},{"_154":170,"_156":157,"_158":159,"_160":171,"_162":172,"_164":165,"_166":173,"_167":-5},{"_154":168,"_156":157,"_158":159,"_160":169,"_162":163,"_164":165,"_166":35,"_167":-5},{"_154":155,"_156":157,"_158":159,"_160":161,"_162":163,"_164":165,"_166":35,"_167":-5},[676,677],{"_188":679,"_190":197},{"_188":678,"_190":191},{"_192":193,"_95":194,"_122":195},{"_192":198,"_95":194,"_122":199},{"_306":307,"_308":72,"_309":310,"_311":-5,"_312":-5,"_313":-5},[266],[206],[206],{"_266":690},{"_248":249,"_250":251,"_252":253,"_254":255,"_256":257,"_258":259,"_260":261,"_262":263,"_264":265},{},[],{"_247":62},{"_245":62,"_246":206},{"_45":268,"_192":269,"_270":691,"_272":203,"_273":274,"_275":276,"_277":278,"_279":280,"_281":282,"_283":67,"_284":18,"_285":18,"_286":62,"_287":62,"_288":62,"_289":18,"_290":-5,"_166":291,"_292":-5,"_58":692},{"_298":299,"_45":300,"_95":301,"_302":303,"_304":305},{"_294":62,"_295":62,"_296":62,"_297":62},{"_298":299,"_192":299,"_346":-5,"_302":303,"_45":300},[695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712],{"_160":529,"_368":786},{"_160":385,"_368":783},{"_160":519,"_368":780},{"_160":389,"_368":774},{"_160":389,"_368":769},{"_160":389,"_368":764},{"_160":389,"_368":758},{"_160":389,"_368":753},{"_160":407,"_368":748},{"_160":407,"_368":744},{"_160":389,"_368":739},{"_160":389,"_368":736},{"_160":407,"_368":732},{"_160":389,"_368":725},{"_160":407,"_368":723},{"_160":389,"_368":718},{"_160":385,"_368":716},{"_160":367,"_368":713},{"_370":371,"_90":714},{"_373":374,"_375":376,"_106":-5,"_45":377,"_378":715,"_380":381,"_382":383,"_384":116},[],{"_384":717,"_370":371},{"_95":388},{"_391":719,"_370":371,"_393":377},{"_394":720,"_396":-5,"_45":397,"_398":721,"_400":401,"_99":96,"_95":402},[722],{"_403":377},{"_49":405,"_95":406},{"_409":410,"_391":724,"_370":371,"_393":377},{"_45":397,"_400":401,"_49":405,"_95":412},{"_391":726,"_370":371,"_393":377},{"_415":416,"_45":-5,"_398":727,"_192":-5,"_418":728,"_95":420},{"_403":377},{"_324":729,"_422":-5},[730,731],{"_49":427,"_95":426},{"_49":425,"_95":426},{"_409":429,"_391":733,"_370":371,"_393":377},{"_431":425,"_432":433,"_434":734,"_436":437,"_375":35,"_438":35,"_45":439,"_440":441,"_442":443,"_384":116,"_95":444},[735],{"_432":446,"_95":447},{"_391":737,"_370":371,"_393":377},{"_415":416,"_45":450,"_451":452,"_398":738,"_192":454,"_455":-5,"_384":116,"_95":456},{"_403":377},{"_391":740,"_370":371,"_393":377},{"_394":-5,"_459":460,"_45":461,"_398":741,"_463":742,"_95":465},{"_403":377},[743],{"_49":467,"_95":468},{"_409":470,"_391":745,"_370":371,"_393":377},{"_394":746,"_45":461,"_463":747,"_95":465},[],[467],{"_409":475,"_391":749,"_370":371,"_393":377},{"_477":-5,"_394":750,"_45":479,"_95":480},[751],{"_49":482,"_483":752,"_95":49},[],{"_391":754,"_370":371,"_393":377},{"_394":755,"_396":-5,"_45":-5,"_398":756,"_400":-5,"_99":336,"_95":402},[757],{"_403":377},{"_49":482,"_95":426},{"_391":759,"_370":371,"_393":377},{"_394":760,"_396":-5,"_45":-5,"_398":761,"_400":-5,"_99":336,"_95":402},[762,763],{"_403":377},{"_49":497,"_95":426},{"_49":496,"_95":426},{"_391":765,"_370":371,"_393":377},{"_394":766,"_396":-5,"_45":-5,"_398":767,"_400":-5,"_99":502,"_95":402},[768],{"_403":377},{"_49":504,"_95":426},{"_391":770,"_370":371,"_393":377},{"_394":771,"_396":-5,"_45":-5,"_398":772,"_400":-5,"_99":502,"_95":402},[773],{"_403":377},{"_49":510,"_95":426},{"_391":775,"_370":371,"_393":377},{"_394":776,"_396":-5,"_45":-5,"_398":777,"_400":-5,"_99":502,"_95":402},[778,779],{"_403":377},{"_49":518,"_95":426},{"_49":517,"_95":426},{"_370":371,"_90":781},{"_106":-5,"_45":377,"_378":782,"_380":381,"_382":383,"_384":523},[],{"_384":784,"_370":371},{"_526":785,"_95":528},[],{"_531":787},{"_533":-5,"_534":-5,"_535":536,"_537":538,"_436":437,"_539":18,"_540":-5,"_541":-5,"_542":-5,"_543":544,"_45":371,"_545":546,"_192":-5,"_547":-5,"_167":548,"_549":206,"_550":538,"_551":371,"_442":552,"_384":788,"_554":-5,"_80":789,"_556":538},{"_95":388},[],[],[792],{"_45":561,"_562":793,"_564":565,"_566":565,"_394":794,"_384":568,"_396":-5,"_569":35,"_570":795,"_572":573,"_574":-5},{"_99":336,"_192":-5,"_570":799},{"_579":49,"_580":798},{"_575":796},{"_577":797},[],[582],{},{"_95":402,"_99":96,"_394":803},{"_95":591,"_592":802},[],[804],{"_579":49,"_49":405},[807,808,809,810,811,812,813,814,815],{"_605":606,"_272":203,"_607":315,"_283":-5,"_608":266,"_609":-5,"_610":-5,"_611":612,"_613":614,"_615":-5,"_616":-5,"_617":618,"_619":18,"_620":62,"_621":57,"_622":18,"_623":-5},{"_95":402,"_99":336,"_394":816},{"_322":323,"_324":325,"_649":650,"_651":132,"_332":-5,"_95":652},{"_95":633,"_324":648,"_167":-5,"_636":-5,"_638":-5},{"_95":633,"_324":647,"_167":643,"_636":637,"_638":644},{"_95":633,"_324":646,"_167":641,"_636":637,"_638":-5},{"_95":633,"_324":645,"_167":635,"_636":637,"_638":639},{"_95":633,"_324":642,"_167":643,"_636":637,"_638":644},{"_95":633,"_324":640,"_167":641,"_636":637,"_638":-5},{"_95":633,"_324":634,"_167":635,"_636":637,"_638":639},[817],{"_579":49,"_49":655},"task_e_6a82622977688328a91689798f73ffd0",1786929705.466582,"GitHub Mention: feat: shared-state ports with Redis + in-process fallback (#543)","has_generated_title","has_unread_turn","denormalized_metadata",{"_848":35,"_849":850,"_851":-5,"_852":-5,"_843":853,"_854":-5,"_855":-5,"_856":-5,"_857":-5},"task_status_display",{"_837":838,"_839":203,"_133":130,"_840":-5,"_841":842},"creator_workspace","external_pull_requests",[830],{"_45":831,"_832":86,"_833":834,"_835":132},"github-1301193065-552","assistant_turn_id","pull_request",{"_314":315,"_304":316,"_317":318,"_319":62,"_320":18,"_321":18,"_322":323,"_324":325,"_326":276,"_327":130,"_328":329,"_330":331,"_332":333,"_334":-5,"_335":-5,"_336":836,"_338":339,"_340":341,"_342":343,"_344":345},"codex_updated_sha",{"_298":299,"_192":299,"_346":-5,"_302":303,"_45":300},"latest_turn_status_display",{"_403":86,"_115":116,"_843":-5,"_97":844,"_845":-5,"_846":-5,"_847":842,"_117":-5},"environment_label","fix_task_id","initial_intent","pr","diff_stats",[],"image_asset_pointer_citations","issue_counts","intent","num_follow_ups","repo_name_map",{},"last_user_turn_created_at","last_turn_status",{"_858":-5,"_859":-5,"_860":-5},"last_pushed_turn_id","start_branch_name","pushed_branch_name","cached_pull_request_data","files_modified","lines_added","lines_removed","dd",{"_875":876,"_877":878},"disablePrefetch","shouldPrefetchAccount","shouldPrefetchUser","shouldPrefetchSystemHints","manageRootRouteStylesheet","mobileWebHandoffEnabled","promoteCss","disableSSR","stripModulepreloadImports","paidCampaignAppRedirect","statsigGateEvaluationsPromise",["P",874],"traceId","2327067123892731506","traceTime",1787294918251]