\\n\\n\\n\\n\",\"path\":null,\"side\":null,\"line\":null}{\"type\":\"pull_request_comment\",\"body\":\"\\n\\n**This PR introduces a dedicated-box routing feature for specific model families; the changes are operationally significant but security-neutral with one minor concern worth noting.**\\n\\n---\\n\\n### Trust boundaries touched\\n\\n- **TB-002** (Coordinator → Provider WebSocket) — routing gate logic in `registry.go`\\n- **TB-006** (Admin → Coordinator) — new env-var configuration surface in `main.go`\\n\\n---\\n\\n### Threat analysis\\n\\n**T-034** (Provider runs modified code while advertising a trusted identity) — `coordinator/registry/registry.go` lines +2985–2988, +3072–3075\\n\\nℹ️ **Neutral.** `providerExcludedByDedicatedRuleLocked` and `providerHasWarmModelLocked` are routing/scheduling helpers that sit downstream of the APNs code-identity gate (`providerSupportsPrivateTextLocked`). The dedicated-model exclusion fires on catalog shape (`dedicatedModels` substring match), not on any attestation field, so it cannot be manipulated by a provider to bypass or strengthen the `CodeAttested` gate. No change to trust-level assignment.\\n\\n**T-038** (HTTP server config) — `coordinator/cmd/coordinator/main.go`\\n\\nℹ️ **Neutral.** The diff adds only the dedicated-models config block; `http.Server` fields (`ReadHeaderTimeout`, `MaxHeaderBytes`, etc.) are not touched.\\n\\n**T-032 / T-041** (Solana mnemonic / profile signing identity) — `coordinator/cmd/coordinator/main.go`\\n\\nℹ️ **Neutral.** No secrets handling added or modified.\\n\\n---\\n\\n### New attack surface not covered by an existing threat\\n\\n**Operator-controlled routing topology via `EIGENINFERENCE_DEDICATED_MODELS`**\\n\\n`main.go` +168–180: the env-var is read with `os.LookupEnv` and fed into `registry.ParseDedicatedModels`. This is a coordinator-side config variable (not a provider-supplied value), so ADV-001 cannot influence it directly. However:\\n\\n1. **Log-based enumeration** (`main.go` +180): `logger.Info(\\\"dedicated-model routing ENABLED\\\", \\\"patterns\\\", strings.Join(dedicatedModels, \\\",\\\"))` emits the full pattern list to structured logs. If logs are accessible to any consumer or provider (e.g. via a log-aggregation misconfiguration), this leaks the exact routing topology — which model families have dedicated boxes and by implication which do not. This is low severity but worth being deliberate about the log destination.\\n\\n2. **Shed-to-OpenRouter path**: the comment at `main.go` +163 (\\\"sheds to OpenRouter as a transient 429\\\") introduces a new reachable code path where a request for a dedicated model with no available dedicated box gets a different response code and destination than other capacity-shed requests. A consumer (ADV-002) who systematically sends requests for `gemma-4` variants could probe for the presence/absence of dedicated capacity by observing the 429 cadence. This is informational leakage about infrastructure topology — not covered by any existing threat. Severity is low given the non-sensitive nature of the information, but it should be acknowledged.\\n\\nNo existing threat ID maps cleanly to either of these; the closest is T-006 (registry flooding/enumeration), but that targets the provider WebSocket, not consumer-facing response codes.\\n\\n---\\n\\n### Open findings resolved\\n\\nNone — this PR does not touch any SEC-* open finding.\\n\\n---\\n\\n### Summary for reviewers\\n\\nThe core routing logic changes (`providerExcludedByDedicatedRuleLocked` calls in `registry.go`) are straightforward exclusion filters that cannot be influenced by adversarial providers. The only items to consider before merge are:\\n\\n- Confirm `logger.Info` for dedicated-model patterns goes only to operator-accessible sinks, not to any consumer-visible telemetry endpoint.\\n- Decide whether the 429 response for \\\"no dedicated box available\\\" leaks topology information at a sensitivity level that warrants a generic error message instead.\\n\\n---\\n*🔐 Threat model: [`docs/threat-model.yaml`](../blob/HEAD/docs/threat-model.yaml) · Updates on each push to this PR*\",\"path\":null,\"side\":null,\"line\":null}{\"type\":\"pull_request_comment\",\"body\":\"\\n### 💡 Codex Review\\n\\nHere are some automated review suggestions for this pull request.\\n\\n**Reviewed commit:** `a5fc81f361`\\n \\n\\n

ℹ️ About Codex in GitHub

\\n
\\n\\n[Your team has set up Codex to review pull requests in this repo](https://chatgpt.com/codex/cloud/settings/general). Reviews are triggered when you\\n- Open a pull request for review\\n- Mark a draft as ready\\n- Comment \\\"@codex review\\\".\\n\\nIf Codex has suggestions, it will comment; otherwise it will react with 👍.\\n\\n\\n\\n\\nCodex can also answer questions or update the PR. Try commenting \\\"@codex address that feedback\\\".\\n \\n

\",\"path\":null,\"side\":null,\"line\":null}{\"type\":\"pull_request_comment\",\"body\":\"**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Shed saturated dedicated models before queueing**\\n\\nWhen every dedicated provider for a Gemma-4 request exists but is already at concurrency/token capacity, `QuickCapacityCheckWithTTFTForRequest` reports `candidateCount == 0` with `capacityRejections > 0`, so the earlier queue-before-shed block handles the request and this new 429 branch is never reached. With the default queue-before-shed setting, OpenRouter traffic can therefore sit in the 120s dispatch queue instead of receiving the intended `Retry-After` 429 whenever dedicated boxes are present but saturated; the mirrored `handleGenericInference` path has the same ordering.\\n\\nUseful? React with 👍 / 👎.\",\"path\":\"coordinator/api/consumer.go\",\"side\":\"RIGHT\",\"line\":2105}{\"type\":\"pull_request_comment\",\"body\":\"**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Apply routing gates before treating a provider as available**\\n\\nWhen the only advertiser for a dedicated model is private-only, below the public trust floor, runtime-unverified, lacks private-text support, or has a stale challenge, public routing still rejects it in `providerPassesRoutingGatesLockedEx`, but this check returns true because it only checks status and catalog membership. In that context the new preflight branch classifies an unroutable public fleet as `machine_busy` 429 instead of the `no_provider` 503 path, so clients are told to retry/fail over as if capacity might free when no public provider can actually route the model.\\n\\nUseful? React with 👍 / 👎.\",\"path\":\"coordinator/registry/dedicated_models.go\",\"side\":\"RIGHT\",\"line\":135}","text_elements",[],{"_579":863,"_560":561,"_581":567},{"_582":864,"_584":-5,"_42":-5,"_585":-5,"_586":-5,"_96":485,"_92":588},[865],{"_46":859,"_92":866},"input_text",{"_579":868,"_560":561,"_581":567},{"_582":869,"_584":-5,"_42":-5,"_585":-5,"_586":-5,"_96":485,"_92":588},[870,871],{"_46":873,"_92":866},{"_46":872,"_92":866},"\n /workspace/d-inference\n bash\n 2026-06-21\n Etc/UTC\n","# AGENTS.md instructions for /workspace/d-inference\n\n\n# Darkbloom - Decentralized Private Inference\n\nDarkbloom is a decentralized private inference network for Apple Silicon Macs. Consumers use OpenAI-compatible APIs, the coordinator handles routing, auth, billing, attestation, and capacity management, and providers run local inference workloads on macOS hardware using MLX-Swift. All inference is end-to-end encrypted -- the coordinator never sees plaintext prompts.\n\n## Project Structure\n\n```text\ncoordinator/ Go control plane (packages live at top level, not internal/)\n├── cmd/coordinator/ main service entrypoint\n├── api/ HTTP + WebSocket handlers\n│ ├── consumer.go OpenAI-compatible chat/completions/messages/transcriptions/images\n│ ├── provider.go provider registration, heartbeats, attestation, relay\n│ ├── billing_handlers.go Stripe/Solana/referral/pricing endpoints\n│ ├── device_auth.go device code flow for linking providers to user accounts\n│ ├── enroll.go MDM + ACME enrollment profile generation\n│ ├── invite_handlers.go invite code admin/user flows\n│ ├── release_handlers.go binary release registration (GitHub Actions integration)\n│ ├── acme_verify.go ACME device-attest-01 client cert verification\n│ ├── stats.go public network stats\n│ └── server.go route wiring, auth middleware, version gate\n├── attestation/ Secure Enclave + MDA verification\n├── auth/ Privy JWT integration\n├── billing/ Stripe, referrals\n├── e2e/ X25519 request-encryption helpers\n├── mdm/ MicroMDM client + webhook handling\n├── payments/ ledger + pricing\n├── protocol/ WebSocket message types shared with provider\n├── ratelimit/ rate limiting\n├── registry/ provider registry, queueing, routing, reputation, token-budget admission\n├── saferun/ panic-safe goroutine runners\n├── store/ in-memory or Postgres persistence\n├── telemetry/ Datadog DogStatsD metrics\n├── datadog/ dev dashboard JSON definitions\n└── internal/e2e/ coordinator-scoped integration tests\n\ne2e/ System-level E2E testing framework\n├── integration_test.go 12 E2E tests (streaming, billing, encryption, attestation, etc.)\n├── profile_test.go latency profiling tests\n├── benchmark_test.go load benchmarks (posts markdown to PR comments)\n└── testbed/ shared test harness\n ├── coordinator.go Coordinator lifecycle (start/stop, Postgres helpers)\n ├── provider.go Provider lifecycle (binary discovery, start/stop)\n ├── config.go Test configuration (model, provider, request settings)\n ├── suite.go Suite orchestration (multi-provider, user pools)\n ├── events.go Event system (segments, buffers, fan-out)\n ├── instrument.go Request-level instrumentation\n ├── load.go Load generator (concurrency, streaming, metrics)\n ├── assert/ Latency threshold + accounting integrity assertions\n ├── deps/ External dependency lifecycle (ephemeral Postgres)\n └── profile/ Segment stats aggregation, diffing, JSON export\n\nprovider-swift/ Swift provider CLI for Apple Silicon Macs\n├── Sources/ProviderCore/ coordinator client, protocol, hardware, security, inference, server, telemetry, model downloads\n├── Sources/ProviderCoreFoundation/ model manifests, scanner, hashing, publish-safe foundation code\n├── Sources/darkbloom/ CLI (`serve`, `start`, `stop`, `models`, `benchmark`, `status`, `doctor`, `login`, etc.)\n├── Sources/darkbloom-publish/ registry manifest builder used by publish workflow\n├── Sources/darkbloom-enclave-cli/ Secure Enclave attestation/sign helper\n└── Tests/ ProviderCore and ProviderCoreFoundation tests\n\nconsole-ui/ Next.js 16 / React 19 frontend\n├── src/app/ chat, billing, images, models, stats, providers, settings, link, api-console, earn\n├── src/app/api/ chat, images, transcribe, auth/keys, payments/*, invite, models, health, pricing\n├── src/components/ chat UI, sidebar, top bar, trust badge, verification panel, invite banner\n├── src/components/providers/\n│ ├── PrivyClientProvider.tsx\n│ └── ThemeProvider.tsx\n├── src/lib/ API client (api.ts) + Zustand store (store.ts)\n├── src/hooks/ auth (useAuth.ts) + toast (useToast.ts)\n└── proxy.ts Next.js 16 proxy (replaces middleware.ts)\n\nscripts/ build, signing, install, and deploy helpers\n├── install.sh end-user installer served from coordinator (hash + codesign verification)\n├── admin.sh admin CLI (Privy auth, release mgmt, API calls)\n├── publish-model.sh model registry publish workflow\n├── deploy-acme.sh nginx/step-ca helper\n├── fetch-metallib.sh MLX metallib fetcher\n└── entitlements.plist hardened runtime entitlements (hypervisor, network)\n\ndocs/ architecture, deploy runbooks, MDM/ACME notes, threat model\n.github/workflows/ CI (ci.yml), integration tests (integration.yml), Swift release (release-swift.yml),\n model registration (register-model.yml), threat model review (threat-model-review.yml)\n```\n\n## Current Surface Area\n\n- Coordinator HTTP routes include `POST /v1/chat/completions`, `POST /v1/completions`, `POST /v1/messages`, `GET /v1/models`, `GET /v1/models/capacity`, billing/pricing endpoints, invite flows, stats, enrollment, device authorization, and release registration endpoints.\n- Coordinator auth is split between Privy JWTs, API keys, and device-code login (RFC 8628) for provider machines.\n- Routing uses token-budget admission with engine-reported capacity, speculative TTFT dispatch, EWMA TPS tracking, and early 429 with Retry-After for OpenRouter compatibility.\n- Billing logic is split between `coordinator/payments` (ledger + pricing) and `coordinator/billing` (Stripe, referrals).\n- Providers serve text inference through the Swift `darkbloom` CLI with continuous batching via MLX-Swift.\n- Model registry data is DB-backed in the coordinator and points to R2 manifests under `https://models.darkbloom.ai`; model bytes are not hardcoded in the provider or UI.\n- Observability: Datadog metrics (DogStatsD) for attestation, routing, billing, fleet version, and provider capacity. X-Timing header decomposes per-request latency.\n\n## Building And Testing\n\nToolchain versions (Go, Node, Swift, Python, plus `jq`/`gh`/`awscli`/`gcloud`)\nare pinned in [`mise.toml`](mise.toml). Build/test commands are wrapped in the\nroot [`Makefile`](Makefile) — run `make` with no args to list all targets.\n\n### One-time setup\n```bash\nmise install # installs every tool pinned in mise.toml\nmake ui-install # console-ui npm deps\n```\n\n### Coordinator (Go)\n```bash\nmake coordinator-test # cd coordinator && go test ./...\nmake coordinator-build # cd coordinator && go build ./cmd/coordinator\nmake coordinator-build-linux # GOOS=linux GOARCH=amd64 CGO_ENABLED=0 build (EigenCloud)\nmake coordinator # test + build\n```\n\n### Provider (Swift)\n```bash\nmake provider-build # cd provider-swift && swift build\nmake provider-test # cd provider-swift && swift test\nmake provider # build + test\n```\n\n### Console UI (Next.js 16)\n```bash\nmake ui-install # npm install\nmake ui-build # npm run build\nmake ui-lint # npx eslint src/\nmake ui-test # vitest (npm test)\nmake ui # install + lint + test + build\n```\n\n### E2E Integration Tests\n```bash\n# Requires Postgres + Swift provider binary + MLX model downloaded.\nmake e2e-integration # go test ./e2e/... -run TestIntegration -v\nmake e2e-benchmark # go test ./e2e/... -run TestBenchmark -v\n```\n\n### Aggregates\n```bash\nmake test # all unit tests (coordinator + provider + ui)\nmake build # build all components\nmake all # test + build everything\nmake clean # remove built artifacts\n```\n\n## Deploying\n\nCanonical runbook: `docs/coordinator-deploy-runbook.md`\n\nCurrent release-sensitive pieces:\n\n- Prod coordinator runs on EigenCloud (TEE) as app `d-inference` at `api.darkbloom.dev`. Build target: `coordinator/Dockerfile`. Dev coordinator runs on Google Cloud (see `docs/dev-environment.md`).\n- Provider bundle creation lives in `scripts/build-bundle.sh`.\n- App bundle + DMG creation lives in `scripts/bundle-app.sh`.\n- Installer flow lives in `scripts/install.sh`.\n- Provider update checks use `LatestProviderVersion` in `coordinator/api/server.go`, so bundle uploads and version bumps need to stay coordinated.\n- CI release workflow (`release-swift.yml`) signs binaries with Developer ID Application cert, notarizes with Apple, computes SHA-256 hashes after signing, embeds provisioning profile in .app bundle.\n\nQuick coordinator deploy (prod, EigenCloud):\n\n```bash\n# EigenCloud builds from the repo via coordinator/Dockerfile and blue-green deploys.\ngit push origin master\necloud compute app deploy d-inference\ncurl https://api.darkbloom.dev/health\necloud compute app logs d-inference\n```\n\nDev coordinator deploy (Google Cloud): see `docs/dev-environment.md`.\n\n## Important Sync Points\n\n- Protocol changes must be mirrored in both `provider-swift/Sources/ProviderCore/Protocol/` and `coordinator/protocol/messages.go`.\n- Telemetry wire types live in three places and MUST stay aligned:\n - `coordinator/protocol/telemetry.go` (canonical),\n - `provider-swift/Sources/ProviderCore/Telemetry/` (Swift mirror),\n - `console-ui/src/lib/telemetry-types.ts` (TS mirror).\n Symmetry tests in each language pin enum casing and optional-field omission.\n Field allowlist additions need parallel updates in\n `coordinator/api/telemetry_handlers.go`,\n `provider-swift/Sources/ProviderCore/Telemetry/`, and the TS set above.\n- If you change provider bundle semantics, keep `scripts/build-bundle.sh`, `scripts/install.sh`, and `LatestProviderVersion` in sync.\n- If you change install paths or process invocation, update both the CLI and install flow.\n- Device linking changes often span both coordinator device auth endpoints and the provider `login` / `logout` commands.\n- Model registry changes span coordinator registry schema/endpoints, `provider-swift` manifest download/publish code, `scripts/publish-model.sh`, and the console UI. Do not add hardcoded provider `MODEL_CATALOG` lists.\n\n## Common Pitfalls\n\n- `coordinator/coordinator` is a built binary checked into the tree. Do not model changes from it, and do not commit more built artifacts.\n- CI release workflow must compute binary SHA-256 hashes AFTER code signing, not before. Providers verify hashes of the signed binary.\n- Model scan uses fast discovery (no hashing) at startup. Weight hashing is on-demand via `compute_weight_hash()` only for the served model. Don't add hashing back to the scan path.\n- Provider auto-injects ChatML template for models missing `chat_template` field. This is intentional -- Qwen3.5 base models ship without it.\n- Store selection (`cmd/coordinator/main.go`): the coordinator uses the **Postgres** store whenever `EIGENINFERENCE_DATABASE_URL` is set (prod does — durable across restarts/deploys), and refuses to start without it unless `EIGENINFERENCE_ALLOW_MEMORY_STORE=true`. The in-memory store is the dev/test fallback only (state lost on restart). Note: the live provider *registry* (WebSocket connections/attestation) is always in-process and is rebuilt on reconnect regardless of store.\n- Request queue timeout is 120 seconds. Initial attestation challenge is sent immediately on registration, then every 5 minutes.\n- Backend idle timeout is 1 hour (not 10 minutes as some comments may say).\n\n### Coordinator State Model — Multiple Overlapping Views\n\nProvider state lives in several fields that are read by different code paths with different precedence rules. When mutating any of these, trace every reader:\n\n- `BackendCapacity.Slots` is **authoritative** for the scheduler when present (Swift providers). The scheduler derives `slotState`, `modelLoaded`, token budgets, and observed TPS from it. `WarmModels` is only a fallback for legacy providers without `BackendCapacity`.\n- `WarmModels` is updated by heartbeats. It is NOT consulted by `snapshotProviderLocked` or `buildCandidateWithReason` when `BackendCapacity` is non-nil. `TriggerModelSwaps` / `hasWarmProviderLocked` checks it as a fallback. Legacy `ScoreProvider` also reads it for warm bonus, and `/v1/me/providers` copies it into API responses.\n- `CurrentModel` is set from heartbeat `active_model`. A nil/omitted `active_model` means no model is loaded. Stale `CurrentModel` can cause attestation hash mismatches.\n- `pendingModelLoads` is only checked by `TriggerModelSwaps` planning. It is NOT checked by `QuickCapacityCheck`, `ReserveProviderEx`, or `freeMemoryAdmits`. Do not assume pending-load state affects routing decisions.\n- Provider-reported slot states include `\"running\"` (active requests), `\"idle\"` (loaded, no requests), `\"crashed\"`, `\"reloading\"`, and `\"idle_shutdown\"`. The `\"idle\"` state means the model IS loaded — treat it the same as `\"running\"` for warm detection, not as `\"unknown\"`.\n- Providers can hold up to `maxModelSlots` models simultaneously (default 3). Do not assume a model swap evicts all other models.\n- The provider's `ensureModelLoaded` requires `estimatedMemoryGb * 3.0` headroom. The coordinator's `freeMemoryAdmits` uses a different (less conservative) check. A model the coordinator admits can still fail on the provider side.\n\n### Coordinator Mutation Checklist\n\nWhen adding code that mutates provider state or sends commands (`load_model`, etc.):\n\n1. Enumerate every reader of the fields you're mutating (`BackendCapacity.Slots`, `WarmModels`, `CurrentModel`, `pendingModelLoads`).\n2. Check what happens on the failure path — does state get cleaned up on disconnect, timeout, and load failure?\n3. Check concurrent access — heartbeats arrive per-provider on separate goroutines; `TriggerModelSwaps` can race with `drainQueuedRequestsForModels`.\n4. Check the cleanup path — `Disconnect()` must clear any per-provider state you add.\n5. Verify pre-existing invariants: `maxModelSlots`, heartbeat field omission semantics (`nil` vs empty), and the 3x memory gate on the provider side.\n\n## Code Structure & Modularity\n\nKeep the codebase modular, never monolithic.\n\n- Prefer small, single-responsibility files over large catch-all ones. Split by concern: types, pure helpers, data/IO hooks, UI pieces, and a thin orchestrator that wires them together.\n- Group a feature's files into a dedicated module/folder with a thin entry point. Examples: the coordinator's top-level Go packages (`registry/`, `billing/`, `store/`), and `console-ui/src/components/api-keys/` (`constants`, `format`, `limits`, `Modal`, `KeyForm`, `KeyCard`, a `useApiKeys` data hook, and a thin `ApiKeysManager` orchestrator).\n- One file/component should do one thing. If a file mixes several concerns or grows past a few hundred lines, that's a signal to split it.\n- **At the end of every large piece of work, do a refactor pass to make it modular before calling it done.** Extract helpers/types/hooks into focused files, delete dead code, and keep the public entry point thin. The refactor must be behavior-preserving — build, lint, and tests stay green.\n\n## Formatting\n\nA pre-commit hook in `.githooks/pre-commit` checks staged files only. It is enabled via:\n\n```bash\ngit config core.hooksPath .githooks\n```\n\n| Component | Check | Manual fix |\n|-----------|-------|------------|\n| Go (`coordinator/`) | `gofmt -l` | `gofmt -w ` |\n| Swift (`provider-swift/`) | no enforced formatter | `cd provider-swift && swift test` |\n| TypeScript (`console-ui/`) | `npx eslint src/` | `cd console-ui && npx eslint src/ --fix` |\n\n",{"_579":875,"_560":561,"_581":567},{"_582":876,"_584":-5,"_42":-5,"_585":-5,"_586":-5,"_96":877,"_92":588},[878,879],"developer",{"_46":881,"_92":866},{"_46":880,"_92":866},"\n## Skills\nA skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.\n### Available skills\n- imagegen: Generate or edit raster images when the task benefits from AI-created bitmap visuals such as photos, illustrations, textures, sprites, mockups, or transparent-background cutouts. Use when Codex should create a brand-new image, transform an existing image, or derive visual variants from references, and the output should be a bitmap asset rather than repo-native code or vector. Do not use when the task is better handled by editing existing SVG/vector/code-native assets, extending an established icon or logo system, or building the visual directly in HTML/CSS/canvas. (file: /opt/codex/skills/.system/imagegen/SKILL.md)\n- openai-docs: Use when the user asks how to build with OpenAI products or APIs and needs up-to-date official documentation with citations, help choosing the latest model for a use case, or model upgrade and prompt-upgrade guidance; prioritize OpenAI docs MCP tools, use bundled references only as helper context, and restrict any fallback browsing to official OpenAI domains. (file: /opt/codex/skills/.system/openai-docs/SKILL.md)\n- plugin-creator: Create and scaffold plugin directories for Codex with a required `.codex-plugin/plugin.json`, optional plugin folders/files, and baseline placeholders you can edit before publishing or testing. Use when Codex needs to create a new local plugin, add optional plugin structure, or generate or update repo-root `.agents/plugins/marketplace.json` entries for plugin ordering and availability metadata. (file: /opt/codex/skills/.system/plugin-creator/SKILL.md)\n- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /opt/codex/skills/.system/skill-creator/SKILL.md)\n- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /opt/codex/skills/.system/skill-installer/SKILL.md)\n### How to use skills\n- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.\n- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.\n- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.\n- How to use a skill (progressive disclosure):\n 1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.\n 2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.\n 3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.\n 4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.\n 5) If `assets/` or templates exist, reuse them instead of recreating from scratch.\n- Coordination and sequencing:\n - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.\n - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.\n- Context hygiene:\n - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.\n - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.\n - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.\n- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.\n","\nFilesystem sandboxing defines which files can be read or written. `sandbox_mode` is `danger-full-access`: No filesystem sandboxing - all commands are permitted. Network access is enabled.\nApproval policy is currently never. Do not provide the `sandbox_permissions` for any reason, commands will be rejected.\n","turn/started",{"_560":561,"_87":884},{"_103":-5,"_42":567,"_568":885,"_570":571,"_572":886},[],"inProgress",{"_572":888,"_560":561},{"_889":890,"_92":891},"activeFlags",[],"active","thread/started",{"_894":895},"thread",{"_896":-5,"_897":-5,"_898":899,"_900":901,"_612":613,"_902":15,"_903":-5,"_904":-5,"_905":-5,"_906":907,"_42":561,"_908":909,"_367":-5,"_910":-5,"_255":911,"_912":385,"_913":-5,"_617":914,"_572":915,"_916":-5,"_77":917,"_918":901},"agentNickname","agentRole","cliVersion","0.128.0","createdAt",1782003777,"ephemeral","extra","forkedFromId","gitInfo","historyMode","legacy","modelProvider","openai","parentThreadId","/opt/codex/sessions/2026/06/21/rollout-2026-06-21T01-02-57-019ee7b3-c145-73f1-9f69-24d6842d1d25.jsonl","preview","recencyAt","vscode",{"_92":576},"threadSource",[],"updatedAt",[],"messages",[922],{"_42":923,"_924":925,"_926":927,"_928":927,"_582":929,"_572":930,"_584":-5,"_931":32,"_932":933,"_934":935,"_936":-5},"85837fc1-0bf4-4b79-9a70-2eec8b4f51b5","author",{"_96":485,"_367":-5,"_932":945},"create_time",1782003776.825742,"update_time",{"_941":46,"_942":943},"finished_successfully","weight","metadata",{"_937":938},"recipient","all","channel","l1239dk1",{"_939":940},"inline_comments",[],"content_type","parts",[944],"Thanks @chatgpt-codex-connector — addressed both findings in `f28e89a9`:\n\n**① Shed saturated dedicated models before queueing** — ✅ fixed. Dedicated-family models now **bypass queue-before-shed when their dedicated boxes are saturated** (`candidateCount==0 && capacityRejections>0`) and fast-429 with `Retry-After` instead of sitting in the 120s queue — holding an OpenRouter request that long would blow its TTFT SLA. Applied in both `handleChatCompletions` and `handleGenericInference`; non-dedicated models keep queue-before-shed unchanged. New test: `TestDedicatedSaturatedBoxFast429` (saturated dedicated box + queue-before-shed left ON → immediate 429).\n\n**② Apply routing gates before treating a provider as available** — intentionally **left at 429** (product decision). For a dedicated model, we prefer a transient `429 + Retry-After` (clean OpenRouter failover) over an \"honest\" `503` even in the narrow case where the only advertiser is structurally unroutable (private-only / sub-trust / unverified). Routing itself is unaffected — the gate still excludes those providers; this only concerns the shed status code, and 429 is the desired behavior for OpenRouter compatibility.\n\nNet effect: every \"can't serve a dedicated model right now\" case (no dedicated box, or dedicated boxes saturated) now sheds as **429 + Retry-After**; only a model genuinely absent from the fleet returns 503.\nfeat(coordinator): route Gemma 4 only to dedicated providers; 429 shed\n\n## What & why\n\nGemma 4 destabilizes when it shares a Mac with other models/work (memory contention, OOM/jetsam, ragged-batch NaN, Metal resource-count leaks). This restricts **a dedicated model family (default `gemma-4`) to providers whose entire advertised catalog is that family** — i.e. machines dedicated to Gemma 4. Those dedicated boxes are the only routing candidates *and* the only providers counted when deciding whether to accept or shed a request to OpenRouter.\n\nCoordinator-only — no provider/Swift or protocol changes.\n\n## Routing behavior: before → after\n\n**Before** — a Gemma 4 request could land on any box serving it (including mixed boxes where it contends), and \"no provider\" shed as `503`:\n\n```mermaid\nflowchart TD\n Req[\"Gemma 4 request\"] --> Gate{\"Provider serves gemma-4?\"}\n Gate -->|\"Dedicated box ✓
(gemma-4 only)\"| Cand[\"Candidate pool\"]\n Gate -->|\"Mixed box ✓
(gemma-4 + qwen/gpt-oss)\"| Cand\n Cand --> Cap{\"Capacity?\"}\n Cap -->|\"Yes\"| Route[\"Route to best score —
⚠️ often a MIXED box →
Gemma 4 contends → OOM / NaN / leak\"]\n Cap -->|\"All busy\"| R429[\"429 rate_limit\"]\n Cap -->|\"None eligible\"| R503[\"❌ 503 model_unavailable
(OpenRouter may derank us)\"]\n```\n\n**After** — mixed boxes are filtered out at the single shared gate (routing *and* capacity preflight), so only dedicated boxes are eligible; \"no dedicated box\" sheds as `429`:\n\n```mermaid\nflowchart TD\n Req[\"Gemma 4 request\"] --> Gate{\"Provider serves gemma-4?\"}\n Gate -->|\"Dedicated box ✓\"| Ded{\"Dedicated to gemma-4?
(advertised catalog == gemma-4 only)\"}\n Gate -->|\"Mixed box ✓\"| Ded\n Ded -->|\"No — mixed\"| Drop[\"🚫 Excluded from routing
AND capacity count
(self-route to own box exempt)\"]\n Ded -->|\"Yes — dedicated\"| Cand[\"Candidate pool =
dedicated boxes only\"]\n Cand --> Cap{\"Capacity?\"}\n Cap -->|\"Yes\"| Route[\"✅ Route to a DEDICATED box →
Gemma 4 isolated, no contention\"]\n Cap -->|\"No dedicated box free\"| R429[\"✅ 429 + Retry-After →
clean OpenRouter failover\"]\n```\n\n## How\n\nOne shared predicate, `registry.providerExcludedByDedicatedRuleLocked(p, model)` (*model is a dedicated family AND the provider isn't dedicated to it*), applied at **every** public provider-selection site so routing, shedding, and pre-warming can never drift:\n\n- **`providerPassesRoutingGatesLockedEx`** — the documented single gate shared by the dispatch hot path (`snapshotProviderLockedEx`) **and** the OpenRouter capacity preflight (`quickCapacityCheck`).\n- **`coldSpillProviderEligibleLocked`**, **`warmPoolCandidateLocked`**, **`modelLoadCandidatePendingLocked`** — so the autoscaler/swap planner never pre-warms a dedicated model onto a mixed box.\n- **`providerHasWarmModelLocked`** — so a warm *mixed* box isn't counted as warm capacity for a dedicated model (which would suppress warming a real dedicated box).\n\nA caller self-routing to its **own** machine is exempt (`!selfRouteOwner`). Eligibility is catalog-based, not load-based, so an **idle** dedicated box stays eligible (cold-loads on demand).\n\n## 429, not 503\n\nWhen the fleet serves the model but no dedicated box is available (none exist, or all busy), the preflight `no_eligible_provider` shed now returns **`429 rate_limit_exceeded` + `Retry-After`** instead of `503` — for OpenRouter this is a clean transient failover rather than an \"unhealthy endpoint\" signal. Applied in **both** `handleChatCompletions` and the generic `/v1/completions` + `/v1/messages` path. A genuinely-absent model still `503`s (guarded by `IsDedicatedModel(model) && HasProviderForModel(model, serials...)`).\n\n## Config & rollout\n\n- `EIGENINFERENCE_DEDICATED_MODELS` — comma-separated, case-insensitive substring patterns. **Unset → default `gemma-4` (ON)**; set to `none`/empty → disabled.\n- The registry field defaults nil (OFF), so unit tests and the e2e testbed (both build via `registry.New`) are unaffected — the rule only activates via `cmd/coordinator/main.go` at startup.\n\n> [!IMPORTANT]\n> **Default is ON.** Once deployed, if the fleet has no Gemma-4-only boxes, all Gemma 4 traffic sheds to OpenRouter (429). Ensure dedicated machines exist (operationally: only Gemma 4 downloaded on them) or set `EIGENINFERENCE_DEDICATED_MODELS=none`. Coordinator deploy is human-only.\n\n## Testing\n\n- New: 15 registry tests (parse, predicate, routing-selects-only-dedicated, never-falls-back-to-mixed, preflight shed counts only dedicated boxes, self-route exempt, default-off, warm-pool/swap/warm-detection gating) + a real-HTTP test asserting **429 + Retry-After** for both `/v1/chat/completions` and `/v1/completions`, with a 503 control for a truly-absent model. Tests fail without the change; baselines confirm the dedicated rule is the differentiator.\n- `go build ./...`, linux cross-compile, and the **full coordinator suite** pass; `gofmt` clean.\n\n## Known follow-ups (non-blocking, cosmetic)\n\n- `ModelCapacitySnapshot` / `/v1/models` / the OpenRouter feed are display-only and left ungated, so they over-report Gemma 4 capacity by counting mixed boxes. No routing/admission impact.\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)\n\n[vc]: #dZASsr2oD4u7pBpydIsKkfg7EdXlJpCsuO6w0z/XoUg=:eyJpc01vbm9yZXBvIjp0cnVlLCJ0eXBlIjoiZ2l0aHViIiwicHJvamVjdHMiOlt7Im5hbWUiOiJkLWluZmVyZW5jZSIsInByb2plY3RJZCI6InByal84YTNCaFVhbjBkcUppa1hUb2huYnJwQnFhNkVIIiwicm9vdERpcmVjdG9yeSI6ImNvbnNvbGUtdWkiLCJpbnNwZWN0b3JVcmwiOiJodHRwczovL3ZlcmNlbC5jb20vZWlnZW4tbGFicy9kLWluZmVyZW5jZS9GRG42Mk1ScjR3NGlIS3c4aUtqU3dOOW53M0tMIiwicHJldmlld1VybCI6ImQtaW5mZXJlbmNlLWdpdC1mZWF0LWdlbW1hNC1kZWRpY2F0ZWQtcm91dGluZy1laWdlbi1sYWJzLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiUEVORElORyJ9LHsibmFtZSI6ImQtaW5mZXJlbmNlLWxhbmRpbmciLCJwcm9qZWN0SWQiOiJwcmpfUUZKa0xma1pEUkRqV0ZsREpEQ1FTMWR4MkR2ViIsInJvb3REaXJlY3RvcnkiOiJsYW5kaW5nIiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL2VpZ2VuLWxhYnMvZC1pbmZlcmVuY2UtbGFuZGluZy9BZTV1NnpVTFJNODVXa29QNlVqd3hCR0xDSlA3IiwicHJldmlld1VybCI6ImQtaW5mZXJlbmNlLWxhbmRpbmctZ2l0LWZlYXQtZ2VtbWE0LWRlZGljYXRlZC05MGNhOTMtZWlnZW4tbGFicy52ZXJjZWwuYXBwIiwibmV4dENvbW1pdFN0YXR1cyI6IkRFUExPWUVEIn0seyJuYW1lIjoiZC1pbmZlcmVuY2UtY29uc29sZS11aS1kZXYiLCJwcm9qZWN0SWQiOiJwcmpfMTJORnVUcElVNmdxS095R3lVU0FlRlF1QWhkcyIsInJvb3REaXJlY3RvcnkiOiJjb25zb2xlLXVpIiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL2VpZ2VuLWxhYnMvZC1pbmZlcmVuY2UtY29uc29sZS11aS1kZXYvQmEyUDl6bTlwd2Rra2tKWTg1VlBWVUtQTkdpcSIsInByZXZpZXdVcmwiOiJkLWluZmVyZW5jZS1jb25zb2xlLXVpLWRldi1naXQtZmVhdC1nZW1tYTQtZGUtODViYjFlLWVpZ2VuLWxhYnMudmVyY2VsLmFwcCIsIm5leHRDb21taXRTdGF0dXMiOiJQRU5ESU5HIn1dLCJyZXF1ZXN0UmV2aWV3VXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL3ZlcmNlbC1hZ2VudC9yZXF1ZXN0LXJldmlldz9vd25lcj1MYXlyLUxhYnMmcmVwbz1kLWluZmVyZW5jZSZwcj00MzQifQ==\nThe latest updates on your projects. Learn more about [Vercel for GitHub](https://vercel.link/github-learn-more).\n\n| Project | Deployment | Actions | Updated (UTC) |\n| :--- | :----- | :------ | :------ |\n| [d-inference](https://vercel.com/eigen-labs/d-inference) | ![Building](https://vercel.com/static/status/building.svg) [Building](https://vercel.com/eigen-labs/d-inference/FDn62MRr4w4iHKw8iKjSwN9nw3KL) | [Preview](https://d-inference-git-feat-gemma4-dedicated-routing-eigen-labs.vercel.app) | Jun 21, 2026 1:02am |\n| [d-inference-console-ui-dev](https://vercel.com/eigen-labs/d-inference-console-ui-dev) | ![Building](https://vercel.com/static/status/building.svg) [Building](https://vercel.com/eigen-labs/d-inference-console-ui-dev/Ba2P9zm9pwdkkkJY85VPVUKPNGiq) | [Preview](https://d-inference-console-ui-dev-git-feat-gemma4-de-85bb1e-eigen-labs.vercel.app) | Jun 21, 2026 1:02am |\n| [d-inference-landing](https://vercel.com/eigen-labs/d-inference-landing) | ![Ready](https://vercel.com/static/status/ready.svg) [Ready](https://vercel.com/eigen-labs/d-inference-landing/Ae5u6zULRM85WkoP6UjwxBGLCJP7) | [Preview](https://d-inference-landing-git-feat-gemma4-dedicated-90ca93-eigen-labs.vercel.app) | Jun 21, 2026 1:02am |\n\n

\n\n\n\n\n\n\n**This PR introduces a dedicated-box routing feature for specific model families; the changes are operationally significant but security-neutral with one minor concern worth noting.**\n\n---\n\n### Trust boundaries touched\n\n- **TB-002** (Coordinator → Provider WebSocket) — routing gate logic in `registry.go`\n- **TB-006** (Admin → Coordinator) — new env-var configuration surface in `main.go`\n\n---\n\n### Threat analysis\n\n**T-034** (Provider runs modified code while advertising a trusted identity) — `coordinator/registry/registry.go` lines +2985–2988, +3072–3075\n\nℹ️ **Neutral.** `providerExcludedByDedicatedRuleLocked` and `providerHasWarmModelLocked` are routing/scheduling helpers that sit downstream of the APNs code-identity gate (`providerSupportsPrivateTextLocked`). The dedicated-model exclusion fires on catalog shape (`dedicatedModels` substring match), not on any attestation field, so it cannot be manipulated by a provider to bypass or strengthen the `CodeAttested` gate. No change to trust-level assignment.\n\n**T-038** (HTTP server config) — `coordinator/cmd/coordinator/main.go`\n\nℹ️ **Neutral.** The diff adds only the dedicated-models config block; `http.Server` fields (`ReadHeaderTimeout`, `MaxHeaderBytes`, etc.) are not touched.\n\n**T-032 / T-041** (Solana mnemonic / profile signing identity) — `coordinator/cmd/coordinator/main.go`\n\nℹ️ **Neutral.** No secrets handling added or modified.\n\n---\n\n### New attack surface not covered by an existing threat\n\n**Operator-controlled routing topology via `EIGENINFERENCE_DEDICATED_MODELS`**\n\n`main.go` +168–180: the env-var is read with `os.LookupEnv` and fed into `registry.ParseDedicatedModels`. This is a coordinator-side config variable (not a provider-supplied value), so ADV-001 cannot influence it directly. However:\n\n1. **Log-based enumeration** (`main.go` +180): `logger.Info(\"dedicated-model routing ENABLED\", \"patterns\", strings.Join(dedicatedModels, \",\"))` emits the full pattern list to structured logs. If logs are accessible to any consumer or provider (e.g. via a log-aggregation misconfiguration), this leaks the exact routing topology — which model families have dedicated boxes and by implication which do not. This is low severity but worth being deliberate about the log destination.\n\n2. **Shed-to-OpenRouter path**: the comment at `main.go` +163 (\"sheds to OpenRouter as a transient 429\") introduces a new reachable code path where a request for a dedicated model with no available dedicated box gets a different response code and destination than other capacity-shed requests. A consumer (ADV-002) who systematically sends requests for `gemma-4` variants could probe for the presence/absence of dedicated capacity by observing the 429 cadence. This is informational leakage about infrastructure topology — not covered by any existing threat. Severity is low given the non-sensitive nature of the information, but it should be acknowledged.\n\nNo existing threat ID maps cleanly to either of these; the closest is T-006 (registry flooding/enumeration), but that targets the provider WebSocket, not consumer-facing response codes.\n\n---\n\n### Open findings resolved\n\nNone — this PR does not touch any SEC-* open finding.\n\n---\n\n### Summary for reviewers\n\nThe core routing logic changes (`providerExcludedByDedicatedRuleLocked` calls in `registry.go`) are straightforward exclusion filters that cannot be influenced by adversarial providers. The only items to consider before merge are:\n\n- Confirm `logger.Info` for dedicated-model patterns goes only to operator-accessible sinks, not to any consumer-visible telemetry endpoint.\n- Decide whether the 429 response for \"no dedicated box available\" leaks topology information at a sensitivity level that warrants a generic error message instead.\n\n---\n*🔐 Threat model: [`docs/threat-model.yaml`](../blob/HEAD/docs/threat-model.yaml) · Updates on each push to this PR*\n\n### 💡 Codex Review\n\nHere are some automated review suggestions for this pull request.\n\n**Reviewed commit:** `a5fc81f361`\n \n\n

ℹ️ About Codex in GitHub

\n
\n\n[Your team has set up Codex to review pull requests in this repo](https://chatgpt.com/codex/cloud/settings/general). Reviews are triggered when you\n- Open a pull request for review\n- Mark a draft as ready\n- Comment \"@codex review\".\n\nIf Codex has suggestions, it will comment; otherwise it will react with 👍.\n\n\n\n\nCodex can also answer questions or update the PR. Try commenting \"@codex address that feedback\".\n \n

\n**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Shed saturated dedicated models before queueing**\n\nWhen every dedicated provider for a Gemma-4 request exists but is already at concurrency/token capacity, `QuickCapacityCheckWithTTFTForRequest` reports `candidateCount == 0` with `capacityRejections > 0`, so the earlier queue-before-shed block handles the request and this new 429 branch is never reached. With the default queue-before-shed setting, OpenRouter traffic can therefore sit in the 120s dispatch queue instead of receiving the intended `Retry-After` 429 whenever dedicated boxes are present but saturated; the mirrored `handleGenericInference` path has the same ordering.\n\nUseful? React with 👍 / 👎.\n**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Apply routing gates before treating a provider as available**\n\nWhen the only advertiser for a dedicated model is private-only, below the public trust floor, runtime-unverified, lacks private-text support, or has a stale challenge, public routing still rejects it in `providerPassesRoutingGatesLockedEx`, but this check returns true because it only checks status and catalog membership. In that context the new preflight branch classifies an unroutable public fleet as `machine_busy` 429 instead of the `no_provider` 503 path, so clients are told to retry/fail over as if capacity might free when no public provider can actually route the model.\n\nUseful? React with 👍 / 👎.",{},"created",1782003899.4331996,"Completing the task","event_type","info",{"_92":588,"_96":93,"_582":4500},{"_92":953,"_954":955},"partial_repo_snapshot","files",[956,957,958],{"_255":1182,"_960":-5,"_961":1183,"_963":-5},{"_255":1076,"_960":-5,"_961":1077,"_963":-5},{"_255":959,"_960":-5,"_961":962,"_963":-5},"coordinator/registry/dedicated_models.go","contents","line_range_contents",[964],"missing_reason",{"_965":685,"_966":967,"_582":968,"_969":59},"line_range_start","line_range_end",142,[970,385,971,385,972,973,974,975,976,977,978,979,385,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,385,995,996,997,998,999,1000,1001,987,988,989,990,1002,992,1003,1004,1005,1006,1007,992,1008,994,385,1009,1010,1011,1012,1013,1014,992,1015,1016,1017,1018,990,992,1019,994,385,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,989,990,1030,1031,1032,990,992,1033,994,385,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,992,1046,994,385,1047,1048,1049,1050,1051,1052,1053,1054,1055,994,385,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,992,1052,1053,1067,1068,989,990,1069,1070,1071,1072,1073,1074,990,992,1075,994],"contains_end_of_file","package registry","import \"strings\"","// Dedicated-model routing isolates a model family (e.g. Gemma 4) to providers","// that run ONLY that family. A \"dedicated\" provider is one whose entire","// advertised catalog matches the family pattern; nothing else can ever load on","// that machine, so the isolated model never contends for memory or compute with","// other models on the same box. The patterns are configured at startup","// (EIGENINFERENCE_DEDICATED_MODELS, default \"gemma-4\") and applied in the single","// routing/preflight gate providerPassesRoutingGatesLockedEx, which both the","// dispatch hot path and the OpenRouter capacity preflight funnel through.","// ParseDedicatedModels parses a comma-separated list of model-family patterns","// into normalized (trimmed, lowercased, non-empty) substring patterns. Used by","// the coordinator entrypoint to read EIGENINFERENCE_DEDICATED_MODELS. An empty","// or all-blank input yields a nil slice (feature disabled).","func ParseDedicatedModels(csv string) []string {","\tvar out []string","\tfor _, p := range strings.Split(csv, \",\") {","\t\tp = strings.ToLower(strings.TrimSpace(p))","\t\tif p == \"\" {","\t\t\tcontinue","\t\t}","\t\tout = append(out, p)","\t}","\treturn out","}","// SetDedicatedModels configures the dedicated-model routing patterns. Patterns","// are matched case-insensitively as substrings of the resolved build id. An","// empty (or all-blank) list disables the feature. Called once at startup before","// the coordinator begins serving.","func (r *Registry) SetDedicatedModels(patterns []string) {","\tnormalized := make([]string, 0, len(patterns))","\tfor _, p := range patterns {","\t\tnormalized = append(normalized, p)","\tr.mu.Lock()","\tdefer r.mu.Unlock()","\tif len(normalized) == 0 {","\t\tr.dedicatedModels = nil","\t\treturn","\tr.dedicatedModels = normalized","// dedicatedPatternForLocked returns the first configured pattern that the given","// resolved build id contains, or \"\", false when the model is not a dedicated","// model (or the feature is disabled). Caller holds r.mu.","func (r *Registry) dedicatedPatternForLocked(model string) (string, bool) {","\tif len(r.dedicatedModels) == 0 {","\t\treturn \"\", false","\tm := strings.ToLower(model)","\tfor _, p := range r.dedicatedModels {","\t\tif strings.Contains(m, p) {","\t\t\treturn p, true","\treturn \"\", false","// providerDedicatedToPatternLocked reports whether EVERY catalog-allowed model","// the provider advertises matches pattern — i.e. the box is dedicated to that","// model family. A provider that advertises no catalog-allowed model is not","// dedicated (returns false). Non-catalog-allowed advertised builds (stale or","// unknown) are ignored so they cannot spuriously disqualify a dedicated box.","// Caller holds r.mu AND p.mu (mirrors providerServesCatalogModelLocked).","func (r *Registry) providerDedicatedToPatternLocked(p *Provider, pattern string) bool {","\tadvertised := 0","\tfor _, m := range p.Models {","\t\tif !r.modelAllowedByCatalogLocked(m) {","\t\tadvertised++","\t\tif !strings.Contains(strings.ToLower(m.ID), pattern) {","\t\t\treturn false","\treturn advertised > 0","// providerExcludedByDedicatedRuleLocked reports whether the dedicated-box rule","// excludes this provider from PUBLIC routing/warming for the model: true when","// the model is a dedicated family AND the provider is not dedicated to it. It is","// the single predicate shared by the routing gate, the capacity preflight, the","// cold-spill check, the warm-pool target picker, and the model-swap planner, so","// every place that selects or pre-warms a provider for a public request applies","// the same constraint and cannot drift. (Self-route exemption is the caller's","// concern — only the routing gate honors it.) Caller holds r.mu AND p.mu.","func (r *Registry) providerExcludedByDedicatedRuleLocked(p *Provider, model string) bool {","\tpat, ok := r.dedicatedPatternForLocked(model)","\tif !ok {","\t\treturn false","\treturn !r.providerDedicatedToPatternLocked(p, pat)","// IsDedicatedModel reports whether the resolved build id is governed by the","// dedicated-model routing rule. The consumer layer uses it to classify a","// \"no eligible provider\" shed for such a model as a transient 429 (rather than","// a 503): with no dedicated box available, the request must fail over cleanly.","func (r *Registry) IsDedicatedModel(model string) bool {","\tr.mu.RLock()","\tdefer r.mu.RUnlock()","\t_, ok := r.dedicatedPatternForLocked(model)","\treturn ok","// HasProviderForModel reports whether any online, non-untrusted provider","// advertises a catalog-allowed build of the resolved model id (ignoring","// dedicated-box and capacity gates). It distinguishes \"the fleet serves this","// model but no dedicated/available box can take it right now\" (transient — a","// 429 shed) from \"the model is absent from the fleet entirely\" (a 503). When","// allowedSerials is non-empty the check is restricted to providers whose","// attested serial is in the set, mirroring the routing candidate filter.","func (r *Registry) HasProviderForModel(model string, allowedSerials ...string) bool {","\tallowedSet := make(map[string]struct{}, len(allowedSerials))","\tfor _, s := range allowedSerials {","\t\tallowedSet[s] = struct{}{}","\tfor _, p := range r.providers {","\t\tif len(allowedSet) > 0 && !providerMatchesAllowedSerial(p, allowedSet) {","\t\tp.mu.Lock()","\t\teligible := p.Status != StatusOffline && p.Status != StatusUntrusted &&","\t\t\tr.providerServesCatalogModelLocked(p, model)","\t\tp.mu.Unlock()","\t\tif eligible {","\t\t\treturn true","\treturn false","coordinator/api/dedicated_models_test.go",[1078],{"_965":685,"_966":1079,"_582":1080,"_969":59},168,[1081,385,1082,1083,1084,1085,1086,1087,1088,385,1089,1090,1091,385,1092,1093,1094,1095,1096,1097,1098,992,1099,1100,1101,1097,1098,992,1102,1103,1104,994,385,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,385,1116,1117,385,1118,1119,1120,1121,1122,1123,1124,1125,1126,385,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1132,385,1137,1138,1097,1139,992,1140,1141,992,1142,1143,992,1144,1145,992,385,1146,1147,1097,1148,992,1149,1150,992,1151,1152,992,385,1153,1154,1155,1156,1097,1157,992,1099,1100,1101,1097,1158,992,1102,1159,1160,1161,992,1162,1163,992,994,385,1164,1165,1166,1167,1168,1111,1112,1113,1169,385,1116,1117,385,1118,1170,1121,1122,1124,1125,1126,385,1127,1128,1171,1172,1173,1174,1175,1176,1177,1132,1133,1134,1135,1178,1132,385,1138,1097,1179,992,1140,1180,992,1144,1181,992,994],"package api","import (","\t\"context\"","\t\"io\"","\t\"net/http\"","\t\"strings\"","\t\"testing\"","\t\"time\"","\t\"github.com/eigeninference/d-inference/coordinator/protocol\"","\t\"nhooyr.io/websocket\"",")","// chatRequestWithHeaders posts a chat completion and returns status, body, and","// the Retry-After header (adaptiveChatRequest drops headers).","func chatRequestWithHeaders(ctx context.Context, baseURL, model string) (int, string, string, error) {","\tbody := `{\"model\":\"` + model + `\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}],\"stream\":true,\"max_tokens\":64}`","\treq, err := http.NewRequestWithContext(ctx, http.MethodPost, baseURL+\"/v1/chat/completions\", strings.NewReader(body))","\tif err != nil {","\t\treturn 0, \"\", \"\", err","\treq.Header.Set(\"Authorization\", \"Bearer test-key\")","\treq.Header.Set(\"Content-Type\", \"application/json\")","\tresp, err := http.DefaultClient.Do(req)","\tdefer resp.Body.Close()","\tdata, _ := io.ReadAll(resp.Body)","\treturn resp.StatusCode, string(data), resp.Header.Get(\"Retry-After\"), nil","// TestDedicatedModelShed429NotServiceUnavailable verifies that when a dedicated","// model (gemma-4) is served by the fleet but no DEDICATED box can take the","// request (only a mixed gemma-4+qwen box exists), the coordinator sheds to","// OpenRouter with a transient 429 + Retry-After — NOT a 503. A truly-absent,","// non-dedicated model still returns 503.","func TestDedicatedModelShed429NotServiceUnavailable(t *testing.T) {","\tts, reg := setupAdaptiveCapacityIntegration(t)","\tdefer ts.Close()","\treg.SetDedicatedModels([]string{\"gemma-4\"})","\t// Keep cold-dispatch from spilling to the queue; pin the preflight shed path.","\tt.Setenv(envColdDispatch, \"false\")","\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)","\tdefer cancel()","\tgemma := \"gemma-4-26b-test\"","\tqwen := \"qwen-3-test\"","\t// A single MIXED provider: advertises gemma-4 AND qwen -> not dedicated.","\tconn := connectProvider(t, ctx, ts.URL, []protocol.ModelInfo{","\t\t{ID: gemma, ModelType: \"chat\", Quantization: \"4bit\"},","\t\t{ID: qwen, ModelType: \"chat\", Quantization: \"4bit\"},","\t}, testPublicKeyB64())","\tdefer conn.Close(websocket.StatusNormalClosure, \"done\")","\tp := markOnlyProviderRoutable(t, reg)","\twriteAdaptiveHeartbeat(t, ctx, conn, gemma, &protocol.BackendCapacity{","\t\tTotalMemoryGB: 64,","\t\tSlots: []protocol.BackendSlotCapacity{","\t\t\t{Model: gemma, State: \"running\", MaxConcurrency: 8, ActiveTokenBudgetMax: 32_768},","\t\t},","\t})","\twaitForAdaptiveCondition(t, time.Second, func() bool {","\t\tp.Mu().Lock()","\t\tdefer p.Mu().Unlock()","\t\treturn p.BackendCapacity != nil","\t// Gemma-4 to a mixed-only fleet -> transient 429 + Retry-After (not 503).","\tstatus, body, retryAfter, err := chatRequestWithHeaders(ctx, ts.URL, gemma)","\t\tt.Fatalf(\"gemma request: %v\", err)","\tif status != http.StatusTooManyRequests {","\t\tt.Fatalf(\"gemma status = %d, want 429; body = %s\", status, body)","\tif !strings.Contains(body, \"rate_limit_exceeded\") {","\t\tt.Fatalf(\"gemma body = %s, want rate_limit_exceeded\", body)","\tif retryAfter == \"\" {","\t\tt.Fatalf(\"gemma 429 missing Retry-After header\")","\t// Control: a non-dedicated model absent from the fleet still 503s.","\tstatusC, bodyC, _, err := chatRequestWithHeaders(ctx, ts.URL, \"totally-absent-model\")","\t\tt.Fatalf(\"control request: %v\", err)","\tif statusC != http.StatusServiceUnavailable {","\t\tt.Fatalf(\"control status = %d, want 503; body = %s\", statusC, bodyC)","\tif !strings.Contains(bodyC, \"model_unavailable\") {","\t\tt.Fatalf(\"control body = %s, want model_unavailable\", bodyC)","\t// The legacy /v1/completions endpoint (handleGenericInference) must classify","\t// the dedicated shed identically — 429, not 503.","\tcbody := `{\"model\":\"` + gemma + `\",\"prompt\":\"hello\",\"stream\":true,\"max_tokens\":64}`","\treq, err := http.NewRequestWithContext(ctx, http.MethodPost, ts.URL+\"/v1/completions\", strings.NewReader(cbody))","\t\tt.Fatalf(\"completions request build: %v\", err)","\t\tt.Fatalf(\"completions request: %v\", err)","\tcdata, _ := io.ReadAll(resp.Body)","\tif resp.StatusCode != http.StatusTooManyRequests {","\t\tt.Fatalf(\"/v1/completions status = %d, want 429; body = %s\", resp.StatusCode, string(cdata))","\tif resp.Header.Get(\"Retry-After\") == \"\" {","\t\tt.Fatalf(\"/v1/completions 429 missing Retry-After header\")","// TestDedicatedSaturatedBoxFast429 verifies that when the dedicated box for a","// Gemma 4 request EXISTS but is at capacity, the request is fast-429'd","// immediately instead of sitting in the 120s queue-before-shed window (which is","// left at its default ON) — so OpenRouter fails over within its TTFT SLA.","func TestDedicatedSaturatedBoxFast429(t *testing.T) {","\t// queue-before-shed left at default (ON): dedicated models must still fast-429.","\t// A single DEDICATED gemma-4 box with its token budget nearly exhausted.","\t\tSlots: []protocol.BackendSlotCapacity{{","\t\t\tModel: gemma,","\t\t\tState: \"running\",","\t\t\tMaxConcurrency: 8,","\t\t\tActiveTokenBudgetUsed: 950,","\t\t\tActiveTokenBudgetMax: 1_000,","\t\t}},","\t\treturn p.BackendCapacity != nil && p.BackendCapacity.Slots[0].ActiveTokenBudgetMax == 1_000","\t\tt.Fatalf(\"request: %v (a hang here means the dedicated request was queued instead of fast-429'd)\", err)","\t\tt.Fatalf(\"status = %d, want 429 (dedicated boxes bypass queue-before-shed when saturated); body = %s\", status, body)","\t\tt.Fatalf(\"saturated dedicated 429 missing Retry-After header\")","coordinator/api/consumer.go",[1184],{"_965":685,"_966":1185,"_582":1186,"_969":59},5667,[1081,385,1187,1188,1189,1190,1191,1188,1192,1193,1194,1195,1196,1197,1198,385,1082,1083,1199,1200,1201,1202,1084,1203,1204,1085,1205,1206,1086,1088,385,1207,1208,1209,1089,1210,1211,1212,1213,385,1214,1091,385,1215,1216,1217,1218,1219,1220,385,1221,1222,1223,1224,1225,1226,1227,1228,1229,385,1230,1231,1232,1233,385,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,385,1246,1247,1248,1249,1250,385,1251,1252,1253,1254,1255,1256,1257,385,1258,1259,1260,1261,1091,385,1262,385,1263,1264,1265,1266,1267,1268,1269,994,385,1270,1271,1272,1273,1274,1275,1276,1277,1278,1045,992,1279,1280,1281,992,1282,1283,1284,1285,1286,1287,1288,1289,1290,1291,1292,1293,1294,1295,1296,1297,1298,1299,1300,1132,1301,1302,1303,1304,1305,994,385,1306,1307,1308,1309,1310,1311,1007,992,1312,1313,1097,1314,1007,992,1315,1117,1316,1317,1318,992,994,385,1319,1311,1320,992,1321,994,385,1322,1323,1324,1325,1188,1326,1327,1328,1329,1330,1331,1007,992,1332,1333,1334,1335,1336,992,994,385,1337,1338,1339,1340,1341,1342,1343,1344,1007,992,1345,1346,1007,992,1347,1348,1349,994,385,1350,1351,1352,1353,1354,1355,1188,1356,1357,1358,1359,1360,1007,992,1361,1362,992,1363,1364,1365,1366,1367,1368,1369,992,994,385,1370,1371,1372,1373,1374,1007,992,1375,1376,1377,1378,1379,992,994,385,1380,1381,1382,1383,1384,1385,1386,1387,1188,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,992,1400,1401,1045,992,1402,1403,1305,994,385,1404,1405,1406,1407,1408,1409,1410,1411,992,1412,1413,1414,994,385,1415,1416,1417,1418,1419,385,1420,1421,1422,1423,1424,385,1425,1426,1427,1428,1429,1430,992,1431,994,385,1432,1433,1434,1435,1436,1437,1438,1439,1440,992,1441,1442,1443,994,385,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,992,1457,1458,992,1459,1460,1097,1461,992,1462,994,385,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,992,1473,1474,1472,992,1475,1472,992,1476,1472,992,1477,1478,1479,992,1480,1481,994,385,1482,1471,1472,992,1473,1474,1472,992,1475,1472,992,1476,1472,992,1477,1483,1484,992,1480,1481,994,385,1485,1486,994,385,1487,1488,1489,992,1490,994,385,1491,1492,1493,1494,1495,992,1496,1497,992,1498,1499,992,1500,994,385,1501,1502,1301,1503,1302,1504,1304,994,385,1505,1506,1007,992,1507,994,385,1508,1509,1007,992,1510,1511,1512,1007,992,1513,1514,994,385,1515,1516,1517,1518,1519,992,1520,994,385,1521,1522,1523,1524,1525,1526,992,1527,1528,1529,1530,990,992,1531,1526,992,1532,1097,1526,992,1533,994,385,1534,1535,1536,1537,385,1538,385,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,992,385,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,992,1615,1616,1617,385,1618,1619,1620,1621,990,1622,992,385,1623,1624,1625,1626,1627,1628,990,1629,1630,1631,1632,1633,990,1634,992,1635,1636,1637,1638,1639,1640,990,992,1641,1642,1643,992,1644,1645,992,385,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,992,385,1657,1658,1659,992,385,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,990,992,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,990,992,385,1683,1684,1685,1686,1687,1688,992,385,1689,1097,1685,1686,1687,1690,992,385,1691,1097,1685,1686,1692,992,385,1693,1694,1695,1696,1097,1685,1686,1697,992,1642,1698,992,385,1699,1700,1701,1702,1703,1704,1131,992,385,1705,1706,1707,385,1708,1097,1685,1686,1709,992,1642,1710,992,1711,1685,1686,1687,1712,992,1713,385,1714,994,385,1715,1716,1717,1718,1719,1720,1721,1720,1722,1720,1723,1724,1664,1725,990,1726,1727,1728,992,994,385,1729,1730,1731,1732,1733,1188,1734,1735,1736,1737,1519,992,1716,1738,1739,1740,990,1741,1742,1743,990,1744,1727,1745,1664,1740,990,1746,1742,1743,990,1744,992,994,385,1747,1748,1749,1750,1751,1188,1752,1753,1754,1755,1756,1737,1519,992,1716,1738,1757,1727,1745,1664,1740,990,1758,992,994,385,1759,1760,1761,1762,992,1763,1764,992,1765,1766,992,1767,1768,992,1769,994,385,1770,1771,1772,1773,1774,1760,1761,1775,1776,1777,1778,1779,1780,1781,1782,992,1763,1782,992,1765,1782,992,1767,1783,992,1769,994,385,1784,1785,1786,1787,1788,1215,1789,1790,1091,385,1791,1792,1793,1794,1795,1796,1797,1798,992,1075,994,385,1799,1800,1801,1802,1803,1804,1805,1806,1740,990,1807,1808,990,1809,992,1810,1738,1811,1812,1813,1814,1815,1816,1817,1669,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1823,1669,990,1831,1727,1832,992,994,385,1833,1834,1835,1836,1837,1044,1838,992,1760,1839,1840,1841,1842,989,990,1843,1844,992,1769,994,385,1845,1846,1847,1848,1849,1850,1738,1851,1812,1813,1852,1853,1854,1855,1856,1857,1858,1859,1860,1823,1861,1862,1828,1863,1669,990,1831,1727,1864,992,994,385,1865,1866,1867,1868,1044,1045,992,1869,1870,1841,989,990,1871,1074,990,992,1075,994,385,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1669,990,992,1883,1884,1885,1886,1887,1882,1669,990,992,1075,994,385,1888,1889,1890,1891,1892,1893,1894,1895,994,385,1896,1897,1898,1411,992,1899,1900,1411,992,1901,1902,994,385,1903,1904,1905,1906,1907,1669,1908,990,992,1909,1910,992,1911,994,385,1912,1913,1914,1915,1916,1841,989,990,1917,1918,1919,1920,1921,1922,1923,1924,990,992,1925,1926,992,385,1927,1928,1929,1930,1841,1931,990,1932,1933,989,990,1934,989,990,1935,1936,992,1937,1938,992,1939,994,385,1940,1941,1915,1942,1943,1944,990,992,1945,994,385,1946,1947,1948,1949,1950,1951,385,1952,1953,385,1954,1955,1956,1957,1958,1959,1960,1961,992,1962,1963,992,1964,1965,1961,992,1941,1966,1942,1943,1944,990,992,1967,1961,992,1968,1969,992,1970,994,385,1971,1972,1973,1974,1975,1976,1977,1978,385,1979,1980,1981,1904,1905,1908,990,992,1982,994,385,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,994,385,1994,1995,1045,992,1996,1997,992,1998,1999,2000,2001,2002,990,2003,1132,1097,2004,2005,2006,2007,2008,2009,1045,992,2010,1045,992,2011,2012,2013,2014,2015,992,1305,994,385,2016,2017,2018,2019,2020,2021,2022,992,2023,2024,2025,2026,992,2027,2028,992,2029,994,385,2030,2031,2032,2033,2034,2035,994,385,2036,1624,1045,992,2037,2038,2039,994,385,2040,1624,1411,992,2037,2038,2041,994,385,2042,2043,2044,2045,2046,2047,990,992,2048,994,385,2049,2050,2051,2052,2053,2054,1045,992,2055,2056,992,1075,994,385,2057,1344,2058,992,2059,2060,2061,2062,2063,992,2064,2065,2063,992,2066,2067,2068,2069,2070,2071,2072,2073,2074,2075,990,992,2076,2077,2078,992,2079,2080,2081,994,385,2082,2083,2084,2085,2086,2087,2088,2089,2090,2091,2092,2093,2094,2095,2096,2097,2098,1882,1669,990,1045,992,2099,2100,2101,2102,992,1305,994,385,2103,2104,2105,2106,992,993,994,385,2107,1810,2108,1411,1738,2109,1812,2110,1814,2111,1854,2112,2113,1823,1856,2114,2115,1860,1823,2116,2117,2118,2119,1823,1669,990,2120,1727,2121,1664,2122,990,2123,992,994,385,2124,2125,1738,2126,1812,2127,2128,2129,1816,1817,1669,2130,2131,2132,2133,2134,2135,2136,2137,2138,2136,2139,2140,2141,2142,2143,2144,2145,2146,2136,2147,2148,2139,2149,2150,2151,2152,2153,2154,2155,2156,2157,2158,2142,2159,2144,2139,2160,2161,2162,2142,2163,2164,1860,2165,2166,1823,1817,1669,385,2167,2168,1817,1669,2169,2170,1669,2171,2172,2173,2174,990,2175,2176,990,2177,1727,2178,992,994,385,2179,2180,2181,2182,992,2183,2184,2185,1841,989,990,2186,2187,2188,2189,2190,2191,1860,1823,2192,1669,2193,2194,2195,1669,2196,2197,1669,2198,2199,2200,2174,989,990,2201,992,2202,994,385,2203,2204,1044,2205,992,2206,2207,2208,2209,2210,990,2211,2212,2213,2214,2215,2216,992,2217,994,385,2218,2219,1044,1526,992,2220,1044,1526,992,2221,2222,2223,2224,2211,2225,2226,990,992,2227,994,385,2228,2229,1097,2230,992,385,2231,2232,2233,2234,2235,2236,2237,2238,992,2239,2230,2240,2241,992,2242,2230,2243,2244,992,2245,2246,992,2202,994,385,2247,1188,2248,2249,2250,1188,2251,2252,2253,2254,2255,385,2256,2257,2258,1044,1007,992,2259,2260,2261,385,2262,2263,2264,2265,2266,2267,2268,2269,2270,2271,2272,2273,2274,2275,2276,2277,1007,992,385,2278,2279,1909,2267,2268,2269,2280,2271,2272,2273,2274,2281,2275,2276,2282,2283,1007,992,385,2284,1097,2267,2268,2269,2280,2271,2272,2273,2274,2275,2276,2285,1007,992,2286,2287,992,385,2288,2289,2290,2291,2292,2293,385,2294,2295,2296,2297,2298,2299,2300,1044,2267,2268,2301,2302,2303,2272,2273,2274,2275,2276,2304,2305,1007,992,2306,385,2307,2308,2309,2310,2311,2312,2313,2314,2315,2316,2317,2318,2319,2320,2321,2322,1007,992,385,2323,385,2324,2325,2326,2327,2328,2329,2330,2331,2332,2333,2334,2335,2336,1669,990,2337,2338,2339,2340,2341,2342,990,2343,992,385,2344,2345,2346,2347,2348,2349,2350,2287,992,385,2351,2352,2353,2354,2355,2356,2357,1007,992,385,2099,2358,1664,2359,2360,2361,2362,2363,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2174,2374,2375,990,2376,992,385,2377,2378,2379,2380,2381,1044,1007,992,385,2382,2383,2384,2385,2386,2387,2388,2389,2390,2391,2392,2393,2394,2395,2396,2397,2398,2359,2360,2399,2400,2401,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2174,2402,2375,990,2403,2404,1664,1667,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,1669,2375,990,992,2426,385,2427,2428,2429,2430,990,992,385,2431,2432,2433,2267,2434,2435,2436,2437,2438,2439,2440,2441,2442,2443,2444,2445,2446,2447,2276,2448,2449,1007,992,385,2450,2451,2452,2453,2454,2375,990,2455,2456,2457,2458,2459,2460,2461,2101,2462,2463,2464,2465,2466,2467,2468,2469,2470,2471,2472,2473,2474,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2136,2497,2498,2499,1823,1669,990,2500,2501,2502,2503,2504,2505,2506,2375,990,2507,2508,2509,2454,2510,2359,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2174,2530,2531,2532,2375,990,2470,2533,2534,2535,2536,2537,2538,2539,2540,2541,2542,2543,2544,2545,2546,2547,2548,2549,2423,2550,2551,2552,2553,2554,2555,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2577,2578,2579,1669,990,2580,2581,2582,2583,2584,2585,2586,2587,2588,2589,2590,2591,2592,2593,2594,2595,2596,2586,2597,2534,2535,2598,2599,2600,2601,2602,2603,2604,2605,2606,2607,2608,2552,2553,2554,2609,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2610,2578,2579,2423,2611,2612,2613,2614,2615,2616,2552,2553,2554,2617,2405,2556,2557,2618,2619,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2620,2621,2622,2579,1669,990,2623,2624,2625,2626,2627,2628,2629,2630,2631,2632,2633,2634,2635,2472,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2136,2497,2498,2499,1823,2423,2636,2637,2632,2633,2638,2554,2405,2556,2639,2640,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2570,2571,2572,2573,2574,2641,2420,2642,2579,1669,990,992,385,2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2433,2267,2434,2672,2673,2674,2438,2439,2440,2441,2442,2443,2444,2445,2446,2675,2447,2276,2676,2677,1007,992,385,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697,2698,2699,2700,2701,2702,2703,2704,992,2705,994,385,2706,2707,2708,2709,2710,2711,2712,1007,992,385,2713,1044,2714,1007,992,385,2715,2716,2717,2718,2719,992,2720,385,2721,2722,2723,385,2724,2725,2726,2727,2728,2729,2730,385,2731,2732,2733,2734,2735,385,2736,2737,2738,2739,2740,989,990,2741,2742,990,2743,2744,2745,2746,2747,2748,2749,2750,2751,2752,1669,2753,2754,2755,990,992,385,2756,2757,2758,2759,385,2760,2761,2762,1816,2763,2764,2765,2766,2767,2768,2769,2770,2771,2772,2773,2157,2494,2774,2775,2496,2136,2165,1823,2776,2777,2778,2779,2780,2781,1823,2782,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2136,2806,2807,2808,2809,2805,2136,2810,2811,2812,2813,2814,2815,2816,2805,2817,2808,2809,2805,2818,2819,2820,2821,2822,2823,2824,2825,2826,2827,2828,2829,2830,2494,2831,2775,2136,2832,2833,2780,1823,2579,1669,2834,2835,2836,2837,2838,2839,2763,2840,2165,1823,1669,2841,385,2751,2842,2843,1823,1669,2844,2845,2846,2847,2848,2849,2850,2851,1817,1669,2852,2853,2854,2751,2855,2856,1860,1823,1669,2751,2857,2858,2859,2860,2861,2862,1860,1823,1669,2863,2864,2755,385,2865,1816,1817,1669,2866,2867,2868,2869,2870,2871,2872,2873,2874,2174,2875,2755,2375,385,2876,2877,2878,2879,2880,2755,2375,385,2881,2375,990,992,994,385,2882,2713,1044,2714,1007,992,385,2883,2718,2719,992,385,2884,2885,2886,2887,385,2739,2888,2889,990,992,385,2758,2759,385,2760,2761,2762,1816,2890,2891,2763,2892,2893,2894,2895,2136,2896,1823,2897,2777,2778,2898,2781,1823,2783,2899,2579,1669,2900,2839,2763,2840,2165,1823,1669,2841,385,2865,1816,1817,1669,2866,2867,2868,2869,2901,2375,385,2876,2877,2878,2879,2902,2375,385,2881,2375,990,992,994,385,2903,2904,2905,2906,2907,2908,1117,385,2909,2739,2888,2910,990,992,385,2760,2761,2762,1816,2763,2764,2765,2766,2767,2911,2912,2913,2914,2805,2915,2496,2136,2165,1823,2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,2944,2945,2946,2937,2947,2802,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,2962,2963,2964,2965,2966,2967,2968,2964,2965,2966,2969,2970,2971,2936,2937,2972,2973,2974,2975,2976,2977,2937,2802,2978,2979,2980,2802,2981,2982,2983,2805,2136,1823,385,2984,2985,2763,2986,2987,2988,2989,2990,2496,2136,2991,2992,2993,2994,2995,2136,2996,2997,2998,2999,3000,3001,3002,2994,3003,3004,2136,1823,2579,1669,3005,385,2865,1816,1817,1669,2866,2867,3006,3007,3008,3009,1669,3010,2375,385,3011,3012,3013,3014,3015,2423,3016,3017,1669,2375,990,992,994,385,3018,3019,3020,3021,3022,1007,992,3023,1044,1007,992,3024,3025,3026,3027,1669,990,992,994,385,3028,3029,3030,992,3031,3032,3033,990,992,3023,1044,1007,992,3024,3034,1841,989,990,3035,3036,990,3037,3038,990,992,994,385,3039,3040,3041,3042,3043,3044,3045,3046,992,385,3047,3048,3049,990,3050,992,3051,3052,992,3053,3054,992,3055,3056,3057,990,992,994,385,3058,3059,3060,1007,992,3061,3062,3063,990,1007,992,3064,994,385,3065,3066,3067,3068,3069,3070,3071,3072,1669,990,992,3073,3074,3075,3076,3077,3078,990,3079,992,3080,3081,992,3082,994,385,3083,3084,3085,3086,3087,3088,3089,3090,3091,3092,3093,3094,3095,3096,3097,3098,3099,3100,1440,992,385,3101,3102,1440,992,385,1941,385,3103,3104,3105,3106,3107,1944,990,992,385,3108,3109,3110,3111,3112,3113,3114,3115,3116,3117,3118,3119,2802,2805,3120,3121,3122,2805,3123,3124,3125,3126,3127,3128,3129,3119,2802,3130,3131,3132,3122,2805,3133,3134,2805,2136,1823,1669,3135,3136,1669,990,992,385,1967,1440,992,385,3137,1097,1440,992,3138,994,385,3139,3140,3141,3142,3143,3144,3145,994,385,3146,3147,3148,3149,3150,3151,3152,3153,385,3154,3155,3156,3157,989,990,385,3158,3159,989,990,385,3160,1841,989,990,3161,3162,3163,3164,3165,3166,3167,3168,3169,3170,3171,3172,3173,3174,3175,3176,3163,3164,3165,3177,3178,990,3179,989,990,385,3180,3181,3182,1669,3183,3184,3185,3186,1669,3187,3188,3189,3190,3191,3192,3193,3194,1669,3195,3196,1858,3197,3198,3199,3200,3201,2136,3202,1823,3203,3204,1823,3205,3206,1823,3207,3208,3209,1823,3210,1669,990,992,385,3211,3212,3213,3214,3215,3216,2750,3217,990,992,3218,3219,3220,3221,3222,3223,3224,1669,990,992,3225,994,385,3226,3227,3228,3229,3230,3231,3232,3233,3234,3235,3236,3237,3238,3239,3240,3241,1007,992,3242,1044,1007,992,3243,3244,3245,992,3246,1007,992,3247,3248,3249,994,385,3250,3251,3252,3253,3254,3255,3256,3257,3258,3259,3260,3261,994,385,3262,3263,3264,3265,3266,3267,3268,3269,3260,3270,3271,1045,992,3272,3273,992,3274,1045,992,3275,994,385,3276,3277,3278,3279,3280,3281,1188,3282,3283,3284,3285,3286,3287,3288,3289,3290,1188,3291,3292,3293,3294,3295,3296,3297,3298,3299,3300,3301,3302,3303,990,3304,3305,1882,1669,990,992,3306,3307,3308,1045,992,3309,3310,3311,3312,3313,3314,3315,992,3316,1045,992,3317,1045,992,3318,1045,992,3319,1045,992,3320,3321,1032,990,3322,1032,990,3323,3324,3325,3326,3327,3328,3329,1823,3330,3331,3329,1823,1828,3332,3333,1669,990,992,1305,994,385,3334,3088,3335,3336,3337,992,3338,3337,992,3339,3337,992,3340,3337,992,3341,994,385,3342,3343,3344,3345,3346,3347,3348,3349,3350,992,3351,3352,992,3353,1097,1411,992,3354,994,385,3355,3356,3357,3358,3359,3088,3360,3337,992,3361,3338,3337,992,3362,3363,3364,3365,3366,1669,990,992,3367,994,385,3368,3369,3370,3371,3372,3373,3374,3375,3376,3377,3378,3379,2136,1823,1669,990,992,3351,3352,992,3353,1097,1411,992,3354,994,385,3380,3381,3382,3383,3384,3385,994,385,3386,3387,3388,3389,3390,3391,992,3392,3393,992,3394,3395,992,3396,994,385,3397,3398,3399,992,3400,3401,992,1982,994,385,3402,3403,3404,3405,3406,3407,992,994,385,3408,3409,3410,3411,3412,3413,1727,1526,992,994,385,3414,3415,994,385,3416,3417,3418,3419,3420,3421,3422,3423,3424,3425,2276,3426,992,3427,3419,3428,3429,3430,3431,3432,3433,3434,3435,3425,2276,3426,992,3436,3437,3438,3439,3440,990,3441,3442,3419,3443,3444,3445,3446,3447,3448,2276,3426,992,3449,994,385,3450,3451,3452,3453,3454,3455,2101,3456,992,3457,3458,3459,992,3460,3461,992,3462,3463,992,994,385,3464,3465,3466,3467,3468,3469,3470,3471,3472,3473,3474,992,3475,3476,3477,3478,992,3479,994,385,3480,3481,1526,992,3482,994,385,3483,3484,3485,3486,3487,3488,992,3489,994,385,3490,3491,3492,3493,992,3494,3495,3496,992,385,3497,3151,3498,3499,3500,3501,3502,992,385,3503,3504,3505,3506,3507,3508,3509,992,3510,3511,992,3512,3476,3513,3514,992,3515,994,385,3516,3517,3518,3519,992,3418,3520,992,385,3521,3522,992,3466,385,3523,3524,3525,3526,3527,3528,3529,3530,3531,1177,3532,3533,3534,3535,1131,992,385,3536,3537,3538,3539,3540,3541,990,992,385,3476,3477,3478,992,385,3479,994,385,3542,1188,3543,3544,3545,3546,3547,3548,3549,3550,3551,3552,3553,3554,1007,992,3555,3556,992,994,385,3557,3558,3559,3560,3561,3562,3563,3564,3565,3566,3567,3568,3569,3570,3571,3572,3573,1097,3574,3575,992,385,3576,3577,3578,989,990,3579,3580,3581,3582,3583,3584,3585,3586,3587,990,3588,3589,3590,3591,3592,3593,3594,3595,3596,990,3597,3598,3599,3600,3601,1669,990,3602,3603,3604,3605,3606,3607,1923,3608,989,990,385,3609,3610,3611,3612,3613,3614,3615,1669,990,385,3616,3617,3618,3619,3620,990,3621,3622,3623,3624,3625,3626,3627,990,3628,3629,3630,3631,3632,3633,990,3634,3635,3636,3637,3638,990,3639,3640,3641,3637,3642,990,3643,3644,992,3645,994,385,3646,3647,3648,3649,3650,385,3651,3652,3653,3654,3655,992,385,3656,3657,3658,3659,1097,2230,992,385,3660,3661,3662,385,3663,3664,3665,3666,3667,989,990,3668,989,990,3621,3669,3670,3671,3672,3673,990,3674,3675,3676,3677,3678,2750,3679,3680,3681,990,3682,3683,3684,3685,3686,1669,990,3687,3688,990,385,3628,3689,3690,3691,3692,3693,3694,3695,990,385,3696,3697,3698,3699,3700,385,3701,3641,3637,3642,990,3702,385,3703,992,385,3704,994,385,3705,3706,3707,1097,3708,3709,1007,992,385,3710,3711,3712,1132,994,385,3713,3714,3715,3716,3717,3718,3719,1097,3708,3709,1007,992,3720,3721,3722,2375,990,992,3723,3724,994,385,3725,3726,3727,3728,3729,3730,3731,3732,1007,992,385,3733,1097,3734,1007,992,3735,3736,3737,1132,994,385,3738,3739,3740,3741,3729,3730,3742,1007,992,385,3743,3744,992,3745,3746,1007,992,385,3747,3748,3749,1007,992,385,3750,3751,1007,992,3752,385,3753,994,385,3754,385,3755,3756,3757,3758,3759,3760,3761,3762,3763,3764,3765,3766,994,385,3767,3768,385,3769,3770,385,3771,3772,3773,3774,3775,3776,3777,3778,3779,3780,3781,3782,3783,3784,3785,3786,3787,992,3788,3789,3790,3791,3792,3793,3794,3795,3796,990,3797,3798,992,3479,994,385,3799,3800,3801,3802,3803,1727,3804,992,3805,3806,992,3807,3808,992,3809,3810,992,3811,3812,992,3813,3814,992,3815,994,385,3816,3817,3818,3819,3820,3821,1798,992,3822,3823,1074,990,992,1075,994,385,3824,3825,3826,3827,3828,3829,3830,3820,3831,3832,992,3833,3789,3834,3835,3836,3837,990,3838,3839,992,3840,994,385,3841,3842,3729,3730,3742,1007,992,3843,1097,3844,1007,992,3845,3846,3847,992,3848,994,385,3849,3850,3851,3729,3730,3731,3732,1007,992,385,3852,3853,3854,3855,3856,2375,990,992,3857,3858,1007,992,385,3859,3860,3861,3862,3863,3864,3865,3866,3867,992,3868,3869,3870,992,385,3871,1097,3734,1007,992,3872,3873,3874,1132,994,385,3875,3876,3729,3730,3742,1007,992,3718,3877,1097,3878,1007,992,3879,994,385,3880,3881,3882,3883,3884,3885,3886,1007,992,3879,994,385,3887,3888,3889,3729,3730,3742,1007,992,3718,3890,1097,3878,1007,992,385,3891,3892,3893,3894,3895,1007,992,3896,3858,1007,992,3897,3858,1007,992,385,3898,3899,3900,3901,3902,1097,3903,1007,992,3901,3904,994,385,3905,3906,3907,3908,3909,3910,3911,990,3912,992,3913,3914,3915,3916,990,3917,992,3918,3919,3920,3921,990,3922,992,3923,3924,3925,2750,3926,3927,3928,1669,3929,3930,1669,3931,3932,990,992,3933,3924,3934,2750,3935,3936,3937,1669,3938,990,992,3939,3940,3941,3942,990,3943,992,3944,3945,3946,3947,3948,3949,3950,3951,2423,3952,3953,3954,1823,3955,1669,990,992,3956,3924,3957,2750,3958,3959,3960,1669,3961,990,992,3815,994,385,3962,3963,3729,3730,3742,1007,992,3718,3901,3964,3878,1007,992,3901,3753,994,385,3965,3966,3967,3968,3729,3730,3742,1007,992,3718,3969,3970,3901,3971,1097,3878,1007,992,3901,3872,3873,3874,1132,994,385,3972,3973,3974,3975,3976,3977,3978,3979,3980,3981,1132,994,385,3982,3983,3984,3985,3986,3987,3988,3989,1007,992,385,3990,3991,3992,3993,3994,3995,3996,3997,3998,3999,4000,4001,990,2101,4002,4003,4004,4005,990,4006,3993,4007,4008,990,992,4009,1097,4010,1007,992,4011,4012,994,385,4013,385,4014,4015,4016,2657,4017,4018,385,4019,4020,4021,4022,4023,1132,994,385,4024,4025,4026,4027,4028,2657,4029,385,4030,4031,4032,4033,4034,4035,4036,4037,1669,4038,4039,4040,1669,4041,4042,4043,4044,4045,4046,4047,2174,990,992,385,4048,4049,1132,994,385,4050,1188,4051,4052,4053,4054,4055,4056,992,4055,4057,1007,992,385,4058,4059,4060,4061,385,4062,4063,4064,4065,4066,4067,4068,4069,4070,990,992,385,4071,4072,4073,4074,4075,4076,4077,4078,4079,4080,4081,4082,4083,2420,4084,4085,1669,990,992,385,4086,4087,992,385,4088,4089,4090,4091,4092,4093,4094,4095,1132,994,385,4096,385,4097,4098,4099,4100,4101,4102,994,385,4103,4104,4105,4106,994,385,4107,4108,4109,4110,994,385,4111,4112,4113,4114,2255,385,4115,4116,4117,4118,2258,1044,1007,992,2259,2260,2261,385,2284,1097,2267,2268,2269,2280,2271,2272,2273,2274,2275,2276,2285,1007,992,4119,4120,992,385,4121,2293,385,4122,4123,4124,4125,4126,1044,2267,2268,2301,2302,2303,2272,2273,2274,2275,2276,2304,2305,1007,992,4127,2317,385,2432,2267,2268,2301,4128,4129,2272,2273,4130,4131,2275,2276,2448,2449,1007,992,385,4132,4133,4134,2312,2316,2321,4135,1007,992,385,4136,4137,4138,4139,2330,2331,2341,4140,990,2343,992,4141,385,2351,2352,2353,2354,4142,2356,2357,1007,992,385,4143,4144,385,4145,2381,1044,1007,992,385,4146,4147,4148,2657,2658,2388,2389,4149,2393,2395,4150,2398,2359,2360,2399,2400,2401,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2174,2402,2375,990,2403,2404,1664,1667,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,1669,2375,990,992,2428,2429,4151,990,992,2426,4152,4153,2434,4154,4155,4156,2438,4157,2440,2441,2442,2443,2444,2445,2446,4158,4159,2447,4160,990,992,4161,4162,4163,4164,4165,4166,4167,4168,4169,992,385,4170,4171,2452,2453,2454,2375,990,2455,4172,4173,2101,2469,2470,2471,2472,2473,2474,1669,990,4174,4175,4176,2506,2375,990,2507,2454,2510,2359,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2174,2530,2531,2532,2375,990,2470,2533,2534,2535,4177,4178,2539,4179,2543,4180,4181,2548,2549,2423,2552,2553,2554,2555,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2577,2578,2579,1669,990,2580,4182,4183,2586,4184,4185,4186,4187,4188,4189,2534,2535,2598,2599,2600,2601,4190,4191,4192,4193,2552,2553,2554,2609,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2610,2578,2579,2423,4194,4195,2552,2553,2554,2617,2405,2556,2557,2618,2619,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2620,2621,2622,2579,1669,990,4196,2623,2624,4197,4198,4199,4200,2632,2633,2634,2635,2472,2423,4201,4202,2632,2633,2638,2554,2405,2556,2639,2640,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2570,2571,2572,2573,2574,2641,2420,2642,2579,1669,990,992,385,1568,4203,4204,1577,1578,1579,1580,1581,1582,1583,1594,1595,1596,1597,1598,1585,1586,1590,4205,4206,4207,4208,4209,4210,4211,4212,4213,4214,4215,4216,992,385,1604,1605,1606,1607,4217,4218,1613,4219,992,4220,1617,385,1672,4221,4222,4223,4224,1675,1676,1677,1678,1679,1680,1681,1682,990,992,385,4225,4226,4227,4228,4229,4230,4231,990,385,4232,4233,4234,4235,4236,4237,4238,4239,4240,990,385,4241,4242,4243,990,385,4244,4245,4246,4247,4248,4249,4250,4251,4252,4253,4254,4255,4256,1823,1817,1669,990,385,4257,4258,992,1624,1629,4259,4260,1632,4261,2454,4262,2375,990,385,4263,4264,4265,1627,2454,2510,2530,2531,2532,2375,990,4266,4267,4268,4269,4270,990,4271,4272,4273,4274,2454,4275,4276,4277,4278,4279,2423,2576,4280,2578,1669,2375,990,4281,4282,4283,4284,4285,4286,4287,4288,2434,4289,4290,4291,4292,2443,2444,2445,2446,4293,4294,4295,4296,4297,990,4298,4299,1664,4300,4301,4302,4303,2554,2579,1669,4304,4273,4305,4274,2454,4306,4277,4278,4307,2423,2576,4308,2578,1669,2375,990,4281,4309,992,4310,4311,4312,4313,4314,4315,992,4316,4317,1284,4318,4319,4320,4321,1293,1294,1295,1296,4322,4323,4324,4325,4326,4327,992,4328,1635,1636,1637,1638,1639,1640,990,992,1641,4329,4330,4331,4332,4333,1654,4334,1656,992,4335,1658,1659,992,4336,4337,4338,1665,4339,2454,1667,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,4340,2423,4341,4342,2425,1669,1667,4343,1669,2375,990,992,385,4344,385,4345,4346,4347,4348,2670,4349,1686,1685,2433,4350,2267,2434,2672,2673,2674,2438,2439,2440,2441,2442,2443,2444,2445,2446,4351,2447,2276,2676,2677,1007,992,385,1684,1686,1685,2433,4352,4353,4354,1007,992,385,1689,1097,1686,1685,2433,4355,4356,1007,992,385,1691,1097,1686,1685,2433,4355,4357,1007,992,385,4358,4359,4360,4361,1097,1686,1685,2433,4355,4362,1007,992,4363,385,1699,1700,1701,1702,1703,1704,1131,992,385,1705,4364,4365,1711,1686,1685,2433,4366,4367,1007,992,1713,385,4368,4369,4370,4371,4372,4373,4374,385,4375,4376,4377,4378,4379,4380,4381,4382,385,4383,4384,4385,4386,4387,4385,4388,4389,4390,4391,2750,4392,4393,4248,4249,4394,4395,2554,4396,4397,4398,4399,4400,1823,4401,2579,1828,4402,1669,990,4403,4385,4404,4405,4406,1685,2433,4407,4408,4409,4410,4411,990,4412,1007,4413,4404,4405,4406,1685,2433,4414,4415,4416,1007,4417,4385,4404,4405,4406,1685,2433,4418,4419,1007,992,385,4420,4421,4422,2761,2762,4423,4424,4425,4426,4402,2423,2763,4427,4428,4429,4430,4431,4432,4433,4434,4435,4436,4437,2136,4438,2781,2165,4439,1823,1669,4440,4423,1638,1639,4441,4339,2454,4442,4443,3007,3008,3009,1669,3010,2375,4444,1638,1639,4441,4339,2454,4445,4446,4447,4448,4449,4450,4451,4452,2375,2881,4423,1638,1639,4441,4339,2454,4453,4454,2375,990,992,385,4455,4404,4405,4406,1685,2433,4456,4457,1007,992,4458,385,4459,4460,4461,4462,4463,4464,2750,4465,4466,4467,2174,990,4404,4405,4406,4468,385,4469,4470,4471,992,4472,4473,4474,4475,2101,4476,992,994,385,4477,4478,4479,4480,994,385,4481,4482,4483,4484,4485,4486,4487,4488,992,4489,4490,4491,990,4492,4493,990,992,4494,4495,992,994,385,4496,4497,385,4498,4499],"// Consumer-facing API handlers for the Darkbloom coordinator.","//","// This file implements the OpenAI-compatible HTTP endpoints that consumers","// use to send inference requests. The coordinator acts as a trusted routing","// layer between consumers and providers.","// Trust model:","// The coordinator runs in a Confidential VM, providing hardware-encrypted","// memory. Consumers may additionally sender-seal requests to the","// coordinator's X25519 key. The coordinator decrypts for routing purposes","// but never logs prompt content, then re-encrypts each request to the","// selected provider's X25519 public key before forwarding over the","// WebSocket. Providers are attested via Secure Enclave challenge-response.","\t\"crypto/sha256\"","\t\"encoding/json\"","\t\"errors\"","\t\"fmt\"","\t\"log\"","\t\"math\"","\t\"regexp\"","\t\"strconv\"","\t\"github.com/eigeninference/d-inference/coordinator/auth\"","\t\"github.com/eigeninference/d-inference/coordinator/internal/e2e\"","\t\"github.com/eigeninference/d-inference/coordinator/payments\"","\t\"github.com/eigeninference/d-inference/coordinator/registry\"","\t\"github.com/eigeninference/d-inference/coordinator/saferun\"","\t\"github.com/eigeninference/d-inference/coordinator/store\"","\t\"github.com/google/uuid\"","\t\"github.com/eigeninference/d-inference/coordinator/api/types\"","const (","\t// inferenceTimeout is the maximum time to wait between chunks (streaming)","\t// or for the full response (non-streaming). For streaming, the deadline","\t// resets on each received chunk so long-running generations don't time out.","\t// 10 minutes allows 32k tokens at ~55 tok/s on slower hardware.","\tinferenceTimeout = 600 * time.Second","\t// preambleContentTimeout is the budget from the first boilerplate chunk to","\t// the first CONTENT chunk when the TTFT deadline has already expired. A","\t// provider that produced only preamble (role delta / Responses lifecycle)","\t// has written ZERO bytes to the client, so a role-then-stall zombie must","\t// fail over instead of pinning the request for the full inferenceTimeout.","\t// 90s comfortably covers the measured pre-content tail (vision prefill is","\t// 6-30s); genuine cold model loads signal via AcceptedCh and keep the full","\t// inferenceTimeout.","\tpreambleContentTimeout = 90 * time.Second","\t// chunkBufferSize is the channel buffer size for SSE chunks flowing from","\t// the provider to the consumer. A larger buffer prevents dropped chunks","\t// when the consumer reads slowly.","\tchunkBufferSize = 256","\t// maxDispatchAttempts is a SAFETY CEILING on per-request provider failover,","\t// not the normal stopping point. A request keeps failing over to fresh","\t// healthy providers until one succeeds, OR candidates are exhausted (every","\t// failed provider is excluded from re-selection, so dispatchPrimary returns","\t// outcomeFailFast on the next attempt once no eligible provider remains), OR","\t// the request's deadline/context fires (run() checks r.Context() each","\t// attempt). This ceiling only guards against a pathological retry path that","\t// fails to exclude a provider (an unbounded hot loop); it is set well above","\t// any realistic per-request fault count. Retries never re-queue — only the","\t// first attempt may wait for capacity — so failover stays fast, walking the","\t// immediately-available healthy providers rather than waiting on busy ones.","\tmaxDispatchAttempts = 64","\t// speculativeTimerRatio is the fraction of the TTFT deadline at which","\t// the coordinator launches a speculative backup dispatch. The primary","\t// provider gets this fraction of the deadline before the backup is","\t// started, and then both race until one produces the first chunk.","\tspeculativeTimerRatio = 0.5","\t// maxHeldBoilerplate bounds how many pre-content boilerplate chunks the","\t// dispatch loop holds per provider before committing anyway. Real","\t// preambles are one chunk (chat role delta) or two (Responses","\t// created/in_progress), so the cap exists only to stop a misbehaving","\t// provider from growing the held buffer for the whole inference window.","\t// Past the cap the chunk commits the dispatch — the pre-deferral behavior.","\tmaxHeldBoilerplate = 8","\t// cancelWriteTimeout bounds how long a cancel write to the provider can","\t// block. Using context.Background() unbounded here risks hanging the HTTP","\t// handler goroutine when a WebSocket is half-dead.","\tcancelWriteTimeout = 2 * time.Second","var thinkBlockPattern = regexp.MustCompile(`(?is)(.*?)\\s*`)","// ttftDeadline returns the TTFT budget for a request based on prompt size.","// Base: 5 seconds + 1ms per estimated input token. This meets the OpenRouter","// SLA of TTFT < 5s + 1ms/input_token.","func ttftDeadline(estimatedPromptTokens int) time.Duration {","\tbase := 5 * time.Second","\tperToken := time.Duration(estimatedPromptTokens) * time.Millisecond","\treturn base + perToken","// shedIfModelRejected answers a public/prefer-owner request with 429 +","// Retry-After when its requested alias or resolved build is in the operator","// reject set (EIGENINFERENCE_REJECT_MODELS). This is a deterministic","// per-model circuit breaker: it takes an unhealthy model out of rotation before","// rate-limit, reservation, or routing work, so aggregators see rate limiting","// rather than dropped/cancelled streams. Exclusive self-route bypasses the shed","// because it never falls back to the public fleet.","func (s *Server) shedIfModelRejected(w http.ResponseWriter, r *http.Request, parsed map[string]any, policy selfRoutePolicy, publicModel, model string, stream bool, estimatedPromptTokens, requestedMaxTokens int, requiresVision, hasTools bool) bool {","\tif policy.enabled || !s.modelShed(model, publicModel) {","\tretryAfter := s.estimateRetryAfter(model)","\tif retryAfter <= 0 {","\t\tretryAfter = 30","\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:model_shed\"})","\ts.recordRejection(rejectionInfo{","\t\tr: r,","\t\tstage: \"model_shed\",","\t\treasonCode: \"model_shed\",","\t\thttpStatus: http.StatusTooManyRequests,","\t\tkeyID: keyIDFromContext(r.Context()),","\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\trequestedModel: publicModel,","\t\tresolvedModel: model,","\t\tstream: stream,","\t\testimatedPromptTokens: estimatedPromptTokens,","\t\trequestedMaxTokens: requestedMaxTokens,","\t\trequiresVision: requiresVision,","\t\thasTools: hasTools,","\t\tselfRouteOnly: policy.enabled,","\t\tpreferOwner: policy.prefer,","\t\tretryAfterMs: retryAfter * 1000,","\t\tparams: rejectionSamplingParams(parsed),","\tw.Header().Set(\"Retry-After\", strconv.Itoa(retryAfter))","\twriteJSON(w, http.StatusTooManyRequests, errorResponse(\"rate_limit_exceeded\",","\t\tfmt.Sprintf(\"model %q is temporarily rate-limited — retry after %ds\", publicModel, retryAfter),","\t\twithCode(\"rate_limit_exceeded\")))","\treturn true","// sendProviderCancel sends a Cancel message for the given request to the","// provider with a bounded timeout so a half-dead WebSocket doesn't hang the","// caller. Errors are logged at debug level because a disconnect race is the","// expected case — the provider may already be gone.","func (s *Server) sendProviderCancel(provider *registry.Provider, requestID string) {","\tif provider == nil || provider.Conn == nil {","\tcancelMsg := protocol.CancelMessage{Type: protocol.TypeCancel, RequestID: requestID}","\tcancelData, err := json.Marshal(cancelMsg)","\t\ts.logger.Error(\"failed to marshal cancel message\", \"request_id\", requestID, \"error\", err)","\tctx, cancel := context.WithTimeout(context.Background(), cancelWriteTimeout)","\tif err := provider.EnqueueText(ctx, cancelData); err != nil {","\t\ts.logger.Debug(\"failed to send cancel (provider may have disconnected)\",","\t\t\t\"request_id\", requestID, \"error\", err)","func writeProviderInferenceRequest(ctx context.Context, provider *registry.Provider, data []byte) error {","\t\treturn errors.New(\"provider websocket is not connected\")","\treturn provider.WriteText(ctx, data)","// cancelDispatch cleans up a speculative dispatch participant that lost the","// race (or a failed/timed-out attempt): removes the pending request, marks the","// provider idle, sends a cancel over WebSocket so the provider stops generating","// tokens, and refunds this attempt's provider-specific reservation top-up.","// The top-up refund only runs if THIS call actually removed the pending request","// (RemovePending returned non-nil). If settlement (handleComplete) already","// claimed it via its own RemovePending, we must not also refund — that would","// double-credit the consumer.","func (s *Server) cancelDispatch(provider *registry.Provider, pr *registry.PendingRequest) {","\tif provider == nil || pr == nil {","\tremoved := provider.RemovePending(pr.RequestID)","\ts.registry.SetProviderIdle(provider.ID)","\ts.sendProviderCancel(provider, pr.RequestID)","\tif removed != nil {","\t\ts.refundProviderExtra(pr)","// refundProviderExtra refunds the provider-specific surcharge charged on top of","// the shared base reservation when an attempt is abandoned. It is idempotent:","// after refunding it resets ReservedMicroUSD to the base so a second call (or a","// later settlement) cannot double-refund. The shared base is never refunded","// here — that is handled once by refundReservation (full failure) or by the","// winning attempt's settlement.","func (s *Server) refundProviderExtra(pr *registry.PendingRequest) {","\tif pr == nil {","\textra := pr.ReservedMicroUSD - pr.BaseReservedMicroUSD","\tif extra <= 0 {","\t_ = s.store.Credit(pr.ConsumerKey, extra, store.LedgerRefund, \"reservation_extra_refund:\"+pr.RequestID)","\tpr.ReservedMicroUSD = pr.BaseReservedMicroUSD","\ts.ddIncr(\"billing.reservation_extra_refunds\", []string{\"model:\" + pr.Model})","// noteInferenceError feeds BOTH circuit breakers for a provider-side error","// received on a pending request's ErrorCh (any phase, pre- or post-commit):","// - the shape-keyed inference-error breaker (counts only sickness-shaped","// 500/502/504 for the (provider, model, shape) triple), and","// - the per-provider node-health breaker, which also counts fault-shaped","// 503s (errStr classifies capacity-503 vs fault-503).","// It emits the cool-down metric on the inference-error transition and the","// provider_breaker_open metric on the node-health transition into quarantine.","// errStr is the provider's error message (\"\" for synthetic timeouts).","func (s *Server) noteInferenceError(providerID string, pr *registry.PendingRequest, statusCode int, errStr string) {","\tif providerID == \"\" || pr == nil {","\tif s.registry.RecordInferenceError(providerID, pr.Model, statusCode, pr.Traits.CooldownShape()) {","\t\ts.ddIncr(\"routing.cooldown_entered\", []string{\"model:\" + pr.Model})","\t// Feed EVERY provider terminal into the per-provider node-health breaker (not","\t// just the shape-keyed 5xx the inference-error breaker counts) so a node","\t// fault-503ing ~all of its requests gets quarantined fleet-wide. errStr lets","\t// the breaker tell a capacity-503 (ignored) from a fault-503 (counted). Both","\t// breakers coexist.","\tif opened, _ := s.registry.RecordProviderOutcome(providerID, false, statusCode, errStr); opened {","\t\ts.ddIncr(\"routing.provider_breaker_open\", []string{\"model:\" + pr.Model})","// noteInferenceSuccess clears the inference-error strike state for the serving","// provider-model pair on a clean completion (streaming relay ended without a","// provider error; non-streaming response assembled OK).","func (s *Server) noteInferenceSuccess(pr *registry.PendingRequest) {","\tif pr == nil || pr.ProviderID == \"\" {","\ts.registry.RecordInferenceSuccess(pr.ProviderID, pr.Model, pr.Traits.CooldownShape())","\t// A clean completion proves the node is healthy — close its node-health","\t// breaker (and reset the exponential backoff) if it had tripped.","\tif _, closed := s.registry.RecordProviderOutcome(pr.ProviderID, true, 200, \"\"); closed {","\t\ts.ddIncr(\"routing.provider_breaker_closed\", []string{\"model:\" + pr.Model})","// noteDispatchProviderError records a provider error received while the","// dispatch loop had NOT yet committed to that provider: it feeds the","// inference-error breaker, refunds the failed attempt's provider-specific","// reservation top-up, and, when boilerplate chunks from that provider were","// being held (deferred commit), discards them and emits the pre-content","// failover counter — the invisible-retry signal that replaces what used to be","// an in-band SSE error after a premature commit. Returns true when held","// chunks were discarded so callers skip their generic retry counter.","// The refund lives here because both ErrorCh senders (handleInferenceError and","// registry.Disconnect's pending flush) remove the pending request BEFORE","// pushing the error, so the arm's cancelDispatch sees RemovePending()==nil and","// skips its own refund — without this the custom-price surcharge reserved by","// reserveAdditionalForProvider would be stranded for the failed attempt.","// refundProviderExtra is idempotent (it resets ReservedMicroUSD to the base),","// so arms where cancelDispatch did refund are safe, and a failed pre-commit","// attempt never reaches settlement (its channels are closed and it is neither","// pending nor parked), so this can never double-credit against a settle.","func (s *Server) noteDispatchProviderError(provider *registry.Provider, pr *registry.PendingRequest, statusCode int, errStr string, held *[]string) (discardedHeld bool) {","\tif provider != nil {","\t\ts.noteInferenceError(provider.ID, pr, statusCode, errStr)","\ts.refundProviderExtra(pr)","\tif held == nil || len(*held) == 0 {","\t*held = nil","\ts.ddIncr(\"inference.dispatches\", []string{\"status:retry_precontent\"})","// failedProviderVersion reads a provider's reported binary version under its","// lock (mirroring the policy.prefer owner reads). Captured when an attempt","// fails so the next attempt's Traits.AvoidVersion can steer the retry to a","// different build — a deterministic per-version bug must not burn every retry","// on identical binaries.","func failedProviderVersion(p *registry.Provider) string {","\tif p == nil {","\t\treturn \"\"","\tp.Mu().Lock()","\tdefer p.Mu().Unlock()","\treturn p.Version","// errModelTooLarge is the dispatch error returned when providers serve the","// requested model but none of them has enough total memory to ever load it.","// Distinct from \"no provider available\" so the caller rejects fast instead of","// queuing for 120s — queueing can't help a model that will never fit.","const errModelTooLarge = \"model too large for any available provider\"","// errTTFTTooSlow is the dispatch error returned when providers are available","// but all of them exceed the per-request TTFT ceiling. Distinct from","// \"no provider available\" so the caller returns a retryable 429 instead of","// queueing for a provider that would miss the OpenRouter SLA target.","const errTTFTTooSlow = \"all available providers exceed the TTFT target\"","// consumerModel returns the model name to echo back to the consumer: the public","// alias they requested when set, otherwise the concrete build id (raw-id","// requests and any internal caller that didn't populate PublicModel).","func consumerModel(pr *registry.PendingRequest) string {","\tif pr.PublicModel != \"\" {","\t\treturn pr.PublicModel","\treturn pr.Model","// rewriteChunkModel replaces the concrete build id in a streamed SSE chunk's","// \"model\" field with the public alias the consumer requested, so streaming","// responses never expose the underlying build/quant. No-op when the request","// used a raw build id (PublicModel == Model) or no alias was set. Uses a","// precise key+value string replace (both compact and spaced JSON forms) to","// avoid parsing every chunk on the hot path.","func rewriteChunkModel(chunk string, pr *registry.PendingRequest) string {","\tif pr.PublicModel == \"\" || pr.PublicModel == pr.Model {","\t\treturn chunk","\tchunk = strings.ReplaceAll(chunk, `\"model\":\"`+pr.Model+`\"`, `\"model\":\"`+pr.PublicModel+`\"`)","\tchunk = strings.ReplaceAll(chunk, `\"model\": \"`+pr.Model+`\"`, `\"model\": \"`+pr.PublicModel+`\"`)","\treturn chunk","// resolveRequestedModel maps the consumer-requested model — which may be a","// public alias like \"gemma-4-26b\" — to the concrete build id used for routing,","// billing, and serving, returning the public name to echo back to the consumer.","// When the request used an alias it rewrites parsed[\"model\"] and returns an","// updated rawBody so the provider receives the concrete build. Raw build ids","// pass through unchanged (publicModel == buildModel). ok=false means the alias","// currently has no usable build; the caller should surface a model_unavailable","// error.","func (s *Server) resolveRequestedModel(parsed map[string]any, rawBody []byte, requested string, allowedProviderSerials []string, policy selfRoutePolicy) (buildModel, publicModel string, newRawBody []byte, ok bool) {","\tbuildID, isAlias, resolved := s.registry.ResolveModelConstrained(","\t\trequested, allowedProviderSerials, policy.ownerAccountID, policy.enabled, policy.prefer)","\tif !resolved {","\t\treturn \"\", requested, rawBody, false","\tif !isAlias {","\t\treturn requested, requested, rawBody, true","\tparsed[\"model\"] = buildID","\trb, err := marshalForwardBody(parsed)","\t\trb = rawBody","\treturn buildID, requested, rb, true","// maybeFallbackAliasCapacity keeps public aliases available during a desired-build","// saturation event. Alias resolution intentionally prefers Desired when it is","// routable, but if every desired provider is transiently full and Previous has","// immediate capacity, route this request to Previous instead of returning a fast","// 429. Hard constraints and permanent model-too-large failures are handled by the","// caller and do not use this fallback. The TTFT estimate for Previous is also","// returned so the caller does not need to recompute it.","func (s *Server) maybeFallbackAliasCapacity(parsed map[string]any, publicModel, currentModel string, estimatedPromptTokens, requestedMaxTokens int, traits registry.RequestTraits, requiresVision bool, allowedProviderSerials []string) (string, int, int, int, time.Duration, bool, bool) {","\tif publicModel == \"\" || publicModel == currentModel {","\t\treturn currentModel, 0, 0, 0, 0, false, false","\ttarget, ok := s.registry.AliasTarget(publicModel)","\tif !ok || target.Desired != currentModel || target.Previous == \"\" {","\tif s.modelShed(target.Previous, publicModel) {","\tif !s.registry.IsModelInCatalog(target.Previous) {","\tcandidates, rejections, tooLarge, bestTTFT, hasTTFT := s.registry.QuickCapacityCheckWithTTFTForRequest(target.Previous, estimatedPromptTokens, requestedMaxTokens, traits, requiresVision, allowedProviderSerials...)","\tif candidates <= 0 {","\t\treturn currentModel, candidates, rejections, tooLarge, bestTTFT, hasTTFT, false","\tparsed[\"model\"] = target.Previous","\treturn target.Previous, candidates, rejections, tooLarge, bestTTFT, hasTTFT, true","func (s *Server) maybeFallbackAliasTTFT(parsed map[string]any, publicModel, currentModel string, estimatedPromptTokens, requestedMaxTokens int, ttftThreshold time.Duration, traits registry.RequestTraits, requiresVision bool, allowedProviderSerials []string) (string, int, int, int, time.Duration, bool, bool) {","\tif candidates <= 0 || ttftTooSlow(bestTTFT, hasTTFT, ttftThreshold) {","\t\treturn target.Previous, candidates, rejections, tooLarge, bestTTFT, hasTTFT, false","func ttftTooSlow(bestTTFT time.Duration, hasTTFT bool, threshold time.Duration) bool {","\treturn hasTTFT && bestTTFT > threshold","func fasterTTFTEstimate(primaryModel string, primary time.Duration, alternateModel string, alternate time.Duration, alternateOK bool) (string, time.Duration) {","\tif alternateOK && alternate < primary {","\t\treturn alternateModel, alternate","\treturn primaryModel, primary","func (s *Server) estimateTTFTRetryAfter(model string, bestTTFT, threshold time.Duration) int {","\toverage := bestTTFT - threshold","\tseconds := int(math.Ceil(overage.Seconds()))","\tif base := s.estimateRetryAfter(model); seconds < base {","\t\tseconds = base","\tif seconds < 2 {","\t\tseconds = 2","\tif seconds > 30 {","\t\tseconds = 30","\treturn seconds","func (s *Server) writeTTFTTooSlow(w http.ResponseWriter, model, publicModel string, bestTTFT, threshold time.Duration) {","\tretryAfter := s.estimateTTFTRetryAfter(model, bestTTFT, threshold)","\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:ttft_429\"})","\t\tfmt.Sprintf(\"all providers for model %q are above the %ds TTFT target (best estimate %.1fs); retry after %ds\", publicModel, int(math.Ceil(threshold.Seconds())), bestTTFT.Seconds(), retryAfter),","func (s *Server) triggerWarmPool() {","\tif s == nil || s.registry == nil {","\ts.registry.RequestWarmPoolTrigger()","func (s *Server) recordWarmPoolQueueState(model string) {","\tif s == nil || s.registry == nil || s.registry.Queue() == nil {","\tdepth, oldest := s.registry.Queue().QueueStats(model)","\tif depth <= 0 {","\t\ts.registry.RecordWarmPoolQueueCleared(model)","\ts.registry.RecordWarmPoolQueueEnqueued(model, depth, oldest)","\ts.triggerWarmPool()","// ttftMsForRejection converts a pre-flight TTFT estimate to milliseconds for the","// rejection ledger, returning 0 when the pre-flight produced no estimate.","func ttftMsForRejection(bestTTFT time.Duration, hasTTFT bool) float64 {","\tif !hasTTFT {","\t\treturn 0","\treturn float64(bestTTFT.Milliseconds())","// rejectionSamplingParams captures only the non-content sampling knobs already","// parsed from an inbound request body for the rejection ledger. It never","// includes prompt/message/input content. Returns nil when none are present.","func rejectionSamplingParams(parsed map[string]any) json.RawMessage {","\tif parsed == nil {","\t\treturn nil","\tknobs := make(map[string]any, 4)","\tfor _, k := range []string{\"temperature\", \"top_p\", \"presence_penalty\", \"frequency_penalty\"} {","\t\tif v, ok := parsed[k]; ok {","\t\t\tknobs[k] = v","\tif len(knobs) == 0 {","\tb, err := json.Marshal(knobs)","\treturn b","// dispatchOneProvider encrypts and sends an inference request to a single","// provider. It returns the pending request and provider on success, or an","// error string on failure. The excludeProviders set is updated on failure.","// selfRoutePolicy and its resolvers live in self_route.go.","type routeDecisionRecorder func(*registry.Provider, *registry.PendingRequest, registry.RoutingDecision)","func (s *Server) dispatchOneProvider(","\tr *http.Request,","\tmodel string,","\tpublicModel string,","\trawBody []byte,","\tconsumerKey string,","\tconsumerLocation *store.ProviderLocation,","\treservedMicroUSD int64,","\testimatedPromptTokens int,","\trequestedMaxTokens int,","\ttokenAdmission registry.TokenAdmission,","\trequiresVision bool,","\ttraits registry.RequestTraits,","\tallowedProviderSerials []string,","\tisResponsesAPI bool,","\tpolicy selfRoutePolicy,","\ttiming *registry.RequestTiming,","\tserviceReservation bool,","\tcacheAffinityKey string,","\texcludeProviders map[string]struct{},","\tattempt int,","\trecordRoute routeDecisionRecorder,",") (","\tprovider *registry.Provider,","\tpr *registry.PendingRequest,","\tdecision registry.RoutingDecision,","\tlastErr string,","\tlastErrCode int,",") {","\trequestID := uuid.New().String()","\tpr = ®istry.PendingRequest{","\t\tRequestID: requestID,","\t\t// Attempt is stamped at construction — BEFORE the request is encrypted","\t\t// and sent to the provider — so a fast provider that returns","\t\t// inference_complete immediately is correlated to the right route row.","\t\t// Setting it after the send (on the dispatch goroutine) would race the","\t\t// provider WS reader goroutine's handleComplete read of pr.Attempt.","\t\tAttempt: attempt,","\t\tModel: model,","\t\tPublicModel: publicModel,","\t\tConsumerKey: consumerKey,","\t\tKeyID: keyIDFromContext(r.Context()),","\t\tKeyLimitMicroUSD: keyLimitMicroFromContext(r.Context()),","\t\tKeyLimitReset: keyLimitResetFromContext(r.Context()),","\t\tConsumerLocation: consumerLocation,","\t\tIsResponsesAPI: isResponsesAPI,","\t\tEstimatedPromptTokens: estimatedPromptTokens,","\t\tRequiresVision: requiresVision,","\t\tTraits: traits,","\t\tRequestedMaxTokens: requestedMaxTokens,","\t\tTokenAdmission: tokenAdmission,","\t\tCacheAffinityKey: cacheAffinityKey,","\t\tReservedMicroUSD: reservedMicroUSD,","\t\tBaseReservedMicroUSD: reservedMicroUSD,","\t\tServiceReservation: serviceReservation,","\t\tAllowedProviderSerials: allowedProviderSerials,","\t\tSelfRouteOnly: policy.enabled,","\t\tPreferOwner: policy.prefer,","\t\tOwnerAccountID: policy.ownerAccountID,","\t\tFreeSelfRoute: policy.enabled,","\t\tAcceptedCh: make(chan struct{}, 1),","\t\tChunkCh: make(chan string, chunkBufferSize),","\t\tCompleteCh: make(chan protocol.UsageInfo, 1),","\t\tErrorCh: make(chan protocol.InferenceErrorMessage, 1),","\t\tTiming: timing,","\t// Public inference routes (not self-route / prefer-owner) enforce the","\t// OpenRouter TTFT ceiling inside the scheduler. This makes the preflight","\t// check authoritative: the router cannot select a provider whose estimated","\t// TTFT is above the threshold.","\t// Routing v2 (P1 fix): only enforce the TTFT ceiling inside the scheduler when","\t// the HARD gate is on. In soft mode (default) MaxTTFTMs stays 0 so the primary","\t// dispatch serves the best-available provider instead of re-rejecting an","\t// over-threshold request the preflight already chose to soft-serve. (Mirrors","\t// queueMaxTTFTMs, which already returns 0 in soft mode.)","\tif !policy.enabled && !policy.prefer && s.ttftHardReject {","\t\tpr.MaxTTFTMs = float64(ttftDeadline(estimatedPromptTokens).Milliseconds())","\t// Routing v2 W2: soft per-request decode floor (0 = off). Applies to all","\t// routes; it only ranks providers, never rejects.","\tpr.MinDecodeTPS = s.minDecodeTPS","\texcludeList := func() []string {","\t\tids := make([]string, 0, len(excludeProviders))","\t\tfor id := range excludeProviders {","\t\t\tids = append(ids, id)","\t\treturn ids","\tprovider, decision = s.registry.ReserveProviderEx(model, pr, excludeList()...)","\tif provider == nil {","\t\t// Providers serve this model but none can physically fit it: don't make","\t\t// the caller queue/retry for something that will never load.","\t\tif decision.CandidateCount == 0 && decision.CapacityRejections == 0 && decision.ModelTooLargeRejections > 0 {","\t\t\treturn nil, nil, decision, errModelTooLarge, http.StatusServiceUnavailable","\t\t// Providers are available but all exceed the TTFT ceiling. Fail fast","\t\t// with a retryable 429 rather than queueing or routing to a slow","\t\t// provider.","\t\tif decision.TTFTRejections > 0 {","\t\t\treturn nil, nil, decision, errTTFTTooSlow, http.StatusTooManyRequests","\t\treturn nil, nil, decision, \"no provider available\", http.StatusServiceUnavailable","\tpendingCleanup := true","\tcleanupPending := func() {","\t\tif pendingCleanup {","\t\t\tprovider.RemovePending(requestID)","\t\t\ts.registry.SetProviderIdle(provider.ID)","\t\t\tpendingCleanup = false","\tdefer cleanupPending()","\tif pr.Timing != nil {","\t\tpr.Timing.RoutedAt = time.Now()","\tif recordRoute != nil {","\t\trecordRoute(provider, pr, decision)","\t// A request settles FREE when it's served by a machine the caller owns:","\t// exclusive self-route (policy.enabled) always, OR a prefer request whose","\t// SELECTED provider is the caller's own machine (settlement refunds it to","\t// zero). In that case there is no payout and no reservation to top up — and","\t// applying a provider custom price above the platform rate would wrongly 429","\t// the free owned route, so skip both the payout warning and the top-up.","\tsettlesFree := policy.enabled","\tif !settlesFree && policy.prefer {","\t\tprovider.Mu().Lock()","\t\tsettlesFree = policy.ownerAccountID != \"\" && provider.AccountID == policy.ownerAccountID","\t\tprovider.Mu().Unlock()","\tif s.billing != nil && !settlesFree && !providerHasPayoutDestination(provider) {","\t\ts.logger.Warn(\"provider missing payout destination, crediting to internal ledger\",","\t\t\t\"provider_id\", provider.ID)","\t// Free (owned) requests are settled at zero cost (handleComplete), so there","\t// is no reservation to top up for a provider's custom price.","\tif s.billing != nil && !settlesFree {","\t\t_, err := s.reserveAdditionalForProvider(pr, provider)","\t\tif err != nil {","\t\t\tcleanupPending()","\t\t\texcludeProviders[provider.ID] = struct{}{}","\t\t\tif errors.Is(err, store.ErrInsufficientBalance) {","\t\t\t\treturn nil, nil, decision, \"insufficient funds for provider price\", http.StatusPaymentRequired","\t\t\t}","\t\t\ts.logger.Error(\"provider reservation failed (DB error)\", \"provider_id\", provider.ID, \"error\", err)","\t\t\treturn nil, nil, decision, \"service temporarily unavailable — please retry\", http.StatusServiceUnavailable","\t// refundExtra credits back the provider-specific surcharge that","\t// reserveAdditionalForProvider may have added. The caller's","\t// refundReservation only covers the base reservation.","\trefundExtra := func() {","\t\textra := pr.ReservedMicroUSD - reservedMicroUSD","\t\tif extra > 0 {","\t\t\tstart := time.Now()","\t\t\t_ = s.store.Credit(consumerKey, extra, store.LedgerRefund, \"reservation_extra_refund:\"+requestID)","\t\t\ts.ddIncr(\"billing.reservation_extra_refunds\", []string{\"model:\" + model})","\t\t\ts.ddHistogram(\"store.credit.latency_ms\", float64(time.Since(start).Milliseconds()), []string{\"op:reservation_extra_refund\"})","\t\t\tpr.ReservedMicroUSD = reservedMicroUSD","\t// E2E encryption","\tif provider.PublicKey == \"\" {","\t\trefundExtra()","\t\tcleanupPending()","\t\texcludeProviders[provider.ID] = struct{}{}","\t\treturn nil, nil, decision, \"no provider with E2E encryption\", http.StatusServiceUnavailable","\tproviderPubKey, err := e2e.ParsePublicKey(provider.PublicKey)","\t\treturn nil, nil, decision, \"provider public key invalid\", http.StatusServiceUnavailable","\tsessionKeys, err := e2e.GenerateSessionKeys()","\t\treturn nil, nil, decision, \"failed to generate session keys\", http.StatusInternalServerError","\t// Pre-fix providers crash on a vision request carrying sampling penalties;","\t// strip them for those providers only (see bodyForProvider).","\tsealedBody := bodyForProvider(rawBody, requiresVision, provider)","\tencrypted, err := e2e.Encrypt(sealedBody, providerPubKey, sessionKeys)","\t\treturn nil, nil, decision, \"failed to encrypt request\", http.StatusInternalServerError","\t\tpr.Timing.EncryptedAt = time.Now()","\twireMsg := map[string]any{","\t\t\"type\": protocol.TypeInferenceRequest,","\t\t\"request_id\": requestID,","\t\t\"encrypted_body\": map[string]string{","\t\t\t\"ephemeral_public_key\": encrypted.EphemeralPublicKey,","\t\t\t\"ciphertext\": encrypted.Ciphertext,","\tpr.SessionPrivKey = &sessionKeys.PrivateKey","\t// pr.ReservedMicroUSD was already set in the struct literal and may have","\t// been increased by reserveAdditionalForProvider above. Don't overwrite.","\tdata, err := json.Marshal(wireMsg)","\t\treturn nil, nil, decision, \"failed to marshal request\", http.StatusInternalServerError","\t\tpr.Timing.DispatchedAt = time.Now()","\tif err := writeProviderInferenceRequest(r.Context(), provider, data); err != nil {","\t\treturn nil, nil, decision, \"failed to send request to provider\", http.StatusBadGateway","\tpendingCleanup = false","\treturn provider, pr, decision, \"\", 0","func intFromRequestValue(v any) (int, bool) {","\tswitch x := v.(type) {","\tcase int:","\t\treturn x, true","\tcase int32:","\t\treturn int(x), true","\tcase int64:","\tcase float64:","\tcase json.Number:","\t\tn, err := x.Int64()","\t\t\treturn 0, false","\t\treturn int(n), true","\tdefault:","\t\treturn 0, false","// approximateTokenCount returns a rough token estimate for routing and queue","// admission. The len/4 heuristic is a reasonable average for English text","// with GPT-style BPE tokenizers. This value feeds into the scheduler's","// capacity checks (pendingTokenBudget, freeMemoryAdmits) where a tighter","// estimate produces better routing decisions.","// For billing reservation (where underestimation causes provider shortfall),","// use approximateTokenCountUpperBound instead.","func approximateTokenCount(v any) int {","\tif v == nil {","\tcase string:","\t\tif x == \"\" {","\t\t\treturn 0","\t\ttokens := len(x) / 4","\t\tif tokens < 1 {","\t\t\ttokens = 1","\t\treturn tokens","\t\tb, err := json.Marshal(v)","\t\ttokens := len(b) / 4","// approximateTokenCountUpperBound returns a guaranteed upper bound on the","// number of tokens a BPE tokenizer would produce for v. Every BPE vocabulary","// starts with one token per byte and can only merge, so len(text) >= tokens","// for any model family, any language, forever. This is used only for billing","// reservation to ensure the pre-flight debit always covers the actual cost.","// Using len(text) over-reserves by ~3-4x on average for English prose, but","// the difference is refunded immediately after inference completes, so","// consumers are never overcharged — they only need sufficient balance to","// cover the reservation hold.","func approximateTokenCountUpperBound(v any) int {","\t\treturn len(x)","\t\treturn len(b)","func estimatePromptTokens(parsed map[string]any) int {","\ttotal := 0","\tif v, ok := parsed[\"messages\"]; ok {","\t\ttotal += messagesPromptTokens(v)","\tif v, ok := parsed[\"input\"]; ok {","\t\ttotal += inputPromptTokens(v)","\tif v, ok := parsed[\"prompt\"]; ok {","\t\ttotal += approximateTokenCount(v)","\tif total == 0 {","\t\ttotal = approximateTokenCount(parsed)","\treturn total","// estimateBillingPromptTokens returns a guaranteed upper bound on prompt","// tokens for billing reservation. Uses byte-length (not len/4) so the","// pre-flight reservation always covers actual cost. This value must NOT","// be used for routing — see estimatePromptTokens for that.","func estimateBillingPromptTokens(parsed map[string]any) int {","\t\t// Billing MUST stay a guaranteed upper bound (len(bytes) >= tokens for any","\t\t// BPE tokenizer), so it keeps counting full message bytes — including a","\t\t// base64 image's bytes and every non-content field (role, tool_calls,","\t\t// name). Switching to the media-aware flat count here would DROP those","\t\t// fields and under-reserve for tool-calling requests. Over-reservation on a","\t\t// large image is safe (it is refunded after inference); the routing/ITPM","\t\t// estimate (estimatePromptTokens) is the media-aware one.","\t\ttotal += approximateTokenCountUpperBound(v)","\t\ttotal = approximateTokenCountUpperBound(parsed)","// Media prompt-token costs. A vision encoder turns each image/video into a","// bounded number of soft tokens (Gemma 4 caps around a few hundred per image)","// regardless of the base64 byte length, so counting a `data:` URI as text","// inflates the estimate by orders of magnitude — distorting routing admission and","// over-reserving balance. These flat per-media costs keep both sane.","\timagePromptTokenCost = 300","\tvideoPromptTokenCost = 1500","// isMediaPartType reports whether an OpenAI/OpenRouter content-part type denotes","// image or video input.","func isMediaPartType(t string) bool {","\tswitch t {","\t// OpenAI chat (image_url/video_url), OpenAI Responses (input_image/input_video),","\t// and Anthropic /v1/messages content blocks ({\"type\":\"image\"|\"video\",\"source\":…}).","\tcase \"image_url\", \"input_image\", \"image\", \"video_url\", \"input_video\", \"video\":","\t\treturn true","// messageContentTokens estimates ROUTING prompt tokens for one message's","// `content`, counting text parts as text (len/4) and each image/video part as a","// flat media cost (never the base64 length). Used only for the routing/ITPM","// estimate; billing uses approximateTokenCountUpperBound (a guaranteed upper","// bound that intentionally still counts the base64 bytes).","func messageContentTokens(content any) int {","\ttextTokens := func(s string) int {","\t\tif s == \"\" {","\t\tif t := len(s) / 4; t > 0 {","\t\t\treturn t","\t\treturn 1","\tswitch c := content.(type) {","\t\treturn textTokens(c)","\tcase []any:","\t\ttotal := 0","\t\tfor _, part := range c {","\t\t\tpm, ok := part.(map[string]any)","\t\t\tif !ok {","\t\t\t\tcontinue","\t\t\ttyp, _ := pm[\"type\"].(string)","\t\t\tswitch {","\t\t\tcase typ == \"text\" || typ == \"input_text\":","\t\t\t\tif s, ok := pm[\"text\"].(string); ok {","\t\t\t\t\ttotal += textTokens(s)","\t\t\t\t}","\t\t\tcase typ == \"image_url\" || typ == \"input_image\" || typ == \"image\":","\t\t\t\ttotal += imagePromptTokenCost","\t\t\tcase typ == \"video_url\" || typ == \"input_video\" || typ == \"video\":","\t\t\t\ttotal += videoPromptTokenCost","\t\t\tdefault:","\t\t\t\tif b, err := json.Marshal(pm); err == nil {","\t\t\t\t\ttotal += len(b) / 4","\t\treturn total","\t\treturn approximateTokenCount(content)","// messagesPromptTokens sums media-aware routing content tokens across a messages","// array. Falls back to the len/4 heuristic when messages isn't the standard","// array shape.","func messagesPromptTokens(messages any) int {","\tarr, ok := messages.([]any)","\t\treturn approximateTokenCount(messages)","\tfor _, m := range arr {","\t\tmm, ok := m.(map[string]any)","\t\tif !ok {","\t\t\ttotal += approximateTokenCount(m)","\t\ttotal += 4 // small per-message framing (role + delimiters)","\t\ttotal += messageContentTokens(mm[\"content\"])","// inputPromptTokens estimates the Responses API `input` field. A string input","// is plain text (len/4). Structured input is an array of message-like items with","// `content` parts, so reuse the same media-aware content estimator as chat","// messages instead of counting JSON wrapper bytes.","func inputPromptTokens(input any) int {","\tswitch x := input.(type) {","\t\treturn approximateTokenCount(x)","\t\tfor _, item := range x {","\t\t\tswitch m := item.(type) {","\t\t\tcase string:","\t\t\t\ttotal += approximateTokenCount(m)","\t\t\tcase map[string]any:","\t\t\t\tcontent, ok := m[\"content\"]","\t\t\t\tif !ok {","\t\t\t\t\ttotal += approximateTokenCount(m)","\t\t\t\t\tcontinue","\t\t\t\ttotal += 4 // role/type framing, matching messagesPromptTokens.","\t\t\t\ttotal += messageContentTokens(content)","\t\t\t\ttotal += approximateTokenCount(item)","\t\treturn approximateTokenCount(input)","// contentPartsHaveMedia reports whether a `content` value (a content-part array)","// carries any image/video part.","func contentPartsHaveMedia(content any) bool {","\tparts, ok := content.([]any)","\tfor _, part := range parts {","\t\tpm, ok := part.(map[string]any)","\t\tif typ, _ := pm[\"type\"].(string); isMediaPartType(typ) {","// detectMediaRequirement reports whether the request carries image/video input.","// The coordinator sees plaintext at this point (sealedTransport decrypts before","// the handler), so this drives the vision routing gate and the fail-fast \"no","// vision-capable provider\" response. It scans both the Chat Completions","// `messages[].content` parts and the Responses API `input[].content` parts so a","// media request on either surface is gated (never silently routed text-blind).","func detectMediaRequirement(parsed map[string]any) bool {","\tif messages, ok := parsed[\"messages\"].([]any); ok {","\t\tfor _, m := range messages {","\t\t\tif mm, ok := m.(map[string]any); ok && contentPartsHaveMedia(mm[\"content\"]) {","\t\t\t\treturn true","\t// Responses API: `input` may be a string (no media) or an array of items,","\t// each carrying `content` parts in the same image_url/input_image shape.","\tif input, ok := parsed[\"input\"].([]any); ok {","\t\tfor _, item := range input {","\t\t\tif im, ok := item.(map[string]any); ok && contentPartsHaveMedia(im[\"content\"]) {","// requestHasTools reports whether the request carries a non-empty top-level","// \"tools\" array (Chat Completions and Responses API share the field name).","// Drives Traits.HasTools so tool-bearing requests only route to providers whose","// binaries survive tool-schema template rendering (version floor + per-model","// template_render_ok gate in the scheduler).","func requestHasTools(parsed map[string]any) bool {","\ttools, ok := parsed[\"tools\"].([]any)","\treturn ok && len(tools) > 0","func requestCacheAffinityKey(parsed map[string]any) string {","\traw, ok := parsed[\"prompt_cache_key\"].(string)","\tif !ok || raw == \"\" {","\tconst maxPromptCacheKeyBytes = 512","\tif len(raw) > maxPromptCacheKeyBytes {","\tsum := sha256.Sum256([]byte(raw))","\treturn fmt.Sprintf(\"%x\", sum[:])","func estimateRequestedMaxTokens(parsed map[string]any) int {","\tfor _, key := range []string{\"max_tokens\", \"max_completion_tokens\", \"max_output_tokens\"} {","\t\tif n, ok := intFromRequestValue(parsed[key]); ok && n > 0 {","\t\t\tif copies, ok := intFromRequestValue(parsed[\"n\"]); ok && copies > 1 {","\t\t\t\treturn n * copies","\t\t\treturn n","\tif copies, ok := intFromRequestValue(parsed[\"n\"]); ok && copies > 1 {","\t\treturn 256 * copies","\treturn 256","func parseProviderSerialAllowlist(parsed map[string]any) ([]string, bool, error) {","\tvar rawValues []any","\tprovided := false","\tfor _, key := range []string{\"provider_serial\", \"provider_serials\"} {","\t\tv, ok := parsed[key]","\t\tprovided = true","\t\tswitch x := v.(type) {","\t\tcase string:","\t\t\trawValues = append(rawValues, x)","\t\tcase []any:","\t\t\trawValues = append(rawValues, x...)","\t\tdefault:","\t\t\treturn nil, true, fmt.Errorf(\"%s must be a string or array of strings\", key)","\tif !provided {","\t\treturn nil, false, nil","\tseen := make(map[string]struct{}, len(rawValues))","\tids := make([]string, 0, len(rawValues))","\tfor _, raw := range rawValues {","\t\tid, ok := raw.(string)","\t\t\treturn nil, true, fmt.Errorf(\"provider_serials must contain only strings\")","\t\tid = strings.TrimSpace(id)","\t\tif id == \"\" {","\t\tif _, exists := seen[id]; exists {","\t\tseen[id] = struct{}{}","\t\tids = append(ids, id)","\tif len(ids) == 0 {","\t\treturn nil, true, fmt.Errorf(\"provider_serials must include at least one provider serial\")","\treturn ids, true, nil","func stripProviderRoutingFields(parsed map[string]any) bool {","\tchanged := false","\t\tif _, ok := parsed[key]; ok {","\t\t\tdelete(parsed, key)","\t\t\tchanged = true","\treturn changed","// penaltySafeProviderVersion is the first provider release whose VLM penalty","// path handles repetition/presence/frequency penalties without crashing (the","// TokenRing 2D-prompt fix). Providers below it crash on a vision request that","// carries any of these fields, so the coordinator strips them before sealing","// for such a provider. Keep in sync with the release that ships the fix.","const penaltySafeProviderVersion = \"0.6.7\"","// visionPenaltyFields crash the pre-fix VLM penalty path on image requests.","var visionPenaltyFields = []string{\"repetition_penalty\", \"presence_penalty\", \"frequency_penalty\"}","// bodyForProvider returns the request body to seal for `provider`. It equals","// rawBody, except a vision request routed to a pre-fix provider has the","// crash-inducing penalty fields stripped. Fixed providers receive the penalties","// unchanged. Per-provider (not pre-routing) so a retry on a fixed provider keeps","// them. Remove once MIN_PROVIDER_VERSION clears all pre-fix builds.","func bodyForProvider(rawBody []byte, requiresVision bool, provider *registry.Provider) []byte {","\tif !requiresVision {","\t\treturn rawBody","\tif provider.Version != \"\" && !semverLess(provider.Version, penaltySafeProviderVersion) {","\t\treturn rawBody // fixed provider — pass penalties through","\tvar parsed map[string]any","\tif json.Unmarshal(rawBody, &parsed) != nil {","\tfor _, key := range visionPenaltyFields {","\tif !changed {","\tif stripped, err := marshalForwardBody(parsed); err == nil {","\t\treturn stripped","\treturn rawBody","// defaultMaxOutputTokens is the ceiling injected into requests that don't set","// max_tokens. It bounds the worst-case cost of a single inference so the","// pre-flight balance reservation covers the entire generation; without this","// cap a consumer could stream output exceeding their reservation and the","// post-inference charge would fail silently (see GitHub issue #33). Consumers","// who need longer generations must set max_tokens explicitly and carry the","// balance to cover it.","const defaultMaxOutputTokens = 8192","// explicitMaxTokens returns the consumer-specified max output tokens from any","// of the recognized field names, or 0 if none were set.","func explicitMaxTokens(parsed map[string]any) int {","\treturn 0","// reservationCost is the pre-flight worst-case cost for a text inference","// request. It mirrors the platform-price branch of handleComplete's billing","// so the reservation covers any platform-level custom price for the model;","// without this, a platform override above the built-in default would leave","// the reservation short and the post-inference clamp would silently","// undercharge. Provider-specific custom prices are not known until dispatch","// commits to a provider, so a provider that sets a custom price above the","// platform rate accepts revenue capped at the reservation.","func (s *Server) reservationCost(model string, promptTokens, maxTokens int) int64 {","\tcustomIn, customOut, hasCustom := s.store.GetModelPrice(\"platform\", model)","\treturn payments.CalculateCostWithOverrides(model, promptTokens, maxTokens, customIn, customOut, hasCustom)","func (s *Server) refundReservedBalance(pr *registry.PendingRequest, reference string) bool {","\tif pr == nil || pr.ReservedMicroUSD <= 0 {","\tif reference == \"\" {","\t\treference = \"reservation_refund:\" + pr.RequestID","\tstart := time.Now()","\tfinalized, err := pr.FinalizeReservation(func() error {","\t\tif pr.ServiceReservation {","\t\t\ts.releaseServiceReservation(pr, \"refund\")","\t\t\treturn nil","\t\treturn s.store.Credit(pr.ConsumerKey, pr.ReservedMicroUSD, store.LedgerRefund, reference)","\t\ts.logger.Error(\"failed to refund reservation\",","\t\t\t\"request_id\", pr.RequestID,","\t\t\t\"consumer_key\", pr.ConsumerKey,","\t\t\t\"reserved_micro_usd\", pr.ReservedMicroUSD,","\t\t\t\"error\", err,","\t\t)","\tif !finalized {","\ttags := []string{\"model:\" + pr.Model, \"mode:\" + reservationMetricMode(pr.ServiceReservation)}","\ts.ddIncr(\"billing.reservation_refunds\", tags)","\tif !pr.ServiceReservation {","\t\ts.ddIncr(\"billing.reservation_releases\", append(tags, \"reason:refund\"))","\t\ts.ddHistogram(\"store.credit.latency_ms\", float64(time.Since(start).Milliseconds()), []string{\"op:reservation_refund\"})","// estimateRetryAfter returns a suggested wait time in seconds before retrying","// a request for the given model. Based on queue depth as a rough proxy for","// fleet backlog. OpenRouter uses the Retry-After header to schedule retries.","func (s *Server) estimateRetryAfter(model string) int {","\tqueueDepth := s.registry.Queue().QueueSize(model)","\tif queueDepth == 0 {","\t\treturn 2 // Light load, retry soon","\t// Rough estimate: each queued request takes ~3 seconds to drain.","\testimate := queueDepth * 3","\tif estimate < 2 {","\t\testimate = 2","\tif estimate > 30 {","\t\testimate = 30","\treturn estimate","// writeServiceUnavailable writes a retryable 503 with a Retry-After header so","// clients (and OpenRouter) can schedule the retry instead of blind backoff.","func (s *Server) writeServiceUnavailable(w http.ResponseWriter, model string) {","\tw.Header().Set(\"Retry-After\", strconv.Itoa(s.estimateRetryAfter(model)))","\twriteJSON(w, http.StatusServiceUnavailable, errorResponse(\"service_unavailable\",","\t\t\"service temporarily unavailable — please retry\"))","func providerHasPayoutDestination(provider *registry.Provider) bool {","\tprovider.Mu().Lock()","\tdefer provider.Mu().Unlock()","\treturn provider.AccountID != \"\"","func providerPricingKeys(provider *registry.Provider) string {","\treturn provider.AccountID","func (s *Server) providerReservationCost(provider *registry.Provider, model string, promptTokens, maxTokens int) int64 {","\taccountID := providerPricingKeys(provider)","\tif accountID != \"\" {","\t\tcustomIn, customOut, hasCustom := s.store.GetModelPrice(accountID, model)","\t\tif hasCustom {","\t\t\treturn payments.CalculateCostWithOverrides(model, promptTokens, maxTokens, customIn, customOut, true)","\treturn s.reservationCost(model, promptTokens, maxTokens)","// isServiceConsumer reports whether the account is a service/wholesale account","// (e.g. OpenRouter). Such accounts are billed at the advertised platform price,","// so the provider-price reservation top-up and provider custom pricing are","// skipped for them. A failed lookup falls back to false (normal consumer).","func (s *Server) isServiceConsumer(accountID string) bool {","\tif accountID == \"\" {","\tif u, err := s.store.GetUserByAccountID(accountID); err == nil && u != nil {","\t\treturn u.Role == store.RoleService","func (s *Server) reserveAdditionalForProvider(pr *registry.PendingRequest, provider *registry.Provider) (int64, error) {","\t\treturn 0, fmt.Errorf(\"pending request is required\")","\t// Service/wholesale consumers are billed at the platform price at","\t// settlement, so don't top the reservation up to a provider's higher custom","\t// price — the base platform reservation already covers the actual charge.","\tif s.isServiceConsumer(pr.ConsumerKey) {","\t\treturn pr.ReservedMicroUSD, nil","\trequired := s.providerReservationCost(provider, pr.Model, pr.EstimatedPromptTokens, pr.RequestedMaxTokens)","\tif required <= pr.ReservedMicroUSD {","\t// Per-key spend cap re-check against the provider-specific total: the","\t// initial cap check only saw the platform reservation, so a provider whose","\t// custom price exceeds it could otherwise push a capped key over its limit","\t// in a single request. Treat a cap breach like insufficient funds so the","\t// caller excludes this provider (a cheaper one may still fit) and, if none","\t// fit, the request fails with 402. Checked BEFORE charging the top-up.","\tif pr.KeyID != \"\" && pr.KeyLimitMicroUSD != nil {","\t\tsince := store.KeySpendWindowStart(pr.KeyLimitReset, time.Now())","\t\tif s.store.KeySpendSince(pr.KeyID, since)+required > *pr.KeyLimitMicroUSD {","\t\t\treturn pr.ReservedMicroUSD, store.ErrInsufficientBalance","\textra := required - pr.ReservedMicroUSD","\tif err := s.ledger.Charge(pr.ConsumerKey, extra, \"reserve:\"+pr.ConsumerKey); err != nil {","\t\treturn pr.ReservedMicroUSD, err","\tpr.ReservedMicroUSD = required","\ts.ddHistogram(\"billing.reserved_micro_usd\", float64(required), []string{\"model:\" + pr.Model})","\treturn required, nil","// ensureMaxTokensBound injects a max-tokens bound into parsed when the","// consumer didn't specify any max-tokens field, so the outgoing request to","// the provider is bounded by the amount we reserve upfront. The bound is","// the model's max_output_length from the registry (or defaultMaxOutputTokens","// as fallback). The injected field name depends on the API flavor: Responses","// API uses max_output_tokens, everything else uses max_tokens. Returns true","// when an injection occurred, so the caller can re-marshal the outgoing body","// if needed.","func ensureMaxTokensBound(parsed map[string]any, isResponsesAPI bool, bound int) bool {","\tif n := explicitMaxTokens(parsed); n > 0 {","\t\t// Normalize alias fields the provider engine doesn't read: a chat","\t\t// request bounded only via max_completion_tokens (the OpenAI-preferred","\t\t// spelling) must still reach the provider as max_tokens, or the bound","\t\t// is silently ignored.","\t\tif !isResponsesAPI {","\t\t\tif cur, ok := intFromRequestValue(parsed[\"max_tokens\"]); !ok || cur <= 0 {","\t\t\t\tparsed[\"max_tokens\"] = n","\tif isResponsesAPI {","\t\tparsed[\"max_output_tokens\"] = bound","\t} else {","\t\tparsed[\"max_tokens\"] = bound","func copyJSONMap(in map[string]any) map[string]any {","\tout := make(map[string]any, len(in))","\tfor k, v := range in {","\t\tout[k] = v","func responsesContentText(content any) string {","\tcase nil:","\t\treturn c","\t\tparts := make([]string, 0, len(c))","\t\t\tswitch p := part.(type) {","\t\t\t\tif p != \"\" {","\t\t\t\t\tparts = append(parts, p)","\t\t\t\tif text, _ := p[\"text\"].(string); text != \"\" {","\t\t\t\t\tparts = append(parts, text)","\t\t\t\tif p[\"type\"] == \"input_image\" || p[\"type\"] == \"input_file\" {","\t\t\t\t\t// Text models cannot consume binary Responses parts yet.","\t\t\t\t\t// Preserve the turn shape without leaking URLs or blobs.","\t\t\t\t\tparts = append(parts, fmt.Sprintf(\"[%s omitted]\", p[\"type\"]))","\t\treturn strings.Join(parts, \"\\n\")","\t\tb, err := json.Marshal(c)","\t\t\treturn fmt.Sprint(c)","\t\treturn string(b)","func responsesInputToChatMessages(input any) ([]map[string]any, error) {","\tswitch v := input.(type) {","\t\treturn []map[string]any{{\"role\": \"user\", \"content\": v}}, nil","\t\tmessages := make([]map[string]any, 0, len(v))","\t\tfor _, raw := range v {","\t\t\titem, ok := raw.(map[string]any)","\t\t\tif typ, _ := item[\"type\"].(string); typ != \"\" {","\t\t\t\tswitch typ {","\t\t\t\tcase \"message\":","\t\t\t\t\trole, _ := item[\"role\"].(string)","\t\t\t\t\tif role == \"\" {","\t\t\t\t\t\trole = \"user\"","\t\t\t\t\t}","\t\t\t\t\tif role == \"developer\" {","\t\t\t\t\t\trole = \"system\"","\t\t\t\t\tmessages = append(messages, map[string]any{","\t\t\t\t\t\t\"role\": role,","\t\t\t\t\t\t\"content\": responsesContentText(item[\"content\"]),","\t\t\t\t\t})","\t\t\t\tcase \"function_call\":","\t\t\t\t\tcallID, _ := item[\"call_id\"].(string)","\t\t\t\t\tif callID == \"\" {","\t\t\t\t\t\tcallID, _ = item[\"id\"].(string)","\t\t\t\t\tname, _ := item[\"name\"].(string)","\t\t\t\t\targs, _ := item[\"arguments\"].(string)","\t\t\t\t\t\t\"role\": \"assistant\",","\t\t\t\t\t\t\"content\": \"\",","\t\t\t\t\t\t\"tool_calls\": []map[string]any{{","\t\t\t\t\t\t\t\"id\": callID,","\t\t\t\t\t\t\t\"type\": \"function\",","\t\t\t\t\t\t\t\"function\": map[string]any{","\t\t\t\t\t\t\t\t\"name\": name,","\t\t\t\t\t\t\t\t\"arguments\": args,","\t\t\t\t\t\t\t},","\t\t\t\t\t\t}},","\t\t\t\tcase \"function_call_output\":","\t\t\t\t\t\t\"role\": \"tool\",","\t\t\t\t\t\t\"tool_call_id\": callID,","\t\t\t\t\t\t\"content\": responsesContentText(item[\"output\"]),","\t\t\t\tcase \"reasoning\":","\t\t\t\t\t// Reasoning items are model-side metadata, not prompt text.","\t\t\t\tdefault:","\t\t\t\t\treturn nil, fmt.Errorf(\"unsupported Responses input item type %q\", typ)","\t\t\trole, _ := item[\"role\"].(string)","\t\t\tif role == \"\" {","\t\t\tif role == \"developer\" {","\t\t\t\trole = \"system\"","\t\t\tmessages = append(messages, map[string]any{","\t\t\t\t\"role\": role,","\t\t\t\t\"content\": responsesContentText(item[\"content\"]),","\t\t\t})","\t\tif len(messages) == 0 {","\t\t\treturn nil, fmt.Errorf(\"Responses input did not contain any chat-compatible messages\")","\t\treturn messages, nil","\t\treturn nil, fmt.Errorf(\"Responses input must be a string or array\")","func responsesToolsToChatTools(raw any) ([]any, error) {","\ttools, ok := raw.([]any)","\tif !ok || len(tools) == 0 {","\t\treturn nil, nil","\tout := make([]any, 0, len(tools))","\tfor _, rawTool := range tools {","\t\ttool, ok := rawTool.(map[string]any)","\t\ttyp, _ := tool[\"type\"].(string)","\t\tif typ == \"\" || typ == \"function\" {","\t\t\tname, _ := tool[\"name\"].(string)","\t\t\tif name == \"\" {","\t\t\t\tif fn, _ := tool[\"function\"].(map[string]any); fn != nil {","\t\t\t\t\tout = append(out, tool)","\t\t\t\treturn nil, fmt.Errorf(\"function tool is missing name\")","\t\t\tfn := map[string]any{\"name\": name}","\t\t\tif description, ok := tool[\"description\"].(string); ok {","\t\t\t\tfn[\"description\"] = description","\t\t\tif parameters, ok := tool[\"parameters\"]; ok {","\t\t\t\tfn[\"parameters\"] = parameters","\t\t\tout = append(out, map[string]any{","\t\t\t\t\"type\": \"function\",","\t\t\t\t\"function\": fn,","\t\treturn nil, fmt.Errorf(\"unsupported Responses tool type %q\", typ)","\treturn out, nil","func responsesToolChoiceToChat(raw any) (any, error) {","\tchoice, ok := raw.(map[string]any)","\t\treturn raw, nil","\ttyp, _ := choice[\"type\"].(string)","\tif typ == \"function\" {","\t\tname, _ := choice[\"name\"].(string)","\t\tif name == \"\" {","\t\t\treturn nil, fmt.Errorf(\"function tool_choice is missing name\")","\t\treturn map[string]any{","\t\t\t\"type\": \"function\",","\t\t\t\"function\": map[string]any{","\t\t\t\t\"name\": name,","\t\t\t},","\t\t}, nil","\treturn raw, nil","func responsesTextFormatToChatResponseFormat(raw any) any {","\ttext, ok := raw.(map[string]any)","\tformat, ok := text[\"format\"].(map[string]any)","\tswitch format[\"type\"] {","\tcase \"json_object\":","\t\treturn map[string]any{\"type\": \"json_object\"}","\tcase \"json_schema\":","\t\t\t\"type\": \"json_schema\",","\t\t\t\"json_schema\": format,","\treturn nil","func responsesRequestToChatCompletions(parsed map[string]any) (map[string]any, error) {","\tmessages, err := responsesInputToChatMessages(parsed[\"input\"])","\t\treturn nil, err","\tout := copyJSONMap(parsed)","\tdelete(out, \"input\")","\tdelete(out, \"endpoint\")","\tdelete(out, \"max_output_tokens\")","\tdelete(out, \"text\")","\tout[\"messages\"] = messages","\tif maxTokens := explicitMaxTokens(parsed); maxTokens > 0 {","\t\tout[\"max_tokens\"] = maxTokens","\tif tools, err := responsesToolsToChatTools(parsed[\"tools\"]); err != nil {","\t} else if len(tools) > 0 {","\t\tout[\"tools\"] = tools","\tif choice, err := responsesToolChoiceToChat(parsed[\"tool_choice\"]); err != nil {","\t} else if choice != nil {","\t\tout[\"tool_choice\"] = choice","\tif responseFormat := responsesTextFormatToChatResponseFormat(parsed[\"text\"]); responseFormat != nil {","\t\tout[\"response_format\"] = responseFormat","// handleChatCompletions handles POST /v1/chat/completions.","// This is the main inference endpoint. It validates the request, finds an","// available provider for the requested model, forwards the request via","// WebSocket, and either streams SSE chunks or assembles a complete response.","// The raw request body is passed through to the provider, preserving all","// OpenAI-compatible fields (tools, tool_choice, response_format, top_p, etc.)","// that would otherwise be lost if we parsed into a typed struct.","func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {","\ttiming := ®istry.RequestTiming{ReceivedAt: time.Now()}","\t// Shared prelude: read body, normalize tool schemas, parse, require a model,","\t// enforce the per-key model allowlist. (See parseInferencePrelude.)","\tprelude, ok := s.parseInferencePrelude(w, r)","\trawBody := prelude.rawBody","\tparsed := prelude.parsed","\tmodel := prelude.model","\t// Accept either chat completions format (messages) or Responses API","\t// format (input). The provider's backend handles both natively.","\tmessages, _ := parsed[\"messages\"].([]any)","\tinput := parsed[\"input\"]","\tif len(messages) == 0 && input == nil {","\t\ts.recordRejection(rejectionInfo{","\t\t\tr: r,","\t\t\tstage: \"validation\",","\t\t\treasonCode: \"messages_required\",","\t\t\thttpStatus: http.StatusBadRequest,","\t\t\tkeyID: keyIDFromContext(r.Context()),","\t\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\t\trequestedModel: model,","\t\t\tparams: rejectionSamplingParams(parsed),","\t\t})","\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"invalid_request_error\", \"messages or input is required\"))","\t// Multiple choices per request are not supported — fail loudly instead of","\t// silently returning a single choice the consumer didn't ask for.","\t\t\treasonCode: \"bad_param\",","\t\t\tn: copies,","\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"invalid_request_error\",","\t\t\t\"n > 1 is not supported\", withParam(\"n\")))","\tallowedProviderSerials, hasProviderAllowlist, err := parseProviderSerialAllowlist(parsed)","\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"invalid_request_error\", err.Error()))","\tif hasProviderAllowlist && stripProviderRoutingFields(parsed) {","\t\trawBody, _ = marshalForwardBody(parsed)","\t// \"Use my own machine, for free\" opt-in. The signal is the","\t// X-Darkbloom-Route header (OpenAI-client-safe: invisible to the body","\t// schema) OR a per-key hard ceiling. The header can only *request*","\t// self-routing; it cannot name a machine — ownership is matched on the","\t// coordinator-stamped provider AccountID, so nothing here is forgeable.","\tpolicy := s.resolveSelfRoutePolicy(r)","\t// Resolve a public alias (e.g. \"gemma-4-26b\") to a concrete build id, now","\t// that routing constraints (serial allowlist / self-route) are known so the","\t// pick only considers builds the constrained provider set can actually","\t// serve. From here on `model` is the build (routing/billing/serving) while","\t// `publicModel` is echoed back so the consumer never sees the quant.","\tbuildModel, publicModel, resolvedBody, ok := s.resolveRequestedModel(","\t\tparsed, rawBody, model, allowedProviderSerials, policy)","\t\t\tstage: \"model_resolution\",","\t\t\treasonCode: \"model_unavailable\",","\t\t\thttpStatus: http.StatusServiceUnavailable,","\t\twriteJSON(w, http.StatusServiceUnavailable, errorResponse(\"model_unavailable\",","\t\t\tfmt.Sprintf(\"model %q has no available build right now\", model), withParam(\"model\")))","\tmodel, rawBody = buildModel, resolvedBody","\t// Vision gating: a request carrying image/video input must land on a provider","\t// advertising a vision-capable (VLM) build of this model, or the media is","\t// silently dropped and the answer is image-blind. Fail fast with a clear error","\t// when the fleet has no such provider (e.g. before the gemma fleet finishes","\t// updating to 0.6.0); the routing layer enforces the same gate per dispatch.","\trequiresVision := detectMediaRequirement(parsed)","\t// Tool-bearing requests are routed only to providers that can render tool","\t// schemas without crashing (version floor + template_render_ok gate);","\t// detected here, alongside the media gate, while the parsed body is hot.","\thasTools := requestHasTools(parsed)","\tcacheAffinityKey := requestCacheAffinityKey(parsed)","\t// Shared media/tools fail-fast. Chat completions additionally rejects media","\t// sent via the Responses API surface (input-without-messages), because the","\t// Responses→chat lowering doesn't carry image/video parts through.","\tif s.visionToolsFailFast(w, model, publicModel, requiresVision, hasTools,","\t\tinput != nil && len(messages) == 0, policy, allowedProviderSerials) {","\tisResponsesAPI := input != nil && len(messages) == 0","\t// Inject model-specific defaults from the registry: reasoning_parser","\t// and max_tokens bound. Single DB lookup (cached for platform prices).","\tmaxOutputBound := defaultMaxOutputTokens","\t// modelMaxContext is the model's max context window (0 = unknown), used by the","\t// servability gate. Lifted out of the record block so it is in scope at the","\t// preflight below.","\tmodelMaxContext := 0","\tif rec, err := s.store.GetModelRegistryRecord(model); err == nil {","\t\t// Reasoning parser from runtime_parameters.","\t\tif _, hasRP := parsed[\"reasoning_parser\"]; !hasRP && rec.RuntimeParameters != nil {","\t\t\tif rp, ok := rec.RuntimeParameters[\"reasoning_parser\"]; ok {","\t\t\t\tparsed[\"reasoning_parser\"] = rp","\t\t\t\trawBody, _ = marshalForwardBody(parsed)","\t\t// Use the registry's max_output_length as the default max_tokens","\t\t// bound instead of the hardcoded 8192. This lets models like","\t\t// GPT-OSS 20B (32K output) generate longer responses when the","\t\t// consumer omits max_tokens.","\t\tif rec.MaxOutputLength > 0 {","\t\t\tmaxOutputBound = rec.MaxOutputLength","\t\tmodelMaxContext = rec.MaxContextLength","\t// Bound the generation so the pre-flight reservation covers it. If the","\t// consumer didn't set max_tokens, inject the model's max_output_length","\t// (or defaultMaxOutputTokens as fallback). Without this bound the","\t// provider could return more tokens than we reserved for, and the","\t// silent post-inference charge failure would hand the consumer free","\t// inference (GitHub issue #33).","\tif ensureMaxTokensBound(parsed, isResponsesAPI, maxOutputBound) {","\tstream, _ := parsed[\"stream\"].(bool)","\testimatedPromptTokens := estimatePromptTokens(parsed)","\tbillingPromptTokens := estimateBillingPromptTokens(parsed)","\trequestedMaxTokens := estimateRequestedMaxTokens(parsed)","\tdeadline := ttftDeadline(estimatedPromptTokens)","\ttiming.ParsedAt = time.Now()","\tif s.shedIfModelRejected(w, r, parsed, policy, publicModel, model, stream, estimatedPromptTokens, requestedMaxTokens, requiresVision, hasTools) {","\t\tproviderParsed, err := responsesRequestToChatCompletions(parsed)","\t\t\ts.recordRejection(rejectionInfo{","\t\t\t\tr: r,","\t\t\t\tstage: \"validation\",","\t\t\t\treasonCode: \"bad_param\",","\t\t\t\thttpStatus: http.StatusBadRequest,","\t\t\t\tkeyID: keyIDFromContext(r.Context()),","\t\t\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\t\t\trequestedModel: publicModel,","\t\t\t\tresolvedModel: model,","\t\t\t\tstream: stream,","\t\t\t\testimatedPromptTokens: estimatedPromptTokens,","\t\t\t\trequestedMaxTokens: requestedMaxTokens,","\t\t\t\trequiresVision: requiresVision,","\t\t\t\thasTools: hasTools,","\t\t\t\tparams: rejectionSamplingParams(parsed),","\t\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"invalid_request_error\", err.Error()))","\t\t\treturn","\t\trawBody, _ = marshalForwardBody(providerParsed)","\t// Per-account token rate limiting (ITPM/OTPM) — the industry-standard","\t// token throttle alongside RPM. Charged upfront from the input estimate","\t// and the bounded max_tokens (OpenAI-style). Runs before the balance","\t// reservation so a throttled request never touches billing.","\ttokenAdmission, ok := s.applyTokenRateLimitWithAdmission(w, r, estimatedPromptTokens, requestedMaxTokens)","\t// Pre-flight balance reservation — atomically debit the worst-case cost","\t// using the byte-length upper bound for prompt tokens (guaranteed >=","\t// actual tokens for any BPE tokenizer) plus max_tokens we just bounded","\t// the generation to. The post-inference charge refunds any unused","\t// portion. The routing estimate (estimatedPromptTokens, len/4) is kept","\t// separate so scheduler capacity checks aren't over-inflated.","\tvar reservedMicroUSD int64","\tserviceReservation := false","\t// Self-route is free: skip the pre-flight balance reservation and the","\t// per-key spend cap entirely. A zero-balance owner must never be blocked","\t// from running on their own machine, and a self_route_only key never spends.","\tif s.billing != nil && !policy.enabled {","\t\tconsumerKey := consumerKeyFromContext(r.Context())","\t\treservedMicroUSD = s.reservationCost(model, billingPromptTokens, requestedMaxTokens)","\t\t// Per-key spend cap (phase 1) — checked before the reservation so a","\t\t// capped key never debits the account ledger.","\t\tif msg, ok := s.checkKeySpendCap(r.Context(), reservedMicroUSD); !ok {","\t\t\t\tstage: \"balance\",","\t\t\t\treasonCode: \"insufficient_quota\",","\t\t\t\thttpStatus: http.StatusPaymentRequired,","\t\t\twriteJSON(w, http.StatusPaymentRequired, errorResponse(\"insufficient_quota\", msg, withCode(\"insufficient_quota\")))","\t\tvar err error","\t\tserviceReservation, err = s.reserveInitialBalance(consumerKey, model, reservedMicroUSD)","\t\t\t\ts.recordRejection(rejectionInfo{","\t\t\t\t\tr: r,","\t\t\t\t\tstage: \"balance\",","\t\t\t\t\treasonCode: \"insufficient_funds\",","\t\t\t\t\thttpStatus: http.StatusPaymentRequired,","\t\t\t\t\tkeyID: keyIDFromContext(r.Context()),","\t\t\t\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\t\t\t\trequestedModel: publicModel,","\t\t\t\t\tresolvedModel: model,","\t\t\t\t\tstream: stream,","\t\t\t\t\testimatedPromptTokens: estimatedPromptTokens,","\t\t\t\t\trequestedMaxTokens: requestedMaxTokens,","\t\t\t\t\trequiresVision: requiresVision,","\t\t\t\t\thasTools: hasTools,","\t\t\t\t\tparams: rejectionSamplingParams(parsed),","\t\t\t\t})","\t\t\t\twriteJSON(w, http.StatusPaymentRequired, errorResponse(\"insufficient_funds\",","\t\t\t\t\t\"your balance is too low for this request — add funds at /billing or lower max_tokens\", withCode(\"insufficient_quota\")))","\t\t\t} else {","\t\t\t\ts.logger.Error(\"balance reservation failed (DB error)\", \"consumer_key\", consumerKey, \"error\", err)","\t\t\t\ts.writeServiceUnavailable(w, model)","\ttiming.ReservedAt = time.Now()","\t// Refund reservation on early errors (before inference starts).","\trefundReservation := func() {","\t\tif reservedMicroUSD > 0 {","\t\t\ts.releaseInitialReservation(consumerKeyFromContext(r.Context()), model, reservedMicroUSD, serviceReservation)","\t// Reject requests for models not in the catalog.","\tif !s.registry.IsModelInCatalog(model) {","\t\trefundReservation()","\t\t\tr: r,","\t\t\tstage: \"model_resolution\",","\t\t\treasonCode: \"model_not_found\",","\t\t\thttpStatus: http.StatusNotFound,","\t\t\tkeyID: keyIDFromContext(r.Context()),","\t\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\t\trequestedModel: publicModel,","\t\t\tresolvedModel: model,","\t\t\tstream: stream,","\t\t\testimatedPromptTokens: estimatedPromptTokens,","\t\t\trequestedMaxTokens: requestedMaxTokens,","\t\t\trequiresVision: requiresVision,","\t\t\thasTools: hasTools,","\t\t\tparams: rejectionSamplingParams(parsed),","\t\twriteJSON(w, http.StatusNotFound, errorResponse(\"model_not_found\",","\t\t\tfmt.Sprintf(\"model %q is not available — see /v1/models for supported models\", publicModel), withParam(\"model\")))","\t// Self-route pre-flight: confirm the caller owns an online machine that can","\t// serve this model, with precise errors and no fallback to the paid fleet.","\tif policy.enabled {","\t\tif s.selfRouteUnavailable(w, r, policy.ownerAccountID, model) {","\t\t\trefundReservation()","\t} else if policy.prefer {","\t\t// Prefer mode: SKIP the public fleet pre-flight. QuickCapacityCheck has","\t\t// no owner-trust relaxation, so it would spuriously 429/503 a request","\t\t// whose own (idle, possibly un-enrolled / private-only) machine could","\t\t// serve it while the public fleet is busy. Dispatch does owned-first","\t\t// routing with a paid public fallback and the normal queue, which is the","\t\t// correct gate for prefer.","\t\tttftThreshold := deadline","\t\t// Pre-flight capacity check: can ANY provider serve this model right","\t\t// now? If not, return 429 immediately rather than queueing for up to","\t\t// 120s. OpenRouter treats 429 as \"rate limited\" (no uptime penalty) vs","\t\t// 503 which counts as downtime. Fast 429s also preserve our TTFT","\t\t// metrics. Self-route skips this fleet-wide gate — it queues on the","\t\t// owner's machine instead (handled below).","\t\tcandidateCount, capacityRejections, modelTooLarge, bestTTFT, hasTTFT := s.registry.QuickCapacityCheckWithTTFTForRequest(model, estimatedPromptTokens, requestedMaxTokens, registry.RequestTraits{HasTools: hasTools}, requiresVision, allowedProviderSerials...)","\t\tif candidateCount == 0 && capacityRejections > 0 {","\t\t\tif fallbackModel, fallbackCandidates, fallbackRejections, fallbackTooLarge, fallbackTTFT, fallbackHasTTFT, switched := s.maybeFallbackAliasCapacity(parsed, publicModel, model, estimatedPromptTokens, requestedMaxTokens, registry.RequestTraits{HasTools: hasTools}, requiresVision, allowedProviderSerials); switched {","\t\t\t\tmodel = fallbackModel","\t\t\t\tcandidateCount, capacityRejections, modelTooLarge = fallbackCandidates, fallbackRejections, fallbackTooLarge","\t\t\t\tbestTTFT, hasTTFT = fallbackTTFT, fallbackHasTTFT","\t\t\t\tif isResponsesAPI {","\t\t\t\t\tproviderParsed, err := responsesRequestToChatCompletions(parsed)","\t\t\t\t\tif err != nil {","\t\t\t\t\t\trefundReservation()","\t\t\t\t\t\ts.recordRejection(rejectionInfo{","\t\t\t\t\t\t\tr: r,","\t\t\t\t\t\t\tstage: \"validation\",","\t\t\t\t\t\t\treasonCode: \"bad_param\",","\t\t\t\t\t\t\thttpStatus: http.StatusBadRequest,","\t\t\t\t\t\t\tkeyID: keyIDFromContext(r.Context()),","\t\t\t\t\t\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\t\t\t\t\t\trequestedModel: publicModel,","\t\t\t\t\t\t\tresolvedModel: model,","\t\t\t\t\t\t\tstream: stream,","\t\t\t\t\t\t\testimatedPromptTokens: estimatedPromptTokens,","\t\t\t\t\t\t\trequestedMaxTokens: requestedMaxTokens,","\t\t\t\t\t\t\trequiresVision: requiresVision,","\t\t\t\t\t\t\thasTools: hasTools,","\t\t\t\t\t\t\tparams: rejectionSamplingParams(parsed),","\t\t\t\t\t\t})","\t\t\t\t\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"invalid_request_error\", err.Error()))","\t\t\t\t\t\treturn","\t\t\t\t\trawBody, _ = marshalForwardBody(providerParsed)","\t\t\t\t} else {","\t\t\t\t\trawBody, _ = marshalForwardBody(parsed)","\t\t// Smart early-429 for structurally-unservable long prompts","\t\t// (prompt+max_tokens beyond the model context window or any provider's","\t\t// token budget). Gated (default off) and fail-open. Runs AFTER the alias","\t\t// capacity fallback so an alias whose Previous build still has capacity","\t\t// fails over first; an unservable request is then rejected with an","\t\t// uptime-neutral 429 (OpenRouter fails over) instead of admit→5xx.","\t\tif s.shedIfUnservable(w, r, parsed, publicModel, model, modelMaxContext, stream, estimatedPromptTokens, requestedMaxTokens, requiresVision, hasTools, allowedProviderSerials, refundReservation) {","\t\tif candidateCount == 0 && capacityRejections == 0 && modelTooLarge > 0 {","\t\t\t// Providers serve this model but none can ever fit it — non-retryable.","\t\t\t// Surface a clear 503 instead of a 429 the client would retry forever.","\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:model_too_large\"})","\t\t\t\tr: r,","\t\t\t\tstage: \"preflight_capacity\",","\t\t\t\treasonCode: \"model_too_large\",","\t\t\t\thttpStatus: http.StatusServiceUnavailable,","\t\t\t\tkeyID: keyIDFromContext(r.Context()),","\t\t\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\t\t\trequestedModel: publicModel,","\t\t\t\tresolvedModel: model,","\t\t\t\tstream: stream,","\t\t\t\testimatedPromptTokens: estimatedPromptTokens,","\t\t\t\trequestedMaxTokens: requestedMaxTokens,","\t\t\t\trequiresVision: requiresVision,","\t\t\t\thasTools: hasTools,","\t\t\t\tparams: rejectionSamplingParams(parsed),","\t\t\t\tservabilityComputed: true,","\t\t\t\tcandidateCount: candidateCount,","\t\t\t\tcapacityRejections: capacityRejections,","\t\t\t\tmodelTooLargeRejections: modelTooLarge,","\t\t\t\tbestTTFTMs: ttftMsForRejection(bestTTFT, hasTTFT),","\t\t\twriteJSON(w, http.StatusServiceUnavailable, errorResponse(\"model_unavailable\",","\t\t\t\tfmt.Sprintf(\"model %q is too large for any currently available provider\", publicModel),","\t\t\t\twithCode(\"model_unavailable\")))","\t\t\t// Routing v2 W3: feed the autoscaler the demand the preflight sees.","\t\t\ts.registry.RecordWarmPoolCapacityReject(model)","\t\t\ts.triggerWarmPool()","\t\t\t// Queue-before-shed (default on): providers exist for this model but","\t\t\t// all are at capacity right now. Rather than an immediate 429, let the","\t\t\t// request fall through to the normal dispatch+queue path so a slot","\t\t\t// freeing — or a cold load completing — within the queue window serves","\t\t\t// it. The dispatch/queue path still returns a 429 when the queue is","\t\t\t// full or the wait times out (true saturation). The reservation is","\t\t\t// kept for dispatch.","\t\t\t// Dedicated-family models (e.g. Gemma 4) bypass queue-before-shed when","\t\t\t// their dedicated boxes are saturated: holding an OpenRouter request in","\t\t\t// the 120s queue would blow its TTFT SLA, so shed immediately with a","\t\t\t// 429 + Retry-After for a clean failover rather than waiting on a","\t\t\t// dedicated slot that may not free in time.","\t\t\tif s.queueBeforeShedEnabled() && !s.registry.IsDedicatedModel(model) {","\t\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:capacity_queue_spill\"})","\t\t\t\t// Fast-shed: immediate 429 (always for dedicated models; for every","\t\t\t\t// model when queue-before-shed is disabled).","\t\t\t\tretryAfter := s.estimateRetryAfter(model)","\t\t\t\tw.Header().Set(\"Retry-After\", strconv.Itoa(retryAfter))","\t\t\t\trefundReservation()","\t\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:capacity_429\"})","\t\t\t\t\tr: r,","\t\t\t\t\tstage: \"preflight_capacity\",","\t\t\t\t\treasonCode: \"machine_busy\",","\t\t\t\t\thttpStatus: http.StatusTooManyRequests,","\t\t\t\t\tkeyID: keyIDFromContext(r.Context()),","\t\t\t\t\tconsumerKeyHash: store.HashKey(consumerKeyFromContext(r.Context())),","\t\t\t\t\trequestedModel: publicModel,","\t\t\t\t\tresolvedModel: model,","\t\t\t\t\tstream: stream,","\t\t\t\t\testimatedPromptTokens: estimatedPromptTokens,","\t\t\t\t\trequestedMaxTokens: requestedMaxTokens,","\t\t\t\t\trequiresVision: requiresVision,","\t\t\t\t\thasTools: hasTools,","\t\t\t\t\tretryAfterMs: retryAfter * 1000,","\t\t\t\t\tparams: rejectionSamplingParams(parsed),","\t\t\t\t\tservabilityComputed: true,","\t\t\t\t\tcandidateCount: candidateCount,","\t\t\t\t\tcapacityRejections: capacityRejections,","\t\t\t\t\tmodelTooLargeRejections: modelTooLarge,","\t\t\t\t\tbestTTFTMs: ttftMsForRejection(bestTTFT, hasTTFT),","\t\t\t\twriteJSON(w, http.StatusTooManyRequests, errorResponse(\"rate_limit_exceeded\",","\t\t\t\t\tfmt.Sprintf(\"all providers for model %q are at capacity — retry after %ds\", publicModel, retryAfter),","\t\t\t\t\twithCode(\"rate_limit_exceeded\")))","\t\t\t\treturn","\t\tif candidateCount == 0 && capacityRejections == 0 && modelTooLarge == 0 {","\t\t\t// No provider is even structurally eligible right now: the model's","\t\t\t// whole pool is offline/untrusted, trait-gated (below the tools floor","\t\t\t// / render-broken), or — the case the shape-keyed breaker introduces —","\t\t\t// every serving provider is in inference-error cooldown for THIS","\t\t\t// request shape.","\t\t\t//","\t\t\t// Routing v2 W3 cold-dispatch (default on): before shedding, check","\t\t\t// whether an idle on-disk provider could be WARMED to serve this model","\t\t\t// (and would then pass admission for these traits). If so, spill the","\t\t\t// request into the queue instead of 503'ing — the enqueue path kicks","\t\t\t// the model-swap machinery, and the queued request drains onto the","\t\t\t// provider once the cold load completes. Note that an idle, fitting","\t\t\t// cold provider is already a scheduler candidate (slot \"unknown\" is","\t\t\t// eligible), so this branch usually only fires for genuinely","\t\t\t// unservable demand; it is the safety valve for the narrow window","\t\t\t// where a loadable cold provider is not yet a candidate.","\t\t\t// Feed the autoscaler the demand regardless of outcome.","\t\t\tif s.coldDispatchEnabled() && s.coldSpillAvailable(model, registry.RequestTraits{HasTools: hasTools}, requiresVision, allowedProviderSerials) {","\t\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:cold_dispatch_spill\"})","\t\t\t\t// Fall through to dispatch+queue; reservation kept.","\t\t\t} else if s.registry.IsDedicatedModel(model) && s.registry.HasProviderForModel(model, allowedProviderSerials...) {","\t\t\t\t// Dedicated-box model (e.g. Gemma 4): the fleet DOES serve this","\t\t\t\t// model, but no provider DEDICATED to it can take the request right","\t\t\t\t// now — either none are dedicated, or the dedicated ones are busy/","\t\t\t\t// cooling. That is transient capacity pressure, not an absent model,","\t\t\t\t// so shed to OpenRouter as a 429 + Retry-After (clean failover)","\t\t\t\t// rather than a 503 (which can get the endpoint marked unhealthy /","\t\t\t\t// deranked). Mirrors the capacity_429 path above.","\t\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:dedicated_capacity_429\"})","\t\t\t\t\tfmt.Sprintf(\"no provider dedicated to model %q is available right now — retry after %ds\", publicModel, retryAfter),","\t\t\t\t// None of these clear by a slot freeing up, so queueing for up to","\t\t\t\t// 120s only adds misleading latency before the same error. Fail","\t\t\t\t// fast with a retryable 503 + Retry-After (OpenRouter treats 503","\t\t\t\t// as unavailable, not a uptime-penalised error here because the","\t\t\t\t// body is explicit). This mirrors the trait fast-fails above for","\t\t\t\t// the transient-cooldown case they cannot see.","\t\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:no_eligible_provider\"})","\t\t\t\t\treasonCode: \"no_provider\",","\t\t\t\t\thttpStatus: http.StatusServiceUnavailable,","\t\t\t\twriteJSON(w, http.StatusServiceUnavailable, errorResponse(\"model_unavailable\",","\t\t\t\t\tfmt.Sprintf(\"no provider for model %q is available right now — retry after %ds\", publicModel, retryAfter),","\t\t\t\t\twithCode(\"model_unavailable\")))","\t\tif ttftTooSlow(bestTTFT, hasTTFT, ttftThreshold) {","\t\t\tif !s.ttftHardReject {","\t\t\t\t// Soft TTFT gate (default): a provider passed every routing and","\t\t\t\t// capacity gate, and pr.MaxTTFTMs is left 0 in soft mode, so the","\t\t\t\t// dispatch path serves the best-available provider instead of","\t\t\t\t// re-rejecting (P1 fix). Do NOT divert to an older alias build here","\t\t\t\t// (P2 fix) — the desired build is routable. A soft-serve over the","\t\t\t\t// deadline is still a TTFT near-miss, so feed the autoscaler so it","\t\t\t\t// grows warm capacity for this model.","\t\t\t\ts.registry.RecordWarmPoolTTFTMiss(model, ttftThreshold)","\t\t\t\ts.triggerWarmPool()","\t\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:ttft_soft_served\"})","\t\t\t} else if fallbackModel, _, _, _, fallbackTTFT, fallbackHasTTFT, switched := s.maybeFallbackAliasTTFT(parsed, publicModel, model, estimatedPromptTokens, requestedMaxTokens, ttftThreshold, registry.RequestTraits{HasTools: hasTools}, requiresVision, allowedProviderSerials); switched {","\t\t\t\t// Hard TTFT gate, no faster alias: shed with a 429 + Retry-After,","\t\t\t\t// and feed the autoscaler a TTFT-miss so warm capacity grows.","\t\t\t\tretryModel, retryTTFT := fasterTTFTEstimate(model, bestTTFT, fallbackModel, fallbackTTFT, fallbackHasTTFT)","\t\t\t\t\tstage: \"routing_ttft\",","\t\t\t\t\treasonCode: \"ttft_too_slow\",","\t\t\t\t\tbestTTFTMs: float64(retryTTFT.Milliseconds()),","\t\t\t\ts.writeTTFTTooSlow(w, retryModel, publicModel, retryTTFT, ttftThreshold)","\t// Dispatch to a provider with speculative TTFT-aware dispatch. On the","\t// first attempt we dispatch to the best provider (primary), and start a","\t// speculative timer at 50% of the TTFT deadline. If the primary hasn't","\t// produced a first chunk by the speculative timer, a backup provider is","\t// dispatched in parallel and both race. If the primary fails outright","\t// (error before the speculative timer), up to maxDispatchAttempts","\t// sequential retries are performed without speculation.","\t//","\t// No HTTP response is written until a provider starts generating, so","\t// retries and speculative dispatch are invisible to the consumer.","\t// Dispatch is driven by the per-request state machine in dispatch.go: it","\t// picks a provider (or queues), runs the speculative TTFT-aware first-chunk","\t// wait with an invisible backup race + failover up to maxDispatchAttempts,","\t// commits exactly once, then writes attestation/timing headers and streams.","\tconsumerKey := consumerKeyFromContext(r.Context())","\tconsumerLocation := s.requestLocation(r)","\t// Final cap on the body we'll seal. The read cap (parseInferencePrelude)","\t// bounded the request as received, but rawBody has since been re-marshaled at","\t// several points — alias resolution, allowlist/routing-field stripping,","\t// reasoning_parser + max_tokens injection, Responses→chat lowering, and the","\t// alias-capacity fallback above. The coordinator seals this body and sends it","\t// as ONE WebSocket frame; a body over the cap produces a frame the provider","\t// rejects by tearing down its session and cancelling every unrelated in-flight","\t// request (see maxInferenceBodyBytes / CoordinatorClient.maxInboundMessageBytes).","\t// This is the single point where rawBody is frozen into dispatchState, so the","\t// check here covers every upstream mutation; an oversized request gets a clean","\t// 413 instead of disconnecting a provider mid-flight. The reservation is held","\t// at this point, so refund before returning.","\tif len(rawBody) > maxInferenceBodyBytes {","\t\t\tstage: \"validation\",","\t\t\treasonCode: \"payload_too_large\",","\t\t\thttpStatus: http.StatusRequestEntityTooLarge,","\t\t\trequestBodyBytes: len(rawBody),","\t\twriteJSON(w, http.StatusRequestEntityTooLarge, errorResponse(\"invalid_request_error\",","\t\t\tfmt.Sprintf(\"request body exceeds the %d-byte limit\", maxInferenceBodyBytes)))","\td := &dispatchState{","\t\ts: s,","\t\tw: w,","\t\tr: r,","\t\tmodel: model,","\t\tpublicModel: publicModel,","\t\trawBody: rawBody,","\t\tconsumerKey: consumerKey,","\t\tconsumerLocation: consumerLocation,","\t\treservedMicroUSD: reservedMicroUSD,","\t\ttokenAdmission: tokenAdmission,","\t\tserviceReservation: serviceReservation,","\t\testimatedPromptTokens: estimatedPromptTokens,","\t\trequestedMaxTokens: requestedMaxTokens,","\t\trequiresVision: requiresVision,","\t\thasTools: hasTools,","\t\tisResponsesAPI: isResponsesAPI,","\t\tstream: stream,","\t\tpolicy: policy,","\t\tallowedProviderSerials: allowedProviderSerials,","\t\tcacheAffinityKey: cacheAffinityKey,","\t\ttiming: timing,","\t\tdeadline: deadline,","\t\tspeculativeAt: time.Duration(float64(deadline) * speculativeTimerRatio),","\t\trefundReservation: refundReservation,","\t\t// Track providers that failed during retry so we don't dispatch to them again.","\t\texcludeProviders: make(map[string]struct{}),","\td.run()","// handleStreamingResponseWithFirstChunk streams SSE chunks to the consumer.","// Any firstChunks (held preamble + first content chunk) are written in order","// before reading further chunks from the channel. This allows the dispatch","// loop to \"peek\" at chunks for retry decisions without losing them.","func (s *Server) handleStreamingResponseWithFirstChunk(w http.ResponseWriter, r *http.Request, pr *registry.PendingRequest, firstChunks []string, headerWritten bool) {","\tif pr.IsResponsesAPI {","\t\ts.handleResponsesStreamingResponseWithFirstChunk(w, r, pr, firstChunks, headerWritten)","\tflusher, ok := w.(http.Flusher)","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"internal_error\", \"streaming not supported\"))","\t// When a prefill keepalive already committed the SSE 200 header, skip the","\t// header write (a second WriteHeader would be superfluous) and stream straight","\t// into the held first chunks; otherwise write it now.","\tif !headerWritten {","\t\twriteSSEResponseHeader(w, pr.RequestID)","\tflusher.Flush()","\t// Detect Responses API format to skip appending chat-completions-style","\t// termination events (SE signature chunk + [DONE]).","\tsawResponsesAPI := false","\t// The terminal include_usage chunk lacks the reasoning breakdown; we hold its","\t// parsed object and re-emit it at stream end with the provider's authoritative","\t// reasoning count (CompleteCh) spliced in — matching the non-streaming/Responses","\t// paths. Held as a parsed map so it is decoded exactly once. Declared before the","\t// first-chunk write because a zero-delta completion (empty/filtered output) can","\t// make the include_usage frame the very first chunk.","\tvar pendingUsage map[string]any","\t// The chunk carrying the terminal finish_reason is held the same way: the","\t// provider engine reports \"stop\" even when generation hit the max-tokens","\t// bound, so the coordinator re-derives \"length\" from the authoritative","\t// token counts (CompleteCh) before forwarding it.","\tvar pendingFinish map[string]any","\t// Write the chunks that were already consumed during dispatch (held","\t// preamble first, then the committing content chunk), each through the","\t// same per-chunk special-casing the relay loop below applies.","\tfor _, firstChunk := range firstChunks {","\t\tif firstChunk == \"\" || isSSEDoneChunk(firstChunk) {","\t\tif isResponsesAPIEventChunk(firstChunk) {","\t\t\tsawResponsesAPI = true","\t\t// A usage-only first chunk (no content/reasoning deltas streamed before it)","\t\t// is still terminal usage — hold it so the reasoning breakdown is spliced in","\t\t// at stream end instead of being emitted raw without reasoning_tokens.","\t\tif obj, isUsage := parseUsageOnlyStreamChunk(firstChunk); !sawResponsesAPI && isUsage {","\t\t\tpendingUsage = obj","\t\t} else if obj, isFinish := parseFinishStreamChunk(normalizeSSEChunk(firstChunk)); !sawResponsesAPI && isFinish {","\t\t\tpendingFinish = obj","\t\t} else {","\t\t\tif !sawResponsesAPI {","\t\t\t\tfirstChunk = normalizeSSEChunk(firstChunk)","\t\t\tfirstChunk = rewriteChunkModel(firstChunk, pr)","\t\t\tfmt.Fprintf(w, \"%s\\n\\n\", firstChunk)","\t\t\tflusher.Flush()","\t// Use a timer that resets on each chunk so long-running generations","\t// (e.g. chain-of-thought models) don't hit a global timeout.","\ttimer := time.NewTimer(inferenceTimeout)","\tdefer timer.Stop()","\tfor {","\t\tselect {","\t\tcase chunk, ok := <-pr.ChunkCh:","\t\t\t\tselect {","\t\t\t\tcase errMsg, ok := <-pr.ErrorCh:","\t\t\t\t\tif ok && errMsg.Error != \"\" {","\t\t\t\t\t\ts.refundReservedBalance(pr, \"provider_error:\"+pr.RequestID)","\t\t\t\t\t\ts.noteInferenceError(pr.ProviderID, pr, errMsg.StatusCode, errMsg.Error)","\t\t\t\t\t\ts.ddIncr(\"inference.in_band_error\", []string{\"model:\" + pr.Model, \"reason:provider_error\"})","\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, postCommitProviderErrorOutcome(pr, errMsg))","\t\t\t\t\t\terrData, _ := json.Marshal(map[string]any{","\t\t\t\t\t\t\t\"error\": map[string]any{","\t\t\t\t\t\t\t\t\"message\": errMsg.Error,","\t\t\t\t\t\t\t\t\"type\": \"provider_error\",","\t\t\t\t\t\tfmt.Fprintf(w, \"data: %s\\n\\n\", errData)","\t\t\t\t\t\tflusher.Flush()","\t\t\t\tif s.refundReservedBalance(pr, \"provider_incomplete:\"+pr.RequestID) {","\t\t\t\t\ts.ddIncr(\"inference.in_band_error\", []string{\"model:\" + pr.Model, \"reason:provider_incomplete\"})","\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, postCommitProviderIncompleteOutcome(pr))","\t\t\t\t\tfmt.Fprintf(w, \"data: {\\\"error\\\":{\\\"message\\\":\\\"provider ended without completion\\\",\\\"type\\\":\\\"provider_error\\\"}}\\n\\n\")","\t\t\t\t\tflusher.Flush()","\t\t\t\t\treturn","\t\t\t\t// Channel closed — inference complete.","\t\t\t\ts.noteInferenceSuccess(pr)","\t\t\t\t// For Responses API streams, the provider already sent","\t\t\t\t// \"response.completed\" as the terminal event. Adding","\t\t\t\t// extra chunks would break SDK parsers.","\t\t\t\tif !sawResponsesAPI {","\t\t\t\t\t// Emit the held finish/usage chunks with the authoritative token","\t\t\t\t\t// counts (CompleteCh) spliced in: the finish chunk gets its","\t\t\t\t\t// finish_reason corrected to \"length\" when generation hit the","\t\t\t\t\t// max-tokens bound, and the usage chunk gets the reasoning","\t\t\t\t\t// breakdown. This select runs once, at stream end: the provider's","\t\t\t\t\t// inferenceComplete (which populates CompleteCh) is what ends the","\t\t\t\t\t// stream, so it is effectively already buffered — the timeout is a","\t\t\t\t\t// fallback, not a hot-path wait.","\t\t\t\t\tvar usage protocol.UsageInfo","\t\t\t\t\tif pendingUsage != nil || pendingFinish != nil {","\t\t\t\t\t\tselect {","\t\t\t\t\t\tcase u, uok := <-pr.CompleteCh:","\t\t\t\t\t\t\tif uok {","\t\t\t\t\t\t\t\tusage = u","\t\t\t\t\t\t\t}","\t\t\t\t\t\tcase <-time.After(2 * time.Second):","\t\t\t\t\t\tcase <-r.Context().Done():","\t\t\t\t\t\t}","\t\t\t\t\tif pendingFinish != nil {","\t\t\t\t\t\tif out := finalizeFinishChunk(pendingFinish, usage, pr); out != \"\" {","\t\t\t\t\t\t\tfmt.Fprintf(w, \"%s\\n\\n\", out)","\t\t\t\t\t\t\tflusher.Flush()","\t\t\t\t\tif pendingUsage != nil {","\t\t\t\t\t\t// Ride the SE signature on the held usage chunk (a complete,","\t\t\t\t\t\t// well-formed chat.completion.chunk) instead of emitting a","\t\t\t\t\t\t// separate bare event that strict SDK parsers reject.","\t\t\t\t\t\tif pr.SESignature != \"\" {","\t\t\t\t\t\t\tpendingUsage[\"se_signature\"] = pr.SESignature","\t\t\t\t\t\t\tpendingUsage[\"response_hash\"] = pr.ResponseHash","\t\t\t\t\t\tif out := finalizeUsageChunk(pendingUsage, usage, pr); out != \"\" {","\t\t\t\t\t} else if pr.SESignature != \"\" {","\t\t\t\t\t\t// No held usage chunk to ride on: emit the signature as a","\t\t\t\t\t\t// fully-shaped chat.completion.chunk (id/object/created/model/","\t\t\t\t\t\t// choices) so strict decoders parse it; the extra fields are","\t\t\t\t\t\t// additive. It precedes the single [DONE] below.","\t\t\t\t\t\tsigEvent, _ := json.Marshal(map[string]any{","\t\t\t\t\t\t\t\"id\": \"chatcmpl-\" + pr.RequestID,","\t\t\t\t\t\t\t\"object\": \"chat.completion.chunk\",","\t\t\t\t\t\t\t\"created\": time.Now().Unix(),","\t\t\t\t\t\t\t\"model\": consumerModel(pr),","\t\t\t\t\t\t\t\"choices\": []any{},","\t\t\t\t\t\t\t\"se_signature\": pr.SESignature,","\t\t\t\t\t\t\t\"response_hash\": pr.ResponseHash,","\t\t\t\t\t\tfmt.Fprintf(w, \"data: %s\\n\\n\", sigEvent)","\t\t\t\t\t// Exactly one terminator, after every coordinator-appended event.","\t\t\t\t\tfmt.Fprint(w, \"data: [DONE]\\n\\n\")","\t\t\t// Every chunk is a liveness signal — re-arm the idle timeout up front,","\t\t\t// before deciding whether to forward or hold it, so holding the terminal","\t\t\t// usage chunk still resets the window that bounds the wait for the","\t\t\t// provider's inference_complete (which closes ChunkCh after billing).","\t\t\t// One reset covers both the forward and hold paths.","\t\t\tif !timer.Stop() {","\t\t\t\tcase <-timer.C:","\t\t\ttimer.Reset(inferenceTimeout)","\t\t\t\tif isResponsesAPIEventChunk(chunk) {","\t\t\t\t\tsawResponsesAPI = true","\t\t\t// Swallow the provider's own \"data: [DONE]\" terminator. The","\t\t\t// coordinator appends terminal events of its own (held usage with","\t\t\t// the reasoning breakdown, SE signature) and then emits exactly ONE","\t\t\t// [DONE] — forwarding the provider's produced a stream shaped","\t\t\t// `...usage, [DONE], signature, [DONE]`, and third-party SDKs treat","\t\t\t// the first [DONE] as final (MacPaw/OpenAI then chokes parsing the","\t\t\t// signature event).","\t\t\tif !sawResponsesAPI && isSSEDoneChunk(chunk) {","\t\t\t// Hold the terminal usage chunk (chat completions only) so we can splice","\t\t\t// in the reasoning breakdown at stream end; forwarding it inline would","\t\t\t// emit it without reasoning_tokens.","\t\t\t\tif obj, isUsage := parseUsageOnlyStreamChunk(chunk); isUsage {","\t\t\t\t\tpendingUsage = obj","\t\t\t\tchunk = normalizeSSEChunk(chunk)","\t\t\t\t// Hold the chunk carrying the terminal finish_reason so it can be","\t\t\t\t// corrected to \"length\" against the authoritative token counts at","\t\t\t\t// stream end (the provider engine always reports \"stop\").","\t\t\t\tif obj, isFinish := parseFinishStreamChunk(chunk); isFinish {","\t\t\t\t\tpendingFinish = obj","\t\t\tchunk = rewriteChunkModel(chunk, pr)","\t\t\tfmt.Fprintf(w, \"%s\\n\\n\", chunk)","\t\tcase errMsg, ok := <-pr.ErrorCh:","\t\t\ts.refundReservedBalance(pr, \"provider_error:\"+pr.RequestID)","\t\t\ts.noteInferenceError(pr.ProviderID, pr, errMsg.StatusCode, errMsg.Error)","\t\t\ts.ddIncr(\"inference.in_band_error\", []string{\"model:\" + pr.Model, \"reason:provider_error\"})","\t\t\ts.updateInferenceRouteOutcomeForPending(pr, postCommitProviderErrorOutcome(pr, errMsg))","\t\t\terrData, _ := json.Marshal(map[string]any{","\t\t\t\t\"error\": map[string]any{","\t\t\t\t\t\"message\": errMsg.Error,","\t\t\t\t\t\"type\": \"provider_error\",","\t\t\t\t},","\t\t\tfmt.Fprintf(w, \"data: %s\\n\\n\", errData)","\t\tcase <-timer.C:","\t\t\ts.refundReservedBalance(pr, \"provider_timeout:\"+pr.RequestID)","\t\t\ts.ddIncr(\"inference.in_band_error\", []string{\"model:\" + pr.Model, \"reason:timeout\"})","\t\t\ts.updateInferenceRouteOutcomeForPending(pr, postCommitStreamTimeoutOutcome(pr))","\t\t\tfmt.Fprintf(w, \"data: {\\\"error\\\":{\\\"message\\\":\\\"request timed out\\\",\\\"type\\\":\\\"timeout\\\"}}\\n\\n\")","\t\tcase <-r.Context().Done():","func (s *Server) handleResponsesStreamingResponseWithFirstChunk(w http.ResponseWriter, r *http.Request, pr *registry.PendingRequest, firstChunks []string, headerWritten bool) {","\t// Skip the header write when a prefill keepalive already committed the SSE 200.","\tresponseID := \"resp_\" + strings.ReplaceAll(pr.RequestID, \"-\", \"\")","\tcreatedAt := time.Now().Unix()","\temitter := newResponsesStreamEmitter(w, flusher, pr, responseID, createdAt)","\temitter.start()","\t\tif firstChunk != \"\" {","\t\t\temitter.handleChunk(firstChunk)","\t\t\t\tvar usage protocol.UsageInfo","\t\t\t\tcompleted := false","\t\t\t\tcase u, ok := <-pr.CompleteCh:","\t\t\t\t\tif ok {","\t\t\t\t\t\tusage = u","\t\t\t\t\t\tcompleted = true","\t\t\t\tcase <-time.After(2 * time.Second):","\t\t\t\tif !completed && s.refundReservedBalance(pr, \"provider_incomplete:\"+pr.RequestID) {","\t\t\t\t\temitter.emitError(\"provider_error\", \"provider ended without completion\")","\t\t\t\temitter.finish(usage)","\t\t\temitter.handleChunk(chunk)","\t\t\temitter.emitError(\"provider_error\", errMsg.Error)","\t\t\temitter.emitError(\"timeout\", \"request timed out\")","// handleNonStreamingResponseWithFirstChunk collects all chunks from the","// provider and assembles them into a single OpenAI-compatible JSON response.","// Any firstChunks (held preamble + first content chunk consumed during","// dispatch) seed the collected chunks in order.","func (s *Server) handleNonStreamingResponseWithFirstChunk(w http.ResponseWriter, r *http.Request, pr *registry.PendingRequest, firstChunks []string) {","\tctx, cancel := context.WithTimeout(r.Context(), inferenceTimeout)","\tvar chunks []string","\t\t\tchunks = append(chunks, firstChunk)","\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preResponseProviderErrorOutcome(pr, errMsg))","\t\t\t\t\t\tstatusCode := errMsg.StatusCode","\t\t\t\t\t\tif statusCode == 0 {","\t\t\t\t\t\t\tstatusCode = http.StatusBadGateway","\t\t\t\t\t\twriteJSON(w, statusCode, errorResponse(\"provider_error\", errMsg.Error))","\t\t\t\t// The provider forwards the raw backend response as a single","\t\t\t\t// chunk. Detect complete responses (object=chat.completion","\t\t\t\t// or object=response) and pass through directly — this is","\t\t\t\t// format-agnostic and works for chat completions, Responses","\t\t\t\t// API, or any future endpoint without parsing.","\t\t\t\tif len(chunks) == 1 {","\t\t\t\t\traw := strings.TrimPrefix(chunks[0], \"data: \")","\t\t\t\t\tvar obj map[string]any","\t\t\t\t\tif err := json.Unmarshal([]byte(raw), &obj); err == nil {","\t\t\t\t\t\tobjType, _ := obj[\"object\"].(string)","\t\t\t\t\t\t// Complete responses have object=chat.completion or","\t\t\t\t\t\t// object=response. Delta chunks have object=chat.completion.chunk.","\t\t\t\t\t\tif objType == \"chat.completion\" || objType == \"response\" {","\t\t\t\t\t\t\tvar completeUsage protocol.UsageInfo","\t\t\t\t\t\t\tselect {","\t\t\t\t\t\t\tcase u, ok := <-pr.CompleteCh:","\t\t\t\t\t\t\t\tif !ok {","\t\t\t\t\t\t\t\t\ts.refundReservedBalance(pr, \"provider_incomplete:\"+pr.RequestID)","\t\t\t\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preResponseProviderIncompleteOutcome(pr))","\t\t\t\t\t\t\t\t\twriteJSON(w, http.StatusBadGateway, errorResponse(\"provider_error\", \"provider ended without completion\"))","\t\t\t\t\t\t\t\t\treturn","\t\t\t\t\t\t\t\t}","\t\t\t\t\t\t\t\tcompleteUsage = u","\t\t\t\t\t\t\tcase <-ctx.Done():","\t\t\t\t\t\t\t\tif errors.Is(ctx.Err(), context.DeadlineExceeded) {","\t\t\t\t\t\t\t\t\ts.refundReservedBalance(pr, \"provider_timeout:\"+pr.RequestID)","\t\t\t\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preResponseTimeoutOutcome(pr, \"usage_timeout_before_response\"))","\t\t\t\t\t\t\t\t\twriteJSON(w, http.StatusGatewayTimeout, errorResponse(\"timeout\", \"timed out waiting for usage info\"))","\t\t\t\t\t\t\t\t} else {","\t\t\t\t\t\t\t\t\ts.refundReservedBalance(pr, \"client_gone:\"+pr.RequestID)","\t\t\t\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, clientGoneBeforeResponseOutcome(pr))","\t\t\t\t\t\t\t\treturn","\t\t\t\t\t\t\tif objType == \"chat.completion\" {","\t\t\t\t\t\t\t\tnormalizeCompleteChatResponse(obj, consumerModel(pr))","\t\t\t\t\t\t\t\t// The provider engine reports \"stop\" even when generation","\t\t\t\t\t\t\t\t// hit the max-tokens bound — correct it from the","\t\t\t\t\t\t\t\t// authoritative token counts.","\t\t\t\t\t\t\t\trewriteRawFinishReason(obj, completeUsage, pr.RequestedMaxTokens)","\t\t\t\t\t\t\t\t// Keep the passthrough path consistent with the","\t\t\t\t\t\t\t\t// SSE-reconstruction path: surface the provider's","\t\t\t\t\t\t\t\t// accurate reasoning-token count if its raw usage","\t\t\t\t\t\t\t\t// object didn't already carry one.","\t\t\t\t\t\t\t\tinjectReasoningDetailIntoRawUsage(obj, completeUsage)","\t\t\t\t\t\t\t\tif pr.IsResponsesAPI {","\t\t\t\t\t\t\t\t\tvar chatResp types.ChatCompletionResponse","\t\t\t\t\t\t\t\t\tb, err := json.Marshal(obj)","\t\t\t\t\t\t\t\t\tif err != nil {","\t\t\t\t\t\t\t\t\t\tlog.Printf(\"WARN: failed to marshal chat response for Responses API conversion: %v\", err)","\t\t\t\t\t\t\t\t\t\twriteJSON(w, http.StatusBadGateway, errorResponse(\"provider_error\", \"invalid provider response\"))","\t\t\t\t\t\t\t\t\t\treturn","\t\t\t\t\t\t\t\t\t}","\t\t\t\t\t\t\t\t\tif err := json.Unmarshal(b, &chatResp); err != nil {","\t\t\t\t\t\t\t\t\t\tlog.Printf(\"WARN: failed to unmarshal chat response into typed struct: %v\", err)","\t\t\t\t\t\t\t\t\trespObj := chatCompletionToResponses(chatResp, consumerModel(pr), pr.SESignature, pr.ResponseHash)","\t\t\t\t\t\t\t\t\ts.noteInferenceSuccess(pr)","\t\t\t\t\t\t\t\t\twriteJSON(w, http.StatusOK, respObj)","\t\t\t\t\t\t\t} else {","\t\t\t\t\t\t\t\t// Native passthrough (object==\"response\"): the provider","\t\t\t\t\t\t\t\t// echoed the concrete build id; rewrite it to the public","\t\t\t\t\t\t\t\t// alias so the consumer never sees the quant/build.","\t\t\t\t\t\t\t\tif pr.PublicModel != \"\" {","\t\t\t\t\t\t\t\t\tobj[\"model\"] = consumerModel(pr)","\t\t\t\t\t\t\tif pr.SESignature != \"\" {","\t\t\t\t\t\t\t\tobj[\"se_signature\"] = pr.SESignature","\t\t\t\t\t\t\t\tobj[\"response_hash\"] = pr.ResponseHash","\t\t\t\t\t\t\ts.noteInferenceSuccess(pr)","\t\t\t\t\t\t\twriteJSON(w, http.StatusOK, obj)","\t\t\t\t\t\t\treturn","\t\t\t\t// Fallback: SSE delta chunks — reconstruct into response.","\t\t\t\tmsg := extractMessage(chunks)","\t\t\t\tcase usage, ok := <-pr.CompleteCh:","\t\t\t\t\tif !ok {","\t\t\t\t\t\ts.refundReservedBalance(pr, \"provider_incomplete:\"+pr.RequestID)","\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preResponseProviderIncompleteOutcome(pr))","\t\t\t\t\t\twriteJSON(w, http.StatusBadGateway, errorResponse(\"provider_error\", \"provider ended without completion\"))","\t\t\t\t\tvar resp any","\t\t\t\t\tif pr.IsResponsesAPI {","\t\t\t\t\t\tresp = buildResponsesResponse(pr.RequestID, consumerModel(pr), msg, usage, pr.RequestedMaxTokens, pr.SESignature, pr.ResponseHash)","\t\t\t\t\t} else {","\t\t\t\t\t\tresp = buildNonStreamingResponse(pr.RequestID, consumerModel(pr), msg, usage, pr.RequestedMaxTokens, pr.SESignature, pr.ResponseHash)","\t\t\t\t\ts.noteInferenceSuccess(pr)","\t\t\t\t\twriteJSON(w, http.StatusOK, resp)","\t\t\t\tcase <-ctx.Done():","\t\t\t\t\tif errors.Is(ctx.Err(), context.DeadlineExceeded) {","\t\t\t\t\t\ts.refundReservedBalance(pr, \"provider_timeout:\"+pr.RequestID)","\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preResponseTimeoutOutcome(pr, \"usage_timeout_before_response\"))","\t\t\t\t\t\twriteJSON(w, http.StatusGatewayTimeout, errorResponse(\"timeout\", \"timed out waiting for usage info\"))","\t\t\t\t\t\ts.refundReservedBalance(pr, \"client_gone:\"+pr.RequestID)","\t\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, clientGoneBeforeResponseOutcome(pr))","\t\t\tchunks = append(chunks, chunk)","\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preResponseProviderErrorOutcome(pr, errMsg))","\t\t\tstatusCode := errMsg.StatusCode","\t\t\tif statusCode == 0 {","\t\t\t\tstatusCode = http.StatusBadGateway","\t\t\twriteJSON(w, statusCode, errorResponse(\"provider_error\", errMsg.Error))","\t\tcase <-ctx.Done():","\t\t\tif errors.Is(ctx.Err(), context.DeadlineExceeded) {","\t\t\t\ts.refundReservedBalance(pr, \"provider_timeout:\"+pr.RequestID)","\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preResponseTimeoutOutcome(pr, \"response_timeout_before_response\"))","\t\t\t\twriteJSON(w, http.StatusGatewayTimeout, errorResponse(\"timeout\", \"request timed out\"))","\t\t\t\ts.refundReservedBalance(pr, \"client_gone:\"+pr.RequestID)","\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, clientGoneBeforeResponseOutcome(pr))","// rewriteRawFinishReason corrects a provider-reported \"stop\" finish_reason to","// \"length\" on a raw chat.completion object when the authoritative token counts","// show generation consumed the entire max-tokens budget.","func rewriteRawFinishReason(obj map[string]any, usage protocol.UsageInfo, requestedMax int) {","\tif !truncatedByMaxTokens(usage, requestedMax) {","\tchoices, ok := obj[\"choices\"].([]any)","\tfor _, rawChoice := range choices {","\t\tif choice, ok := rawChoice.(map[string]any); ok {","\t\t\tif fr, _ := choice[\"finish_reason\"].(string); fr == \"stop\" {","\t\t\t\tchoice[\"finish_reason\"] = \"length\"","func normalizeCompleteChatResponse(obj map[string]any, requestedModel string) {","\tif requestedModel != \"\" {","\t\tobj[\"model\"] = requestedModel","\tfor _, key := range []string{\"system_fingerprint\"} {","\t\tif v, ok := obj[key]; ok && v == nil {","\t\t\tdelete(obj, key)","\t\tchoice, ok := rawChoice.(map[string]any)","\t\tif message, ok := choice[\"message\"].(map[string]any); ok {","\t\t\tnormalizeCompleteMessage(message)","\t\tif delta, ok := choice[\"delta\"].(map[string]any); ok {","\t\t\tnormalizeCompleteMessage(delta)","func normalizeCompleteMessage(message map[string]any) {","\tvar extractedReasoning string","\tif content, ok := message[\"content\"]; !ok || content == nil {","\t\tmessage[\"content\"] = \"\"","\t} else if contentText, ok := content.(string); ok {","\t\tcleaned, reasoning := stripThinkBlocks(contentText)","\t\tmessage[\"content\"] = cleaned","\t\textractedReasoning = reasoning","\tif rc, ok := message[\"reasoning_content\"]; ok {","\t\tif rcText, ok := rc.(string); ok && rcText != \"\" {","\t\t\tmergeReasoningField(message, rcText)","\t\tdelete(message, \"reasoning_content\")","\tif reasoning, ok := message[\"reasoning\"]; ok && reasoning == nil {","\t\tdelete(message, \"reasoning\")","\tif extractedReasoning != \"\" {","\t\tmergeReasoningField(message, extractedReasoning)","\tfor _, key := range []string{\"tool_calls\", \"refusal\"} {","\t\tif v, ok := message[key]; ok && v == nil {","\t\t\tdelete(message, key)","func mergeReasoningField(message map[string]any, reasoning string) {","\treasoning = strings.TrimSpace(reasoning)","\tif reasoning == \"\" {","\tif existing, ok := message[\"reasoning\"].(string); ok && strings.TrimSpace(existing) != \"\" {","\t\tif existing != reasoning && !strings.Contains(existing, reasoning) {","\t\t\tmessage[\"reasoning\"] = existing + \"\\n\\n\" + reasoning","\tmessage[\"reasoning\"] = reasoning","func stripThinkBlocks(text string) (string, string) {","\tmatches := thinkBlockPattern.FindAllStringSubmatch(text, -1)","\treasoningParts := make([]string, 0, len(matches)+1)","\tfound := len(matches) > 0","\tfor _, match := range matches {","\t\tif len(match) > 1 {","\t\t\tif part := strings.TrimSpace(match[1]); part != \"\" {","\t\t\t\treasoningParts = append(reasoningParts, part)","\tcleaned := thinkBlockPattern.ReplaceAllString(text, \"\")","\tlower := strings.ToLower(cleaned)","\tif idx := strings.Index(lower, \"\"); idx >= 0 {","\t\tfound = true","\t\tif part := strings.TrimSpace(cleaned[idx+len(\"\"):]); part != \"\" {","\t\t\treasoningParts = append(reasoningParts, part)","\t\tcleaned = cleaned[:idx]","\tif !found {","\t\treturn text, \"\"","\treturn strings.TrimSpace(cleaned), strings.Join(reasoningParts, \"\\n\\n\")","// normalizeSSEChunk fixes fields in SSE chunks to match the OpenAI spec.","// Some backends (e.g. vllm-mlx) emit \"content\":null instead of \"content\":\"\",","// and include \"usage\":null which strict parsers (ForgeCode, Codex) reject","// because they expect usage to be either absent or a full object.","func normalizeSSEChunk(chunk string) string {","\tline := strings.TrimPrefix(chunk, \"data: \")","\t// Only trigger the expensive JSON parse for fields we actually fix.","\t// \"finish_reason\":null appears on every chunk but we don't touch it,","\t// so checking for generic \":null\" causes unnecessary JSON round-trips.","\tneedsNullFix := strings.Contains(line, `\"content\":null`) ||","\t\tstrings.Contains(line, `\"tool_calls\":null`) ||","\t\tstrings.Contains(line, `\"usage\":null`) ||","\t\tstrings.Contains(line, `\"reasoning\":null`) ||","\t\tstrings.Contains(line, `\"reasoning_content\":null`) ||","\t\tstrings.Contains(line, `\"refusal\":null`) ||","\t\tstrings.Contains(line, `\"system_fingerprint\":null`)","\tneedsReasoningDedup := strings.Contains(line, `\"reasoning_content\"`)","\tif !needsNullFix && !needsReasoningDedup {","\tvar raw map[string]json.RawMessage","\tif err := json.Unmarshal([]byte(line), &raw); err != nil {","\t// Remove top-level null fields (usage, system_fingerprint, etc.)","\t// ForgeCode expects usage to be absent or a full object, not null.","\tfor _, key := range []string{\"usage\", \"system_fingerprint\"} {","\t\tif v, ok := raw[key]; ok && string(v) == \"null\" {","\t\t\tdelete(raw, key)","\t// Fix null fields inside choices[].delta","\tif choicesRaw, ok := raw[\"choices\"]; ok {","\t\tvar choices []map[string]json.RawMessage","\t\tif err := json.Unmarshal(choicesRaw, &choices); err == nil {","\t\t\tfor i, choice := range choices {","\t\t\t\tif deltaRaw, ok := choice[\"delta\"]; ok {","\t\t\t\t\tvar delta map[string]json.RawMessage","\t\t\t\t\tif err := json.Unmarshal(deltaRaw, &delta); err == nil {","\t\t\t\t\t\tfor _, field := range []string{\"content\", \"reasoning_content\", \"reasoning\", \"refusal\"} {","\t\t\t\t\t\t\tif v, ok := delta[field]; ok && string(v) == \"null\" {","\t\t\t\t\t\t\t\tdelta[field] = json.RawMessage(`\"\"`)","\t\t\t\t\t\t\t\tchanged = true","\t\t\t\t\t\tif v, ok := delta[\"tool_calls\"]; ok && string(v) == \"null\" {","\t\t\t\t\t\t\tdelta[\"tool_calls\"] = json.RawMessage(`[]`)","\t\t\t\t\t\t\tchanged = true","\t\t\t\t\t\t// Emit BOTH \"reasoning\" and \"reasoning_content\" so both","\t\t\t\t\t\t// AI SDK (reads reasoning_content) and ForgeCode/other","\t\t\t\t\t\t// clients (reads reasoning) see reasoning tokens.","\t\t\t\t\t\tif _, hasR := delta[\"reasoning\"]; hasR {","\t\t\t\t\t\t\tif _, hasRC := delta[\"reasoning_content\"]; !hasRC {","\t\t\t\t\t\t\t\t// Only reasoning exists — copy to reasoning_content for AI SDK.","\t\t\t\t\t\t\t\tdelta[\"reasoning_content\"] = delta[\"reasoning\"]","\t\t\t\t\t\t} else if rc, hasRC := delta[\"reasoning_content\"]; hasRC {","\t\t\t\t\t\t\t// Only reasoning_content exists — add reasoning alias.","\t\t\t\t\t\t\tdelta[\"reasoning\"] = rc","\t\t\t\t\t\tif changed {","\t\t\t\t\t\t\tchoices[i][\"delta\"], _ = json.Marshal(delta)","\t\t\tif changed {","\t\t\t\traw[\"choices\"], _ = json.Marshal(choices)","\tout, err := json.Marshal(raw)","\treturn \"data: \" + string(out)","// extractedMessage holds the reconstructed assistant message from SSE chunks,","// including text content, reasoning, and any tool calls.","type extractedMessage struct {","\tContent string `json:\"content\"`","\tReasoning string `json:\"reasoning,omitempty\"`","\tToolCalls []map[string]any `json:\"tool_calls,omitempty\"`","\tFinishReason string `json:\"-\"`","// extractMessage parses SSE data lines and reconstructs the full assistant","// message from streaming chunks, including content, reasoning, and tool_calls.","func extractMessage(chunks []string) extractedMessage {","\tvar contentBuilder strings.Builder","\tvar reasoningBuilder strings.Builder","\tfinishReason := \"\"","\t// Tool calls are indexed — accumulate argument fragments by index.","\ttoolCallMap := map[int]map[string]any{}","\tfor _, chunk := range chunks {","\t\tline := strings.TrimPrefix(chunk, \"data: \")","\t\tline = strings.TrimSpace(line)","\t\tif line == \"\" || line == \"[DONE]\" {","\t\tvar parsed map[string]json.RawMessage","\t\tif err := json.Unmarshal([]byte(line), &parsed); err != nil {","\t\tchoicesRaw, ok := parsed[\"choices\"]","\t\tvar choices []struct {","\t\t\tDelta struct {","\t\t\t\tContent string `json:\"content\"`","\t\t\t\tReasoning string `json:\"reasoning\"`","\t\t\t\tReasoningContent string `json:\"reasoning_content\"`","\t\t\t\tToolCalls []struct {","\t\t\t\t\tIndex int `json:\"index\"`","\t\t\t\t\tID string `json:\"id,omitempty\"`","\t\t\t\t\tType string `json:\"type,omitempty\"`","\t\t\t\t\tFunction struct {","\t\t\t\t\t\tName string `json:\"name,omitempty\"`","\t\t\t\t\t\tArguments string `json:\"arguments,omitempty\"`","\t\t\t\t\t} `json:\"function,omitempty\"`","\t\t\t\t} `json:\"tool_calls,omitempty\"`","\t\t\t} `json:\"delta\"`","\t\t\tMessage struct {","\t\t\t} `json:\"message\"`","\t\t\tFinishReason *string `json:\"finish_reason\"`","\t\tif err := json.Unmarshal(choicesRaw, &choices); err != nil {","\t\tfor _, c := range choices {","\t\t\tif c.FinishReason != nil && *c.FinishReason != \"\" {","\t\t\t\tfinishReason = *c.FinishReason","\t\t\tif c.Delta.Content != \"\" {","\t\t\t\tcontentBuilder.WriteString(c.Delta.Content)","\t\t\t} else if c.Message.Content != \"\" {","\t\t\t\tcontentBuilder.WriteString(c.Message.Content)","\t\t\tif c.Delta.Reasoning != \"\" {","\t\t\t\treasoningBuilder.WriteString(c.Delta.Reasoning)","\t\t\t} else if c.Delta.ReasoningContent != \"\" {","\t\t\t\treasoningBuilder.WriteString(c.Delta.ReasoningContent)","\t\t\t} else if c.Message.Reasoning != \"\" {","\t\t\t\treasoningBuilder.WriteString(c.Message.Reasoning)","\t\t\t} else if c.Message.ReasoningContent != \"\" {","\t\t\t\treasoningBuilder.WriteString(c.Message.ReasoningContent)","\t\t\tfor _, tc := range c.Delta.ToolCalls {","\t\t\t\texisting, ok := toolCallMap[tc.Index]","\t\t\t\t\texisting = map[string]any{","\t\t\t\t\t\t\"index\": tc.Index,","\t\t\t\t\t\t\"function\": map[string]any{","\t\t\t\t\t\t\t\"arguments\": \"\",","\t\t\t\t\t\t},","\t\t\t\t\ttoolCallMap[tc.Index] = existing","\t\t\t\tif tc.ID != \"\" {","\t\t\t\t\texisting[\"id\"] = tc.ID","\t\t\t\tif tc.Type != \"\" {","\t\t\t\t\texisting[\"type\"] = tc.Type","\t\t\t\tfn := existing[\"function\"].(map[string]any)","\t\t\t\tif tc.Function.Name != \"\" {","\t\t\t\t\tfn[\"name\"] = tc.Function.Name","\t\t\t\tfn[\"arguments\"] = fn[\"arguments\"].(string) + tc.Function.Arguments","\tcontent := contentBuilder.String()","\treasoning := reasoningBuilder.String()","\tif cleaned, extractedReasoning := stripThinkBlocks(content); extractedReasoning != \"\" {","\t\tcontent = cleaned","\t\tif strings.TrimSpace(reasoning) != \"\" {","\t\t\treasoning += \"\\n\\n\" + extractedReasoning","\t\t\treasoning = extractedReasoning","\tmsg := extractedMessage{Content: content, Reasoning: reasoning, FinishReason: finishReason}","\tif len(toolCallMap) > 0 {","\t\tmsg.ToolCalls = make([]map[string]any, 0, len(toolCallMap))","\t\tfor i := range len(toolCallMap) {","\t\t\tif tc, ok := toolCallMap[i]; ok {","\t\t\t\tdelete(tc, \"index\")","\t\t\t\tmsg.ToolCalls = append(msg.ToolCalls, tc)","\treturn msg","// resolveReasoningTokens returns the reasoning-token count to report.","// It prefers the provider's tokenizer-accurate count","// (UsageInfo.ReasoningTokens) and falls back to the coarse \"all","// completion tokens\" estimate only for older providers that emit","// reasoning content without a count — so a reasoning response never","// reports zero reasoning tokens, while up-to-date providers report the","// real split.","// injectReasoningDetailIntoRawUsage splices","// completion_tokens_details.reasoning_tokens into a passthrough","// chat.completion object when the provider reported an accurate","// reasoning-token count (UsageInfo.ReasoningTokens) and the raw usage","// object didn't already carry the detail. It never overrides a value the","// provider already supplied, and is a no-op when there is no reasoning","// count or no usage object.","func injectReasoningDetailIntoRawUsage(obj map[string]any, usage protocol.UsageInfo) {","\tif usage.ReasoningTokens <= 0 {","\tusageObj, ok := obj[\"usage\"].(map[string]any)","\tdetails, _ := usageObj[\"completion_tokens_details\"].(map[string]any)","\tif details == nil {","\t\tdetails = map[string]any{}","\tif _, exists := details[\"reasoning_tokens\"]; exists {","\tdetails[\"reasoning_tokens\"] = usage.ReasoningTokens","\tusageObj[\"completion_tokens_details\"] = details","\tobj[\"usage\"] = usageObj","// parseUsageOnlyStreamChunk decodes a terminal include_usage chunk (empty choices","// + a non-null usage object, carrying the final usage and no content delta) and","// returns the parsed object. ok is false for any other chunk. Parsing here once","// lets the caller hold the object and finalize it at stream end without re-parsing.","// isSSEDoneChunk reports whether a provider stream chunk is the SSE","// \"data: [DONE]\" terminator (with or without the data: prefix). The","// coordinator owns stream termination — provider terminators are swallowed","// so coordinator-appended events (held usage, SE signature) never trail a","// [DONE] that SDKs treat as final.","func isSSEDoneChunk(chunk string) bool {","\tline := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(chunk), \"data:\"))","\treturn line == \"[DONE]\"","// isResponsesAPIEventChunk reports whether a streamed chunk is a Responses API","// SSE event (its parsed top-level \"type\" is a \"response.*\" event). It parses","// rather than substring-matches: a chat.completion content delta whose text","// quotes \"response.created\"/\"response.output_text.delta\" (e.g. a user asking","// about the Responses API) must NOT be misread as a Responses stream, which","// would make the relay skip chat-completions termination handling (usage","// splicing, [DONE] swallowing, normalizeSSEChunk) and corrupt the stream.","func isResponsesAPIEventChunk(chunk string) bool {","\t// Cheap gate: every Responses event names a response.* type at top level.","\tif !strings.Contains(line, `\"response.`) {","\tvar ev struct {","\t\tType string `json:\"type\"`","\tif err := json.Unmarshal([]byte(line), &ev); err != nil {","\treturn strings.HasPrefix(ev.Type, \"response.\")","// isBoilerplateChunk reports whether a streamed provider chunk carries no","// consumer-visible output yet: the preamble emitted BEFORE the failure-prone","// work (media decode, template render, vision prefill) begins. The dispatch","// loop holds such chunks instead of committing on them, so a provider that","// dies after its preamble is retried invisibly instead of surfacing an","// in-band SSE error with zero retries.","// Boilerplate is exactly:","// - a chat.completion.chunk whose choices[].delta carries ONLY the assistant","// role — content/reasoning/refusal absent, null, or \"\" (some backends ride","// an empty content along with the role), tool_calls absent/null/empty,","// finish_reason null, no usage object; or","// - a Responses API response.created / response.in_progress lifecycle event","// (the parsed top-level \"type\" equals exactly one of those — NOT a mere","// substring match: a chat content delta whose text quotes \"response.created\"","// must still commit).","// Everything else — content or tool_call deltas, finish chunks, usage-only","// chunks, [DONE], complete responses, unparseable data — commits the dispatch.","func isBoilerplateChunk(chunk string) bool {","\tline := strings.TrimPrefix(strings.TrimPrefix(chunk, \"data: \"), \"data:\")","\tline = strings.TrimSpace(line)","\t// Responses API lifecycle preamble: classify ONLY when the parsed top-level","\t// \"type\" is exactly response.created / response.in_progress. A chat content","\t// delta that merely mentions that text (e.g. a user asking about the","\t// Responses API) parses as a chat.completion.chunk and falls through to the","\t// role-only logic below — it is NOT boilerplate.","\tif strings.Contains(line, `\"response.created\"`) || strings.Contains(line, `\"response.in_progress\"`) {","\t\tvar ev struct {","\t\t\tType string `json:\"type\"`","\t\tif err := json.Unmarshal([]byte(line), &ev); err == nil {","\t\t\tif ev.Type == \"response.created\" || ev.Type == \"response.in_progress\" {","\t// Cheap gate: the role preamble always names the role; chunks that can't","\t// be it (content deltas, finish chunks, [DONE], garbage) skip the parse.","\tif !strings.Contains(line, `\"role\"`) {","\tvar parsed struct {","\t\tObject string `json:\"object\"`","\t\tUsage json.RawMessage `json:\"usage\"`","\t\tChoices []struct {","\t\t\tDelta map[string]json.RawMessage `json:\"delta\"`","\t\t\tFinishReason *string `json:\"finish_reason\"`","\t\t} `json:\"choices\"`","\tif err := json.Unmarshal([]byte(line), &parsed); err != nil {","\tif parsed.Object != \"chat.completion.chunk\" {","\tif len(parsed.Usage) > 0 && string(parsed.Usage) != \"null\" {","\tif len(parsed.Choices) == 0 {","\tfor _, choice := range parsed.Choices {","\t\tif choice.FinishReason != nil {","\t\tif _, hasRole := choice.Delta[\"role\"]; !hasRole {","\t\tfor field, v := range choice.Delta {","\t\t\tswitch field {","\t\t\tcase \"role\":","\t\t\t\t// The preamble itself.","\t\t\tcase \"content\", \"reasoning_content\", \"reasoning\", \"refusal\":","\t\t\t\tif s := string(v); s != `\"\"` && s != \"null\" {","\t\t\t\t\treturn false","\t\t\tcase \"tool_calls\":","\t\t\t\tif s := string(v); s != \"null\" && s != \"[]\" {","\t\t\t\t// Unknown delta payload — assume it's real output.","\t\t\t\treturn false","func parseUsageOnlyStreamChunk(chunk string) (obj map[string]any, ok bool) {","\t// Cheap gate: skip the parse for content deltas and usage:null chunks.","\tif !strings.Contains(line, `\"usage\"`) || strings.Contains(line, `\"usage\":null`) {","\t\treturn nil, false","\tif err := json.Unmarshal([]byte(line), &obj); err != nil {","\tif u, uok := obj[\"usage\"].(map[string]any); !uok || u == nil {","\tif choices, _ := obj[\"choices\"].([]any); len(choices) != 0 {","\treturn obj, true","// finalizeUsageChunk renders the held terminal usage chunk for chat-completions","// streaming: it splices the provider's authoritative reasoning count into","// completion_tokens_details (no-op when there is none), strips a null","// system_fingerprint, and rewrites the build id to the public alias — marshalling","// ONCE (obj is already parsed). Returns \"\" if it can't be marshalled.","func finalizeUsageChunk(obj map[string]any, usage protocol.UsageInfo, pr *registry.PendingRequest) string {","\tinjectReasoningDetailIntoRawUsage(obj, usage)","\tif v, present := obj[\"system_fingerprint\"]; present && v == nil {","\t\tdelete(obj, \"system_fingerprint\")","\tif pr.PublicModel != \"\" && pr.PublicModel != pr.Model {","\t\tobj[\"model\"] = pr.PublicModel","\tb, err := json.Marshal(obj)","\treturn \"data: \" + string(b)","// parseFinishStreamChunk decodes a chunk whose choices carry a non-null","// finish_reason (the terminal content chunk). ok is false for any other","// chunk. The parsed object is held by the caller and finalized at stream end","// once the authoritative token counts are known.","func parseFinishStreamChunk(chunk string) (map[string]any, bool) {","\tif !strings.Contains(line, `\"finish_reason\":\"`) {","\tvar obj map[string]any","\tchoices, _ := obj[\"choices\"].([]any)","\tfor _, c := range choices {","\t\tif m, ok := c.(map[string]any); ok {","\t\t\tif fr, _ := m[\"finish_reason\"].(string); fr != \"\" {","\t\t\t\treturn obj, true","\treturn nil, false","// finalizeFinishChunk renders the held terminal finish chunk: when the","// authoritative completion-token count shows generation hit the max-tokens","// bound, a provider-reported \"stop\" is corrected to \"length\" (the engine","// doesn't distinguish natural stop from truncation). Also rewrites the build","// id to the public alias. Returns \"\" if it can't be marshalled.","func finalizeFinishChunk(obj map[string]any, usage protocol.UsageInfo, pr *registry.PendingRequest) string {","\tif truncatedByMaxTokens(usage, pr.RequestedMaxTokens) {","\t\tif choices, ok := obj[\"choices\"].([]any); ok {","\t\t\tfor _, c := range choices {","\t\t\t\tif m, ok := c.(map[string]any); ok {","\t\t\t\t\tif fr, _ := m[\"finish_reason\"].(string); fr == \"stop\" {","\t\t\t\t\t\tm[\"finish_reason\"] = \"length\"","// truncatedByMaxTokens reports whether generation consumed the entire","// max-tokens budget. requestedMax is the effective bound — the consumer's","// explicit max_tokens or the coordinator-injected default — so hitting it","// means the engine cut generation short.","func truncatedByMaxTokens(usage protocol.UsageInfo, requestedMax int) bool {","\treturn requestedMax > 0 && usage.CompletionTokens >= requestedMax","// effectiveFinishReason resolves the finish_reason for a reconstructed","// response. The provider engine reports \"stop\" unconditionally, so a","// truncation-aware reason is re-derived from the authoritative token counts.","func effectiveFinishReason(extracted string, hasToolCalls bool, usage protocol.UsageInfo, requestedMax int) string {","\tif extracted != \"\" && extracted != \"stop\" {","\t\treturn extracted","\tif truncatedByMaxTokens(usage, requestedMax) {","\t\treturn \"length\"","\tif hasToolCalls {","\t\treturn \"tool_calls\"","\treturn \"stop\"","func resolveReasoningTokens(usage protocol.UsageInfo, reasoning string) uint64 {","\tif usage.ReasoningTokens > 0 {","\t\treturn uint64(usage.ReasoningTokens)","\tif reasoning != \"\" {","\t\treturn uint64(usage.CompletionTokens)","func buildResponsesUsage(promptTokens, completionTokens uint64, reasoningTokens uint64) types.ResponsesUsage {","\treturn types.ResponsesUsage{","\t\tInputTokens: int(promptTokens),","\t\tInputTokensDetail: types.ResponsesUsageDetail{},","\t\tOutputTokens: int(completionTokens),","\t\tOutputTokensDetail: types.ResponsesUsageDetail{ReasoningTokens: int(reasoningTokens)},","func buildResponsesIncompleteDetails(finishReason string) *types.ResponsesIncompleteDetail {","\tswitch finishReason {","\tcase \"length\":","\t\treturn &types.ResponsesIncompleteDetail{Reason: \"max_output_tokens\"}","\tcase \"content_filter\":","\t\treturn &types.ResponsesIncompleteDetail{Reason: \"content_filter\"}","func responseItemID(prefix, requestID string, index int) string {","\treturn fmt.Sprintf(\"%s_%s_%d\", prefix, strings.ReplaceAll(requestID, \"-\", \"\"), index)","func appendResponsesOutputItems(output []any, requestID string, msg extractedMessage) []any {","\tindex := len(output)","\tif msg.Reasoning != \"\" {","\t\toutput = append(output, map[string]any{","\t\t\t\"type\": \"reasoning\",","\t\t\t\"id\": responseItemID(\"rs\", requestID, index),","\t\t\t\"summary\": []map[string]any{{","\t\t\t\t\"type\": \"summary_text\",","\t\t\t\t\"text\": msg.Reasoning,","\t\t\t}},","\t\tindex++","\tif msg.Content != \"\" || len(msg.ToolCalls) == 0 {","\t\t\t\"type\": \"message\",","\t\t\t\"role\": \"assistant\",","\t\t\t\"status\": \"completed\",","\t\t\t\"id\": responseItemID(\"msg\", requestID, index),","\t\t\t\"content\": []map[string]any{{","\t\t\t\t\"type\": \"output_text\",","\t\t\t\t\"text\": msg.Content,","\t\t\t\t\"annotations\": []any{},","\tfor _, tc := range msg.ToolCalls {","\t\tfn, _ := tc[\"function\"].(map[string]any)","\t\tcallID, _ := tc[\"id\"].(string)","\t\tif callID == \"\" {","\t\t\tcallID = responseItemID(\"call\", requestID, index)","\t\tname, _ := fn[\"name\"].(string)","\t\targs, _ := fn[\"arguments\"].(string)","\t\t\t\"type\": \"function_call\",","\t\t\t\"id\": responseItemID(\"fc\", requestID, index),","\t\t\t\"call_id\": callID,","\t\t\t\"name\": name,","\t\t\t\"arguments\": args,","\t\t\t\"status\": \"completed\",","\treturn output","// finalizeResponsesEnvelope fills the spec-required envelope fields of a","// Responses object: status derived from incomplete_details, and the","// always-present defaults (tool_choice, tools, metadata, parallel_tool_calls).","func finalizeResponsesEnvelope(r *types.ResponsesResponse) {","\tif r.IncompleteDetail != nil {","\t\tr.Status = \"incomplete\"","\t\tr.Status = \"completed\"","\tr.ParallelToolCalls = true","\tif r.ToolChoice == nil {","\t\tr.ToolChoice = \"auto\"","\tif r.Tools == nil {","\t\tr.Tools = []any{}","\tif r.Metadata == nil {","\t\tr.Metadata = map[string]any{}","func buildResponsesResponse(requestID, model string, msg extractedMessage, usage protocol.UsageInfo, requestedMax int, seSignature, responseHash string) types.ResponsesResponse {","\treasoningTokens := resolveReasoningTokens(usage, msg.Reasoning)","\tfinishReason := effectiveFinishReason(msg.FinishReason, len(msg.ToolCalls) > 0, usage, requestedMax)","\tresp := types.ResponsesResponse{","\t\tID: \"resp_\" + strings.ReplaceAll(requestID, \"-\", \"\"),","\t\tObject: \"response\",","\t\tCreatedAt: time.Now().Unix(),","\t\tModel: model,","\t\tOutput: appendResponsesOutputItems(nil, requestID, msg),","\t\tUsage: buildResponsesUsage(uint64(usage.PromptTokens), uint64(usage.CompletionTokens), reasoningTokens),","\t\tIncompleteDetail: buildResponsesIncompleteDetails(finishReason),","\tfinalizeResponsesEnvelope(&resp)","\tif seSignature != \"\" {","\t\tresp.SESignature = seSignature","\t\tresp.ResponseHash = responseHash","\treturn resp","func firstChoice(resp types.ChatCompletionResponse) *types.ChatCompletionChoice {","\tif len(resp.Choices) == 0 {","\treturn &resp.Choices[0]","func chatUsageToResponsesUsage(resp types.ChatCompletionResponse, reasoning string) types.ResponsesUsage {","\treasoningTokens := 0","\tif d := resp.Usage.CompletionTokensDetails; d != nil && d.ReasoningTokens > 0 {","\t\treasoningTokens = d.ReasoningTokens","\t} else if reasoning != \"\" {","\t\treasoningTokens = resp.Usage.CompletionTokens","\treturn buildResponsesUsage(uint64(resp.Usage.PromptTokens), uint64(resp.Usage.CompletionTokens), uint64(reasoningTokens))","func chatCompletionToResponses(resp types.ChatCompletionResponse, requestedModel, seSignature, responseHash string) types.ResponsesResponse {","\trequestID := strings.TrimPrefix(resp.ID, \"chatcmpl-\")","\tif requestID == \"\" {","\t\trequestID = uuid.NewString()","\tcreated := int(resp.Created)","\tif created <= 0 {","\t\tcreated = int(time.Now().Unix())","\tmsg := extractedMessage{}","\tif choice := firstChoice(resp); choice != nil {","\t\tfinishReason = choice.FinishReason","\t\tmsg.Content = choice.Message.Content","\t\tmsg.Reasoning = choice.Message.Reasoning","\t\tmsg.ToolCalls = choice.Message.ToolCalls","\tr := types.ResponsesResponse{","\t\tID: \"resp_\" + strings.ReplaceAll(requestID, \"-\", \"\"),","\t\tObject: \"response\",","\t\tCreatedAt: int64(created),","\t\tModel: requestedModel,","\t\tOutput: appendResponsesOutputItems(nil, requestID, msg),","\t\tUsage: chatUsageToResponsesUsage(resp, msg.Reasoning),","\tif finishReason != \"\" && finishReason != \"stop\" {","\t\tr.IncompleteDetail = buildResponsesIncompleteDetails(finishReason)","\tfinalizeResponsesEnvelope(&r)","\t\tr.SESignature = seSignature","\t\tr.ResponseHash = responseHash","\treturn r","func buildNonStreamingResponse(requestID, model string, msg extractedMessage, usage protocol.UsageInfo, requestedMax int, seSignature, responseHash string) types.ChatCompletionResponse {","\tmessage := types.ChatCompletionMessage{","\t\tRole: \"assistant\",","\t\tContent: msg.Content,","\t\tmessage.Reasoning = msg.Reasoning","\tif len(msg.ToolCalls) > 0 {","\t\tmessage.ToolCalls = msg.ToolCalls","\tresp := types.ChatCompletionResponse{","\t\tID: \"chatcmpl-\" + requestID,","\t\tObject: \"chat.completion\",","\t\tCreated: time.Now().Unix(),","\t\tModel: model,","\t\tChoices: []types.ChatCompletionChoice{{","\t\t\tIndex: 0,","\t\t\tMessage: message,","\t\t\tFinishReason: finishReason,","\t\tUsage: types.ChatCompletionUsage{","\t\t\tPromptTokens: usage.PromptTokens,","\t\t\tCompletionTokens: usage.CompletionTokens,","\t\t\tTotalTokens: usage.PromptTokens + usage.CompletionTokens,","\t// Surface the OpenAI-standard reasoning-token breakdown when present","\t// so non-streaming chat-completions consumers can read it (the","\t// streaming path carries it on the provider's verbatim usage chunk).","\tif rt := resolveReasoningTokens(usage, msg.Reasoning); rt > 0 {","\t\tresp.Usage.CompletionTokensDetails = &types.CompletionTokensDetails{","\t\t\tReasoningTokens: int(rt),","// handleListModels handles GET /v1/models.","// Returns a deduplicated list of models across all connected providers,","// including attestation metadata (trust level, Secure Enclave status,","// provider count) for each model. Capacity fields (routable_providers,","// warm_providers, can_accept) are included from the live capacity snapshot.","// hideAliasBuild marks a concrete build id as hidden from the standalone model","// listing — a build behind a public alias is only ever exposed through that","// alias. Only builds that are actually in the catalog are hidden: a build absent","// from the catalog can't appear in the listing anyway, and adding it would","// pollute the hidden set (e.g. an alias pointing at a not-yet-registered build).","// Empty ids are ignored.","func hideAliasBuild(hidden map[string]struct{}, catalogByID map[string]store.SupportedModel, buildID string) {","\tif buildID == \"\" {","\tif _, inCatalog := catalogByID[buildID]; inCatalog {","\t\thidden[buildID] = struct{}{}","// aliasModelEntries builds the consumer-facing /v1/models entries for active","// public aliases and returns the set of underlying build ids those aliases","// cover (so the caller can hide them from the default listing). The hidden set","// covers EVERY build an alias references — desired, previous, and the retired","// lineage — so a concrete quant build never appears as its own entry once it is","// behind an alias. Each alias entry derives its metadata from its primary build","// — the desired build, or the previous build if the desired one isn't in the","// catalog yet — and aggregates live capacity across the desired and previous","// builds so the alias's routable/warm counts reflect every quant currently","// serving it (retired builds are hide-only and never contribute capacity).","func (s *Server) aliasModelEntries(","\tcapByModel map[string]*registry.ModelCapacity,","\tcatalogByID map[string]store.SupportedModel,","\tregistryByID map[string]store.ModelRegistryEntry,",") ([]types.ModelEntry, map[string]struct{}) {","\thidden := make(map[string]struct{})","\taliases, err := s.store.ListModelAliases()","\t\ts.logger.Error(\"model registry: failed to list aliases\", \"error\", err)","\t\treturn nil, hidden","\tentries := make([]types.ModelEntry, 0, len(aliases))","\tfor _, a := range aliases {","\t\tif !a.Active || a.DesiredBuild == \"\" {","\t\t// A consumer must only ever see the alias, never a concrete build behind","\t\t// it. Hide EVERY build this alias references — desired, previous, AND the","\t\t// retired lineage — from the standalone listing, even if the alias itself","\t\t// isn't advertisable right now. (Capacity below aggregates only the","\t\t// routable desired/previous members; retired builds are hide-only.)","\t\thideAliasBuild(hidden, catalogByID, a.DesiredBuild)","\t\thideAliasBuild(hidden, catalogByID, a.PreviousBuild)","\t\tfor _, b := range a.RetiredBuilds {","\t\t\thideAliasBuild(hidden, catalogByID, b)","\t\t// Primary build = the desired build when it's in the catalog, else the","\t\t// previous build (so the alias keeps a real entry while the desired build","\t\t// is mid-registration). An alias whose builds are all out of catalog","\t\t// resolves to nothing and must not be advertised (it would 503).","\t\tmembers := make([]string, 0, 2)","\t\tdesiredInCatalog := false","\t\tif _, ok := catalogByID[a.DesiredBuild]; ok {","\t\t\tmembers = append(members, a.DesiredBuild)","\t\t\tdesiredInCatalog = true","\t\tpreviousInCatalog := false","\t\tif a.PreviousBuild != \"\" {","\t\t\tif _, ok := catalogByID[a.PreviousBuild]; ok {","\t\t\t\tmembers = append(members, a.PreviousBuild)","\t\t\t\tpreviousInCatalog = true","\t\tvar primary string","\t\tswitch {","\t\tcase desiredInCatalog:","\t\t\tprimary = a.DesiredBuild","\t\tcase previousInCatalog:","\t\t\tprimary = a.PreviousBuild","\t\t\t// No in-catalog build backs this alias — don't advertise it.","\t\troutable, warm := 0, 0","\t\tcanAccept := false","\t\tfor _, b := range members {","\t\t\tif cap, ok := capByModel[b]; ok {","\t\t\t\troutable += cap.RoutableProviders","\t\t\t\twarm += cap.WarmProviders","\t\t\t\tcanAccept = canAccept || cap.CanAccept","\t\tcm := catalogByID[primary]","\t\treg, hasReg := registryByID[primary]","\t\tdisplayName := a.DisplayName","\t\tif displayName == \"\" {","\t\t\tdisplayName = cm.DisplayName","\t\tmetadata := types.ModelMetadata{","\t\t\tModelType: cm.ModelType,","\t\t\tQuantization: \"\", // an alias spans quants; omit the per-build quant","\t\t\tDisplayName: displayName,","\t\t\tRoutableProviders: routable,","\t\t\tWarmProviders: warm,","\t\t\tCanAccept: canAccept,","\t\tentry := types.ModelEntry{","\t\t\tID: a.AliasID,","\t\t\tObject: \"model\",","\t\t\tOwnedBy: \"eigeninference\",","\t\t\tName: displayName,","\t\t\tMetadata: metadata,","\t\t// Pricing / context / features come from the primary build's registry","\t\t// entry. Quantization is intentionally left blank on the alias.","\t\tprimaryQuant := \"\"","\t\tif hasReg {","\t\t\tprimaryQuant = reg.Quantization","\t\ts.openRouterModelFieldsFor(primary, primaryQuant, reg, hasReg).applyToModelEntry(&entry)","\t\tentry.Quantization = \"\"","\t\tvar caps []string","\t\t\tcaps = reg.Capabilities","\t\tentry.InputModalities, entry.OutputModalities = deriveModalities(cm.ModelType, caps)","\t\tentries = append(entries, entry)","\treturn entries, hidden","// listModelEntries assembles the consumer-facing model entries shared by","// GET /v1/models and GET /v1/models/{id}. includeBuilds also lists the raw","// quant builds hidden behind public aliases (ops/debug).","func (s *Server) listModelEntries(includeBuilds bool) ([]types.ModelEntry, error) {","\tmodels := s.registry.ListModels()","\t// Build a lookup of capacity data keyed by model ID.","\tcapacities := s.registry.ModelCapacitySnapshot()","\tcapByModel := make(map[string]*registry.ModelCapacity, len(capacities))","\tfor i := range capacities {","\t\tcapByModel[capacities[i].ModelID] = &capacities[i]","\t// Filter to only show models from the active catalog, and capture the richer","\t// registry entries used to populate the OpenRouter provider fields. These","\t// lookups are shared with the dedicated /v1/models/openrouter feed.","\tcatalogByID, registryByID, err := s.activeCatalogLookups()","\t// Public aliases are the consumer-facing model names; their underlying","\t// quant builds are hidden by default so consumers never see the quant.","\taliasEntries, hiddenBuilds := s.aliasModelEntries(capByModel, catalogByID, registryByID)","\tdata := make([]types.ModelEntry, 0, len(models)+len(aliasEntries))","\tdata = append(data, aliasEntries...)","\tfor _, m := range models {","\t\tcm, inCatalog := catalogByID[m.ID]","\t\tif len(catalogByID) > 0 && !inCatalog {","\t\tif _, hidden := hiddenBuilds[m.ID]; hidden && !includeBuilds {","\t\t\tModelType: m.ModelType,","\t\t\tQuantization: m.Quantization,","\t\t\tProviderCount: m.Providers,","\t\t\tAttestedProviders: m.AttestedProviders,","\t\t\tTrustLevel: string(m.TrustLevel),","\t\t// Add capacity fields from live snapshot.","\t\tif cap, ok := capByModel[m.ID]; ok {","\t\t\tmetadata.RoutableProviders = cap.RoutableProviders","\t\t\tmetadata.WarmProviders = cap.WarmProviders","\t\t\tmetadata.CanAccept = cap.CanAccept","\t\t\tmetadata.RoutableProviders = 0","\t\t\tmetadata.WarmProviders = 0","\t\t\tmetadata.CanAccept = false","\t\tif m.Attestation != nil {","\t\t\tmetadata.Attestation = &types.ModelAttestation{","\t\t\t\tSecureEnclave: m.Attestation.SecureEnclave,","\t\t\t\tSIPEnabled: m.Attestation.SIPEnabled,","\t\t\t\tSecureBoot: m.Attestation.SecureBoot,","\t\tif inCatalog && cm.DisplayName != \"\" {","\t\t\tmetadata.DisplayName = cm.DisplayName","\t\t\tID: m.ID,","\t\t\tObject: \"model\",","\t\t\tCreated: 0,","\t\t\tOwnedBy: \"eigeninference\",","\t\t\tName: metadata.DisplayName,","\t\t\tHuggingFaceID: m.ID, // model IDs are HuggingFace paths","\t\t\tMetadata: metadata,","\t\t// OpenRouter provider fields (quantization, per-token pricing, sampling","\t\t// params, and registry-sourced metadata), shared with the dedicated","\t\t// /v1/models/openrouter feed.","\t\treg, hasReg := registryByID[m.ID]","\t\ts.openRouterModelFieldsFor(m.ID, m.Quantization, reg, hasReg).applyToModelEntry(&entry)","\t\t// Modalities are derived from the model's capabilities (text by default).","\t\tentry.InputModalities, entry.OutputModalities = deriveModalities(m.ModelType, caps)","\t\tdata = append(data, entry)","\treturn data, nil","func (s *Server) handleListModels(w http.ResponseWriter, r *http.Request) {","\t// Pass ?include_builds=1 (ops/debug) to also list the raw quant builds.","\tdata, err := s.listModelEntries(r.URL.Query().Get(\"include_builds\") == \"1\")","\t\ts.logger.Error(\"model registry: failed to list active models\", \"error\", err)","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"internal_error\", \"failed to list models\"))","\twriteJSON(w, http.StatusOK, types.ModelListResponse{","\t\tObject: \"list\",","\t\tData: data,","// handleGetModel handles GET /v1/models/{id...} — the OpenAI \"retrieve model\"","// endpoint. Model IDs may contain slashes (HuggingFace paths), hence the","// wildcard path segment. Hidden quant builds are retrievable by their exact","// id, matching the behavior of requesting one for inference.","func (s *Server) handleGetModel(w http.ResponseWriter, r *http.Request) {","\tid := r.PathValue(\"id\")","\tdata, err := s.listModelEntries(true)","\tfor _, entry := range data {","\t\tif entry.ID == id {","\t\t\twriteJSON(w, http.StatusOK, entry)","\twriteJSON(w, http.StatusNotFound, errorResponse(\"model_not_found\",","\t\tfmt.Sprintf(\"model %q not found\", id), withParam(\"model\")))","// handleCreateKey handles POST /v1/auth/keys — creates a new consumer API key.","// Requires Privy authentication. The key is linked to the user's account so","// requests made with the key are billed to the same account.","func (s *Server) handleCreateKey(w http.ResponseWriter, r *http.Request) {","\tuser := auth.UserFromContext(r.Context())","\tif user == nil {","\t\twriteJSON(w, http.StatusUnauthorized, errorResponse(\"auth_error\",","\t\t\t\"API key creation requires a Privy account — authenticate with a Privy access token\"))","\tkey, err := s.store.CreateKeyForAccount(user.AccountID)","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"server_error\", \"failed to create key\"))","\twriteJSON(w, http.StatusOK, types.CreateKeyResponse{","\t\tAPIKey: key,","\t\tAccountID: user.AccountID,","// handleRevokeKey handles DELETE /v1/auth/keys — revokes an API key.","// The caller must own the key (same account). Requires Privy auth so a","// compromised API key cannot revoke legitimate keys.","func (s *Server) handleRevokeKey(w http.ResponseWriter, r *http.Request) {","\t\twriteJSON(w, http.StatusUnauthorized, errorResponse(\"auth_error\", \"authentication required\"))","\tvar body struct {","\t\tKey string `json:\"key\"`","\tif err := json.NewDecoder(r.Body).Decode(&body); err != nil || body.Key == \"\" {","\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"bad_request\", \"provide {\\\"key\\\": \\\"sk-db-...\\\"}\"))","\towner := s.store.GetKeyAccount(body.Key)","\tif owner != user.AccountID {","\t\twriteJSON(w, http.StatusForbidden, errorResponse(\"forbidden\", \"you can only revoke your own keys\"))","\tif !s.store.RevokeKey(body.Key) {","\t\twriteJSON(w, http.StatusNotFound, errorResponse(\"not_found\", \"key not found or already revoked\"))","\ts.invalidateAPIKeyCache(body.Key)","\twriteJSON(w, http.StatusOK, types.RevokeKeyResponse{Status: \"revoked\"})","// ── Multi-key management handlers (GET/POST/PATCH/DELETE /v1/keys) ────","// createAPIKeyRequest is the POST /v1/keys (and rotate inherit) body. Money is","// supplied in USD; the wire never sees the secret after the create response.","type createAPIKeyRequest struct {","\tName string `json:\"name\"`","\tLimitUSD *float64 `json:\"limit_usd\"`","\tLimitReset string `json:\"limit_reset\"`","\tRPMLimit *int64 `json:\"rpm_limit\"`","\tITPMLimit *int64 `json:\"itpm_limit\"`","\tOTPMLimit *int64 `json:\"otpm_limit\"`","\tAllowedModels []string `json:\"allowed_models\"`","\tSelfRouteOnly bool `json:\"self_route_only\"`","\tExpiresAt *time.Time `json:\"expires_at\"`","// usdToMicro converts a USD dollar amount to micro-USD (rounded).","func usdToMicro(usd float64) int64 { return int64(math.Round(usd * 1_000_000)) }","// microToUSD converts micro-USD to a USD float.","func microToUSD(micro int64) float64 { return float64(micro) / 1_000_000 }","// apiKeyToResponse projects a stored key into its masked API representation,","// computing the current-window spend and remaining budget.","func (s *Server) apiKeyToResponse(k *store.APIKey) types.APIKeyResponse {","\tresp := types.APIKeyResponse{","\t\tID: k.ID,","\t\tName: k.Name,","\t\tLabel: k.Label,","\t\tDisabled: k.Disabled,","\t\tLimitReset: store.NormalizeResetWindow(k.LimitReset),","\t\tRPMLimit: k.RPMLimit,","\t\tITPMLimit: k.ITPMLimit,","\t\tOTPMLimit: k.OTPMLimit,","\t\tAllowedModels: k.AllowedModels,","\t\tSelfRouteOnly: k.SelfRouteOnly,","\t\tExpiresAt: k.ExpiresAt,","\t\tCreatedAt: k.CreatedAt,","\t\tLastUsedAt: k.LastUsedAt,","\tsince := store.KeySpendWindowStart(resp.LimitReset, time.Now())","\tspent := s.store.KeySpendSince(k.ID, since)","\tresp.UsageUSD = microToUSD(spent)","\tif k.LimitMicroUSD != nil {","\t\tlimitUSD := microToUSD(*k.LimitMicroUSD)","\t\tresp.LimitUSD = &limitUSD","\t\tremaining := *k.LimitMicroUSD - spent","\t\tif remaining < 0 {","\t\t\tremaining = 0","\t\tremUSD := microToUSD(remaining)","\t\tresp.RemainingUSD = &remUSD","// validateKeyLimitInputs sanity-checks user-supplied limit values. Returns a","// human-readable error string (empty when valid).","func validateKeyLimitInputs(reset string, limitUSD *float64, rpm, itpm, otpm *int64, expiresAt *time.Time) string {","\tswitch reset {","\tcase \"\", store.KeyResetNone, store.KeyResetDaily, store.KeyResetWeekly, store.KeyResetMonthly:","\t\treturn \"limit_reset must be one of: none, daily, weekly, monthly\"","\tif limitUSD != nil && *limitUSD < 0 {","\t\treturn \"limit_usd must be >= 0\"","\tif rpm != nil && *rpm < 0 {","\t\treturn \"rpm_limit must be >= 0\"","\tif itpm != nil && *itpm < 0 {","\t\treturn \"itpm_limit must be >= 0\"","\tif otpm != nil && *otpm < 0 {","\t\treturn \"otpm_limit must be >= 0\"","\tif expiresAt != nil && !expiresAt.IsZero() && expiresAt.Before(time.Now()) {","\t\treturn \"expires_at must be in the future\"","\treturn \"\"","// keyModelAllowed returns false when the calling key restricts models via an","// allow-list that does not include the requested model. Account-scoped/legacy","// keys (no key in context) and keys without an allow-list always pass.","func (s *Server) keyModelAllowed(ctx context.Context, model string) bool {","\tk := apiKeyFromContext(ctx)","\tif k == nil || len(k.AllowedModels) == 0 {","\tfor _, m := range k.AllowedModels {","\t\tif m == model {","// checkKeySpendCap reports whether charging additionalMicroUSD to the calling","// key would exceed its per-key spend cap in the current window. It returns","// (message, ok); ok=false means the request must be rejected with 402. The","// per-account balance ledger is still the hard atomic ceiling — this is the","// soft, per-key sub-cap, enforced against settled usage (so concurrent","// in-flight requests are eventually-consistent, never over the account balance).","func (s *Server) checkKeySpendCap(ctx context.Context, additionalMicroUSD int64) (string, bool) {","\tif k == nil || k.ID == \"\" || k.LimitMicroUSD == nil {","\t\treturn \"\", true","\tsince := store.KeySpendWindowStart(k.LimitReset, time.Now())","\tif spent+additionalMicroUSD > *k.LimitMicroUSD {","\t\twindow := store.NormalizeResetWindow(k.LimitReset)","\t\tif window == store.KeyResetNone {","\t\t\twindow = \"total\"","\t\treturn fmt.Sprintf(\"API key spend limit reached (%s cap $%.2f, used $%.2f) — raise this key's limit or use another key\",","\t\t\twindow, microToUSD(*k.LimitMicroUSD), microToUSD(spent)), false","\treturn \"\", true","// handleListAPIKeys handles GET /v1/keys — lists the caller's keys (masked).","func (s *Server) handleListAPIKeys(w http.ResponseWriter, r *http.Request) {","\tkeys, err := s.store.ListAPIKeys(user.AccountID)","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"server_error\", \"failed to list keys\"))","\tout := make([]types.APIKeyResponse, 0, len(keys))","\tfor i := range keys {","\t\tout = append(out, s.apiKeyToResponse(&keys[i]))","\twriteJSON(w, http.StatusOK, types.APIKeyListResponse{Object: \"list\", Data: out})","// handleCreateAPIKey handles POST /v1/keys — mints a new named, optionally","// limited key. The raw secret is returned exactly once.","func (s *Server) handleCreateAPIKey(w http.ResponseWriter, r *http.Request) {","\tvar req createAPIKeyRequest","\tif r.Body != nil {","\t\t// A missing/empty body is allowed (creates a default unnamed key).","\t\tif err := json.NewDecoder(r.Body).Decode(&req); err != nil && !errors.Is(err, io.EOF) {","\t\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"bad_request\", \"invalid JSON body\"))","\tif msg := validateKeyLimitInputs(req.LimitReset, req.LimitUSD, req.RPMLimit, req.ITPMLimit, req.OTPMLimit, req.ExpiresAt); msg != \"\" {","\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"bad_request\", msg))","\topts := store.APIKeyCreate{","\t\tName: strings.TrimSpace(req.Name),","\t\tLimitReset: store.NormalizeResetWindow(req.LimitReset),","\t\tRPMLimit: req.RPMLimit,","\t\tITPMLimit: req.ITPMLimit,","\t\tOTPMLimit: req.OTPMLimit,","\t\tAllowedModels: req.AllowedModels,","\t\tSelfRouteOnly: req.SelfRouteOnly,","\t\tExpiresAt: req.ExpiresAt,","\tif req.LimitUSD != nil {","\t\tm := usdToMicro(*req.LimitUSD)","\t\topts.LimitMicroUSD = &m","\traw, rec, err := s.store.CreateAPIKey(user.AccountID, opts)","\twriteJSON(w, http.StatusOK, types.CreateAPIKeyResponse{","\t\tKey: raw,","\t\tData: s.apiKeyToResponse(rec),","// handleGetAPIKey handles GET /v1/keys/{id}.","func (s *Server) handleGetAPIKey(w http.ResponseWriter, r *http.Request) {","\tk, err := s.store.GetAPIKeyByID(user.AccountID, id)","\t\twriteJSON(w, http.StatusNotFound, errorResponse(\"not_found\", \"key not found\"))","\twriteJSON(w, http.StatusOK, s.apiKeyToResponse(k))","// handleGetCallingKey handles GET /v1/key — returns the metadata for the API","// key used to authenticate the request (OpenRouter parity).","func (s *Server) handleGetCallingKey(w http.ResponseWriter, r *http.Request) {","\tk := apiKeyFromContext(r.Context())","\tif k == nil || k.ID == \"\" {","\t\twriteJSON(w, http.StatusNotFound, errorResponse(\"not_found\",","\t\t\t\"no key metadata — this endpoint requires API key authentication\"))","// handleUpdateAPIKey handles PATCH /v1/keys/{id} — sparse update of a key's","// name, disabled flag, limits, reset window, expiry, and model allow-list.","func (s *Server) handleUpdateAPIKey(w http.ResponseWriter, r *http.Request) {","\texisting, err := s.store.GetAPIKeyByID(user.AccountID, id)","\t// Decode into a presence map so we can distinguish \"field omitted\" (leave","\t// unchanged) from \"field set to null\" (clear the limit).","\tvar patch map[string]json.RawMessage","\tif err := json.NewDecoder(r.Body).Decode(&patch); err != nil {","\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"bad_request\", \"invalid JSON body\"))","\tif msg := applyKeyPatch(existing, patch); msg != \"\" {","\tif msg := validateKeyLimitInputs(existing.LimitReset, nil, existing.RPMLimit, existing.ITPMLimit, existing.OTPMLimit, existing.ExpiresAt); msg != \"\" {","\t// Bump the auth-cache generation before AND after the mutation so a","\t// concurrent request cannot keep authenticating with a stale (e.g.","\t// just-disabled) cached record.","\ts.invalidateAllAPIKeyCache()","\tupdated, err := s.store.UpdateAPIKey(user.AccountID, id, *existing)","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"server_error\", \"failed to update key\"))","\twriteJSON(w, http.StatusOK, s.apiKeyToResponse(updated))","// applyKeyPatch merges a presence-aware PATCH body into an existing key record.","// Returns a human-readable error string on invalid input (empty when ok).","func applyKeyPatch(k *store.APIKey, patch map[string]json.RawMessage) string {","\tif raw, ok := patch[\"name\"]; ok {","\t\tvar name string","\t\tif err := json.Unmarshal(raw, &name); err != nil {","\t\t\treturn \"invalid value for name\"","\t\tk.Name = strings.TrimSpace(name)","\tif raw, ok := patch[\"disabled\"]; ok {","\t\tvar disabled bool","\t\tif err := json.Unmarshal(raw, &disabled); err != nil {","\t\t\treturn \"invalid value for disabled\"","\t\tk.Disabled = disabled","\tif raw, ok := patch[\"limit_reset\"]; ok {","\t\tvar reset string","\t\tif err := json.Unmarshal(raw, &reset); err != nil {","\t\t\treturn \"invalid value for limit_reset\"","\t\tk.LimitReset = store.NormalizeResetWindow(reset)","\tif raw, ok := patch[\"limit_usd\"]; ok {","\t\tif string(raw) == \"null\" {","\t\t\tk.LimitMicroUSD = nil","\t\t\tvar usd float64","\t\t\tif err := json.Unmarshal(raw, &usd); err != nil {","\t\t\t\treturn \"invalid value for limit_usd\"","\t\t\tif usd < 0 {","\t\t\t\treturn \"limit_usd must be >= 0\"","\t\t\tm := usdToMicro(usd)","\t\t\tk.LimitMicroUSD = &m","\tif raw, ok := patch[\"allowed_models\"]; ok {","\t\t\tk.AllowedModels = nil","\t\t\tvar models []string","\t\t\tif err := json.Unmarshal(raw, &models); err != nil {","\t\t\t\treturn \"invalid value for allowed_models\"","\t\t\tk.AllowedModels = models","\tif raw, ok := patch[\"self_route_only\"]; ok {","\t\tvar v bool","\t\tif err := json.Unmarshal(raw, &v); err != nil {","\t\t\treturn \"invalid value for self_route_only\"","\t\tk.SelfRouteOnly = v","\tfor field, dst := range map[string]**int64{","\t\t\"rpm_limit\": &k.RPMLimit,","\t\t\"itpm_limit\": &k.ITPMLimit,","\t\t\"otpm_limit\": &k.OTPMLimit,","\t} {","\t\tif raw, ok := patch[field]; ok {","\t\t\tif string(raw) == \"null\" {","\t\t\t\t*dst = nil","\t\t\t\tvar v int64","\t\t\t\tif err := json.Unmarshal(raw, &v); err != nil {","\t\t\t\t\treturn \"invalid value for \" + field","\t\t\t\t*dst = &v","\tif raw, ok := patch[\"expires_at\"]; ok {","\t\t\tk.ExpiresAt = nil","\t\t\tvar t time.Time","\t\t\tif err := json.Unmarshal(raw, &t); err != nil {","\t\t\t\treturn \"invalid value for expires_at (use RFC 3339)\"","\t\t\tk.ExpiresAt = &t","// handleDeleteAPIKey handles DELETE /v1/keys/{id} — permanently deletes a key.","func (s *Server) handleDeleteAPIKey(w http.ResponseWriter, r *http.Request) {","\tif err := s.store.RevokeAPIKeyByID(user.AccountID, id); err != nil {","// handleRotateAPIKey handles POST /v1/keys/{id}/rotate — mints a fresh secret","// carrying the same limits and metadata, then deletes the old key. The new","// secret is returned exactly once.","func (s *Server) handleRotateAPIKey(w http.ResponseWriter, r *http.Request) {","\t// Bump the auth-cache generation before AND after the mutation so the old","\t// secret stops authenticating the instant rotation commits.","\traw, rec, err := s.store.RotateAPIKey(user.AccountID, id)","// handleHealth handles GET /health.","// Returns the coordinator's status and the number of connected providers.","// This endpoint does not require authentication.","func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {","\twriteJSON(w, http.StatusOK, types.HealthResponse{","\t\tStatus: \"ok\",","\t\tProviders: s.registry.ProviderCount(),","\t\tVersion: BuildVersion,","\t\tBuildCommit: BuildCommit,","\t\tBuildDate: BuildDate,","// handleVersion returns the latest provider CLI version and download URL.","// Providers call GET /api/version to check if they need to update.","// If a release is registered in the store, uses that. Otherwise falls back","// to the hardcoded LatestProviderVersion.","func (s *Server) handleVersion(w http.ResponseWriter, r *http.Request) {","\tconst cacheKey = \"api_version:v1\"","\tif cached, ok := s.readCache.Get(cacheKey); ok {","\t\twriteCachedJSON(w, cached)","\tvar resp types.VersionResponse","\t// Try release table first.","\tif release := s.store.GetLatestRelease(\"macos-arm64\"); release != nil {","\t\tresp = types.VersionResponse{","\t\t\tVersion: release.Version,","\t\t\tPlatform: release.Platform,","\t\t\tBackend: release.Backend,","\t\t\tDownloadURL: release.URL,","\t\t\tBinaryHash: release.BinaryHash,","\t\t\tBundleHash: release.BundleHash,","\t\t\tMetallibHash: release.MetallibHash,","\t\t\tChangelog: release.Changelog,","\t\t// Fallback to hardcoded version + coordinator download.","\t\tscheme := \"https\"","\t\tif r.TLS == nil && !strings.Contains(r.Host, \"darkbloom.dev\") {","\t\t\tscheme = \"http\"","\t\tdownloadURL := fmt.Sprintf(\"%s://%s/dl/eigeninference-bundle-macos-arm64.tar.gz\", scheme, r.Host)","\t\t\tVersion: LatestProviderVersion,","\t\t\tDownloadURL: downloadURL,","\tbody, err := json.Marshal(resp)","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"internal_error\", \"failed to encode version\"))","\ts.readCache.Set(cacheKey, body, time.Minute)","\twriteCachedJSON(w, body)","// --- payment handlers ---","// handleBalance handles GET /v1/payments/balance.","// Returns the consumer's current balance in both micro-USD and USD.","func (s *Server) handleBalance(w http.ResponseWriter, r *http.Request) {","\tbalance := s.ledger.Balance(consumerKey)","\twithdrawable := s.store.GetWithdrawableBalance(consumerKey)","\twriteJSON(w, http.StatusOK, types.BalanceResponse{","\t\tBalanceMicroUSD: balance,","\t\tBalanceUSD: fmt.Sprintf(\"%.6f\", float64(balance)/1_000_000),","\t\tWithdrawableMicroUSD: withdrawable,","\t\tWithdrawableUSD: fmt.Sprintf(\"%.6f\", float64(withdrawable)/1_000_000),","// handleUsage handles GET /v1/payments/usage.","// Returns the consumer's inference usage history with per-request costs.","// Tries in-memory ledger first (has full detail), falls back to store","// ledger history (persists across restarts but has less detail).","func (s *Server) handleUsage(w http.ResponseWriter, r *http.Request) {","\tentries := s.ledger.Usage(consumerKey)","\t// If in-memory usage is empty (coordinator restarted), build from","\t// the persisted usage table which has full request details.","\tif len(entries) == 0 {","\t\tusageRecords := s.store.UsageByConsumer(consumerKey)","\t\tfor _, u := range usageRecords {","\t\t\tjobID := u.RequestID","\t\t\tif jobID == \"\" {","\t\t\t\tjobID = u.ProviderID","\t\t\tmodel := u.Model","\t\t\tif u.PublicModel != \"\" {","\t\t\t\tmodel = u.PublicModel","\t\t\tentries = append(entries, payments.UsageEntry{","\t\t\t\tJobID: jobID,","\t\t\t\tModel: model,","\t\t\t\tPromptTokens: u.PromptTokens,","\t\t\t\tCompletionTokens: u.CompletionTokens,","\t\t\t\tCostMicroUSD: u.CostMicroUSD,","\t\t\t\tTimestamp: u.CreatedAt,","\twriteJSON(w, http.StatusOK, types.UsageResponse{","\t\tUsage: entries,","// handleProviderEarnings handles GET /v1/provider/earnings?wallet=0x...","// Returns the provider's balance and payout history.","// No API key auth required — providers identify by provider address.","func (s *Server) handleProviderEarnings(w http.ResponseWriter, r *http.Request) {","\twallet := r.URL.Query().Get(\"wallet\")","\tif wallet == \"\" {","\t\twallet = r.Header.Get(\"X-Provider-Wallet\")","\t\twriteJSON(w, http.StatusBadRequest, errorResponse(\"invalid_request_error\", \"wallet address required (query param ?wallet=0x... or X-Provider-Wallet header)\"))","\t// Look up balance by provider address","\tbalance := s.ledger.Balance(wallet)","\thistory := s.ledger.LedgerHistory(wallet)","\tpayouts := s.ledger.AllPayouts()","\t// Filter payouts to this wallet","\tvar walletPayouts []payments.Payout","\tvar totalEarned int64","\tvar totalJobs int","\tfor _, p := range payouts {","\t\tif p.ProviderAddress == wallet {","\t\t\twalletPayouts = append(walletPayouts, p)","\t\t\ttotalEarned += p.AmountMicroUSD","\t\t\ttotalJobs++","\t// If no explicit payout records exist (for example, legacy rows created","\t// before provider_payouts was introduced), reconstruct from persisted","\t// ledger entries with payout type and the wallet as account ID.","\tif len(walletPayouts) == 0 {","\t\tledgerEntries := s.store.LedgerHistory(wallet)","\t\tfor _, le := range ledgerEntries {","\t\t\tif le.Type == store.LedgerPayout && le.Reference != \"\" {","\t\t\t\twalletPayouts = append(walletPayouts, payments.Payout{","\t\t\t\t\tProviderAddress: wallet,","\t\t\t\t\tAmountMicroUSD: le.AmountMicroUSD,","\t\t\t\t\tJobID: le.Reference,","\t\t\t\t\tTimestamp: le.CreatedAt,","\t\t\t\t\tSettled: true,","\t\t\t\ttotalEarned += le.AmountMicroUSD","\t\t\t\ttotalJobs++","\tif walletPayouts == nil {","\t\twalletPayouts = []payments.Payout{}","\twriteJSON(w, http.StatusOK, types.ProviderEarningsResponse{","\t\tBalanceMicroUSD: balance,","\t\tBalanceUSD: fmt.Sprintf(\"%.6f\", float64(balance)/1_000_000),","\t\tTotalEarnedMicroUSD: totalEarned,","\t\tTotalEarnedUSD: fmt.Sprintf(\"%.6f\", float64(totalEarned)/1_000_000),","\t\tTotalJobs: totalJobs,","\t\tPayouts: walletPayouts,","\t\tLedger: history,","// --- helpers ---","// writeJSON serializes v as JSON and writes it to the response with the","// given HTTP status code. Sets Content-Type to application/json.","func writeJSON(w http.ResponseWriter, status int, v any) {","\tw.Header().Set(\"Content-Type\", \"application/json\")","\tw.WriteHeader(status)","\tjson.NewEncoder(w).Encode(v)","// handleCompletions handles POST /v1/completions.","// Proxies OpenAI-compatible text completions to the provider's vllm-mlx server.","func (s *Server) handleCompletions(w http.ResponseWriter, r *http.Request) {","\ts.handleGenericInference(w, r, \"/v1/completions\")","// handleAnthropicMessages handles POST /v1/messages.","// Proxies Anthropic-compatible messages API to the provider's vllm-mlx server.","func (s *Server) handleAnthropicMessages(w http.ResponseWriter, r *http.Request) {","\ts.handleGenericInference(w, r, \"/v1/messages\")","// handleGenericInference is the shared dispatch for completions and Anthropic endpoints.","// It reads the raw request body, extracts model/stream, sets the endpoint field,","// and reuses the same E2E encryption + provider routing as chat completions.","func (s *Server) handleGenericInference(w http.ResponseWriter, r *http.Request, endpoint string) {","\t// Shared prelude: read body, normalize tool schemas (Anthropic /v1/messages","\t// bodies carry a top-level \"tools\" array too; the provider body is rebuilt","\t// from parsed below, so normalizing before the unmarshal covers it), parse,","\t// require a model, enforce the per-key model allowlist.","\tif hasProviderAllowlist {","\t\tstripProviderRoutingFields(parsed)","\t// \"Use my own machine, for free\" opt-in (see handleChatCompletions).","\t// Resolve a public alias to a concrete build id, constraint-aware (after","\t// allowlist/self-route are known). resolveRequestedModel rewrites","\t// parsed[\"model\"] to the build; this handler builds the provider body fresh","\t// from `parsed` (inferenceBody below), so rawBody isn't threaded here.","\tbuildModel, publicModel, _, ok := s.resolveRequestedModel(parsed, rawBody, model, allowedProviderSerials, policy)","\tmodel = buildModel","\t\t\treasonCode: \"model_not_found\",","\t\t\thttpStatus: http.StatusNotFound,","\t\t\trequestedModel: publicModel,","\t\t\tresolvedModel: model,","\t// Shared media/tools fail-fast (see visionToolsFailFast). Completions and","\t// Anthropic bodies share the top-level \"tools\" field; neither has the","\t// Responses-API media surface, so rejectResponsesMedia is false here.","\t\tfalse, policy, allowedProviderSerials) {","\t// Completions and Anthropic messages both use the max_tokens field (never","\t// max_output_tokens, which is Responses API only). Inject a default if","\t// unset so the pre-flight reservation bounds the generation.","\tgenericMaxOutput := defaultMaxOutputTokens","\t\t\tgenericMaxOutput = rec.MaxOutputLength","\tensureMaxTokensBound(parsed, false, genericMaxOutput)","\tgenericDeadline := ttftDeadline(estimatedPromptTokens)","\t// Inject the endpoint so the provider knows which local path to forward to.","\tparsed[\"endpoint\"] = endpoint","\t// Per-account token rate limiting (ITPM/OTPM), before the reservation.","\t// Pre-flight balance reservation — same worst-case-cost reservation as","\t// handleChatCompletions, using the byte-length upper bound for prompt","\t// tokens so the reservation always covers actual cost.","\t// Self-route is free: skip the reservation and per-key spend cap.","\t\t// Per-key spend cap (phase 1) — checked before the reservation.","\t\t\ts.releaseInitialReservation(consumerKey, model, reservedMicroUSD, serviceReservation)","\trejectionForGeneric := func(stage, reason string, status, retryAfterMs int) rejectionInfo {","\t\treturn rejectionInfo{","\t\t\tstage: stage,","\t\t\treasonCode: reason,","\t\t\thttpStatus: status,","\t\t\tconsumerKeyHash: store.HashKey(consumerKey),","\t\t\tselfRouteOnly: policy.enabled,","\t\t\tpreferOwner: policy.prefer,","\t\t\tretryAfterMs: retryAfterMs,","\trejectionForGenericWithDecision := func(stage, reason string, status, retryAfterMs int, decision registry.RoutingDecision) rejectionInfo {","\t\tinfo := rejectionForGeneric(stage, reason, status, retryAfterMs)","\t\tinfo.servabilityComputed = true","\t\tinfo.candidateCount = decision.CandidateCount","\t\tinfo.capacityRejections = decision.CapacityRejections","\t\tinfo.modelTooLargeRejections = decision.ModelTooLargeRejections","\t\tinfo.visionRejections = decision.VisionRejections","\t\tinfo.bestTTFTMs = decision.BestTTFTMs","\t\treturn info","\t// Self-route pre-flight (precise errors, no paid fallback); otherwise the","\t// fleet-wide capacity 429 (same logic as handleChatCompletions).","\t\t// Prefer mode skips the public fleet pre-flight (no owner-trust","\t\t// relaxation there); owned-first dispatch + paid fallback + queue gate it.","\t\t// Smart early-429 for structurally-unservable long prompts (mirrors","\t\t// handleChatCompletions): runs AFTER the alias capacity fallback. Gated","\t\t// (default off), fail-open.","\t\t\t// Queue-before-shed (default on): all providers for this model are at","\t\t\t// capacity right now. Fall through to the dispatch+queue path so a slot","\t\t\t// it; the queue path still 429s on a full queue or wait timeout.","\t\t\t// saturated — fast-429 for a clean OpenRouter failover instead of a","\t\t\t// 120s queue that would blow the TTFT SLA. Mirrors handleChatCompletions.","\t\t\t// No structurally-eligible provider right now (offline, trait-gated,","\t\t\t// or shape-cooled by the inference-error breaker).","\t\t\t// Routing v2 W3 cold-dispatch (default on): if an idle on-disk provider","\t\t\t// could be warmed to serve this model (and would pass admission for","\t\t\t// these traits), spill into the queue instead of 503'ing — the enqueue","\t\t\t// path kicks the model-swap machinery and the queued request drains","\t\t\t// onto the provider once the cold load completes. Feed the autoscaler","\t\t\t// the demand regardless of outcome.","\t\t\t\t// Dedicated-box model served by the fleet but no dedicated box is","\t\t\t\t// available right now: transient capacity, so shed with 429 +","\t\t\t\t// Retry-After (not 503). Mirrors the chat-completions preflight so","\t\t\t\t// /v1/completions and /v1/messages classify the shed identically.","\t\t\t\t// Queueing cannot help — fail fast with a retryable 503 instead of","\t\t\t\t// a 120s queue. Mirrors the chat-completions preflight.","\t\tttftThreshold := genericDeadline","\t\t\t\t// Soft TTFT gate (default): serve the best-available provider","\t\t\t\t// (MaxTTFTMs is 0 in soft mode — P1 fix); do not divert to an older","\t\t\t\t// alias build (P2 fix). Feed the autoscaler a near-miss to grow","\t\t\t\t// warm capacity for this model.","\t\t\t\t// Hard TTFT gate, no faster alias: 429 + Retry-After, and feed the","\t\t\t\t// autoscaler a TTFT-miss so warm capacity grows.","\tpr := ®istry.PendingRequest{","\t\tRequestID: requestID,","\t\t// Single-attempt path: no retry loop, so no AvoidVersion to thread.","\t\tTraits: registry.RequestTraits{HasTools: hasTools},","\t\tRequestedMaxTokens: requestedMaxTokens,","\t\tTokenAdmission: tokenAdmission,","\t\tReservedMicroUSD: reservedMicroUSD,","\t\tBaseReservedMicroUSD: reservedMicroUSD,","\t\tServiceReservation: serviceReservation,","\t\tAcceptedCh: make(chan struct{}, 1),","\t\tChunkCh: make(chan string, chunkBufferSize),","\t\tCompleteCh: make(chan protocol.UsageInfo, 1),","\t\tErrorCh: make(chan protocol.InferenceErrorMessage, 1),","\t\tTiming: timing,","\t// Routing v2 (P1 fix): enforce the TTFT ceiling only in HARD mode; soft mode","\t// leaves MaxTTFTMs 0 so dispatch serves the best-available provider.","\t\tpr.MaxTTFTMs = float64(genericDeadline.Milliseconds())","\t// Routing v2 W2: soft per-request decode floor (0 = off).","\t// reserveAdditionalForProvider may have added on top of the base","\t// reservation. Without this, failing after the extra charge leaks","\t// the difference between pr.ReservedMicroUSD and the original","\t// reservedMicroUSD.","\tvar provider *registry.Provider","\tvar decision registry.RoutingDecision","\tvar excludeProviders []string","\tfor attempt := 0; attempt < 3; attempt++ {","\t\tprovider, decision = s.registry.ReserveProviderEx(model, pr, excludeProviders...)","\t\tif provider == nil {","\t\t\tbreak","\t\t// Settles FREE when served by the caller's own machine: exclusive","\t\t// self-route always, or a prefer request whose selected provider is owned","\t\t// (settlement refunds to zero). Skip the payout warning + custom-price","\t\t// top-up then (the top-up could otherwise 429 the free owned route).","\t\tsettlesFree := policy.enabled","\t\tif !settlesFree && policy.prefer {","\t\t\tprovider.Mu().Lock()","\t\t\tsettlesFree = policy.ownerAccountID != \"\" && provider.AccountID == policy.ownerAccountID","\t\t\tprovider.Mu().Unlock()","\t\tif s.billing != nil && !settlesFree && !providerHasPayoutDestination(provider) {","\t\t\ts.logger.Warn(\"provider missing payout destination, crediting to internal ledger\",","\t\t\t\t\"provider_id\", provider.ID)","\t\t// Custom pricing check — provider may charge more than the platform","\t\t// rate. Skipped for free (owned) requests, which settle at zero cost.","\t\tif s.billing != nil && !settlesFree {","\t\t\tif _, err := s.reserveAdditionalForProvider(pr, provider); err != nil {","\t\t\t\tprovider.RemovePending(requestID)","\t\t\t\ts.registry.SetProviderIdle(provider.ID)","\t\t\t\texcludeProviders = append(excludeProviders, provider.ID)","\t\t\t\tif !errors.Is(err, store.ErrInsufficientBalance) {","\t\t\t\t\ts.logger.Error(\"provider reservation failed (DB error)\",","\t\t\t\t\t\t\"request_id\", requestID,","\t\t\t\t\t\t\"provider_id\", provider.ID,","\t\t\t\t\t\t\"error\", err,","\t\t\t\t\t)","\t\t// Provider passed all checks.","\t\tbreak","\t\t// with a retryable 429 rather than queueing for a provider that would","\t\t// miss the OpenRouter SLA target.","\t\t\tbestTTFT := time.Duration(decision.BestTTFTMs * float64(time.Millisecond))","\t\t\ts.writeTTFTTooSlow(w, model, publicModel, bestTTFT, genericDeadline)","\t\t// No online provider can physically fit this model — queueing/retrying","\t\t// can't help, so fast-fail with a clear, non-retryable error instead of","\t\t// blocking for 120s then 503-ing. Mirrors the streaming dispatch path.","\t\tqueuedReq := ®istry.QueuedRequest{","\t\t\tRequestID: requestID,","\t\t\tModel: model,","\t\t\tPending: pr,","\t\t\tResponseCh: make(chan *registry.Provider, 1),","\t\ttiming.QueuedAt = time.Now()","\t\tif err := s.registry.Queue().Enqueue(queuedReq); err != nil {","\t\t\tretryAfter := s.estimateRetryAfter(model)","\t\t\tw.Header().Set(\"Retry-After\", strconv.Itoa(retryAfter))","\t\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:over_capacity\"})","\t\t\ts.recordRejection(rejectionForGenericWithDecision(\"queue\", \"queue_full\", http.StatusTooManyRequests, retryAfter*1000, decision))","\t\t\tif policy.enabled {","\t\t\t\twriteJSON(w, http.StatusTooManyRequests, errorResponse(\"machine_busy\",","\t\t\t\t\t\"your machine is at capacity — retry shortly\", withCode(\"machine_busy\")))","\t\t\t\t\tfmt.Sprintf(\"all providers for model %q are at capacity and queue is full\", publicModel),","\t\ts.recordWarmPoolQueueState(model)","\t\t// Routing v2 W3: the model now has queued demand — proactively warm a cold","\t\t// provider for it (TriggerModelSwaps) instead of waiting for the next","\t\t// heartbeat, so the queued request drains onto it sooner.","\t\ts.kickColdDispatch(model)","\t\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:queued\"})","\t\trouteState := &dispatchState{","\t\t\ts: s,","\t\t\tmodel: model,","\t\t\tpublicModel: publicModel,","\t\t\tconsumerKey: consumerKey,","\t\t\tconsumerLocation: consumerLocation,","\t\t\tpolicy: policy,","\t\t\tcacheAffinityKey: cacheAffinityKey,","\t\t\trequestID: requestID,","\t\t\tattempt: pr.Attempt,","\t\t\tpr: pr,","\t\trouteState.recordRoutingDecisionFor(nil, pr, requestID, pr.Attempt, decision, \"\", \"queued\")","\t\tprovider, err = s.registry.Queue().WaitForProviderContext(r.Context(), queuedReq)","\t\t\tif errors.Is(err, context.Canceled) {","\t\t\t\ts.recordWarmPoolQueueState(model)","\t\t\t\ts.emitClientGone(model, estimatedPromptTokens, providerChipFamily(provider), phaseBeforeFirstToken)","\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, pendingRouteOutcome(pr, \"cancelled\", \"client_gone\", 0))","\t\t\ts.updateInferenceRouteOutcomeForPending(pr, pendingRouteOutcome(pr, \"timeout\", \"queue_timeout\", http.StatusTooManyRequests))","\t\t\ts.registry.RecordWarmPoolQueueTimeout(model, time.Since(queuedReq.EnqueuedAt))","\t\t\ts.recordRejection(rejectionForGenericWithDecision(\"queue\", \"queue_timeout\", http.StatusTooManyRequests, retryAfter*1000, decision))","\t\t\t\t\t\"your machine is at capacity (timed out waiting for a free slot) — retry shortly\", withCode(\"machine_busy\")))","\t\t\t\t\tfmt.Sprintf(\"all providers for model %q are at capacity (queue timeout)\", publicModel),","\t\tdecision = queuedReq.Decision","\ttiming.RoutedAt = time.Now()","\ts.ddIncr(\"routing.decisions\", []string{\"model:\" + model, \"model_type:\" + s.registry.ModelType(model), \"outcome:selected\"})","\ts.ddIncr(\"routing.provider_selected\", []string{\"provider_id:\" + provider.ID, \"model:\" + model})","\ts.ddHistogram(\"routing.cost_ms\", decision.CostMs, []string{\"model:\" + model, \"provider_id:\" + provider.ID})","\tif decision.EffectiveTPS > 0 {","\t\ts.ddGauge(\"routing.effective_decode_tps\", decision.EffectiveTPS, []string{\"provider_id:\" + provider.ID})","\trouteState := &dispatchState{","\t\ts: s,","\t\tmodel: model,","\t\tpublicModel: publicModel,","\t\tconsumerKey: consumerKey,","\t\tconsumerLocation: consumerLocation,","\t\tpolicy: policy,","\t\tcacheAffinityKey: cacheAffinityKey,","\t\trequestID: requestID,","\t\tattempt: pr.Attempt,","\t\tprovider: provider,","\t\tpr: pr,","\trouteState.recordRoutingDecisionFor(provider, pr, requestID, pr.Attempt, decision, \"\", \"\")","\t// Settles FREE when served by the caller's own machine (exclusive self-route,","\t// or a prefer request whose selected provider is owned — settlement refunds","\t// to zero). Skip the payout warning + custom-price top-up then.","\tsettlesFreeDirect := policy.enabled","\tif !settlesFreeDirect && policy.prefer {","\t\tsettlesFreeDirect = policy.ownerAccountID != \"\" && provider.AccountID == policy.ownerAccountID","\tif s.billing != nil && !settlesFreeDirect && !providerHasPayoutDestination(provider) {","\t// Free (owned) requests settle at zero cost — no provider-price top-up.","\tif s.billing != nil && !settlesFreeDirect {","\t\tif _, err := s.reserveAdditionalForProvider(pr, provider); err != nil {","\t\t\trefundExtra()","\t\t\t\t\t\"your balance is too low for this provider price — add funds at /billing or lower max_tokens\", withCode(\"insufficient_quota\")))","\t\t\t\ts.logger.Error(\"provider reservation failed (DB error)\", \"consumer_key\", consumerKey, \"error\", err)","\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, dispatchFailedPendingRouteOutcome(pr, \"provider_error\", http.StatusServiceUnavailable))","\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, dispatchFailedPendingRouteOutcome(pr, \"insufficient_funds\", http.StatusPaymentRequired))","\tinferenceBody, _ := marshalForwardBody(parsed)","\t// Re-check the cap on the FINAL body we'll seal (the input cap bounded the","\t// read; this body was re-marshaled after mutation). A body over the cap seals","\t// into a frame the provider rejects by tearing down its session — return a","\t// clean 413 instead (see maxInferenceBodyBytes). Billing is already reserved","\tif len(inferenceBody) > maxInferenceBodyBytes {","\t\ts.updateInferenceRouteOutcomeForPending(pr, dispatchFailedPendingRouteOutcome(pr, \"payload_too_large\", http.StatusRequestEntityTooLarge))","\t\t\trequestBodyBytes: len(inferenceBody),","\t\ts.updateInferenceRouteOutcomeForPending(pr, dispatchFailedPendingRouteOutcome(pr, \"encryption_missing\", http.StatusServiceUnavailable))","\t\twriteJSON(w, http.StatusServiceUnavailable, errorResponse(\"encryption_required\",","\t\t\t\"no provider with E2E encryption available\"))","\t\ts.updateInferenceRouteOutcomeForPending(pr, dispatchFailedPendingRouteOutcome(pr, \"encryption_error\", http.StatusInternalServerError))","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"encryption_error\", \"provider public key invalid\"))","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"encryption_error\", \"failed to generate session keys\"))","\t// Version-gated penalty strip for vision requests (Anthropic /v1/messages","\t// carries image blocks); this handler seals separately from dispatchOneProvider.","\tinferenceBody = bodyForProvider(inferenceBody, requiresVision, provider)","\tencrypted, err := e2e.Encrypt(inferenceBody, providerPubKey, sessionKeys)","\t\twriteJSON(w, http.StatusInternalServerError, errorResponse(\"encryption_error\", \"failed to encrypt request\"))","\ttiming.EncryptedAt = time.Now()","\tdata, _ := json.Marshal(wireMsg)","\ttiming.DispatchedAt = time.Now()","\t\ts.updateInferenceRouteOutcomeForPending(pr, dispatchFailedPendingRouteOutcome(pr, \"provider_error\", http.StatusBadGateway))","\t\twriteJSON(w, http.StatusBadGateway, errorResponse(\"provider_error\", \"failed to send request to provider\"))","\ts.logger.Info(\"inference request dispatched\",","\t\t\"request_id\", requestID,","\t\t\"model\", model,","\t\t\"provider_id\", provider.ID,","\t\t\"endpoint\", endpoint,","\t\t\"stream\", stream,","\t)","\t// Dynamic TTFT deadline — wait for the first chunk or accepted signal","\t// before committing. This mirrors the chat completions path but without","\t// speculative dispatch (single attempt). If the provider misses the","\t// TTFT deadline, the request fails instead of streaming forever.","\tttftTimer := time.NewTimer(genericDeadline)","\tvar firstChunk string","\tcommitted := false","\taccepted := false","\tselect {","\tcase <-pr.AcceptedCh:","\t\tttftTimer.Stop()","\t\taccepted = true","\tcase chunk, ok := <-pr.ChunkCh:","\t\tif ok {","\t\t\tfirstChunk = chunk","\t\t\tpr.MarkFirstChunkArrived()","\t\t\tcommitted = true","\t\t\tselect {","\t\t\tcase errMsg := <-pr.ErrorCh:","\t\t\t\ts.sendProviderCancel(provider, requestID)","\t\t\t\trefundExtra()","\t\t\t\ts.noteInferenceError(provider.ID, pr, errMsg.StatusCode, errMsg.Error)","\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preCommitProviderErrorOutcome(pr, errMsg))","\t\t\t\tstatusCode := errMsg.StatusCode","\t\t\t\tif statusCode == 0 {","\t\t\t\t\tstatusCode = http.StatusBadGateway","\t\t\t\twriteJSON(w, statusCode, errorResponse(\"provider_error\", errMsg.Error))","\t\t\t\tcommitted = true","\tcase errMsg := <-pr.ErrorCh:","\t\tprovider.RemovePending(requestID)","\t\ts.registry.SetProviderIdle(provider.ID)","\t\ts.sendProviderCancel(provider, requestID)","\t\ts.noteInferenceError(provider.ID, pr, errMsg.StatusCode, errMsg.Error)","\t\ts.updateInferenceRouteOutcomeForPending(pr, preCommitProviderErrorOutcome(pr, errMsg))","\t\tstatusCode := errMsg.StatusCode","\t\tif statusCode == 0 {","\t\t\tstatusCode = http.StatusBadGateway","\t\twriteJSON(w, statusCode, errorResponse(\"provider_error\", errMsg.Error))","\tcase <-ttftTimer.C:","\t\ts.ddIncr(\"inference.dispatches\", []string{\"status:timeout\"})","\t\ts.updateInferenceRouteOutcomeForPending(pr, pendingRouteOutcome(pr, \"timeout\", \"first_chunk_timeout\", http.StatusGatewayTimeout))","\t\twriteJSON(w, http.StatusGatewayTimeout, errorResponse(\"timeout\", \"provider did not respond within TTFT deadline\"))","\tcase <-r.Context().Done():","\t\ts.emitClientGone(model, estimatedPromptTokens, providerChipFamily(provider), phaseBeforeFirstToken)","\t\ts.updateInferenceRouteOutcomeForPending(pr, pendingRouteOutcome(pr, \"cancelled\", \"client_gone\", 0))","\t// If provider accepted (model reload), wait for first chunk with extended deadline.","\tif accepted && !committed {","\t\tchunkTimer := time.NewTimer(inferenceTimeout)","\t\t\tchunkTimer.Stop()","\t\t\tif ok {","\t\t\t\tfirstChunk = chunk","\t\t\t\tpr.MarkFirstChunkArrived()","\t\t\t\tcase errMsg := <-pr.ErrorCh:","\t\t\t\t\tprovider.RemovePending(requestID)","\t\t\t\t\ts.registry.SetProviderIdle(provider.ID)","\t\t\t\t\ts.sendProviderCancel(provider, requestID)","\t\t\t\t\trefundExtra()","\t\t\t\t\trefundReservation()","\t\t\t\t\ts.noteInferenceError(provider.ID, pr, errMsg.StatusCode, errMsg.Error)","\t\t\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preCommitProviderErrorOutcome(pr, errMsg))","\t\t\t\t\tstatusCode := errMsg.StatusCode","\t\t\t\t\tif statusCode == 0 {","\t\t\t\t\t\tstatusCode = http.StatusBadGateway","\t\t\t\t\twriteJSON(w, statusCode, errorResponse(\"provider_error\", errMsg.Error))","\t\t\t\t\tcommitted = true","\t\tcase errMsg := <-pr.ErrorCh:","\t\t\ts.sendProviderCancel(provider, requestID)","\t\t\ts.noteInferenceError(provider.ID, pr, errMsg.StatusCode, errMsg.Error)","\t\t\ts.updateInferenceRouteOutcomeForPending(pr, preCommitProviderErrorOutcome(pr, errMsg))","\t\tcase <-chunkTimer.C:","\t\t\t// Accepted-then-silent is a provider-at-fault 504 — feed the","\t\t\t// breaker (single-attempt path: no retry here, but repeated","\t\t\t// stalls must still accumulate into the routing cooldown). No","\t\t\t// provider message on this synthetic timeout, so errStr is \"\".","\t\t\ts.noteInferenceError(provider.ID, pr, http.StatusGatewayTimeout, \"\")","\t\t\ts.ddIncr(\"inference.dispatches\", []string{\"status:timeout\"})","\t\t\ts.updateInferenceRouteOutcomeForPending(pr, pendingRouteOutcome(pr, \"timeout\", \"accepted_timeout\", http.StatusGatewayTimeout))","\t\t\twriteJSON(w, http.StatusGatewayTimeout, errorResponse(\"timeout\", \"provider accepted but timed out before first chunk\"))","\t\t\ts.emitClientGone(model, estimatedPromptTokens, providerChipFamily(provider), phaseBeforeFirstToken)","\t\t\ts.updateInferenceRouteOutcomeForPending(pr, pendingRouteOutcome(pr, \"cancelled\", \"client_gone\", 0))","\tif !committed {","\t\ts.updateInferenceRouteOutcomeForPending(pr, providerFailedPendingRouteOutcome(pr, \"error\", \"provider_incomplete\", http.StatusServiceUnavailable))","\t\twriteJSON(w, http.StatusServiceUnavailable, errorResponse(\"provider_error\", \"failed to get first chunk from provider\"))","\ts.updateInferenceRouteOutcomeForPending(pr, committedRouteOutcome(pr))","\t// Free the slot, stop the provider, and preserve billing on a mid-stream","\t// disconnect (park-before-remove + post-terminal sweep; see the","\t// chat-completions path for the full rationale).","\tdefer func() {","\t\tif stale := provider.GetPending(requestID); stale != nil {","\t\t\ts.holdForSettlement(stale)","\t\t\trefundPr := pr","\t\t\tsaferun.Go(s.logger, \"api.postTerminalSweep\", func() {","\t\t\t\ts.refundReservedBalance(refundPr, \"post_terminal_sweep:\"+requestID)","\t}()","\tvar firstChunks []string","\tif firstChunk != \"\" {","\t\tfirstChunks = []string{firstChunk}","\tif stream {","\t\t// The generic (/v1/completions, /v1/messages) path does not run prefill","\t\t// keepalives, so the SSE header has not been written yet.","\t\ts.handleStreamingResponseWithFirstChunk(w, r, pr, firstChunks, false)","\t\ts.handleNonStreamingResponseWithFirstChunk(w, r, pr, firstChunks)","// errorDetailOpt carries optional fields for OpenAI-compatible error responses.","type errorDetailOpt struct {","\tparam string // e.g. \"model\", \"max_tokens\"","\tcode string // e.g. \"model_not_found\", \"insufficient_quota\"","// errorResponse builds a standard OpenAI-compatible error response body.","// By default, code is inferred from errType. Callers can override code or","// set param via withParam / withCode helpers.","func errorResponse(errType, message string, opts ...errorDetailOpt) map[string]any {","\tdetail := map[string]any{","\t\t\"type\": errType,","\t\t\"message\": message,","\t\t\"code\": errType, // default: code mirrors type","\tfor _, o := range opts {","\t\tif o.param != \"\" {","\t\t\tdetail[\"param\"] = o.param","\t\tif o.code != \"\" {","\t\t\tdetail[\"code\"] = o.code","\treturn map[string]any{","\t\t\"error\": detail,","// withParam returns an option that sets the \"param\" field on an error response.","func withParam(p string) errorDetailOpt { return errorDetailOpt{param: p} }","// withCode returns an option that overrides the \"code\" field on an error response.","func withCode(c string) errorDetailOpt { return errorDetailOpt{code: c} }",[4501,4502,4503,4504,4505,4506,4507],{"_941":46,"_46":4518},{"_941":4509,"_255":1182,"_965":4516,"_966":4517},{"_941":46,"_46":4515},{"_941":4509,"_255":1076,"_965":4513,"_966":4514},{"_941":46,"_46":4512},{"_941":4509,"_255":959,"_965":4510,"_966":4511},{"_941":46,"_46":4508},"\n\n**Testing**\n\n* ⚠️ `go test ./coordinator/api -run TestDedicatedSaturatedBoxFast429 -count=1` — blocked by environment/toolchain setup: `mise` could not install/resolve required tools because remote downloads failed with tunnel/network errors after trusting `mise.toml`.\n* ✅ `git status --short` — working tree is clean.","repo_file_citation",127,141,"\n* I also confirmed the intentionally retained `HasProviderForModel` behavior only checks online/trusted catalog advertisers, matching the stated product decision to keep this class of dedicated-model shed as transient 429 instead of 503. ",121,166,"\n* I confirmed there is a regression test covering the queue-before-shed bypass for a saturated dedicated Gemma 4 box. ",2105,2137,"### Summary\n\n* No code changes were needed, so I did **not** commit or open a new PR.\n* I confirmed the current branch already includes the saturated dedicated-box fast-429 path for dedicated models, with `Retry-After` and `machine_busy` rejection metadata. ",{"_92":485,"_42":84,"_96":485,"_4521":4522,"_4523":4524,"_4525":4526,"_132":-5,"_119":4527},[83],"input_items",[4546,4547,4548,4549,4550,4551,4552],"external_post_config",{"_4528":4529,"_428":429,"_4530":464,"_440":-5,"_4531":423,"_4532":-5,"_4533":-5,"_4534":4535,"_4536":4537,"_4538":-5,"_4539":-5,"_4540":4541,"_4542":15,"_4543":59,"_4544":54,"_4545":15},"user_model_slug","slug",1782003767.698548,"installation_id",67327998,"pr_number","repo_id","gitlab_service_account_project_id","review_trigger","comment_id",4760494290,"comment_type","issue","pr_write_guard_token","pending_pr_reaction_id","pending_comment_reaction_id",372681301,"comment_if_no_findings","comment_on_error","share_url","is_parallel_review",{"_92":588,"_96":485,"_582":4566},{"_471":472,"_473":474,"_4563":478,"_4564":129,"_481":-5,"_92":4565},{"_92":4553,"_473":4562,"_255":-5,"_4555":-5,"_4557":-5},{"_92":4553,"_473":4561,"_255":-5,"_4555":-5,"_4557":-5},{"_92":4553,"_473":4560,"_255":-5,"_4555":-5,"_4557":-5},{"_92":4553,"_473":4559,"_255":1182,"_4555":4556,"_4557":4516},{"_92":4553,"_473":4554,"_255":959,"_4555":4556,"_4557":4558},"pull_request_comment","**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Apply routing gates before treating a provider as available**\n\nWhen the only advertiser for a dedicated model is private-only, below the public trust floor, runtime-unverified, lacks private-text support, or has a stale challenge, public routing still rejects it in `providerPassesRoutingGatesLockedEx`, but this check returns true because it only checks status and catalog membership. In that context the new preflight branch classifies an unroutable public fleet as `machine_busy` 429 instead of the `no_provider` 503 path, so clients are told to retry/fail over as if capacity might free when no public provider can actually route the model.\n\nUseful? React with 👍 / 👎.","side","RIGHT","line",135,"**_{_{![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)}} Shed saturated dedicated models before queueing**\n\nWhen every dedicated provider for a Gemma-4 request exists but is already at concurrency/token capacity, `QuickCapacityCheckWithTTFTForRequest` reports `candidateCount == 0` with `capacityRejections > 0`, so the earlier queue-before-shed block handles the request and this new 429 branch is never reached. With the default queue-before-shed setting, OpenRouter traffic can therefore sit in the 120s dispatch queue instead of receiving the intended `Retry-After` 429 whenever dedicated boxes are present but saturated; the mirrored `handleGenericInference` path has the same ordering.\n\nUseful? React with 👍 / 👎.","\n### 💡 Codex Review\n\nHere are some automated review suggestions for this pull request.\n\n**Reviewed commit:** `a5fc81f361`\n \n\n

ℹ️ About Codex in GitHub

","\n\n**This PR introduces a dedicated-box routing feature for specific model families; the changes are operationally significant but security-neutral with one minor concern worth noting.**\n\n---\n\n### Trust boundaries touched\n\n- **TB-002** (Coordinator → Provider WebSocket) — routing gate logic in `registry.go`\n- **TB-006** (Admin → Coordinator) — new env-var configuration surface in `main.go`\n\n---\n\n### Threat analysis\n\n**T-034** (Provider runs modified code while advertising a trusted identity) — `coordinator/registry/registry.go` lines +2985–2988, +3072–3075\n\nℹ️ **Neutral.** `providerExcludedByDedicatedRuleLocked` and `providerHasWarmModelLocked` are routing/scheduling helpers that sit downstream of the APNs code-identity gate (`providerSupportsPrivateTextLocked`). The dedicated-model exclusion fires on catalog shape (`dedicatedModels` substring match), not on any attestation field, so it cannot be manipulated by a provider to bypass or strengthen the `CodeAttested` gate. No change to trust-level assignment.\n\n**T-038** (HTTP server config) — `coordinator/cmd/coordinator/main.go`\n\nℹ️ **Neutral.** The diff adds only the dedicated-models config block; `http.Server` fields (`ReadHeaderTimeout`, `MaxHeaderBytes`, etc.) are not touched.\n\n**T-032 / T-041** (Solana mnemonic / profile signing identity) — `coordinator/cmd/coordinator/main.go`\n\nℹ️ **Neutral.** No secrets handling added or modified.\n\n---\n\n### New attack surface not covered by an existing threat\n\n**Operator-controlled routing topology via `EIGENINFERENCE_DEDICATED_MODELS`**\n\n`main.go` +168–180: the env-var is read with `os.LookupEnv` and fed into `registry.ParseDedicatedModels`. This is a coordinator-side config variable (not a provider-supplied value), so ADV-001 cannot influence it directly. However:\n\n1. **Log-based enumeration** (`main.go` +180): `logger.Info(\"dedicated-model routing ENABLED\", \"patterns\", strings.Join(dedicatedModels, \",\"))` emits the full pattern list to structured logs. If logs are accessible to any consumer or provider (e.g. via a log-aggregation misconfiguration), this leaks the exact routing topology — which model families have dedicated boxes and by implication which do not. This is low severity but worth being deliberate about the log destination.\n\n2. **Shed-to-OpenRouter path**: the comment at `main.go` +163 (\"sheds to OpenRouter as a transient 429\") introduces a new reachable code path where a request for a dedicated model with no available dedicated box gets a different response code and destination than other capacity-shed requests. A consumer (ADV-002) who systematically sends requests for `gemma-4` variants could probe for the presence/absence of dedicated capacity by observing the 429 cadence. This is informational leakage about infrastructure topology — not covered by any existing threat. Severity is low given the non-sensitive nature of the information, but it should be acknowledged.\n\nNo existing threat ID maps cleanly to either of these; the closest is T-006 (registry flooding/enumeration), but that targets the provider WebSocket, not consumer-facing response codes.\n\n---\n\n### Open findings resolved\n\nNone — this PR does not touch any SEC-* open finding.\n\n---\n\n### Summary for reviewers\n\nThe core routing logic changes (`providerExcludedByDedicatedRuleLocked` calls in `registry.go`) are straightforward exclusion filters that cannot be influenced by adversarial providers. The only items to consider before merge are:\n\n- Confirm `logger.Info` for dedicated-model patterns goes only to operator-accessible sinks, not to any consumer-visible telemetry endpoint.\n- Decide whether the 429 response for \"no dedicated box available\" leaks topology information at a sensitivity level that warrants a generic error message instead.\n\n---\n*🔐 Threat model: [`docs/threat-model.yaml`](../blob/HEAD/docs/threat-model.yaml) · Updates on each push to this PR*","[vc]: #dZASsr2oD4u7pBpydIsKkfg7EdXlJpCsuO6w0z/XoUg=:eyJpc01vbm9yZXBvIjp0cnVlLCJ0eXBlIjoiZ2l0aHViIiwicHJvamVjdHMiOlt7Im5hbWUiOiJkLWluZmVyZW5jZSIsInByb2plY3RJZCI6InByal84YTNCaFVhbjBkcUppa1hUb2huYnJwQnFhNkVIIiwicm9vdERpcmVjdG9yeSI6ImNvbnNvbGUtdWkiLCJpbnNwZWN0b3JVcmwiOiJodHRwczovL3ZlcmNlbC5jb20vZWlnZW4tbGFicy9kLWluZmVyZW5jZS9GRG42Mk1ScjR3NGlIS3c4aUtqU3dOOW53M0tMIiwicHJldmlld1VybCI6ImQtaW5mZXJlbmNlLWdpdC1mZWF0LWdlbW1hNC1kZWRpY2F0ZWQtcm91dGluZy1laWdlbi1sYWJzLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiUEVORElORyJ9LHsibmFtZSI6ImQtaW5mZXJlbmNlLWxhbmRpbmciLCJwcm9qZWN0SWQiOiJwcmpfUUZKa0xma1pEUkRqV0ZsREpEQ1FTMWR4MkR2ViIsInJvb3REaXJlY3RvcnkiOiJsYW5kaW5nIiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL2VpZ2VuLWxhYnMvZC1pbmZlcmVuY2UtbGFuZGluZy9BZTV1NnpVTFJNODVXa29QNlVqd3hCR0xDSlA3IiwicHJldmlld1VybCI6ImQtaW5mZXJlbmNlLWxhbmRpbmctZ2l0LWZlYXQtZ2VtbWE0LWRlZGljYXRlZC05MGNhOTMtZWlnZW4tbGFicy52ZXJjZWwuYXBwIiwibmV4dENvbW1pdFN0YXR1cyI6IkRFUExPWUVEIn0seyJuYW1lIjoiZC1pbmZlcmVuY2UtY29uc29sZS11aS1kZXYiLCJwcm9qZWN0SWQiOiJwcmpfMTJORnVUcElVNmdxS095R3lVU0FlRlF1QWhkcyIsInJvb3REaXJlY3RvcnkiOiJjb25zb2xlLXVpIiwiaW5zcGVjdG9yVXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL2VpZ2VuLWxhYnMvZC1pbmZlcmVuY2UtY29uc29sZS11aS1kZXYvQmEyUDl6bTlwd2Rra2tKWTg1VlBWVUtQTkdpcSIsInByZXZpZXdVcmwiOiJkLWluZmVyZW5jZS1jb25zb2xlLXVpLWRldi1naXQtZmVhdC1nZW1tYTQtZGUtODViYjFlLWVpZ2VuLWxhYnMudmVyY2VsLmFwcCIsIm5leHRDb21taXRTdGF0dXMiOiJQRU5ESU5HIn1dLCJyZXF1ZXN0UmV2aWV3VXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL3ZlcmNlbC1hZ2VudC9yZXF1ZXN0LXJldmlldz9vd25lcj1MYXlyLUxhYnMmcmVwbz1kLWluZmVyZW5jZSZwcj00MzQifQ==\nThe latest updates on your projects. Learn more about [Vercel for GitHub](https://vercel.link/github-learn-more).\n\n| Project | Deployment | Actions | Updated (UTC) |\n| :--- | :----- | :------ | :------ |\n| [d-inference](https://vercel.com/eigen-labs/d-inference) | ![Building](https://vercel.com/static/status/building.svg) [Building](https://vercel.com/eigen-labs/d-inference/FDn62MRr4w4iHKw8iKjSwN9nw3KL) | [Preview](https://d-inference-git-feat-gemma4-dedicated-routing-eigen-labs.vercel.app) | Jun 21, 2026 1:02am |\n| [d-inference-console-ui-dev](https://vercel.com/eigen-labs/d-inference-console-ui-dev) | ![Building](https://vercel.com/static/status/building.svg) [Building](https://vercel.com/eigen-labs/d-inference-console-ui-dev/Ba2P9zm9pwdkkkJY85VPVUKPNGiq) | [Preview](https://d-inference-console-ui-dev-git-feat-gemma4-de-85bb1e-eigen-labs.vercel.app) | Jun 21, 2026 1:02am |\n| [d-inference-landing](https://vercel.com/eigen-labs/d-inference-landing) | ![Ready](https://vercel.com/static/status/ready.svg) [Ready](https://vercel.com/eigen-labs/d-inference-landing/Ae5u6zULRM85WkoP6UjwxBGLCJP7) | [Preview](https://d-inference-landing-git-feat-gemma4-dedicated-90ca93-eigen-labs.vercel.app) | Jun 21, 2026 1:02am |\n\n

\n\n\n\n","base_ref","head_ref","pull_request_info",[4567],{"_941":46,"_46":4568},"Thanks @chatgpt-codex-connector — addressed both findings in `f28e89a9`:\n\n**① Shed saturated dedicated models before queueing** — ✅ fixed. Dedicated-family models now **bypass queue-before-shed when their dedicated boxes are saturated** (`candidateCount==0 && capacityRejections>0`) and fast-429 with `Retry-After` instead of sitting in the 120s queue — holding an OpenRouter request that long would blow its TTFT SLA. Applied in both `handleChatCompletions` and `handleGenericInference`; non-dedicated models keep queue-before-shed unchanged. New test: `TestDedicatedSaturatedBoxFast429` (saturated dedicated box + queue-before-shed left ON → immediate 429).\n\n**② Apply routing gates before treating a provider as available** — intentionally **left at 429** (product decision). For a dedicated model, we prefer a transient `429 + Retry-After` (clean OpenRouter failover) over an \"honest\" `503` even in the narrow case where the only advertiser is structurally unroutable (private-only / sub-trust / unverified). Routing itself is unaffected — the gate still excludes those providers; this only concerns the shed status code, and 429 is the desired behavior for OpenRouter compatibility.\n\nNet effect: every \"can't serve a dedicated model right now\" case (no dedicated box, or dedicated boxes saturated) now sheds as **429 + Retry-After**; only a model genuinely absent from the fleet returns 503.\n",[4941,4942],[],{"_946":947,"_46":948,"_949":950},{"_920":4932},{"_499":4931},{"_499":4697},{"_463":464,"_461":465,"_466":467,"_468":59,"_469":15,"_470":15,"_471":472,"_473":474,"_475":433,"_476":127,"_477":478,"_479":480,"_481":482,"_483":-5,"_484":-5,"_485":4696,"_487":488,"_489":490,"_491":492,"_493":494},{"_42":123,"_379":380,"_381":382,"_383":15,"_384":385,"_386":387,"_388":-5,"_389":-5,"_390":-5,"_391":15,"_392":4686,"_394":385,"_395":385,"_396":4687,"_398":4688,"_400":4689,"_402":-5,"_403":404,"_405":406,"_407":408,"_409":4690,"_55":-5,"_119":411,"_412":4691,"_414":4692,"_416":417,"_418":15,"_419":15},{"_133":4679},[4579,4580,4581,4582,4583,4584,4585,4586,4587,4588,4589,4590,4591,4592,4593,4594,4595,4596,4597,4598,4599,4600,4601,4602,4603,4604,4605,4606,4607,4608,4609,4610,4611,4612,4613,4614,4615,4616,4617,4618,4619,4620,4621,4622,4623,4624,4625,4626,4627,4628,4629,4630,4631,4632,4633,4634,4635,4636,4637,4638,4639,4640,4641,4642,4643,4644,4645,4646,4647,4648,4649,4650,4651,4652,4653,4654,4655,4656,4657,4658,4659,4660,4661,4662,4663,4664,4665,4666,4667,4668,4669,4670,4671,4672,4673,4674,4675,4676,4677,4678],{"_242":358,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":356,"_244":245,"_246":247,"_248":249,"_250":357,"_252":253,"_254":32,"_255":-5},{"_242":355,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":354,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":353,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":352,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":351,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":350,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":349,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":347,"_244":245,"_246":247,"_248":249,"_250":348,"_252":253,"_254":32,"_255":-5},{"_242":346,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":345,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":344,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":343,"_244":245,"_246":247,"_248":249,"_250":298,"_252":253,"_254":32,"_255":-5},{"_242":342,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":341,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":340,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":339,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":338,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":337,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":336,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":335,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":334,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":333,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":332,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":331,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":330,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":329,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":328,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":327,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":326,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":325,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":324,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":323,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":322,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":321,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":320,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":319,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":318,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":317,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":316,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":315,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":314,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":313,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":312,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":311,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":310,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":309,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":308,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":307,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":306,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":305,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":304,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":303,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":302,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":301,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":300,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":299,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":297,"_244":245,"_246":247,"_248":249,"_250":298,"_252":253,"_254":32,"_255":-5},{"_242":296,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":295,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":294,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":293,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":292,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":291,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":290,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":289,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":288,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":287,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":286,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":284,"_244":245,"_246":247,"_248":249,"_250":285,"_252":253,"_254":32,"_255":-5},{"_242":283,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":282,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":281,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":280,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":279,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":278,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":277,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":276,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":275,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":274,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":273,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":272,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":271,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":270,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":269,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":268,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":267,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":266,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":265,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":264,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":263,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":262,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":261,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":260,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":259,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":258,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":257,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":256,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},{"_242":243,"_244":245,"_246":247,"_248":249,"_250":251,"_252":253,"_254":32,"_255":-5},[4680,4681,4682],{"_363":4685,"_365":376},{"_363":4684,"_365":372},{"_363":4683,"_365":366},{"_367":368,"_92":369,"_119":370},{"_367":373,"_92":369,"_119":374},{"_367":377,"_92":369,"_119":378},[423],{"_423":4693},{},{},[],{"_422":59},{"_420":59,"_421":385},{"_42":425,"_367":382,"_426":4694,"_428":429,"_430":431,"_432":433,"_434":435,"_436":437,"_438":439,"_440":64,"_441":-5,"_442":-5,"_443":-5,"_444":-5,"_445":-5,"_446":-5,"_447":-5,"_254":448,"_449":-5,"_55":4695},{"_455":456,"_42":457,"_92":458,"_459":460,"_461":462},{"_451":15,"_452":15,"_453":15,"_454":15},{"_455":495,"_367":495,"_496":-5,"_459":497,"_42":498},[4698,4699,4700,4701,4702,4703,4704,4705,4706,4707,4708,4709,4710,4711,4712,4713,4714,4715,4716,4717,4718,4719,4720,4721,4722,4723,4724,4725,4726,4727,4728,4729,4730,4731,4732,4733,4734,4735,4736,4737,4738,4739,4740,4741,4742,4743,4744,4745,4746,4747,4748,4749,4750,4751,4752,4753],{"_248":892,"_558":4927},{"_248":573,"_558":4924},{"_248":882,"_558":4921},{"_248":577,"_558":4916},{"_248":577,"_558":4911},{"_248":577,"_558":4907},{"_248":592,"_558":4902},{"_248":592,"_558":4898},{"_248":577,"_558":4894},{"_248":577,"_558":4892},{"_248":592,"_558":4888},{"_248":577,"_558":4886},{"_248":592,"_558":4882},{"_248":577,"_558":4879},{"_248":577,"_558":4877},{"_248":592,"_558":4873},{"_248":577,"_558":4871},{"_248":577,"_558":4869},{"_248":592,"_558":4863},{"_248":577,"_558":4861},{"_248":592,"_558":4857},{"_248":577,"_558":4854},{"_248":577,"_558":4852},{"_248":592,"_558":4848},{"_248":577,"_558":4846},{"_248":592,"_558":4842},{"_248":577,"_558":4839},{"_248":577,"_558":4837},{"_248":577,"_558":4835},{"_248":592,"_558":4831},{"_248":577,"_558":4828},{"_248":577,"_558":4826},{"_248":577,"_558":4824},{"_248":592,"_558":4820},{"_248":577,"_558":4817},{"_248":577,"_558":4815},{"_248":577,"_558":4813},{"_248":592,"_558":4809},{"_248":577,"_558":4806},{"_248":577,"_558":4804},{"_248":577,"_558":4802},{"_248":592,"_558":4798},{"_248":592,"_558":4794},{"_248":577,"_558":4791},{"_248":577,"_558":4789},{"_248":592,"_558":4785},{"_248":577,"_558":4783},{"_248":592,"_558":4779},{"_248":577,"_558":4775},{"_248":577,"_558":4773},{"_248":592,"_558":4767},{"_248":577,"_558":4765},{"_248":592,"_558":4763},{"_248":577,"_558":4759},{"_248":573,"_558":4757},{"_248":557,"_558":4754},{"_560":561,"_87":4755},{"_563":564,"_565":566,"_103":-5,"_42":567,"_568":4756,"_570":571,"_572":113},[],{"_572":4758,"_560":561},{"_92":576},{"_579":4760,"_560":561,"_581":567},{"_582":4761,"_584":-5,"_42":-5,"_585":-5,"_586":587,"_96":93,"_92":588},[4762],{"_46":590,"_92":591},{"_579":4764,"_560":561,"_581":567},{"_42":595,"_586":587,"_46":590,"_92":596},{"_579":4766,"_560":561,"_581":567},{"_599":600,"_42":-5,"_585":-5,"_601":602,"_92":603},{"_579":4768,"_560":561,"_581":567},{"_606":607,"_608":609,"_610":4769,"_612":613,"_565":32,"_614":32,"_42":600,"_615":616,"_617":618,"_572":113,"_92":619},[4770,4771,4772],{"_608":630,"_367":631,"_255":632,"_92":626},{"_608":627,"_367":628,"_255":629,"_92":626},{"_608":623,"_367":624,"_255":625,"_92":626},{"_579":4774,"_560":561,"_581":567},{"_635":636,"_599":600,"_42":-5,"_585":-5,"_367":637,"_638":-5,"_92":639},{"_579":4776,"_560":561,"_581":567},{"_582":-5,"_642":643,"_42":-5,"_585":-5,"_644":4777,"_92":646},[4778],{"_46":648,"_92":649},{"_579":4780,"_560":561,"_581":567},{"_582":4781,"_42":653,"_644":4782,"_92":646},[],[648],{"_579":4784,"_560":561,"_581":567},{"_599":657,"_42":-5,"_585":-5,"_601":658,"_92":603},{"_579":4786,"_560":561,"_581":567},{"_606":-5,"_608":661,"_610":4787,"_612":613,"_565":32,"_614":32,"_42":657,"_615":663,"_617":618,"_572":113,"_92":619},[4788],{"_608":665,"_92":666},{"_579":4790,"_560":561,"_581":567},{"_635":669,"_599":657,"_42":-5,"_585":-5,"_367":637,"_638":-5,"_92":639},{"_579":4792,"_560":561,"_581":567},{"_582":-5,"_642":672,"_42":-5,"_585":-5,"_644":4793,"_92":646},[],{"_579":4795,"_560":561,"_581":567},{"_582":4796,"_42":677,"_644":4797,"_92":646},[],[],{"_579":4799,"_560":561,"_581":567},{"_606":681,"_608":682,"_610":4800,"_612":613,"_565":684,"_614":685,"_42":686,"_615":687,"_617":618,"_572":688,"_92":619},[4801],{"_608":690,"_92":666},{"_579":4803,"_560":561,"_581":567},{"_599":693,"_42":-5,"_585":-5,"_601":694,"_92":603},{"_579":4805,"_560":561,"_581":567},{"_635":697,"_599":693,"_42":-5,"_585":-5,"_367":698,"_638":-5,"_92":639},{"_579":4807,"_560":561,"_581":567},{"_582":-5,"_642":701,"_42":-5,"_585":-5,"_644":4808,"_92":646},[],{"_579":4810,"_560":561,"_581":567},{"_582":4811,"_42":706,"_644":4812,"_92":646},[],[],{"_579":4814,"_560":561,"_581":567},{"_599":710,"_42":-5,"_585":-5,"_601":711,"_92":603},{"_579":4816,"_560":561,"_581":567},{"_635":714,"_599":710,"_42":-5,"_585":-5,"_367":698,"_638":-5,"_92":639},{"_579":4818,"_560":561,"_581":567},{"_582":-5,"_642":717,"_42":-5,"_585":-5,"_644":4819,"_92":646},[],{"_579":4821,"_560":561,"_581":567},{"_582":4822,"_42":722,"_644":4823,"_92":646},[],[],{"_579":4825,"_560":561,"_581":567},{"_599":726,"_42":-5,"_585":-5,"_601":727,"_92":603},{"_579":4827,"_560":561,"_581":567},{"_635":730,"_599":726,"_42":-5,"_585":-5,"_367":698,"_638":-5,"_92":639},{"_579":4829,"_560":561,"_581":567},{"_582":-5,"_642":733,"_42":-5,"_585":-5,"_644":4830,"_92":646},[],{"_579":4832,"_560":561,"_581":567},{"_582":4833,"_42":738,"_644":4834,"_92":646},[],[],{"_579":4836,"_560":561,"_581":567},{"_599":686,"_42":-5,"_585":-5,"_601":742,"_92":603},{"_579":4838,"_560":561,"_581":567},{"_635":745,"_599":686,"_42":-5,"_585":-5,"_367":637,"_638":-5,"_92":639},{"_579":4840,"_560":561,"_581":567},{"_582":-5,"_642":748,"_42":-5,"_585":-5,"_644":4841,"_92":646},[],{"_579":4843,"_560":561,"_581":567},{"_582":4844,"_42":753,"_644":4845,"_92":646},[],[],{"_579":4847,"_560":561,"_581":567},{"_599":757,"_42":-5,"_585":-5,"_601":758,"_92":603},{"_579":4849,"_560":561,"_581":567},{"_606":761,"_608":762,"_610":4850,"_612":613,"_565":32,"_614":685,"_42":757,"_615":764,"_617":618,"_572":688,"_92":619},[4851],{"_608":766,"_92":666},{"_579":4853,"_560":561,"_581":567},{"_635":769,"_599":757,"_42":-5,"_585":-5,"_367":637,"_638":-5,"_92":639},{"_579":4855,"_560":561,"_581":567},{"_582":-5,"_642":772,"_42":-5,"_585":-5,"_644":4856,"_92":646},[],{"_579":4858,"_560":561,"_581":567},{"_582":4859,"_42":777,"_644":4860,"_92":646},[],[],{"_579":4862,"_560":561,"_581":567},{"_599":781,"_42":-5,"_585":-5,"_601":782,"_92":603},{"_579":4864,"_560":561,"_581":567},{"_606":785,"_608":786,"_610":4865,"_612":613,"_565":32,"_614":32,"_42":781,"_615":788,"_617":618,"_572":113,"_92":619},[4866,4867,4868],{"_608":794,"_367":631,"_255":632,"_92":626},{"_608":793,"_367":628,"_255":629,"_92":626},{"_608":792,"_367":624,"_255":625,"_92":626},{"_579":4870,"_560":561,"_581":567},{"_635":797,"_599":781,"_42":-5,"_585":-5,"_367":637,"_638":-5,"_92":639},{"_579":4872,"_560":561,"_581":567},{"_599":800,"_42":-5,"_585":-5,"_601":801,"_92":603},{"_579":4874,"_560":561,"_581":567},{"_606":804,"_608":805,"_610":4875,"_612":613,"_565":32,"_614":32,"_42":800,"_615":807,"_617":618,"_572":113,"_92":619},[4876],{"_608":809,"_255":810,"_811":812,"_92":813},{"_579":4878,"_560":561,"_581":567},{"_635":816,"_599":800,"_42":-5,"_585":-5,"_367":637,"_638":-5,"_92":639},{"_579":4880,"_560":561,"_581":567},{"_582":-5,"_642":819,"_42":-5,"_585":-5,"_644":4881,"_92":646},[],{"_579":4883,"_560":561,"_581":567},{"_582":4884,"_42":824,"_644":4885,"_92":646},[],[],{"_579":4887,"_560":561,"_581":567},{"_599":828,"_42":-5,"_585":-5,"_601":829,"_92":603},{"_579":4889,"_560":561,"_581":567},{"_606":832,"_608":833,"_610":4890,"_612":613,"_565":32,"_614":32,"_42":828,"_615":835,"_617":618,"_572":113,"_92":619},[4891],{"_608":837,"_92":666},{"_579":4893,"_560":561,"_581":567},{"_635":840,"_599":828,"_42":-5,"_585":-5,"_367":637,"_638":-5,"_92":639},{"_579":4895,"_560":561,"_581":567},{"_582":-5,"_642":843,"_42":-5,"_585":-5,"_644":4896,"_92":646},[4897],{"_46":846,"_92":649},{"_579":4899,"_560":561,"_581":567},{"_582":4900,"_42":850,"_644":4901,"_92":646},[],[846],{"_579":4903,"_560":561,"_581":567},{"_854":-5,"_582":4904,"_42":856,"_92":857},[4905],{"_46":859,"_860":4906,"_92":46},[],{"_579":4908,"_560":561,"_581":567},{"_582":4909,"_584":-5,"_42":-5,"_585":-5,"_586":-5,"_96":485,"_92":588},[4910],{"_46":859,"_92":866},{"_579":4912,"_560":561,"_581":567},{"_582":4913,"_584":-5,"_42":-5,"_585":-5,"_586":-5,"_96":485,"_92":588},[4914,4915],{"_46":873,"_92":866},{"_46":872,"_92":866},{"_579":4917,"_560":561,"_581":567},{"_582":4918,"_584":-5,"_42":-5,"_585":-5,"_586":-5,"_96":877,"_92":588},[4919,4920],{"_46":881,"_92":866},{"_46":880,"_92":866},{"_560":561,"_87":4922},{"_103":-5,"_42":567,"_568":4923,"_570":571,"_572":886},[],{"_572":4925,"_560":561},{"_889":4926,"_92":891},[],{"_894":4928},{"_896":-5,"_897":-5,"_898":899,"_900":901,"_612":613,"_902":15,"_903":-5,"_904":-5,"_905":-5,"_906":907,"_42":561,"_908":909,"_367":-5,"_910":-5,"_255":911,"_912":385,"_913":-5,"_617":914,"_572":4929,"_916":-5,"_77":4930,"_918":901},{"_92":576},[],[],[4933],{"_42":923,"_924":4934,"_926":927,"_928":927,"_582":4935,"_572":930,"_584":-5,"_931":32,"_932":4936,"_934":935,"_936":-5},{"_96":485,"_367":-5,"_932":4940},{"_941":46,"_942":4939},{"_937":4937},{"_939":4938},[],[944],{},{"_92":588,"_96":93,"_582":4956},{"_92":953,"_954":4943},[4944,4945,4946],{"_255":1182,"_960":-5,"_961":4953,"_963":-5},{"_255":1076,"_960":-5,"_961":4950,"_963":-5},{"_255":959,"_960":-5,"_961":4947,"_963":-5},[4948],{"_965":685,"_966":967,"_582":4949,"_969":59},[970,385,971,385,972,973,974,975,976,977,978,979,385,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,385,995,996,997,998,999,1000,1001,987,988,989,990,1002,992,1003,1004,1005,1006,1007,992,1008,994,385,1009,1010,1011,1012,1013,1014,992,1015,1016,1017,1018,990,992,1019,994,385,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,989,990,1030,1031,1032,990,992,1033,994,385,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,992,1046,994,385,1047,1048,1049,1050,1051,1052,1053,1054,1055,994,385,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,992,1052,1053,1067,1068,989,990,1069,1070,1071,1072,1073,1074,990,992,1075,994],[4951],{"_965":685,"_966":1079,"_582":4952,"_969":59},[1081,385,1082,1083,1084,1085,1086,1087,1088,385,1089,1090,1091,385,1092,1093,1094,1095,1096,1097,1098,992,1099,1100,1101,1097,1098,992,1102,1103,1104,994,385,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,385,1116,1117,385,1118,1119,1120,1121,1122,1123,1124,1125,1126,385,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1132,385,1137,1138,1097,1139,992,1140,1141,992,1142,1143,992,1144,1145,992,385,1146,1147,1097,1148,992,1149,1150,992,1151,1152,992,385,1153,1154,1155,1156,1097,1157,992,1099,1100,1101,1097,1158,992,1102,1159,1160,1161,992,1162,1163,992,994,385,1164,1165,1166,1167,1168,1111,1112,1113,1169,385,1116,1117,385,1118,1170,1121,1122,1124,1125,1126,385,1127,1128,1171,1172,1173,1174,1175,1176,1177,1132,1133,1134,1135,1178,1132,385,1138,1097,1179,992,1140,1180,992,1144,1181,992,994],[4954],{"_965":685,"_966":1185,"_582":4955,"_969":59},[1081,385,1187,1188,1189,1190,1191,1188,1192,1193,1194,1195,1196,1197,1198,385,1082,1083,1199,1200,1201,1202,1084,1203,1204,1085,1205,1206,1086,1088,385,1207,1208,1209,1089,1210,1211,1212,1213,385,1214,1091,385,1215,1216,1217,1218,1219,1220,385,1221,1222,1223,1224,1225,1226,1227,1228,1229,385,1230,1231,1232,1233,385,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,385,1246,1247,1248,1249,1250,385,1251,1252,1253,1254,1255,1256,1257,385,1258,1259,1260,1261,1091,385,1262,385,1263,1264,1265,1266,1267,1268,1269,994,385,1270,1271,1272,1273,1274,1275,1276,1277,1278,1045,992,1279,1280,1281,992,1282,1283,1284,1285,1286,1287,1288,1289,1290,1291,1292,1293,1294,1295,1296,1297,1298,1299,1300,1132,1301,1302,1303,1304,1305,994,385,1306,1307,1308,1309,1310,1311,1007,992,1312,1313,1097,1314,1007,992,1315,1117,1316,1317,1318,992,994,385,1319,1311,1320,992,1321,994,385,1322,1323,1324,1325,1188,1326,1327,1328,1329,1330,1331,1007,992,1332,1333,1334,1335,1336,992,994,385,1337,1338,1339,1340,1341,1342,1343,1344,1007,992,1345,1346,1007,992,1347,1348,1349,994,385,1350,1351,1352,1353,1354,1355,1188,1356,1357,1358,1359,1360,1007,992,1361,1362,992,1363,1364,1365,1366,1367,1368,1369,992,994,385,1370,1371,1372,1373,1374,1007,992,1375,1376,1377,1378,1379,992,994,385,1380,1381,1382,1383,1384,1385,1386,1387,1188,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,992,1400,1401,1045,992,1402,1403,1305,994,385,1404,1405,1406,1407,1408,1409,1410,1411,992,1412,1413,1414,994,385,1415,1416,1417,1418,1419,385,1420,1421,1422,1423,1424,385,1425,1426,1427,1428,1429,1430,992,1431,994,385,1432,1433,1434,1435,1436,1437,1438,1439,1440,992,1441,1442,1443,994,385,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,992,1457,1458,992,1459,1460,1097,1461,992,1462,994,385,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,992,1473,1474,1472,992,1475,1472,992,1476,1472,992,1477,1478,1479,992,1480,1481,994,385,1482,1471,1472,992,1473,1474,1472,992,1475,1472,992,1476,1472,992,1477,1483,1484,992,1480,1481,994,385,1485,1486,994,385,1487,1488,1489,992,1490,994,385,1491,1492,1493,1494,1495,992,1496,1497,992,1498,1499,992,1500,994,385,1501,1502,1301,1503,1302,1504,1304,994,385,1505,1506,1007,992,1507,994,385,1508,1509,1007,992,1510,1511,1512,1007,992,1513,1514,994,385,1515,1516,1517,1518,1519,992,1520,994,385,1521,1522,1523,1524,1525,1526,992,1527,1528,1529,1530,990,992,1531,1526,992,1532,1097,1526,992,1533,994,385,1534,1535,1536,1537,385,1538,385,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,992,385,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,992,1615,1616,1617,385,1618,1619,1620,1621,990,1622,992,385,1623,1624,1625,1626,1627,1628,990,1629,1630,1631,1632,1633,990,1634,992,1635,1636,1637,1638,1639,1640,990,992,1641,1642,1643,992,1644,1645,992,385,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,992,385,1657,1658,1659,992,385,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,990,992,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,990,992,385,1683,1684,1685,1686,1687,1688,992,385,1689,1097,1685,1686,1687,1690,992,385,1691,1097,1685,1686,1692,992,385,1693,1694,1695,1696,1097,1685,1686,1697,992,1642,1698,992,385,1699,1700,1701,1702,1703,1704,1131,992,385,1705,1706,1707,385,1708,1097,1685,1686,1709,992,1642,1710,992,1711,1685,1686,1687,1712,992,1713,385,1714,994,385,1715,1716,1717,1718,1719,1720,1721,1720,1722,1720,1723,1724,1664,1725,990,1726,1727,1728,992,994,385,1729,1730,1731,1732,1733,1188,1734,1735,1736,1737,1519,992,1716,1738,1739,1740,990,1741,1742,1743,990,1744,1727,1745,1664,1740,990,1746,1742,1743,990,1744,992,994,385,1747,1748,1749,1750,1751,1188,1752,1753,1754,1755,1756,1737,1519,992,1716,1738,1757,1727,1745,1664,1740,990,1758,992,994,385,1759,1760,1761,1762,992,1763,1764,992,1765,1766,992,1767,1768,992,1769,994,385,1770,1771,1772,1773,1774,1760,1761,1775,1776,1777,1778,1779,1780,1781,1782,992,1763,1782,992,1765,1782,992,1767,1783,992,1769,994,385,1784,1785,1786,1787,1788,1215,1789,1790,1091,385,1791,1792,1793,1794,1795,1796,1797,1798,992,1075,994,385,1799,1800,1801,1802,1803,1804,1805,1806,1740,990,1807,1808,990,1809,992,1810,1738,1811,1812,1813,1814,1815,1816,1817,1669,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1823,1669,990,1831,1727,1832,992,994,385,1833,1834,1835,1836,1837,1044,1838,992,1760,1839,1840,1841,1842,989,990,1843,1844,992,1769,994,385,1845,1846,1847,1848,1849,1850,1738,1851,1812,1813,1852,1853,1854,1855,1856,1857,1858,1859,1860,1823,1861,1862,1828,1863,1669,990,1831,1727,1864,992,994,385,1865,1866,1867,1868,1044,1045,992,1869,1870,1841,989,990,1871,1074,990,992,1075,994,385,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1669,990,992,1883,1884,1885,1886,1887,1882,1669,990,992,1075,994,385,1888,1889,1890,1891,1892,1893,1894,1895,994,385,1896,1897,1898,1411,992,1899,1900,1411,992,1901,1902,994,385,1903,1904,1905,1906,1907,1669,1908,990,992,1909,1910,992,1911,994,385,1912,1913,1914,1915,1916,1841,989,990,1917,1918,1919,1920,1921,1922,1923,1924,990,992,1925,1926,992,385,1927,1928,1929,1930,1841,1931,990,1932,1933,989,990,1934,989,990,1935,1936,992,1937,1938,992,1939,994,385,1940,1941,1915,1942,1943,1944,990,992,1945,994,385,1946,1947,1948,1949,1950,1951,385,1952,1953,385,1954,1955,1956,1957,1958,1959,1960,1961,992,1962,1963,992,1964,1965,1961,992,1941,1966,1942,1943,1944,990,992,1967,1961,992,1968,1969,992,1970,994,385,1971,1972,1973,1974,1975,1976,1977,1978,385,1979,1980,1981,1904,1905,1908,990,992,1982,994,385,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,994,385,1994,1995,1045,992,1996,1997,992,1998,1999,2000,2001,2002,990,2003,1132,1097,2004,2005,2006,2007,2008,2009,1045,992,2010,1045,992,2011,2012,2013,2014,2015,992,1305,994,385,2016,2017,2018,2019,2020,2021,2022,992,2023,2024,2025,2026,992,2027,2028,992,2029,994,385,2030,2031,2032,2033,2034,2035,994,385,2036,1624,1045,992,2037,2038,2039,994,385,2040,1624,1411,992,2037,2038,2041,994,385,2042,2043,2044,2045,2046,2047,990,992,2048,994,385,2049,2050,2051,2052,2053,2054,1045,992,2055,2056,992,1075,994,385,2057,1344,2058,992,2059,2060,2061,2062,2063,992,2064,2065,2063,992,2066,2067,2068,2069,2070,2071,2072,2073,2074,2075,990,992,2076,2077,2078,992,2079,2080,2081,994,385,2082,2083,2084,2085,2086,2087,2088,2089,2090,2091,2092,2093,2094,2095,2096,2097,2098,1882,1669,990,1045,992,2099,2100,2101,2102,992,1305,994,385,2103,2104,2105,2106,992,993,994,385,2107,1810,2108,1411,1738,2109,1812,2110,1814,2111,1854,2112,2113,1823,1856,2114,2115,1860,1823,2116,2117,2118,2119,1823,1669,990,2120,1727,2121,1664,2122,990,2123,992,994,385,2124,2125,1738,2126,1812,2127,2128,2129,1816,1817,1669,2130,2131,2132,2133,2134,2135,2136,2137,2138,2136,2139,2140,2141,2142,2143,2144,2145,2146,2136,2147,2148,2139,2149,2150,2151,2152,2153,2154,2155,2156,2157,2158,2142,2159,2144,2139,2160,2161,2162,2142,2163,2164,1860,2165,2166,1823,1817,1669,385,2167,2168,1817,1669,2169,2170,1669,2171,2172,2173,2174,990,2175,2176,990,2177,1727,2178,992,994,385,2179,2180,2181,2182,992,2183,2184,2185,1841,989,990,2186,2187,2188,2189,2190,2191,1860,1823,2192,1669,2193,2194,2195,1669,2196,2197,1669,2198,2199,2200,2174,989,990,2201,992,2202,994,385,2203,2204,1044,2205,992,2206,2207,2208,2209,2210,990,2211,2212,2213,2214,2215,2216,992,2217,994,385,2218,2219,1044,1526,992,2220,1044,1526,992,2221,2222,2223,2224,2211,2225,2226,990,992,2227,994,385,2228,2229,1097,2230,992,385,2231,2232,2233,2234,2235,2236,2237,2238,992,2239,2230,2240,2241,992,2242,2230,2243,2244,992,2245,2246,992,2202,994,385,2247,1188,2248,2249,2250,1188,2251,2252,2253,2254,2255,385,2256,2257,2258,1044,1007,992,2259,2260,2261,385,2262,2263,2264,2265,2266,2267,2268,2269,2270,2271,2272,2273,2274,2275,2276,2277,1007,992,385,2278,2279,1909,2267,2268,2269,2280,2271,2272,2273,2274,2281,2275,2276,2282,2283,1007,992,385,2284,1097,2267,2268,2269,2280,2271,2272,2273,2274,2275,2276,2285,1007,992,2286,2287,992,385,2288,2289,2290,2291,2292,2293,385,2294,2295,2296,2297,2298,2299,2300,1044,2267,2268,2301,2302,2303,2272,2273,2274,2275,2276,2304,2305,1007,992,2306,385,2307,2308,2309,2310,2311,2312,2313,2314,2315,2316,2317,2318,2319,2320,2321,2322,1007,992,385,2323,385,2324,2325,2326,2327,2328,2329,2330,2331,2332,2333,2334,2335,2336,1669,990,2337,2338,2339,2340,2341,2342,990,2343,992,385,2344,2345,2346,2347,2348,2349,2350,2287,992,385,2351,2352,2353,2354,2355,2356,2357,1007,992,385,2099,2358,1664,2359,2360,2361,2362,2363,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2174,2374,2375,990,2376,992,385,2377,2378,2379,2380,2381,1044,1007,992,385,2382,2383,2384,2385,2386,2387,2388,2389,2390,2391,2392,2393,2394,2395,2396,2397,2398,2359,2360,2399,2400,2401,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2174,2402,2375,990,2403,2404,1664,1667,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,1669,2375,990,992,2426,385,2427,2428,2429,2430,990,992,385,2431,2432,2433,2267,2434,2435,2436,2437,2438,2439,2440,2441,2442,2443,2444,2445,2446,2447,2276,2448,2449,1007,992,385,2450,2451,2452,2453,2454,2375,990,2455,2456,2457,2458,2459,2460,2461,2101,2462,2463,2464,2465,2466,2467,2468,2469,2470,2471,2472,2473,2474,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2136,2497,2498,2499,1823,1669,990,2500,2501,2502,2503,2504,2505,2506,2375,990,2507,2508,2509,2454,2510,2359,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2174,2530,2531,2532,2375,990,2470,2533,2534,2535,2536,2537,2538,2539,2540,2541,2542,2543,2544,2545,2546,2547,2548,2549,2423,2550,2551,2552,2553,2554,2555,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2577,2578,2579,1669,990,2580,2581,2582,2583,2584,2585,2586,2587,2588,2589,2590,2591,2592,2593,2594,2595,2596,2586,2597,2534,2535,2598,2599,2600,2601,2602,2603,2604,2605,2606,2607,2608,2552,2553,2554,2609,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2610,2578,2579,2423,2611,2612,2613,2614,2615,2616,2552,2553,2554,2617,2405,2556,2557,2618,2619,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2620,2621,2622,2579,1669,990,2623,2624,2625,2626,2627,2628,2629,2630,2631,2632,2633,2634,2635,2472,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2136,2497,2498,2499,1823,2423,2636,2637,2632,2633,2638,2554,2405,2556,2639,2640,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2570,2571,2572,2573,2574,2641,2420,2642,2579,1669,990,992,385,2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2433,2267,2434,2672,2673,2674,2438,2439,2440,2441,2442,2443,2444,2445,2446,2675,2447,2276,2676,2677,1007,992,385,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697,2698,2699,2700,2701,2702,2703,2704,992,2705,994,385,2706,2707,2708,2709,2710,2711,2712,1007,992,385,2713,1044,2714,1007,992,385,2715,2716,2717,2718,2719,992,2720,385,2721,2722,2723,385,2724,2725,2726,2727,2728,2729,2730,385,2731,2732,2733,2734,2735,385,2736,2737,2738,2739,2740,989,990,2741,2742,990,2743,2744,2745,2746,2747,2748,2749,2750,2751,2752,1669,2753,2754,2755,990,992,385,2756,2757,2758,2759,385,2760,2761,2762,1816,2763,2764,2765,2766,2767,2768,2769,2770,2771,2772,2773,2157,2494,2774,2775,2496,2136,2165,1823,2776,2777,2778,2779,2780,2781,1823,2782,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2136,2806,2807,2808,2809,2805,2136,2810,2811,2812,2813,2814,2815,2816,2805,2817,2808,2809,2805,2818,2819,2820,2821,2822,2823,2824,2825,2826,2827,2828,2829,2830,2494,2831,2775,2136,2832,2833,2780,1823,2579,1669,2834,2835,2836,2837,2838,2839,2763,2840,2165,1823,1669,2841,385,2751,2842,2843,1823,1669,2844,2845,2846,2847,2848,2849,2850,2851,1817,1669,2852,2853,2854,2751,2855,2856,1860,1823,1669,2751,2857,2858,2859,2860,2861,2862,1860,1823,1669,2863,2864,2755,385,2865,1816,1817,1669,2866,2867,2868,2869,2870,2871,2872,2873,2874,2174,2875,2755,2375,385,2876,2877,2878,2879,2880,2755,2375,385,2881,2375,990,992,994,385,2882,2713,1044,2714,1007,992,385,2883,2718,2719,992,385,2884,2885,2886,2887,385,2739,2888,2889,990,992,385,2758,2759,385,2760,2761,2762,1816,2890,2891,2763,2892,2893,2894,2895,2136,2896,1823,2897,2777,2778,2898,2781,1823,2783,2899,2579,1669,2900,2839,2763,2840,2165,1823,1669,2841,385,2865,1816,1817,1669,2866,2867,2868,2869,2901,2375,385,2876,2877,2878,2879,2902,2375,385,2881,2375,990,992,994,385,2903,2904,2905,2906,2907,2908,1117,385,2909,2739,2888,2910,990,992,385,2760,2761,2762,1816,2763,2764,2765,2766,2767,2911,2912,2913,2914,2805,2915,2496,2136,2165,1823,2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,2944,2945,2946,2937,2947,2802,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,2962,2963,2964,2965,2966,2967,2968,2964,2965,2966,2969,2970,2971,2936,2937,2972,2973,2974,2975,2976,2977,2937,2802,2978,2979,2980,2802,2981,2982,2983,2805,2136,1823,385,2984,2985,2763,2986,2987,2988,2989,2990,2496,2136,2991,2992,2993,2994,2995,2136,2996,2997,2998,2999,3000,3001,3002,2994,3003,3004,2136,1823,2579,1669,3005,385,2865,1816,1817,1669,2866,2867,3006,3007,3008,3009,1669,3010,2375,385,3011,3012,3013,3014,3015,2423,3016,3017,1669,2375,990,992,994,385,3018,3019,3020,3021,3022,1007,992,3023,1044,1007,992,3024,3025,3026,3027,1669,990,992,994,385,3028,3029,3030,992,3031,3032,3033,990,992,3023,1044,1007,992,3024,3034,1841,989,990,3035,3036,990,3037,3038,990,992,994,385,3039,3040,3041,3042,3043,3044,3045,3046,992,385,3047,3048,3049,990,3050,992,3051,3052,992,3053,3054,992,3055,3056,3057,990,992,994,385,3058,3059,3060,1007,992,3061,3062,3063,990,1007,992,3064,994,385,3065,3066,3067,3068,3069,3070,3071,3072,1669,990,992,3073,3074,3075,3076,3077,3078,990,3079,992,3080,3081,992,3082,994,385,3083,3084,3085,3086,3087,3088,3089,3090,3091,3092,3093,3094,3095,3096,3097,3098,3099,3100,1440,992,385,3101,3102,1440,992,385,1941,385,3103,3104,3105,3106,3107,1944,990,992,385,3108,3109,3110,3111,3112,3113,3114,3115,3116,3117,3118,3119,2802,2805,3120,3121,3122,2805,3123,3124,3125,3126,3127,3128,3129,3119,2802,3130,3131,3132,3122,2805,3133,3134,2805,2136,1823,1669,3135,3136,1669,990,992,385,1967,1440,992,385,3137,1097,1440,992,3138,994,385,3139,3140,3141,3142,3143,3144,3145,994,385,3146,3147,3148,3149,3150,3151,3152,3153,385,3154,3155,3156,3157,989,990,385,3158,3159,989,990,385,3160,1841,989,990,3161,3162,3163,3164,3165,3166,3167,3168,3169,3170,3171,3172,3173,3174,3175,3176,3163,3164,3165,3177,3178,990,3179,989,990,385,3180,3181,3182,1669,3183,3184,3185,3186,1669,3187,3188,3189,3190,3191,3192,3193,3194,1669,3195,3196,1858,3197,3198,3199,3200,3201,2136,3202,1823,3203,3204,1823,3205,3206,1823,3207,3208,3209,1823,3210,1669,990,992,385,3211,3212,3213,3214,3215,3216,2750,3217,990,992,3218,3219,3220,3221,3222,3223,3224,1669,990,992,3225,994,385,3226,3227,3228,3229,3230,3231,3232,3233,3234,3235,3236,3237,3238,3239,3240,3241,1007,992,3242,1044,1007,992,3243,3244,3245,992,3246,1007,992,3247,3248,3249,994,385,3250,3251,3252,3253,3254,3255,3256,3257,3258,3259,3260,3261,994,385,3262,3263,3264,3265,3266,3267,3268,3269,3260,3270,3271,1045,992,3272,3273,992,3274,1045,992,3275,994,385,3276,3277,3278,3279,3280,3281,1188,3282,3283,3284,3285,3286,3287,3288,3289,3290,1188,3291,3292,3293,3294,3295,3296,3297,3298,3299,3300,3301,3302,3303,990,3304,3305,1882,1669,990,992,3306,3307,3308,1045,992,3309,3310,3311,3312,3313,3314,3315,992,3316,1045,992,3317,1045,992,3318,1045,992,3319,1045,992,3320,3321,1032,990,3322,1032,990,3323,3324,3325,3326,3327,3328,3329,1823,3330,3331,3329,1823,1828,3332,3333,1669,990,992,1305,994,385,3334,3088,3335,3336,3337,992,3338,3337,992,3339,3337,992,3340,3337,992,3341,994,385,3342,3343,3344,3345,3346,3347,3348,3349,3350,992,3351,3352,992,3353,1097,1411,992,3354,994,385,3355,3356,3357,3358,3359,3088,3360,3337,992,3361,3338,3337,992,3362,3363,3364,3365,3366,1669,990,992,3367,994,385,3368,3369,3370,3371,3372,3373,3374,3375,3376,3377,3378,3379,2136,1823,1669,990,992,3351,3352,992,3353,1097,1411,992,3354,994,385,3380,3381,3382,3383,3384,3385,994,385,3386,3387,3388,3389,3390,3391,992,3392,3393,992,3394,3395,992,3396,994,385,3397,3398,3399,992,3400,3401,992,1982,994,385,3402,3403,3404,3405,3406,3407,992,994,385,3408,3409,3410,3411,3412,3413,1727,1526,992,994,385,3414,3415,994,385,3416,3417,3418,3419,3420,3421,3422,3423,3424,3425,2276,3426,992,3427,3419,3428,3429,3430,3431,3432,3433,3434,3435,3425,2276,3426,992,3436,3437,3438,3439,3440,990,3441,3442,3419,3443,3444,3445,3446,3447,3448,2276,3426,992,3449,994,385,3450,3451,3452,3453,3454,3455,2101,3456,992,3457,3458,3459,992,3460,3461,992,3462,3463,992,994,385,3464,3465,3466,3467,3468,3469,3470,3471,3472,3473,3474,992,3475,3476,3477,3478,992,3479,994,385,3480,3481,1526,992,3482,994,385,3483,3484,3485,3486,3487,3488,992,3489,994,385,3490,3491,3492,3493,992,3494,3495,3496,992,385,3497,3151,3498,3499,3500,3501,3502,992,385,3503,3504,3505,3506,3507,3508,3509,992,3510,3511,992,3512,3476,3513,3514,992,3515,994,385,3516,3517,3518,3519,992,3418,3520,992,385,3521,3522,992,3466,385,3523,3524,3525,3526,3527,3528,3529,3530,3531,1177,3532,3533,3534,3535,1131,992,385,3536,3537,3538,3539,3540,3541,990,992,385,3476,3477,3478,992,385,3479,994,385,3542,1188,3543,3544,3545,3546,3547,3548,3549,3550,3551,3552,3553,3554,1007,992,3555,3556,992,994,385,3557,3558,3559,3560,3561,3562,3563,3564,3565,3566,3567,3568,3569,3570,3571,3572,3573,1097,3574,3575,992,385,3576,3577,3578,989,990,3579,3580,3581,3582,3583,3584,3585,3586,3587,990,3588,3589,3590,3591,3592,3593,3594,3595,3596,990,3597,3598,3599,3600,3601,1669,990,3602,3603,3604,3605,3606,3607,1923,3608,989,990,385,3609,3610,3611,3612,3613,3614,3615,1669,990,385,3616,3617,3618,3619,3620,990,3621,3622,3623,3624,3625,3626,3627,990,3628,3629,3630,3631,3632,3633,990,3634,3635,3636,3637,3638,990,3639,3640,3641,3637,3642,990,3643,3644,992,3645,994,385,3646,3647,3648,3649,3650,385,3651,3652,3653,3654,3655,992,385,3656,3657,3658,3659,1097,2230,992,385,3660,3661,3662,385,3663,3664,3665,3666,3667,989,990,3668,989,990,3621,3669,3670,3671,3672,3673,990,3674,3675,3676,3677,3678,2750,3679,3680,3681,990,3682,3683,3684,3685,3686,1669,990,3687,3688,990,385,3628,3689,3690,3691,3692,3693,3694,3695,990,385,3696,3697,3698,3699,3700,385,3701,3641,3637,3642,990,3702,385,3703,992,385,3704,994,385,3705,3706,3707,1097,3708,3709,1007,992,385,3710,3711,3712,1132,994,385,3713,3714,3715,3716,3717,3718,3719,1097,3708,3709,1007,992,3720,3721,3722,2375,990,992,3723,3724,994,385,3725,3726,3727,3728,3729,3730,3731,3732,1007,992,385,3733,1097,3734,1007,992,3735,3736,3737,1132,994,385,3738,3739,3740,3741,3729,3730,3742,1007,992,385,3743,3744,992,3745,3746,1007,992,385,3747,3748,3749,1007,992,385,3750,3751,1007,992,3752,385,3753,994,385,3754,385,3755,3756,3757,3758,3759,3760,3761,3762,3763,3764,3765,3766,994,385,3767,3768,385,3769,3770,385,3771,3772,3773,3774,3775,3776,3777,3778,3779,3780,3781,3782,3783,3784,3785,3786,3787,992,3788,3789,3790,3791,3792,3793,3794,3795,3796,990,3797,3798,992,3479,994,385,3799,3800,3801,3802,3803,1727,3804,992,3805,3806,992,3807,3808,992,3809,3810,992,3811,3812,992,3813,3814,992,3815,994,385,3816,3817,3818,3819,3820,3821,1798,992,3822,3823,1074,990,992,1075,994,385,3824,3825,3826,3827,3828,3829,3830,3820,3831,3832,992,3833,3789,3834,3835,3836,3837,990,3838,3839,992,3840,994,385,3841,3842,3729,3730,3742,1007,992,3843,1097,3844,1007,992,3845,3846,3847,992,3848,994,385,3849,3850,3851,3729,3730,3731,3732,1007,992,385,3852,3853,3854,3855,3856,2375,990,992,3857,3858,1007,992,385,3859,3860,3861,3862,3863,3864,3865,3866,3867,992,3868,3869,3870,992,385,3871,1097,3734,1007,992,3872,3873,3874,1132,994,385,3875,3876,3729,3730,3742,1007,992,3718,3877,1097,3878,1007,992,3879,994,385,3880,3881,3882,3883,3884,3885,3886,1007,992,3879,994,385,3887,3888,3889,3729,3730,3742,1007,992,3718,3890,1097,3878,1007,992,385,3891,3892,3893,3894,3895,1007,992,3896,3858,1007,992,3897,3858,1007,992,385,3898,3899,3900,3901,3902,1097,3903,1007,992,3901,3904,994,385,3905,3906,3907,3908,3909,3910,3911,990,3912,992,3913,3914,3915,3916,990,3917,992,3918,3919,3920,3921,990,3922,992,3923,3924,3925,2750,3926,3927,3928,1669,3929,3930,1669,3931,3932,990,992,3933,3924,3934,2750,3935,3936,3937,1669,3938,990,992,3939,3940,3941,3942,990,3943,992,3944,3945,3946,3947,3948,3949,3950,3951,2423,3952,3953,3954,1823,3955,1669,990,992,3956,3924,3957,2750,3958,3959,3960,1669,3961,990,992,3815,994,385,3962,3963,3729,3730,3742,1007,992,3718,3901,3964,3878,1007,992,3901,3753,994,385,3965,3966,3967,3968,3729,3730,3742,1007,992,3718,3969,3970,3901,3971,1097,3878,1007,992,3901,3872,3873,3874,1132,994,385,3972,3973,3974,3975,3976,3977,3978,3979,3980,3981,1132,994,385,3982,3983,3984,3985,3986,3987,3988,3989,1007,992,385,3990,3991,3992,3993,3994,3995,3996,3997,3998,3999,4000,4001,990,2101,4002,4003,4004,4005,990,4006,3993,4007,4008,990,992,4009,1097,4010,1007,992,4011,4012,994,385,4013,385,4014,4015,4016,2657,4017,4018,385,4019,4020,4021,4022,4023,1132,994,385,4024,4025,4026,4027,4028,2657,4029,385,4030,4031,4032,4033,4034,4035,4036,4037,1669,4038,4039,4040,1669,4041,4042,4043,4044,4045,4046,4047,2174,990,992,385,4048,4049,1132,994,385,4050,1188,4051,4052,4053,4054,4055,4056,992,4055,4057,1007,992,385,4058,4059,4060,4061,385,4062,4063,4064,4065,4066,4067,4068,4069,4070,990,992,385,4071,4072,4073,4074,4075,4076,4077,4078,4079,4080,4081,4082,4083,2420,4084,4085,1669,990,992,385,4086,4087,992,385,4088,4089,4090,4091,4092,4093,4094,4095,1132,994,385,4096,385,4097,4098,4099,4100,4101,4102,994,385,4103,4104,4105,4106,994,385,4107,4108,4109,4110,994,385,4111,4112,4113,4114,2255,385,4115,4116,4117,4118,2258,1044,1007,992,2259,2260,2261,385,2284,1097,2267,2268,2269,2280,2271,2272,2273,2274,2275,2276,2285,1007,992,4119,4120,992,385,4121,2293,385,4122,4123,4124,4125,4126,1044,2267,2268,2301,2302,2303,2272,2273,2274,2275,2276,2304,2305,1007,992,4127,2317,385,2432,2267,2268,2301,4128,4129,2272,2273,4130,4131,2275,2276,2448,2449,1007,992,385,4132,4133,4134,2312,2316,2321,4135,1007,992,385,4136,4137,4138,4139,2330,2331,2341,4140,990,2343,992,4141,385,2351,2352,2353,2354,4142,2356,2357,1007,992,385,4143,4144,385,4145,2381,1044,1007,992,385,4146,4147,4148,2657,2658,2388,2389,4149,2393,2395,4150,2398,2359,2360,2399,2400,2401,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2174,2402,2375,990,2403,2404,1664,1667,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,1669,2375,990,992,2428,2429,4151,990,992,2426,4152,4153,2434,4154,4155,4156,2438,4157,2440,2441,2442,2443,2444,2445,2446,4158,4159,2447,4160,990,992,4161,4162,4163,4164,4165,4166,4167,4168,4169,992,385,4170,4171,2452,2453,2454,2375,990,2455,4172,4173,2101,2469,2470,2471,2472,2473,2474,1669,990,4174,4175,4176,2506,2375,990,2507,2454,2510,2359,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2174,2530,2531,2532,2375,990,2470,2533,2534,2535,4177,4178,2539,4179,2543,4180,4181,2548,2549,2423,2552,2553,2554,2555,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2577,2578,2579,1669,990,2580,4182,4183,2586,4184,4185,4186,4187,4188,4189,2534,2535,2598,2599,2600,2601,4190,4191,4192,4193,2552,2553,2554,2609,2405,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2576,2610,2578,2579,2423,4194,4195,2552,2553,2554,2617,2405,2556,2557,2618,2619,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2420,2620,2621,2622,2579,1669,990,4196,2623,2624,4197,4198,4199,4200,2632,2633,2634,2635,2472,2423,4201,4202,2632,2633,2638,2554,2405,2556,2639,2640,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2570,2571,2572,2573,2574,2641,2420,2642,2579,1669,990,992,385,1568,4203,4204,1577,1578,1579,1580,1581,1582,1583,1594,1595,1596,1597,1598,1585,1586,1590,4205,4206,4207,4208,4209,4210,4211,4212,4213,4214,4215,4216,992,385,1604,1605,1606,1607,4217,4218,1613,4219,992,4220,1617,385,1672,4221,4222,4223,4224,1675,1676,1677,1678,1679,1680,1681,1682,990,992,385,4225,4226,4227,4228,4229,4230,4231,990,385,4232,4233,4234,4235,4236,4237,4238,4239,4240,990,385,4241,4242,4243,990,385,4244,4245,4246,4247,4248,4249,4250,4251,4252,4253,4254,4255,4256,1823,1817,1669,990,385,4257,4258,992,1624,1629,4259,4260,1632,4261,2454,4262,2375,990,385,4263,4264,4265,1627,2454,2510,2530,2531,2532,2375,990,4266,4267,4268,4269,4270,990,4271,4272,4273,4274,2454,4275,4276,4277,4278,4279,2423,2576,4280,2578,1669,2375,990,4281,4282,4283,4284,4285,4286,4287,4288,2434,4289,4290,4291,4292,2443,2444,2445,2446,4293,4294,4295,4296,4297,990,4298,4299,1664,4300,4301,4302,4303,2554,2579,1669,4304,4273,4305,4274,2454,4306,4277,4278,4307,2423,2576,4308,2578,1669,2375,990,4281,4309,992,4310,4311,4312,4313,4314,4315,992,4316,4317,1284,4318,4319,4320,4321,1293,1294,1295,1296,4322,4323,4324,4325,4326,4327,992,4328,1635,1636,1637,1638,1639,1640,990,992,1641,4329,4330,4331,4332,4333,1654,4334,1656,992,4335,1658,1659,992,4336,4337,4338,1665,4339,2454,1667,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,4340,2423,4341,4342,2425,1669,1667,4343,1669,2375,990,992,385,4344,385,4345,4346,4347,4348,2670,4349,1686,1685,2433,4350,2267,2434,2672,2673,2674,2438,2439,2440,2441,2442,2443,2444,2445,2446,4351,2447,2276,2676,2677,1007,992,385,1684,1686,1685,2433,4352,4353,4354,1007,992,385,1689,1097,1686,1685,2433,4355,4356,1007,992,385,1691,1097,1686,1685,2433,4355,4357,1007,992,385,4358,4359,4360,4361,1097,1686,1685,2433,4355,4362,1007,992,4363,385,1699,1700,1701,1702,1703,1704,1131,992,385,1705,4364,4365,1711,1686,1685,2433,4366,4367,1007,992,1713,385,4368,4369,4370,4371,4372,4373,4374,385,4375,4376,4377,4378,4379,4380,4381,4382,385,4383,4384,4385,4386,4387,4385,4388,4389,4390,4391,2750,4392,4393,4248,4249,4394,4395,2554,4396,4397,4398,4399,4400,1823,4401,2579,1828,4402,1669,990,4403,4385,4404,4405,4406,1685,2433,4407,4408,4409,4410,4411,990,4412,1007,4413,4404,4405,4406,1685,2433,4414,4415,4416,1007,4417,4385,4404,4405,4406,1685,2433,4418,4419,1007,992,385,4420,4421,4422,2761,2762,4423,4424,4425,4426,4402,2423,2763,4427,4428,4429,4430,4431,4432,4433,4434,4435,4436,4437,2136,4438,2781,2165,4439,1823,1669,4440,4423,1638,1639,4441,4339,2454,4442,4443,3007,3008,3009,1669,3010,2375,4444,1638,1639,4441,4339,2454,4445,4446,4447,4448,4449,4450,4451,4452,2375,2881,4423,1638,1639,4441,4339,2454,4453,4454,2375,990,992,385,4455,4404,4405,4406,1685,2433,4456,4457,1007,992,4458,385,4459,4460,4461,4462,4463,4464,2750,4465,4466,4467,2174,990,4404,4405,4406,4468,385,4469,4470,4471,992,4472,4473,4474,4475,2101,4476,992,994,385,4477,4478,4479,4480,994,385,4481,4482,4483,4484,4485,4486,4487,4488,992,4489,4490,4491,990,4492,4493,990,992,4494,4495,992,994,385,4496,4497,385,4498,4499],[4957,4958,4959,4960,4961,4962,4963],{"_941":46,"_46":4518},{"_941":4509,"_255":1182,"_965":4516,"_966":4517},{"_941":46,"_46":4515},{"_941":4509,"_255":1076,"_965":4513,"_966":4514},{"_941":46,"_46":4512},{"_941":4509,"_255":959,"_965":4510,"_966":4511},{"_941":46,"_46":4508},[4966,4967,4968,4969,4970,4971,4972],{"_4528":4529,"_428":429,"_4530":464,"_440":-5,"_4531":423,"_4532":-5,"_4533":-5,"_4534":4535,"_4536":4537,"_4538":-5,"_4539":-5,"_4540":4541,"_4542":15,"_4543":59,"_4544":54,"_4545":15},{"_92":588,"_96":485,"_582":4973},{"_471":472,"_473":474,"_4563":478,"_4564":129,"_481":-5,"_92":4565},{"_92":4553,"_473":4562,"_255":-5,"_4555":-5,"_4557":-5},{"_92":4553,"_473":4561,"_255":-5,"_4555":-5,"_4557":-5},{"_92":4553,"_473":4560,"_255":-5,"_4555":-5,"_4557":-5},{"_92":4553,"_473":4559,"_255":1182,"_4555":4556,"_4557":4516},{"_92":4553,"_473":4554,"_255":959,"_4555":4556,"_4557":4558},[4974],{"_941":46,"_46":4568},"task_e_6a3738369fec832cac18eeed6c03764b",1782003766.624801,"GitHub Mention: feat(coordinator): route Gemma 4 only to dedicated providers; 429 shed","has_generated_title","has_unread_turn","denormalized_metadata",{"_5006":32,"_5007":5008,"_5009":-5,"_5010":-5,"_5001":5011,"_5012":-5,"_5013":-5,"_5014":-5,"_5015":-5},"task_status_display",{"_4994":4995,"_4996":382,"_130":127,"_4997":-5,"_4998":4999},"creator_workspace","external_pull_requests",[4987],{"_42":4988,"_4989":83,"_4990":4991,"_4992":129},"github-1182082906-434","assistant_turn_id","pull_request",{"_463":464,"_461":465,"_466":467,"_468":59,"_469":15,"_470":15,"_471":472,"_473":474,"_475":433,"_476":127,"_477":478,"_479":480,"_481":482,"_483":-5,"_484":-5,"_485":4993,"_487":488,"_489":490,"_491":492,"_493":494},"codex_updated_sha",{"_455":495,"_367":495,"_496":-5,"_459":497,"_42":498},"latest_turn_status_display",{"_5000":83,"_112":113,"_5001":-5,"_94":5002,"_5003":-5,"_5004":-5,"_5005":4999,"_114":-5},"environment_label","fix_task_id","initial_intent","pr","turn_id","diff_stats",[],"image_asset_pointer_citations","issue_counts","intent","num_follow_ups","repo_name_map",{},"last_user_turn_created_at","last_turn_status",{"_5016":-5,"_5017":-5,"_5018":-5},"last_pushed_turn_id","start_branch_name","pushed_branch_name","cached_pull_request_data","files_modified","lines_added","lines_removed","dd",{"_5030":5031,"_5032":5033},"disablePrefetch","shouldPrefetchAccount","shouldPrefetchUser","shouldPrefetchSystemHints","promoteCss","disableSSR","stripModulepreloadImports","statsigGateEvaluationsPromise",["P",5029],"traceId","3207451033636989978","traceTime",1784596281101]