A model gateway translates application-owned requests into provider calls and provider outcomes back into stable application results. It centralizes capabilities, timeouts, retry policy, routing, budgets, telemetry, and redaction without hiding semantics behind a generic “generate” function. Domain code must not depend on one provider's SDK or model name.
The adapter chooses provider request fields, parses usage, categorizes refusal or truncation, and records the provider request ID. The use case chooses what classification means and whether confidence is sufficient.
Create model aliases such as fast-classifier, grounded-writer, and reasoning-reviewer. Resolve an alias through configuration and policy using required capabilities: structured output, context size, tool support, region, data handling, latency SLO, evaluated quality, and price ceiling. A fallback is eligible only if it passed the same feature evaluation and data-governance policy.
Estimate before calling, then reconcile actual usage. Reject or degrade before exceeding a hard budget. Provider pricing and limits are volatile; load current configuration and link operators to official pages rather than hardcoding claims in domain logic.
Retry selected transient network, rate, and provider errors with bounded exponential backoff, jitter, and respect for server guidance. Do not retry invalid schemas, safety refusals, authentication errors, or deterministic client errors. Ensure retry does not duplicate a billable or tool effect; reuse provider idempotency support when available.
The gateway is the anti-corruption layer between probabilistic vendors and stable product semantics. It makes provider replacement possible but does not pretend all models are equivalent. Capability evaluation and routing evidence are part of release management.
Implement a deterministic fake classifier plus one provider adapter interface. Test success, timeout, rate limit, refusal, invalid output, exhausted budget, and eligible fallback.
Acceptance criteria: application tests import no provider SDK, total attempts and cost are bounded, and every outcome maps to a durable AI-run status.