Design a Provider-Neutral AI Harness Contract

A provider-neutral harness stabilizes business lifecycle and control contracts while adapters translate to each SDK. Normalize run requests, events, policy, budgets, checkpoints, and outcomes; preserve provider-native handoffs, built-in tools, sessions, and traces as typed extensions. Portability means controlled substitution and migration, not pretending every runtime is identical.

What will you be able to do?

You will define the harness interfaces that SupportOps depends on and draw the boundary between portable controls and provider capabilities.

What are the core contracts?

python

from dataclasses import dataclass, field
from datetime import datetime
from typing import Any, AsyncIterator, Literal, Protocol


@dataclass(frozen=True)
class RunRequest:
    run_id: str
    tenant_id: str
    actor_id: str
    task: str
    constraints: tuple[str, ...] = ()


@dataclass(frozen=True)
class AgentEvent:
    run_id: str
    sequence: int
    kind: str
    occurred_at: datetime
    data: dict[str, Any] = field(default_factory=dict)


@dataclass(frozen=True)
class RunOutcome:
    status: Literal[
        "succeeded", "waiting_for_approval", "failed", "cancelled", "limited"
    ]
    output: Any | None
    evidence: tuple[str, ...]
    provider_metadata: dict[str, Any]


class ProviderAdapter(Protocol):
    async def stream(self, request: RunRequest) -> AsyncIterator[AgentEvent]: ...
    async def resume(self, checkpoint_id: str) -> AsyncIterator[AgentEvent]: ...

Add explicit contracts for ToolSpec, ToolResult, PolicyDecision, Budget, Checkpoint, and Evaluator. Keep Any at a serialization boundary only; domain outputs should be typed.

What should be portable?

authenticated request identity and tenancy;
domain tool contracts and policy decisions;
budget dimensions and stop reasons;
normalized lifecycle events;
checkpoint ownership and idempotency;
evaluation cases and outcome scorecards; and
business outcome/audit records.

What should remain provider-specific?

native handoff/subagent semantics;
exact stream/message types;
hosted/built-in tools;
server-managed sessions and resume references;
native guardrail/permission features; and
trace identifiers and provider usage fields.

Represent these through extension objects with versioned schemas rather than discarding them.

How do you prove the contract works?

Build a fake adapter that emits deterministic events. Run the same harness contract tests against the fake, OpenAI, and Claude adapters: start, tool proposal, approval pause, resume, limit, cancellation, and failure.

Failure injection: Have an adapter emit run.completed twice or skip sequence numbers. The harness must reject the invalid lifecycle rather than corrupt the outcome.

What mistakes should you avoid?

Returning only final text from adapters.
Designing the interface around one provider’s object model.
Hiding policy inside adapters.
Normalizing away provider evidence needed for debugging.
Coupling domain tests to live models.

Check your understanding

Which contracts should remain stable across providers?
Why preserve provider-native metadata?
What does a fake adapter enable?

Expert extension

Write a versioned adapter capability manifest so orchestration can request required features and fail clearly when a provider lacks them.

Official references

Design a Provider-Neutral AI Harness Contract

What are the core contracts?

python

from dataclasses import dataclass, field
from datetime import datetime
from typing import Any, AsyncIterator, Literal, Protocol


@dataclass(frozen=True)
class RunRequest:
    run_id: str
    tenant_id: str
    actor_id: str
    task: str
    constraints: tuple[str, ...] = ()


@dataclass(frozen=True)
class AgentEvent:
    run_id: str
    sequence: int
    kind: str
    occurred_at: datetime
    data: dict[str, Any] = field(default_factory=dict)


@dataclass(frozen=True)
class RunOutcome:
    status: Literal[
        "succeeded", "waiting_for_approval", "failed", "cancelled", "limited"
    ]
    output: Any | None
    evidence: tuple[str, ...]
    provider_metadata: dict[str, Any]


class ProviderAdapter(Protocol):
    async def stream(self, request: RunRequest) -> AsyncIterator[AgentEvent]: ...
    async def resume(self, checkpoint_id: str) -> AsyncIterator[AgentEvent]: ...

Add explicit contracts for ToolSpec, ToolResult, PolicyDecision, Budget, Checkpoint, and Evaluator. Keep Any at a serialization boundary only; domain outputs should be typed.

What should remain provider-specific?

native handoff/subagent semantics;

exact stream/message types;

hosted/built-in tools;

server-managed sessions and resume references;

native guardrail/permission features; and

trace identifiers and provider usage fields.

Represent these through extension objects with versioned schemas rather than discarding them.

How do you prove the contract works?

Failure injection: Have an adapter emit run.completed twice or skip sequence numbers. The harness must reject the invalid lifecycle rather than corrupt the outcome.