The Missing Control Loop for AI Agents: Intent, Evidence, and Replay

The OpenAI Build Week build story behind CausalGate, from intent contracts to evidence-backed replay

OpenAI Build Week 2026 submission note: I wrote this article as the public build story for CausalGate’s Developer Tools entry. The project was submitted July 19, 2026, before the July 21 deadline. Official submission: https://devpost.com/software/causalgate

An AI agent can produce a reasonable answer while its tools do the wrong thing. It can read protected data, call a destination outside the approved scope, or complete the visible task through an execution path nobody intended. A transcript may still look fine.

I built CausalGate for that gap. It records the agent's declared intent, captures ordered execution evidence, identifies the point where action diverged, applies intent-based authorization, and reruns the same scenario under protection. The result is not a generic risk score. It is a causal record and a scoped release decision.

This post has two parts. The product story, architecture, demo, and adoption paths are free. The paid section contains the practical evaluation runbook I use to decide whether an agent change is ready to promote.

You can watch the complete 2 minute 56 second demo on YouTube, inspect the public MIT-licensed repository, or run the deterministic demo without an API key.

The problem is not another bad answer

Most agent evaluation still concentrates on model output. Did the response follow instructions? Was it helpful? Did it contain unsafe text? Those questions matter, but they stop before the tools begin to create effects.

The harder question is operational: did the execution remain inside the authority that the user, organization, and runtime actually granted? That requires more than a final transcript. It requires the original intent contract, tool arguments, policy decisions, data classifications, destinations, effects, and links between each claim and the event that supports it.

Figure 1 illustrates The Intent Visibility Gap. It shows why a successful-looking answer cannot prove that the underlying execution respected purpose, data boundaries, or destination limits.

Figure 1: The Intent Visibility Gap

Traditional logs usually tell you that a tool ran. They do not always tell you whether that action was necessary for the user's purpose, whether the agent was authorized to take it, or which earlier decision made the harmful effect possible. CausalGate treats those questions as first-class product behavior.

What CausalGate records

A run begins with a typed intent contract. The contract identifies the subject, purpose, allowed actions, protected resources, expected effects, and limits. CausalGate then records normalized events as the agent calls tools and receives policy decisions.

Deterministic detectors analyze that evidence. They do not ask a model to invent a verdict. Each finding points back to supporting event IDs and explains the divergence in concrete terms. The system then assembles an Intent Causal Record that ties contract, execution, findings, authorization, and limitations together.

Figure 2 illustrates Evidence Becomes a Causal Record. The important design choice is the chain of custody: every conclusion remains connected to the contract and ordered events that produced it.

Figure 2: Evidence Becomes a Causal Record

This makes the product useful for more than incident review. The same record can support QA, security review, release approval, and audit evidence. A reviewer can challenge a finding without having to trust an opaque score.

The demo includes a synthetic vulnerable scenario with eight evidence-linked findings. It is intentionally deterministic, so judges and developers can reproduce the same result without a paid API or external system. Optional GPT-5.6 analysis is explicit, ephemeral BYOK. The key is sent only for that request and is not stored by CausalGate.

Intent-Based Access Control

Role-based access answers whether an identity can call a tool. CausalGate adds a second question: can this agent take this action, for this purpose, on this resource, at this destination, with these effects, in the current runtime context?

The authorization engine normalizes tool calls against an ontology and evaluates an intersection of identity, agent authority, intent grant, organization policy, and runtime context. It records allow, deny, and approval-required outcomes as evidence. Unknown mappings fail closed.

Figure 3 illustrates Authority Is an Intersection. No single credential or plausible tool call is enough; the action proceeds only when every required authority boundary agrees.

Figure 3: Authority Is an Intersection

This matters because agent failures often look individually legitimate. A read may be allowed, and an outbound send may be allowed, yet the combination may violate the declared purpose. Purpose and effect have to remain attached to authority throughout the run.

Replay before release

Finding a problem is only half the control loop. Teams still need to know whether the proposed protection actually changes behavior without hiding evidence or breaking the intended task.

CausalGate reruns the same scenario with protection enabled. It compares baseline and replay findings, preserves the authorization record, and emits a scoped recommendation. In the included scenario, the vulnerable run produces eight findings while the protected replay produces zero, leading to a promote recommendation for that tested scope.

Figure 4 illustrates Replay Turns a Fix Into Evidence. A fix earns promotion by changing the observed execution under the same scenario, not by sounding convincing in a review comment.

Figure 4: Replay Turns a Fix Into Evidence

Replay does not prove universal safety. It proves something narrower and more useful: this protection changed this scenario, under this contract and runtime, with this evidence. CausalGate keeps that boundary explicit instead of converting one passing run into a broad guarantee.

SDK and SaaS are two views of the same engine

The Python SDK is for teams that need control inside the agent runtime. They install the package, wrap execution, declare the intent contract, record events, authorize tool calls, generate the causal record, and use replay results in CI or a release workflow.

The smallest adoption path remains familiar:

from causalgate import CausalGate

gate = CausalGate()
run = gate.start_run(contract)
gate.record(run, event)
report = gate.analyze(run)

The contract and event objects are typed, and the engine produces deterministic evidence. The likely failure point is incomplete instrumentation: if a tool bypasses the recorder or authorizer, the record cannot claim complete mediation. CausalGate reports that limitation rather than hiding it.

The SaaS version uses the same engine as a shared control plane. Teams send normalized, tenant-authorized evidence to a hosted service, review causal records in the UI, compare baseline and replay runs, manage policy versions, and connect promote or block decisions to deployment systems. The current repository is an alpha reference implementation and judge profile, not a claim that multi-tenant production hardening is finished.

Figure 5 illustrates One Engine, Two Adoption Paths. The SDK optimizes for embedded enforcement and developer control; SaaS optimizes for shared review, governance, and evidence retention across many agents and teams.

Figure 5: One Engine, Two Adoption Paths

This split also creates a clean privacy boundary. Sensitive raw payloads can remain inside the customer's runtime while the hosted layer receives normalized events, digests, redacted evidence, and explicit limitations. Production SaaS still requires durable tenant storage, workload identity, shared atomic permit state, private workers, and managed audit retention. Those boundaries are documented rather than implied.

What Codex and GPT-5.6 changed

This was not a one-prompt wrapper. During the Build Week submission period, I used Codex and GPT-5.6 to turn an existing technical direction into a coherent, testable product. The work included tracing the PRD and architecture into the running system, fixing authorization-evidence corruption, making optional live analysis fail safely, closing SQLite connection leaks, adding regression tests, clarifying the SDK and SaaS boundaries, completing browser QA, preparing the public deployment, and documenting where the implementation still differs from the product design. The dated commits and primary-session evidence are linked from the Devpost submission and repository build log.

I used Codex as an engineering collaborator across product design, architecture, implementation, debugging, documentation, browser QA, and release preparation. The useful part was not raw code generation. It was the loop: inspect the PRD and architecture, trace the implementation, reproduce failures, patch the smallest responsible surface, run the full suite, and compare the product against the written contract.

GPT-5.6 was used where broader reasoning mattered, including the causal model, authorization ontology, product boundary, SDK and SaaS framing, and the review of gaps between the PRD and implementation. Deterministic code still owns enforcement and release decisions. Optional live GPT-5.6 analysis adds semantic investigation, but it does not replace the recorded evidence or policy engine.

That distinction is important. I do not want a second model to declare the first model safe. I want models to help humans understand evidence while deterministic controls retain authority.

A release gate you can inspect

The repository currently passes 80 backend tests with 90 percent coverage, 9 UI tests, a production TypeScript build, and browser QA. The public demo runs with synthetic data and no account requirement. Docker and Cloud Run deployment profiles use a single instance because the judge profile stores SQLite state locally and keeps a process-local permit ledger.

Figure 6 illustrates The Release Decision. The final output remains operational: promote the tested scope, hold for missing evidence, or block because a material divergence remains.

Figure 6: The Release Decision

The biggest product lesson is simple. Agent safety becomes actionable when intent, authority, evidence, replay, and release decisions live in one loop. Without that loop, teams collect impressive traces and still make release decisions by intuition.

For paid subscribers: the practical release runbook continues below.

It turns the architecture into seven concrete checks for contract quality, complete mediation, vulnerable baselines, evidence links, controlled replay, scoped decisions, and durable records.

Subscribe to unlock the runbook: https://kenhuangus.substack.com/subscribe?coupon=302342d9.