← All posts

API contract diffs on every push - reading OpenAPI, GraphQL and protobuf against the fork point

Rezee reads a repository's API contracts on every branch push, keyed by blob sha, and diffs a pull request's contract against the commit it forked from. The rules that call a change breaking, why the base branch's tip was the wrong "before", and why accepting a break is bound to one commit.

Sep 26, 2026 · 16 min read · Kash Gohil · LinkedIn · X

A pull request that removes a field from an API response is often three lines of YAML. The reviewer sees email disappear from a schema and has to know three more things before the diff means anything. Which endpoint uses that schema? Does a client read that field? Is this the request side or the response side? None of that is on the screen. It is in the reviewer's head, or nowhere.

We wanted the review to say it in words: "GET /users/{id} 200 response field email removed", marked breaking, at the top of its own tab. That sentence needs four things. It needs the contract read at two commits. It needs the right two commits. It needs rules that decide what breaks a client. And it needs a gate that can stop the merge without becoming a label that anybody can add.

This post is those four things, in the order we built them, with the one bug that would have made the whole feature lie.

A contract is a blob, so key it by the blob

Every branch push already triggers a hook in makima, Rezee's API. That hook refreshes the searchable file paths and queues pipelines. The contract index runs there too, and a failure in it costs the index, never the push. It reads OpenAPI (and its Swagger ancestor), GraphQL SDL and protobuf, because those are the three formats teams use to write an API down.

The table is keyed by the file's bytes, not by its path or its commit:

CREATE TABLE IF NOT EXISTS api_specs (
	id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
	repo_id uuid NOT NULL REFERENCES repositories(id) ON DELETE CASCADE,
	blob_sha text NOT NULL,
	parser text NOT NULL,
	format api_spec_format NOT NULL,
	title text,
	version text,
	-- Null when the file could not be read as a spec; `problem` says why.
	document jsonb,
	problem text,
	first_path text NOT NULL,
	first_commit text NOT NULL,
	created_at timestamptz NOT NULL DEFAULT now()
);

CREATE UNIQUE INDEX IF NOT EXISTS api_specs_blob_idx
	ON api_specs (repo_id, blob_sha, parser);

The same openapi.yaml at a hundred commits is one row. A push that does not touch the contract writes nothing. The hook asks denji, our git server, for the names of the files that changed between before and after. That call returns names only, with no patch. The hook keeps the names that look like contracts. For each one, it checks for a row with that blob sha before it parses anything. Bytes that were seen before are never parsed again.

parser is in the key on purpose. It holds a version string, specs@2 today. When we changed what a parse stores (field locations and descriptions arrived in a later slice), we changed the string. Old rows then stop matching, and the next read parses the file again. A row written by an older parser is never trusted to have fields it never had.

The index covers every branch, not only main. That looks wasteful at first, so the code says why:

// Every branch, not only the default one. The contract diff (ISSUE-957) reads a
// pull request's base and head, and a head is a feature branch by definition -
// indexing only main would leave the one side of every diff it exists for
// unread.

Tags are skipped. A tag names a commit that some branch push already indexed.

What counts as a contract is decided by file name. openapi.yaml, swagger.json, billing.openapi.yaml, anything ending in .graphql, .gql or .proto. A YAML file is a candidate only when its name says OpenAPI, because every docker-compose.yml is YAML too. Directories named node_modules, vendor and third_party are skipped, because a vendored .proto from a dependency is not this repository's API. A repository that keeps its contracts somewhere unusual lists globs in .rezee/api.yml. A file that exists but lists nothing is treated as absent, not as "index nothing". An empty list is far more likely to be half-written than to be a request to turn the feature off.

Two smaller decisions came from the same instinct. A spec that fails to parse is still stored, with a problem string, so a broken contract is visible as broken instead of quietly leaving the index. And one pass reads at most 50 contract files. A repository past that gets a partial flag on the answer, not a silent cut.

What is stored is a normalised document, not the parser's own tree. A parser tree carries source positions. Store that, and every spec reads as changed whenever somebody moves a comment.

The contract nobody edits

The first version had a hole, and it was in the part that looked most efficient.

The hook reads the files that a push changed. A contract that was committed last year does not change. Nobody edits a contract to make it appear in a new feature. So every repository that already had an API had no rows at all, and would have none until somebody happened to touch the file.

The fix is a check before the diff. If a repository has never had a contract indexed, the hook reads the whole tree instead of the changed files:

let candidates: string[] | null = null;
if (before && !ZERO.test(before) && (await everIndexed(repo.id))) {
	const changed = await denji.changedFiles(repo.ws, repo.name, before, after);
	if (!changed.truncated && !changed.files.includes(SPEC_CONFIG_PATH))
		candidates = changed.files;
}
if (candidates === null)
	candidates = (await denji.treePaths(repo.ws, repo.name, after)).paths;

The same fallback covers three other cases. A new branch has no before. A diff past denji's cap is incomplete. A change to .rezee/api.yml can turn a file that did not move into a contract. A repository with no contract at all pays one tree listing per push for this, and stops paying it once it gains one. We also shipped a backfill script, so existing repositories did not have to wait for their next push.

Reading at an arbitrary commit does not depend on any of this. specsAt lists the tree at the ref, finds each blob's row, and parses on the spot anything that no push indexed. That is what lets the API page on Code show a repository's contract at any commit, including commits from before the feature existed.

The fork point, not the tip

This is the bug that would have made the feature lie. It hides in the obvious choice of field.

denji's compare endpoint already returns a baseSha, and it looks like the "before" side of a contract diff. It is the tip of the base branch. A pull request is not reviewed against the tip of its base. It is reviewed against the commit it forked from, which is what a three-dot diff shows on the Files tab.

Here is what goes wrong. aki branches labels-filter from main in the web repository on Monday. On Tuesday, kash lands a change on main that adds an assignee field to the GET /issues/{number} response. aki's branch does not contain that change. It was never rebased. Now diff the contract at the tip of main against the head of labels-filter:

`GET /issues/{number}` 200 response field `assignee` removed    breaking

aki never touched assignee. The base moved on, and a diff against the tip shows the base's own changes as the branch undoing them. With a protection that requires the contract check, aki's pull request would be blocked for a break it did not make.

The fix was a small endpoint on denji that answers one question:

func MergeBase(dir, base, head string) (*MergeBaseResult, error) {
	if err := guardRef(base); err != nil {
		return nil, err
	}
	if err := guardRef(head); err != nil {
		return nil, err
	}
	sha, err := runTrim(dir, "merge-base", "--", base, head)
	if err != nil || sha == "" {
		// Unrelated histories, or a ref that does not resolve: either way there
		// is no fork point to name.
		return nil, ErrNotFound
	}
	return &MergeBaseResult{SHA: sha}, nil
}

The contract diff now reads "before" at the merge base and "after" at the head. It reads only the contract files that the pull request touched. A pull request that changes no contract costs one merge-base and one changed-files list. Git answers both without reading a blob.

One more case needed care. A file counts as a contract if either side's .rezee/api.yml says so. A pull request that deletes a contract also deletes it from the head's view of the world, so only the base's rules still call it a spec. Deleting a contract is reported as breaking.

Breaking is a rule you can read

The diff is not a text diff. It walks two normalised documents and names each change. Each change carries breaking: true or false, and the rules that set it are in one file that a reviewer can read.

The line the rules draw is the one a client sees. Taking away something a client receives, or demanding something new from a client, breaks it. Adding something a client may ignore does not. Most rules then depend on direction. The same change breaks a request and not a response, or the other way round:

for (const name of Object.keys(propsA)) {
	if (name in propsB) continue;
	out.push({
		pointer: `${pointer}/${name}`,
		kind: "removed",
		// A client reading a field that stopped coming breaks. A server that
		// stopped reading a field a client sends does not.
		breaking: direction === "response",
		summary: `${label} field \`${name}\` removed`,
	});
}

The full set for OpenAPI:

GraphQL follows the same idea through nullability. On an output field, String to String! breaks nobody, because a client that handled null still works. On an input field the safe direction is the opposite. A new non-null input field with no default is a value that every existing caller now fails to send. Protobuf has one rule that the other formats do not need:

// The number is the field on the wire; changing it is a different
// field wearing the old name.
if (x.id !== y.id)

Every change has a pointer, such as GET /users/{id}/responses/200/body/email. Review threads on the Contract tab anchor to that pointer, not to a line. The same change sits on different lines in YAML and in JSON, and a reviewer who reads the contract diff is not reading lines at all. The migration that added this is one nullable column on pr_review_threads and a partial index. Every existing thread stays exactly what it was.

The walk has limits, and the post should say them. Local $refs (#/components/...) are followed. References to other files are not. Schemas are walked eight levels deep, and a change below that is real but unreported. Both limits are written in the code, beside the walk.

A break is a decision, not a label

A diff that nobody has to read is a decoration. So a branch protection can require it.

Protections in Rezee list required checks by name, and the merge gate matches those names against pipeline jobs. The contract check is not a job. Nothing runs. The answer is already computed from two commits. We still gave it a job's clothes: a name, rezee/api-contract, and the same status values. A repository configures it in the same place it configures everything else. The name has a namespace because required_checks is free text. A repository is free to have its own job called contract, and that job must not satisfy this check by accident.

The gate takes the name out of the list it asks the pipeline about. Without that step, the check would be looked for among a run's jobs, read as missing forever, and block every merge. Then it computes the check itself. Three outcomes:

• success when the pull request breaks nothing, or when somebody accepted the breaks on this head.
• failure when it breaks something and nobody has accepted it.
• missing when the contract cannot be read, for example on unrelated histories.

missing blocks. A gate that passes because it could not ask the question is worse than one that stops. Every other unknown in the merge gate already goes that way.

Then the way past it. A breaking change is sometimes the right change. The usual mechanism is a label like breaking-ok, and a label has two problems. Anybody who can label can add it. And it outlives the commit it was added for. Push another breaking change after the label, and it passes a gate that somebody opened for a different change.

So accepting a break writes a row that names one commit:

CREATE TABLE IF NOT EXISTS pr_contract_waivers (
	id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
	pr_id uuid NOT NULL REFERENCES pull_requests(id) ON DELETE CASCADE,
	head_sha text NOT NULL,
	-- Kept when the person is removed: who accepted a breaking change is the one
	-- thing this row exists to remember.
	accepted_by uuid REFERENCES users(id) ON DELETE SET NULL,
	reason text NOT NULL DEFAULT '',
	withdrawn_at timestamptz,
	created_at timestamptz NOT NULL DEFAULT now()
);

It is a signed act in the plain sense: a person's name, their reason and the commit they read. The route asks for repository write access, because this is the decision to ship it. The caller sends the head sha it read the changes at. If the branch moved between reading and accepting, the route answers 409, because what would be waived is not what was read. The acceptance goes into the audit log as contract.accepted with the head sha, the number of breaking changes and the reason. Withdrawing it sets withdrawn_at. The row stays, because it happened.

The gate only reads a waiver for the current head. Any push after an acceptance asks the question again.

One call for an agent

An agent writing a client usually learns an API from call sites. That teaches it what the code did last month, not what the contract says now. The alternative, handing it the whole OpenAPI document, spends a context window on YAML.

So the contract is also flattened into endpoints, and an MCP tool answers the question in one call:

name: "api_endpoint_search",
description:
	"Find an API endpoint this repository declares, by name or by what it " +
	"does, and get what it sends and receives in the same answer. Use this " +
	"before writing or changing a client: it is what the contract says " +
	"today, where reading call sites tells you what the code used to do. " + ...

Each endpoint arrives with local $refs already followed. Each field says where it goes: path, query, header, cookie or body. The ID in the URL and the email in the JSON stop looking like peers. Each field keeps the description its author wrote. The answer names the status the response belongs to. It also carries an example body as JSON, because the shape of a payload is faster to read from one instance than from a list of types. The tool sits in the Reviewer kit next to api_specs_list and api_spec_get. How one tool list serves every agent is in one tool surface.

Protobuf needed a fix here. Its only prose is comments, and protobufjs drops them unless you ask twice: alternateCommentMode when parsing and keepComments when serialising. Until that commit, every gRPC field arrived with no description at all.

The requests already in the repository

Many teams that write a contract also commit requests for it: Bruno's .bru files, .http files that VS Code and JetBrains run, Hurl, and Postman exports. Rezee reads those at a ref and shows each request under the endpoint it calls. Nothing is imported or stored. The files are in git, already reviewed, and already the thing the team edits.

Postman was nearly a mistake. Its exports are JSON, and claiming .json would have spent the read budget on lockfiles and tsconfig.json before reaching one request. Postman writes .postman_collection.json on every export, so that suffix is all we match.

A request matches an endpoint by method and path shape. /issues/42 matches GET /issues/{number}, because a segment the contract writes as a template matches any value. A collection's own {{issue}} variable matches a template segment too, because both sides stand in for a value. A request that matches no endpoint is shown with no endpoint, not dropped. "This calls something the contract does not describe" is worth seeing.

Replaying them against what was deployed

The last slice closes a gap on Ship. A deploy can succeed and the API can still stop answering. The deployment cell is green, and it is wrong.

rezee contract check replays the committed requests against a running service and records the result on the deployment:

id=$(rezee deploy start production --id)
./deploy.sh
rezee deploy finish "$id"
rezee contract check "$id" --base-url https://api.example.com

The requests are sent from wherever the deploy ran, not from Rezee. We record deployments. We do not make them, and reaching into somebody's network to send requests to a private hostname is the same overreach under a different verb. The place that ran the deploy is also the only place that can reach a private environment. Variables like {{token}} are filled from that process's environment, so a secret stays in the CI environment and is never sent to Rezee. A variable with no value is left as written, so a misconfigured request is visible instead of being sent to a URL with an empty middle.

The result lands on three new columns of deployments, and Ship's Overview grid shows "Contract failing (3)" beside the deploy state, not instead of it. What landed did land. A failing check also exits non-zero, because a CI step that reports red and exits green leaves the pipeline claiming the deploy was fine.

We should be plain about what "passing" means. A request passes when it answers below 400. The check does not compare the response body against the schema. It answers "does the deployed service still answer the requests the team wrote", which is the question a green cell was hiding. Shape validation is a different and larger feature.

What it cost

• Two paths to the same row. The push hook and the read path both parse on a miss. The unique index makes that safe under a race: two pushes of the same file both insert, the index keeps one, and both read it back. It is still two callers to keep honest.
• The rules are opinions. Removing an optional request parameter is not breaking here. A team that treats unknown parameters as errors would disagree. We wrote the rules where a reviewer can read them, and that is the most we can promise.
• missing blocks. A repository with a protection that requires the check, and a branch with unrelated history, cannot merge until somebody changes the protection. That is the price of a gate that never passes by not asking.
• The obvious "before" was wrong. Using compare's baseSha would have produced confident, specific, wrong output, which is worse than no output. The merge-base endpoint landed before the pull request's contract was served. The suite keeps a test named "the before side is the fork, not the base branch's tip" so it stays that way.

FAQ

Does Rezee run a CI job to compute the contract diff?

No. The diff is computed from two commits that git already has. It appears as a required check called rezee/api-contract so a protection can name it, but nothing is scheduled or executed.

Which contract formats does Rezee read?

OpenAPI 3 and Swagger 2 in YAML or JSON, GraphQL SDL, and protobuf. Files are found by name, or by globs in .rezee/api.yml when a repository keeps them somewhere unusual.

Why compare against the merge base instead of the base branch?

Because the base branch keeps moving. A diff against its tip shows every contract change that landed on the base since the fork as the pull request undoing it. The merge base is the commit the branch started from, which is the same "before" the Files tab uses.

Can a breaking change still be merged?

Yes. Somebody with write access accepts the breaks on the current head commit, optionally with a reason. The acceptance is recorded in the audit log, and any later push asks again.

Does rezee contract check validate response bodies?

No. It replays the committed .bru, .http and .hurl requests and records which ones answered below 400. It catches a deploy whose API stopped answering. It does not check the shape of what came back.