Mirroring OSV hourly - and naming which live build carries each advisory

Rezee reads lockfiles on every branch push, mirrors OSV's advisories for five ecosystems every hour, and joins both to Ship's deployments, so a finding names the environments that run it. How the mirror stays incremental, how a version is matched, and how a fix becomes a pull request.

Oct 1, 2026 · 15 min read · Kash Gohil · LinkedIn · X

Every dependency scanner can tell you that main has a vulnerable package. That is not the question you ask on the morning an advisory lands. The question is whether production runs it. Say staging has the fix and production is two deploys behind. The answer to "are we exposed" is yes, and the scanner that looked at main says no.

Answering that needs three facts in one place. You need what is in a commit, what is known to be broken, and which commit is live in which environment. Rezee already had the third: Ship records every deployment against the commit it shipped. So we built the other two, and the feature became a join.

This post covers five parts. The lockfile index. The OSV mirror and how it stays cheap. The version matching and where it is approximate. The join to deployments. And how a finding becomes a pull request. It ends with the fourteen advisories the feature found in Rezee's own repository, and why our own fixer could not clear them all.

The lockfile index rides an existing machine

The dependency index reuses the machine we built for API contracts. The same branch push hook runs it. The same keying applies: a lockfile is stored by its blob sha, so a lockfile nobody touched is one row across every commit that carries it.

The packages are rows, not a JSON document:

CREATE TABLE IF NOT EXISTS dependency_packages (
	lockfile_id uuid NOT NULL REFERENCES dependency_lockfiles(id) ON DELETE CASCADE,
	-- Position in the lockfile's graph; `requires` points at these.
	idx integer NOT NULL,
	ecosystem dependency_ecosystem NOT NULL,
	name text NOT NULL,
	version text NOT NULL,
	direct boolean NOT NULL,
	dev boolean NOT NULL,
	declared_by text[] NOT NULL DEFAULT '{}',
	requires integer[] NOT NULL DEFAULT '{}',
	PRIMARY KEY (lockfile_id, idx)
);

CREATE INDEX IF NOT EXISTS dependency_packages_name_idx
	ON dependency_packages (ecosystem, name);

We chose rows so that an advisory can ask "which lockfiles hold this package" of an index, instead of unpacking every graph. requires holds positions in the same lockfile, so the graph survives without a second table.

The first slice read bun.lock, package-lock.json and go.mod. The same day it grew to pnpm, Yarn 1 and 4, Cargo, uv, pom.xml, gradle.lockfile and Gradle version catalogs, then C and C++ files and git submodules. Each ecosystem has its own reader. All of them produce one graph shape.

Two fields matter later. direct is true, false or unknown. It is unknown for gradle.lockfile and Yarn 1, because those files list what was resolved and say nothing about why. Calling those packages transitive would be a guess. And every transitive package gets a via: the shortest chain that pulls it in, from a direct dependency down to its parent. A finding that says minimatch 3.0.4 is half an answer. One that says it arrives through eslint is something you can act on.

Mirroring OSV: the zip once, the csv after

OSV publishes each ecosystem two ways. all.zip holds every entry. modified_id.csv lists every id with the time it last changed, newest first. We mirror five ecosystems: npm, Go, crates.io, PyPI and Maven.

The first pass for an ecosystem reads the zip. npm's zip is about two hundred megabytes. So it is streamed through an unzipper and written in batches of 500 as it unpacks. The order inside the loop is the part that matters:

const reader = res.body.getReader();
for (;;) {
	const { done, value } = await reader.read();
	unzip.push(value ?? new Uint8Array(), done);
	// The unzipper calls back synchronously, so what it unpacked from this
	// chunk is all in `pending` now; writing it before the next read is what
	// keeps a slow database from letting the download run ahead of it.
	while (pending.length >= BATCH || (done && pending.length > 0)) {
		const batch = pending.splice(0, BATCH);
		await writeEntries(batch);
		written += batch.length;
	}
	if (done) break;
}

Writing before the next read is the back-pressure. Without it, a slow database lets the download fill memory.

Every pass after the first reads the csv. Because it is newest first, the reader stops at the first line at or before the newest modified time the mirror holds, and cancels the stream. It then fetches only those entries, one JSON file each. If more than 2,000 entries changed, it stops counting and reads the zip again, because one large download is cheaper than thousands of small ones.

advisory_sources records how far each ecosystem got. The update has one guard that is easy to miss:

// Never backwards: a full pass over a bucket that lags the csv
// must not make the next incremental pass re-read what it has.
modified: sql`greatest(${advisorySources.modified}, excluded.modified)`,

The mirror keeps only what matching needs: the id, summary, aliases, severity, dates, and each affected package with its ranges. The full advisory text stays on osv.dev, one click away. The details are most of the bytes across about a quarter of a million entries. Without them, all five ecosystems took about 150 MB when we measured it. The tables are global, not per workspace, because an advisory is public and the same for everybody.

Severity needed a decision. GitHub's advisories carry their own rating, and we use it when it is there. Others carry a CVSS v3 vector, and we compute the base score with the specification's formula. An id that starts with MAL- is a malicious package, and we rate it critical, because it is somebody else's code running with yours. Anything else has no severity, and the UI says so instead of guessing.

The mirror runs at boot and then every hour. An installation with no route to the internet turns it off with REZE_ADVISORY_SYNC=0. That installation then shows no advisories, not stale ones it cannot call stale.

That last point is why the mirror has two alerts. A stale mirror is quiet in the worst way. The Vulnerable view reads "nothing found", which is the answer a broken sync and a clean repository give alike. So one alert fires if a pass has never finished six hours after makima started. Six hours covers npm's first full download on a slow link. The other fires if the last good pass is more than six hours old. It runs hourly, so six missed passes is a failing mirror, not a slow one.

Is this version affected?

OSV writes an affected range as ordered events: introduced, fixed, last_affected and limit. You sort the events, walk them up to the version, and the last one you passed decides:

let affected = false;
for (const { kind, at } of keyed) {
	const cmp = at === "0" ? 1 : compare(version, at);
	if (cmp === null) return null;
	if (kind === "introduced" && cmp >= 0) affected = true;
	else if (kind === "fixed" && cmp >= 0) affected = false;
	else if (kind === "last_affected" && cmp > 0) affected = false;
	else if (kind === "limit" && cmp >= 0) affected = false;
}
return affected;

compare is where the ecosystems differ, and where we had to be honest about precision.

npm, Cargo and Go use semver. We use Bun's comparator, with two fixes. Go writes a v prefix and OSV does not, so the prefix is dropped. A partial version like 1.2 is padded to 1.2.0, because Bun would otherwise sort it after 1.2.0.

PyPI and Maven have their own ordering rules, PEP 440 and Maven's ComparableVersion, and both have corners. Their OSV entries also list every affected version explicitly. So the list is checked first, and a version on it is affected with no ordering at all. Only when the list does not answer do we use a loose ordering. Numbers compare as numbers. Known qualifiers like rc and post compare by where they sort against a release. Anything else compares as text. It is good enough for what the lists miss. It is not PEP 440, and the code says so.

GIT ranges are not read. They are commit ranges, and evaluating them needs the upstream project's history.

The result type has three values, not two. Affected, not affected, or null when nothing can order this version. null is a different answer from no, and the matcher treats it that way.

Some files declare what they accept, not what was resolved. Cargo.toml says serde = "1". vcpkg says >=. For those, we test the lowest version the requirement allows and mark a hit as possible, not affected. What actually gets built may be past the fix.

The matching happens at read time, as a join between the stored packages and the mirror. Nothing about a commit is stored as "vulnerable". An advisory published this morning applies to last month's commits with no re-index. Withdrawn advisories drop out of the same query.

Before we trusted any of this, we checked it against OSV's own query API. On Rezee's bun.lock and on fixtures for every other format, both sides found the same 30 and 11 findings. None were missed and none were extra.

Which environment runs it

"Live" uses Ship's meaning. It is the newest successful deployment of a repository into an environment that is not archived. Postgres's DISTINCT ON gives one row per pair:

db.selectDistinctOn([deployments.environmentId, deployments.repoId], { ... })
	.from(deployments)
	.innerJoin(environments, eq(environments.id, deployments.environmentId))
	.where(and(
		eq(environments.workspaceId, workspaceId),
		isNull(environments.archivedAt),
		inArray(deployments.repoId, ids),
		eq(deployments.status, "success"),
	))
	.orderBy(deployments.environmentId, deployments.repoId, desc(deployments.startedAt));

For each distinct live commit, the matcher runs once. The results are merged with the findings at the ref being read, keyed by advisory, lockfile, package and version. Each finding then carries the environments whose live build has it.

The merge has one rule that makes the feature worth building. A finding that is fixed on the branch but still deployed stays in the list, with atRef: false, instead of vanishing. Take the web repository. kobeni opened a fix for a vulnerable package last week, and it merged. main no longer has the advisory. Staging deployed the fix yesterday. Production still runs the commit from before it. A scanner reading main shows nothing. Rezee shows the finding, says main is clean, and names production as the one environment still carrying it. That is the case most worth seeing, and it is the one a branch-only view hides.

The same join runs across the workspace on Ship's Advisories tab, filtered by environment if you ask: "what is production exposed to". For each finding there, Rezee also checks the repository's default branch. If the branch still carries it, the row offers to open a fix. If the branch is already clean, the row says a deploy clears it. Agents get the same answer from the advisories_deployed tool.

Nothing here asks an environment what it runs. Rezee records deployments. It does not probe them. If a deploy happened outside Rezee and nobody reported it, the answer reflects the last deploy we know about.

A fix is the package manager's job

Opening a fix pull request was the part with the most ways to go wrong, so most of its design is refusals.

The first rule: a lockfile is a tool's output, so only the tool may write it. Hashes, peer resolution and the new dependencies a version brings cannot be reproduced by editing text. So a fix is a planned command for the ecosystem's own package manager:

case "npm": {
	const flags = "--package-lock-only --ignore-scripts --no-audit --no-fund";
	return at(
		direct
			? `npm install ${spec}${dev ? " --save-dev" : ""} ${flags}...`
			: // npm 7 and later update a transitive package in place, within the
				// ranges its parents allow.
				`npm update ${quote(name)} ${flags}`,
		IMAGES.node,
	);
}

Each command moves one package, skips install scripts and changes files only in the lockfile's directory. Every package name, version and path is checked against a strict pattern before it is quoted into a command. A name that fails the pattern is refused with a sentence, not escaped and hoped for.

Some cases cannot be done, and the plan says so. Bun has no command that moves one transitive package. The plan refuses and names the parent to update instead. Maven and Gradle files have no fixer yet. Findings carry a fixable flag computed by the same rules, so the Open fix button appears only where it can succeed. An earlier version showed the button on a Maven finding, and the button could only fail.

The second rule: the runner never pushes. kobeni, our runner, claims the fix. It clones the commit with the same read-only token a pipeline job gets, and runs the command in a container:

"--pids-limit", "1024",
"--memory", "2g",
"--cpus", "2",
"--security-opt", "no-new-privileges",

The network stays on, because a package manager has to reach its registry. Nothing of ours goes in: no secrets and no runner environment. The runner hands back the tracked files the command changed, from git diff --name-only, up to 60 files. Fixes run only on Docker runners. A shell runner leaves them to one that has a container engine.

The third rule: makima checks the result before anything is committed. It keeps only lockfiles and manifests, whatever else the tool touched. It parses the new lockfile with the index's own reader and tests every resolved copy of the package against the advisories the fix was for. If one is still inside a range, the fix fails with a reason that names it. The reason says the new lockfile "still resolves" the old version, and that the parent "allows no fixed version yet".

Only then does makima create a branch at the base commit, commit the files as the person who asked, and open the pull request. It opens it through one function, and that was a separate commit on purpose. Before it, the route that opened a pull request did the follow-up work inline. That work is the webhook, the pipeline, the search index, issue links and mentions. A server-side caller would have needed a second copy of that list, and a second copy is where a bot's pull request quietly skips its CI. Now the route and the fixer both call openPullRequest, and a fix arrives exactly as a pull request a person opened.

Nothing merges. The pipeline decides whether it is green. A person or a protection rule decides whether it lands.

Batches on a schedule, from a file

One pull request per package is right when you click a finding. It is noise once a week. So fixes run in batches: several fixes in one checkout, one commit, one pull request. A package that cannot move is listed in the body with its reason. It does not hold back the rest.

The schedule lives in the repository:

# .rezee/dependencies.yml
schedule:
  every: week
  day: monday
  at: "06:00"
  timezone: Europe/London
severity: medium
ecosystems: [npm, go]
ignore: ["@types/*"]

Every field is optional. An empty file means a weekly batch on Monday at 06:00 UTC of everything fixable. Removing the file is the off switch. There is no settings screen. The people who can change the schedule should be the people who can change the repository, and the file's history is the schedule's history.

The batch needs an author, and Rezee has no bot account. The pull request is opened as whoever last changed the file, while they can still write to the repository. If they lose access, no pull request opens, and the schedule shows why: change the file again to open these as you. A push of unrelated work leaves the row alone, because the file's blob sha did not change. Otherwise whoever pushed last would quietly become the author of next week's pull request.

The ticker that runs schedules had a bug worth telling. Several makima processes can run the ticker, so a due row is claimed by an update that moves it on. The first version matched the old time exactly:

eq(dependencyUpdateConfigs.nextRunAt, row.nextRunAt as Date),

Postgres stores microseconds. A JavaScript Date keeps milliseconds. For a stored time that carried microseconds, the value read back could never equal the value stored, so the claim matched nothing. That row would stay due on every sweep and never run. The fix claims by whether the row is still due:

lte(dependencyUpdateConfigs.nextRunAt, now),

A second process that reads the same row finds it no longer due and leaves it.

These are security fixes only. A bot that bumps every minor version weekly is what Renovate already does well, and we did not want to build a worse one. Agents can open the same batch with dependency_fix_batch_open, which is in the On-call kit.

Our own fourteen

Rezee's own dependencies carried fourteen advisories. We cleared eleven of them in ISSUE-1012. faker, fflate and tiptap were bumped. Seven transitive packages moved inside the ranges their parents already allow. uuid got an override to version 11.

The transitive moves are the interesting part. Our lockfile is bun.lock, and the fixer refuses transitive bun packages because bun has no command to move one. So we did what the refusal tells you to do, by hand. We wrote the lockfile entries directly, so that nothing else moved and the patch we carry for expo stayed in place. Three advisories remain: esbuild, decode-uri-component and image-size have no safe fix yet. The Vulnerable view still lists them, and it should.

What it cost

Matching at read time is not free. Every read joins packages against the mirror. Ship's workspace view runs it once per distinct live commit per repository. That is cheap for the workspaces we have. A large one will probably need a cache.
The loose ordering is approximate. PyPI and Maven are matched by their enumerated lists first, and that covers most entries. The fallback ordering has known gaps against PEP 440 and Maven's rules, and we chose to say so in the code over pulling in two version libraries.
A fixer that refuses is a fixer that does less. Transitive bun packages, Maven and Gradle need a person. We think a clear refusal beats a lockfile that was edited by something other than its tool.
A mirror is a liability you have to watch. An empty answer from a dead mirror looks exactly like good news. The two alerts and the success gauge exist because of that, not as decoration.

FAQ

Which ecosystems does the OSV mirror cover?

npm, Go, crates.io, PyPI and Maven. The first pass for each reads OSV's all.zip. Every pass after reads modified_id.csv and fetches only the entries that changed, unless more than 2,000 changed.

How does Rezee know which environment runs a vulnerable package?

It takes the newest successful deployment of each repository into each environment that is not archived, reads the lockfiles at that commit, and matches them against the mirror. Rezee does not probe environments. It uses the deployments that were reported to it.

Does Rezee merge dependency fixes automatically?

No. A runner regenerates the lockfile with the ecosystem's own package manager. makima checks that the advisory no longer matches. The pull request is opened as the person who asked. The pipeline and the team decide whether it merges.

Why can some findings not be fixed automatically?

There are three reasons. No fixed version exists. Or the file is one Rezee cannot regenerate yet (Maven, Gradle). Or the package is a transitive bun dependency, which bun has no command to move on its own. The finding says which, and names the parent to update when that is the way through.

What happens if the advisory mirror stops?

Two alerts fire: one when a pass has never finished six hours after start, and one when the last good pass is more than six hours old. An installation that turns the mirror off shows no advisories, not stale ones.