VeristReplay + diff for AI decisions

See exactly what changes when you update a prompt or model — before it hits production.

terminal

$ verist test

classify-ticket · 10 baselines

  ✓ 8 unchanged  ~ 2 changed

  ticket-002
    category: "urgent" → "normal"
    confidence: 0.92 → 0.71
  ticket-009
    category: "urgent" → "normal"
    confidence: 0.88 → 0.65

  ⚠ 2 regressions — review before shipping

Deterministic Replay

Exact reproduction from stored artifacts. Same inputs + recorded LLM responses = identical output every time.

Recompute with Diffs

Re-run with a new prompt or model against the same inputs. See structured, field-level diffs of what changed.

Human Override Preservation

Three-layer state model — computed, overlay, effective. Human corrections are never overwritten by recomputation.

Audit Trail

Structured events with full decision provenance. Content-addressable hashing for every artifact.

Declarative Commands

Steps return data describing what happens next — invoke, fanout, review, emit, suspend — without executing side effects.

Schema Validation

Zod-based input/output schemas ensure type safety and catch structural regressions at the boundary.

How it works

change prompt → recompute → see diff → approve → ship

Capture

Run a step, store output as a versioned artifact.

Recompute

Re-run with a new prompt or model against the same inputs.

Diff

See exactly which fields changed, were added, or removed.

Approve

Review the impact, then ship with confidence.

Without Verist

Logs tell you what happened — not what would change
A few test cases can't catch regressions at scale
Recomputation silently overwrites human corrections
Decisions can't be reproduced months later

With Verist

Field-level diffs show impact before you ship
Batch recompute across your full history
Human corrections preserved by design
Exact replay from stored artifacts, any time

Git for AI decisions

Code change→Prompt / model updateTest run→Recompute against historyPR diff→Decision diffMerge→Approve and persist

Minimal API

Define a step, capture a baseline, recompute with a new prompt, and see the diff.

typescript

import { defineStep, run, recompute, formatDiff } from "verist";

const classify = defineStep({
  name: "classify-ticket",
  input: z.object({ text: z.string() }),
  output: z.object({ priority: z.enum(["high", "medium", "low"]) }),
  run: async (input, ctx) => {
    const result = await ctx.adapters.llm.extract(input.text);
    return { output: result };
  },
});

// Capture baseline
const baseline = await run(classify, { text: "..." }, { adapters });

// Recompute with new prompt
const result = await recompute(baseline, classifyV2, { adapters });

// See what changed
console.log(formatDiff(result.outputDiff));