Grade: outage apology email

The job

Jev judge eval (1 score) over a fixed state : runnable via experimental_evaluate.

The prompt

Copy this into Muse and replace anything in [square brackets] with your own details.

// Grade: outage apology email — Jev judge eval (typesafe-ai/jev).
// A startup apologizes for a 4-hour outage; the judge scores the apology.
// Model: typesafe-ai/jev via the Vercel AI Gateway ($0.042/1M input tokens).
// Auth: set AI_GATEWAY_API_KEY (vck_…) in the environment — never paste the key here.
// For zero-data-retention / no-training workloads, add
// providerOptions: { gateway: { … } } to the evaluate call below.

import { experimental_evaluate } from 'ai';

const result = await experimental_evaluate({
  model: 'typesafe-ai/jev',
  state: "Email: Sorry for the downtime today. Our database had an issue and the site was down for about 4 hours. We know many of you were in the middle of launches. Here is what broke, the fix we shipped, and the guardrail we added so it cannot recur. A 10% credit is automatic — no action needed. — CTO",
  questions: {
    "apology": {
      "type": "score",
      "instructions": "Score the quality of this outage apology. Higher is better.",
      "criteria": [
        "Deflects blame or hides what happened.",
        "Admits the outage but offers no cause, remedy, or compensation.",
        "Explains cause and fix but leaves the customer to claim any remedy.",
        "Owns it, explains cause and fix, prevents recurrence, compensates automatically."
      ]
    }
  },
});

console.log(JSON.stringify(result, null, 2));