Grade: onboarding email quality

The job

Jev judge eval (1 score) over a fixed state : runnable via experimental_evaluate.

The prompt

Copy this into Muse and replace anything in [square brackets] with your own details.

// Grade: onboarding email quality — Jev judge eval (typesafe-ai/jev).
// An agent drafts a day-1 onboarding email; the judge scores it.
// Model: typesafe-ai/jev via the Vercel AI Gateway ($0.042/1M input tokens).
// Auth: set AI_GATEWAY_API_KEY (vck_…) in the environment — never paste the key here.
// For zero-data-retention / no-training workloads, add
// providerOptions: { gateway: { … } } to the evaluate call below.

import { experimental_evaluate } from 'ai';

const result = await experimental_evaluate({
  model: 'typesafe-ai/jev',
  state: "Email: Welcome! Here are your 3 first steps: 1) connect your calendar (2 min), 2) invite one teammate, 3) run the sample project. Each links below. Reply to this email any time — a human reads every reply within a day.",
  questions: {
    "onboarding": {
      "type": "score",
      "instructions": "Score this onboarding email. Higher is better.",
      "criteria": [
        "Wall of text or zero clear actions.",
        "Lists actions but no links, time estimates, or human fallback.",
        "Clear actions with links, but no time framing or reply path.",
        "Three crisp actions with links, time estimates, and a human reply path."
      ]
    }
  },
});

console.log(JSON.stringify(result, null, 2));