Grade: project plan completeness

The job

Jev judge eval (1 score) over a fixed state : runnable via experimental_evaluate.

The prompt

Copy this into Muse and replace anything in [square brackets] with your own details.

// Grade: project plan completeness — Jev judge eval (typesafe-ai/jev).
// An agent drafts a launch plan; the judge scores its completeness.
// Model: typesafe-ai/jev via the Vercel AI Gateway ($0.042/1M input tokens).
// Auth: set AI_GATEWAY_API_KEY (vck_…) in the environment — never paste the key here.
// For zero-data-retention / no-training workloads, add
// providerOptions: { gateway: { … } } to the evaluate call below.

import { experimental_evaluate } from 'ai';

const result = await experimental_evaluate({
  model: 'typesafe-ai/jev',
  state: "Plan: 1) Freeze scope Friday. 2) Ana owns staging QA, Ben owns migration dry-run. 3) Rollback: one-click revert to snapshot v41, tested Thursday. 4) Comms: status page + email at T-24h and T+1h. 5) Success: error rate <0.5% and p99 < 400ms for 48h.",
  questions: {
    "completeness": {
      "type": "score",
      "instructions": "Score the completeness of this launch plan. Higher is better.",
      "criteria": [
        "A bare task list with no owners, dates, or rollback.",
        "Tasks plus owners, but no rollback, comms, or success criteria.",
        "Owners, dates, and rollback, but vague comms or success measure.",
        "Owners, dates, tested rollback, comms plan, and numeric success criteria."
      ]
    }
  },
});

console.log(JSON.stringify(result, null, 2));