Grade: code review actionability

The job

Jev judge eval (1 score) over a fixed state : runnable via experimental_evaluate.

The prompt

Copy this into Muse and replace anything in [square brackets] with your own details.

// Grade: code review actionability — Jev judge eval (typesafe-ai/jev).
// A reviewer comments on a PR; the judge scores how actionable the feedback is.
// Model: typesafe-ai/jev via the Vercel AI Gateway ($0.042/1M input tokens).
// Auth: set AI_GATEWAY_API_KEY (vck_…) in the environment — never paste the key here.
// For zero-data-retention / no-training workloads, add
// providerOptions: { gateway: { … } } to the evaluate call below.

import { experimental_evaluate } from 'ai';

const result = await experimental_evaluate({
  model: 'typesafe-ai/jev',
  state: "Review comment on PR #4412: This caching layer looks sketchy. Maybe rethink it? Also the naming is confusing in a few places. Not sure about the Redis TTL stuff either.",
  questions: {
    "actionability": {
      "type": "score",
      "instructions": "Score how actionable this review is for the author. Higher is better.",
      "criteria": [
        "Vague unease with no file, line, or concrete change requested.",
        "Names problem areas but leaves the fix entirely to the author.",
        "Specific issues with suggested fixes, but missing severity or priority.",
        "Specific issues, exact locations, suggested fixes, and clear must-fix vs nit split."
      ]
    }
  },
});

console.log(JSON.stringify(result, null, 2));