Loading pattern preview
BlocksTools & Utilities
Jev Composite Scorecard
Jev Composite Scorecard
Score a pull request description with three atomic Jev score questions, then combine the answers with weights in code.
Use this when one blended rubric hides which factor failed and you want atomic Jev scores merged into a single weighted score in your app.
Loading pattern preview
Problems solved
Technical challenges this implementation handles out of the box.
- Score independent factors instead of one blended rubric
- Combine atomic scores with application-owned weights
- Show weighted-mean scores sitting between rubric levels
Use cases
Products and workflows this pattern is designed to support.
- PR description scorecards
- Content-quality gates
- Teaching atomic questions combined in code
Compare with
Similar patterns, and when each one is the better fit.
Jev EvalsUse this when you need per-PR eval checks for an agent that diff a candidate run against a stored baseline to flag fractional drops.
Jev Output VerifierUse this when you want your own code to hold the confidence floor a draft answer must clear before display, with at most one retry on a miss.
Jev JSON RenderUse this when users request UI edits that add, remove, or replace prepared elements in a json-render spec, and the model must not write new prose.
Jev Best-of-N JudgeUse this when you already have several candidate replies to one prompt and need one chosen, or none at all if the pick is too close to call.
Setup
Requirements, wiring steps, and what this pattern adds to your project.
Getting started
Pick how you want to pull this pattern in. Then wire env vars and routes the same way.
- 1
Download → Next.js app
Use Download → Next.js app in the preview toolbar. You get a zip scaffold (Pro downloads rewrite preview `/view/.../api/*` paths to `/api/*` automatically).
- 2
Extract the zip and install dependencies
Unzip the archive, then run `pnpm install` (or npm / yarn) in the project folder.
- 3
Fill .env.local
The download includes .env.local placeholders when needed. Add real values for: AI_GATEWAY_API_KEY.
- 4
Customize the agent and tool files
Adapt prompts, tools, and stop conditions in the generated project for your product.
- 5
Run the app and open the pattern route
Run `pnpm dev`, open the setup/checklist page if present, then navigate to the generated pattern route and verify responses.
Environment variables
AI_GATEWAY_API_KEYGet key
Capabilities
AI SDK APIs
experimental_evaluateProviders
Jev
External services
gatewayvercel-ai-gatewayUpstash
Files added
12 files
app/page.tsxapp/layout.tsxapp/api/scorecard/route.tscomponents/scorecard-demo.tsxcomponents/typesafe-ai.tsxcomponents/score-track.tsxhooks/use-scorecard.tslib/scorecard.tslib/scorecard-questions.tslib/rate-limit.tslib/request-utils.tsREADME.md
Dependencies
7 total
npm packages
5UpstashAISDKlucide-reactNext.jsReact
Registry components
buttontextarea
Critical files
3 files
lib/scorecard.tscombineScores multiplies each atomic score by SCORE_WEIGHTS in application code. Do not ask Jev to produce the composite. Pass at total >= 2.00. Test with Experimental_EvaluationMockModelV4.
lib/scorecard-questions.tsThree score questions with written 4-level rubrics (clarity, completeness, risk). Weights are visible constants: 0.4 / 0.4 / 0.2.
scorecard/route.tsRate-limited experimental_evaluate call through AI Gateway model typesafe-ai/jev with zeroDataRetention. One request, three independent score questions, then combineScores.
Requirements
- Complexity
- Beginner
- Setup time
- ~5 minutes
- Node
- 24+
- Memory
- 256MB
Upstash Redis for deployed demo rate limiting
