Loading pattern preview
BlocksTools & Utilities
Jev Best-of-N Judge
Jev Best-of-N Judge
Three prepared replies sit on the table. A Jev choice lifts the winner only when its probability clears 0.70.
Use this when you already have several candidate replies to one prompt and need one chosen, or none at all if the pick is too close to call.
Loading pattern preview
Problems solved
Technical challenges this implementation handles out of the box.
- Pick among prepared replies with a choice question
- Use the replies themselves as the choice criteria
- Leave every reply on the table when the selected probability is too close
Use cases
Products and workflows this pattern is designed to support.
- Best-of-N answer selection
- Refusing a toss-up instead of auto-shipping
- Teaching a choice question over a closed reply set
Compare with
Similar patterns, and when each one is the better fit.
Jev Output VerifierUse this when you want your own code to hold the confidence floor a draft answer must clear before display, with at most one retry on a miss.
Jev Confidence RouterFreeUse this when incoming messages must land in one of a fixed set of team queues, with low-confidence cases held back for human review.
Jev Composite ScorecardUse this when one blended rubric hides which factor failed and you want atomic Jev scores merged into a single weighted score in your app.
Jev Search FilterFreeUse this when you want semantic search over a known catalog, such as emoji or icons, without building embeddings, scoring each item with a boolean.
Setup
Requirements, wiring steps, and what this pattern adds to your project.
Getting started
Pick how you want to pull this pattern in. Then wire env vars and routes the same way.
- 1
Download → Next.js app
Use Download → Next.js app in the preview toolbar. You get a zip scaffold (Pro downloads rewrite preview `/view/.../api/*` paths to `/api/*` automatically).
- 2
Extract the zip and install dependencies
Unzip the archive, then run `pnpm install` (or npm / yarn) in the project folder.
- 3
Fill .env.local
The download includes .env.local placeholders when needed. Add real values for: AI_GATEWAY_API_KEY.
- 4
Customize the agent and tool files
Adapt prompts, tools, and stop conditions in the generated project for your product.
- 5
Run the app and open the pattern route
Run `pnpm dev`, open the setup/checklist page if present, then navigate to the generated pattern route and verify responses.
Environment variables
AI_GATEWAY_API_KEYGet key
Capabilities
AI SDK APIs
experimental_evaluateProviders
Jev
External services
gatewayvercel-ai-gatewayUpstash
Files added
13 files
app/page.tsxapp/layout.tsxapp/api/best-of-n/route.tscomponents/best-of-n-demo.tsxcomponents/reply-table.tsxcomponents/typesafe-ai.tsxcomponents/probability-bar.tsxhooks/use-best-of-n.tslib/judge-policy.tslib/judge-questions.tslib/rate-limit.tslib/request-utils.tsREADME.md
Dependencies
7 total
npm packages
6UpstashAISDKlucide-reactmotionNext.jsReact
Registry components
button
Critical files
3 files
lib/judge-policy.tsdecideWinner ships only when the selected probability is >= 0.70. A missing distribution, or a selected probability below the floor, is too-close and lifts nobody. Do not call Jev from unit tests. Use Experimental_EvaluationMockModelV4.
lib/judge-questions.tsPREPARED_QUESTIONS is the closed reply set. buildWinnerQuestions copies those replies into the choice criteria. Reject any question that is not in the set. Do not generate replies.
best-of-n/route.tsRate-limited experimental_evaluate through AI Gateway model typesafe-ai/jev with zeroDataRetention. Looks up prepared replies, then returns decideWinner. Does not call generateText.
Requirements
- Complexity
- Beginner
- Setup time
- ~5 minutes
- Node
- 24+
- Memory
- 256MB
Upstash Redis for deployed demo rate limiting
