Loading pattern preview
BlocksTools & Utilities
Jev Best-of-N Judge
Jev Best-of-N Judge
Three prepared replies sit on the table. A Jev choice lifts the winner only when its probability clears 0.70.
Loading pattern preview
Problems solved
Technical challenges this implementation handles out of the box.
- Pick among prepared replies with a choice question
- Use the replies themselves as the choice criteria
- Leave every reply on the table when the selected probability is too close
Use cases
Products and workflows this pattern is designed to support.
- Best-of-N answer selection
- Refusing a toss-up instead of auto-shipping
- Teaching a choice question over a closed reply set
Setup
Requirements, wiring steps, and what this pattern adds to your project.
Getting started
Pick how you want to pull this pattern in. Then wire env vars and routes the same way.
- 1
Download → Next.js app
Use Download → Next.js app in the preview toolbar. You get a zip scaffold (Pro downloads rewrite preview `/view/.../api/*` paths to `/api/*` automatically).
- 2
Extract the zip and install dependencies
Unzip the archive, then run `pnpm install` (or npm / yarn) in the project folder.
- 3
Fill .env.local
The download includes .env.local placeholders when needed. Add real values for: AI_GATEWAY_API_KEY.
- 4
Customize the agent and tool files
Adapt prompts, tools, and stop conditions in the generated project for your product.
- 5
Run the app and open the pattern route
Run `pnpm dev`, open the setup/checklist page if present, then navigate to the generated pattern route and verify responses.
Environment variables
AI_GATEWAY_API_KEYGet key
Capabilities
AI SDK APIs
experimental_evaluateProviders
Jev
External services
gatewayvercel-ai-gatewayUpstash
Files added
13 filesapp/page.tsxapp/layout.tsxapp/api/best-of-n/route.tscomponents/best-of-n-demo.tsxcomponents/reply-table.tsxcomponents/typesafe-ai.tsxcomponents/probability-bar.tsxhooks/use-best-of-n.tslib/judge-policy.tslib/judge-questions.tslib/rate-limit.tslib/request-utils.tsREADME.md
Dependencies
7 totalnpm packages
6UpstashAISDKlucide-reactmotionNext.jsReact
Registry components
button
Critical files
lib/judge-policy.tsdecideWinner ships only when the selected probability is >= 0.70. A missing distribution, or a selected probability below the floor, is too-close and lifts nobody. Do not call Jev from unit tests. Use Experimental_EvaluationMockModelV4.
lib/judge-questions.tsPREPARED_QUESTIONS is the closed reply set. buildWinnerQuestions copies those replies into the choice criteria. Reject any question that is not in the set. Do not generate replies.
best-of-n/route.tsRate-limited experimental_evaluate through AI Gateway model typesafe-ai/jev with zeroDataRetention. Looks up prepared replies, then returns decideWinner. Does not call generateText.
Requirements
- Complexity
- Beginner
- Setup time
- ~5 minutes
- Node
- 24+
- Memory
- 256MB
Upstash Redis for deployed demo rate limiting
