Loading pattern preview
TemplatesTools & Utilities
Local RAG Chat
Local RAG Chat
Load a sample Local RAG index, rank chunks in the browser, and stream a cited answer without a vector database.
Loading pattern preview
Problems solved
Technical challenges this implementation handles out of the box.
- Import a portable embedding index and chat over it in the browser
- Rank local vectors with AI SDK Core cosineSimilarity
- Stream cited answers from bounded selected text without a vector database
Use cases
Products and workflows this pattern is designed to support.
- Query a portable documentation index without a database
- Local RAG demos and workshops
- Cited chat over an exported embedding JSON file
Setup
Requirements, wiring steps, and what this pattern adds to your project.
Getting started
Pick how you want to pull this pattern in. Then wire env vars and routes the same way.
- 1
Download → Next.js app
Use Download → Next.js app in the preview toolbar. You get a zip scaffold (Pro downloads rewrite preview `/view/.../api/*` paths to `/api/*` automatically).
- 2
Extract the zip and install dependencies
Unzip the archive, then run `pnpm install` (or npm / yarn) in the project folder.
- 3
Fill .env.local
The download includes .env.local placeholders when needed. Add real values for: AI_GATEWAY_API_KEY.
- 4
Customize the agent and tool files
Adapt prompts, tools, and stop conditions in the generated project for your product.
- 5
Run the app and open the pattern route
Run `pnpm dev`, open the setup/checklist page if present, then navigate to the generated pattern route and verify responses.
Environment variables
AI_GATEWAY_API_KEYGet key
Capabilities
AI SDK APIs
embedcosineSimilaritystreamTextsafeValidateUIMessagesconvertToModelMessagescreateUIMessageStreamtoUIMessageStreamcreateUIMessageStreamResponseExternal services
gatewayvercel-ai-gatewayUpstash
Files added
29 filesapp/page.tsxapp/layout.tsxapp/api/query-embedding/route.tsapp/api/chat/route.tscomponents/local-rag-chat.tsxcomponents/index-import.tsxcomponents/local-citations.tsxhooks/use-local-rag-chat.tslib/chat-contract.tslib/chunk-text.tslib/embedding-config.tslib/local-rag-index.tslib/rate-limit.tslib/request-utils.tslib/retrieval.tslib/sample-indexes.tsfixtures/northwind-ops-handbook.jsonfixtures/cedar-product-faq.jsonfixtures/atlas-deploy-runbook.jsonfixtures/sources/pto-and-holidays.mdfixtures/sources/tools-and-access.mdfixtures/sources/expense-policy.mdfixtures/sources/cedar/sharing-and-permissions.mdfixtures/sources/cedar/plans-and-billing.mdfixtures/sources/cedar/export-formats.mdfixtures/sources/atlas/rollback-procedure.mdfixtures/sources/atlas/on-call-and-paging.mdfixtures/sources/atlas/feature-flags.mdREADME.md
Dependencies
11 totalnpm packages
7AISDKUpstashlucide-reactNext.jsReactstreamdownZod
Registry components
badgebuttontextareatooltip
Critical files
lib/local-rag-index.tsCopied V1 portable embedding-index contract. Must stay parity-tested against local-rag-ingest. Reject incompatible schema/version/model/dimensions/chunking before cosineSimilarity.
lib/retrieval.tsBrowser-side ranking with AI SDK Core cosineSimilarity, threshold/top-K/context budget selection, and citation helpers. Keep ranking in the browser; never upload the full index to rank on the server.
query-embedding/route.tsStateless Core embed boundary for one query string. Locked model openai/text-embedding-3-small with providerOptions.openai.dimensions=512. Never add @ai-sdk/openai solely for embeddings.
chat/route.tsCited answer boundary: validate retrieval body without embeddings, assign S1..S5 server-side, write non-transient data-retrieval, convert prior data parts to undefined, streamText via gateway openai/gpt-5.4-mini.
Requirements
- Complexity
- Intermediate
- Setup time
- ~10 minutes
- Node
- 24+
- Memory
- 256MB
Upstash Redis for deployed demo rate limitingA completed local-rag-ingest JSON export
