PatternsSDK API

Scrape - Jina AI (advanced)

Loading pattern preview

Loading preview…

Problems solved

Technical challenges this implementation handles out of the box.
  • Fetch clean page markdown through Jina Reader
  • Feed URL content into a tool-calling model flow
  • Demonstrate Jina scrape wiring with the AI SDK

Use cases

Products and workflows this pattern is designed to support.
  • Jina Reader onboarding demos
  • URL-to-markdown research tools
  • Documentation ingestion samples
  • Clean web content extraction prototypes

Setup

Requirements, wiring steps, and what this pattern adds to your project.

Getting started

Pick how you want to pull this pattern in — then wire env vars and routes the same way.

  1. 1

    Install from the preview toolbar

    Shadcn Icon

    Copy the install command above and run it in your project — Pro patterns include a short-lived token.

  2. 2

    Add environment variables to .env.local

    OPENAI_API_KEY

  3. 3

    Point client fetches at your API routes

    Replace preview `/view/.../api/...` paths with `/api/...` in the installed files.

  4. 4

    Customize the agent and tool files

    Adapt prompts, tools, and stop conditions for your product — Copy for AI in the toolbar helps seed that work.

  5. 5

    Run your dev server and open the pattern route

    Install dependencies if needed, then start the app and verify responses.

Environment variables

Capabilities

AI SDK APIs

streamTextconvertToCoreMessagestooluseChat

Providers

OpenAI

External services

Jina

Files added

6 files
  • app/page.tsx
  • app/layout.tsx
  • app/api/tool-scrape-jina/route.ts
  • components/jina-scrape-demo.tsx
  • tools/jina.ts
  • tools/jina-ui.tsx

Dependencies

9 total

npm packages

5
AISDKlucide-reactNext.jsReactZod

Shadcn IconRegistry components

4
alertemptyinput-groupseparator

Critical files

  • tool-scrape-jina/route.ts

    API route implementing a web scraping assistant using streamText with Jina AI tools. Contains the AI pipeline with system prompt, tool binding, and step count limits.

  • tools/jina.ts

    Core scraping engine and tool definitions using Jina AI Reader. Implements scrapeUrl (proxied through Jina service with regex-based parsing) and crawlWebsite (BFS crawler with depth limits).

Requirements

Complexity
Beginner