AI
AI SDK Patterns

Command Palette

Search for a command to run...

PatternsComposeGitHub
All Patterns

Core / SDK

9
  • Structured Output
  • Generative UI
  • JSON Renderer
  • Text Generation
  • Image Generation
  • Streaming Object
  • Code ArtifactPopular
  • Form Generator
  • CSV Editor

Chat

5
  • Streaming ChatPopular
  • Markdown Chat
  • Reasoning Display
  • Chat with Citations
  • Multi-Modal ChatNew

Agents

5
  • Tool CallingPopular
  • Multi-Step AgentPopular
  • Routing Agent
  • Orchestrator Agent
  • Evaluator-Optimizer

Tools

4
  • Web Search Agent
  • RAG PipelinePopular
  • MCP Client AgentNew
  • Text-to-SQLNew

Workflows

7
  • Human-in-the-Loop
  • Sequential Workflow
  • Parallel Workflow
  • Durable Multi-Turn Chat Agent
  • Human-in-the-Loop Approval Workflow
  • Scheduled/Delayed AI Task
  • Refinement Loop
PatternsComposeAI SDK DocsGitHub

Built with AI SDK · shadcn/ui · Next.js

© 2026 AI SDK Patterns

Multi-Modal Chat

Chat with images, files, and text in a single conversation. Drag-and-drop or paste images for vision analysis, attach files for context, and get AI responses that understand all modalities.

Newchatintermediatemultimodalvisionimage-uploadfile-attachmentdrag-drop

Loading interactive preview...

Installation

Option 1: Install via CLI

pnpm dlx shadcn@latest add https://ai-sdk-patterns.vercel.app/r/multimodal-chat

Automatically installs the pattern and its dependencies in your project.

Option 2: Copy or Download

Download the complete pattern as a standalone Next.js project.

Usage

1. Set up environment variables

# .env.local
ANTHROPIC_API_KEY=your_anthropic_key
OPENAI_API_KEY=your_openai_key
GOOGLE_GENERATIVE_AI_API_KEY=your_google_key

Add your AI provider API key to enable real functionality.

2. Run the development server

npm run dev
# or
pnpm dev
# or
yarn dev

Open http://localhost:3000 to see the pattern in action.

3. Customize for your needs

The pattern is ready to use. Modify the components, API routes, and styling to fit your application.

  • Update the UI components in app/page.tsx
  • Modify API logic in app/api/
  • Adjust styling with Tailwind CSS classes
  • Add your own business logic and data sources

Use Cases

Visual QA and Image Analysis

Build apps where users upload photos for identification, analysis, or description — from plant ID to architecture review.

Document Processing Assistants

Create tools that read PDFs, invoices, receipts, and contracts, extracting key information through conversation.

Creative Design Feedback

Build review tools where designers upload mockups and get AI feedback on layout, accessibility, and design principles.

Medical and Scientific Image Review

Develop assistants that analyze medical images, lab results, or scientific diagrams with expert-level context.

Technical Details

Dependencies

• Next.js 16+ (App Router)
• AI SDK v6
• React 19+
• Tailwind CSS
• TypeScript

Files Included

• app/page.tsx
• app/api/multimodal/route.ts
• lib/model.ts

Related patterns

Streaming Chat

Stream text responses from AI models in real-time using streamText and the useChat hook.

Markdown Chat

A polished chat interface with rich markdown rendering — code blocks with syntax highlighting, tables, lists, headings, and inline formatting.

🔒 Demo mode - multi-modal chat with image attachments