ollama-ai-provider-v2 weekly downloads statistics

Ollama Provider V2 for Vercel AI SDK

Use Ollama with the Vercel AI SDK, implementing the official Ollama API. This provider has minimal dependencies and is web-compatible out of the box.

Support this project — Maintained in spare time and free to use under Apache-2.0. If you value this plugin and appreciate it, a donation is appreciated and helps keep it updated with new AI SDK releases.

Why Choose Ollama Provider V2?

• ✅ Minimal Dependencies - Lean codebase with just 2 core dependencies
• ✅ Universal Compatibility - Works seamlessly in Node.js, browsers, and edge environments
• ✅ Direct Ollama Integration - Uses official Ollama API endpoints for maximum compatibility
• ✅ Advanced Features - Tool calling, streaming, thinking mode, embeddings, and completion models
• ✅ Type Safety - Full TypeScript support with comprehensive type definitions
• ✅ Zero Configuration - Works out-of-the-box with sensible defaults
• ✅ Actively Maintained - Regular updates and AI SDK v7 compatibility

Quick Start

npm install ollama-ai-provider-v2 ai

import { ollama } from 'ollama-ai-provider-v2';
import { generateText } from 'ai';

// Works everywhere - Node.js, browsers, edge environments
const { text } = await generateText({
  model: ollama('llama3.2'),
  prompt: 'Write a haiku about coding',
});

console.log(text);

Core Features

Streaming Text Generation

import { streamText } from 'ai';

const { textStream } = await streamText({
  model: ollama('llama3.2'),
  prompt: 'Tell me a story about artificial intelligence',
});

for await (const chunk of textStream) {
  process.stdout.write(chunk);
}

Tool Calling Support

AI SDK 7 defaults to a single generation step. When the model calls a tool, that step ends with empty text and populated toolCalls. Set stopWhen to allow the model to execute tools and then answer.

import { generateText, tool, stepCountIs } from 'ai';
import { z } from 'zod';

const { text, toolCalls, toolResults } = await generateText({
  model: ollama('llama3.2'),
  stopWhen: stepCountIs(5), // allow tool call + follow-up answer
  prompt: 'What is the weather like in San Francisco?',
  tools: {
    getWeather: tool({
      description: 'Get current weather for a location',
      inputSchema: z.object({
        location: z.string().describe('City name'),
        unit: z.enum(['celsius', 'fahrenheit']).optional(),
      }),
      execute: async ({ location, unit = 'celsius' }) => {
        return { temp: 18, unit, condition: 'sunny' };
      },
    }),
  },
});

Reasoning Mode (Thinking)

Unique feature for models that support chain-of-thought reasoning:

const { text } = await generateText({
  model: ollama('deepseek-r1:7b'),
  reasoning: 'high', // AI SDK 7 top-level reasoning option
  prompt: 'Solve this complex math problem step by step: 2x + 5 = 17',
});

You can also use the Ollama-specific provider option:

const { text } = await generateText({
  model: ollama('deepseek-r1:7b'),
  providerOptions: { ollama: { think: true } },
  prompt: 'Solve this complex math problem step by step: 2x + 5 = 17',
});

Advanced Ollama Options

Access Ollama's native parameters while maintaining AI SDK compatibility:

const { text } = await generateText({
  model: ollama('llama3.2'),
  providerOptions: {
    ollama: {
      options: {
        seed: 123,           // Deterministic outputs
        num_ctx: 8192,       // Context window size
        repeat_penalty: 1.1, // Control repetition
        top_k: 40,          // Advanced sampling
        min_p: 0.1,         // Minimum probability
      },
    },
  },
  prompt: 'Write a detailed analysis of Alice in Wonderland',
  temperature: 0.8, // Standard AI SDK parameters work too
});

Embeddings

import { embed } from 'ai';

// Single embedding
const { embedding } = await embed({
  model: ollama.embedding('nomic-embed-text'),
  value: 'Hello world',
});

console.log('Embedding dimensions:', embedding.length);

// Batch embeddings
const texts = ['Hello world', 'How are you?', 'AI is amazing'];
const results = await Promise.all(
  texts.map((text) =>
    embed({
      model: ollama.embedding('all-minilm'),
      value: text,
    })
  )
);

Completion Models

For use cases requiring completion-style generation:

const { text } = await generateText({
  model: ollama.completion('codellama:code'),
  prompt: 'def fibonacci(n):\n    if n <= 1:\n        return n\n    else:\n        return fibonacci(n-1) + fibonacci(n-2)\n\n# Optimize this function:\n',
});

Custom Ollama Instance

Connect to remote Ollama servers or custom configurations:

import { createOllama } from 'ollama-ai-provider-v2';

const customOllama = createOllama({
  baseURL: 'https://my-ollama-server.com/api',
  headers: {
    'Authorization': 'Bearer your-token',
  },
});

const { text } = await generateText({
  model: customOllama('llama3.2'),
  prompt: 'Hello from remote server!',
});

Supported Models

Works with any model in your Ollama installation, including:…