@qvac/ai-sdk-provider weekly downloads statistics

@qvac/ai-sdk-provider

Vercel AI SDK provider for the QVAC local AI runtime.

QVAC is an open-source, cross-platform ecosystem for local-first, peer-to-peer AI — LLMs, embeddings, transcription, translation, speech, OCR, and image generation, all running on the user's own hardware. This package implements the native provider contracts for language, embedding, image, file-reference, transcription, and speech operations against a local qvac serve --openai runtime. It also re-exports QVAC's model metadata so callers can introspect typed model constants without an HTTP round-trip.

The provider is OpenAI-compatible by construction: the OpenAI-compatible transport is its fallback provider, while the SDK's native custom-provider composition adds QVAC-specific files, transcription, and speech capabilities. Existing OpenAI-shaped chat, completion, embedding, and image requests keep the same wire protocol.

Runtime modes:

• External (default): the package wraps a qvac serve --openai HTTP endpoint that you run yourself.
• Managed (mode: 'managed'): the provider synthesizes an ephemeral config from a model list, then spawns (or reuses) a shared qvac serve on a free port and keeps it alive for as long as anything is using it, reaping it automatically once everyone is done. See Managed mode below. Requires the optional @qvac/cli peer dependency.

See the QVAC-19194 epic.

---

Install

bun add @qvac/ai-sdk-provider ai @ai-sdk/openai-compatible
# or: npm install @qvac/ai-sdk-provider ai @ai-sdk/openai-compatible

Runtime requirements:

• Node.js 22 or newer. Node 20 is not supported.
• AI SDK 7 (ai@^7) and OpenAI-compatible provider 3 (@ai-sdk/openai-compatible@^3).
• Provider-v4 contracts. Custom middleware and direct model integrations must use the AI SDK v4 provider interfaces exposed by these versions.
• ESM-only imports. Use import / dynamic import(); CommonJS require() is not supported.

ai and @ai-sdk/openai-compatible are peer dependencies — install those compatible major versions alongside the provider.

---

Quickstart

1. Run qvac serve --openai

You need @qvac/cli installed and a minimal config that preloads at least one chat model:

npm i -g @qvac/cli

cat > qvac.config.json <<'EOF'
{
  "serve": {
    "models": {
      "qwen3-600m": { "model": "QWEN3_600M_INST_Q4", "preload": true }
    }
  }
}
EOF

export QVAC_API_KEY='replace-with-a-random-secret'
qvac serve --openai --no-default --api-key "$QVAC_API_KEY"

By default, qvac serve listens on http://127.0.0.1:11434/v1 (the port may change in a future CLI release — see the Default base URL note below).

2. Use the provider

import { createQvac } from '@qvac/ai-sdk-provider'
import { streamText } from 'ai'

const qvac = createQvac({
  baseURL: 'http://127.0.0.1:11434/v1', // match your `qvac serve` port
  apiKey: process.env.QVAC_API_KEY // must match the serve's --api-key value
})

const { textStream } = streamText({
  model: qvac('qwen3-600m'),
  prompt: 'Write a haiku about local-first AI.'
})

for await (const chunk of textStream) {
  process.stdout.write(chunk)
}

The provider exposes the same surface as any AI SDK provider:

qvac('qwen3-600m') // language model (chat)
qvac.chatModel('qwen3-600m') // explicit chat model
qvac.completionModel('qwen3-600m') // legacy completion
qvac.textEmbeddingModel('embed-gemma') // text embeddings
qvac.imageModel('flux-schnell') // image generation
qvac.transcriptionModel('whisper') // speech to text
qvac.speechModel('tts') // text to speech
qvac.files() // local ephemeral file uploads

Local file references

uploadFile stores bytes in the running QVAC process and returns a qvac provider reference. Passing that reference to a QVAC language model resolves the bytes through the same local runtime immediately before inference. The reference never becomes a cloud URL and expires with QVAC's ephemeral file store.

import { readFileSync } from 'node:fs'
import { generateText, uploadFile } from 'ai'

const { providerReference } = await uploadFile({
  api: qvac,
  data: readFileSync('./sensor.png'),
  mediaType: 'image/png',
  filename: 'sensor.png'
})

const { text } = await generateText({
  model: qvac('vision'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Read the sensor display.' },
        { type: 'file', mediaType: 'image/png', data: providerReference }
      ]
    }
  ]
})

Native speech and transcription

Speech operations use the provider's native model contracts while remaining entirely inside the local QVAC serve:

import { generateSpeech, transcribe } from 'ai'

const speech = await generateSpeech({
  model: qvac.speechModel('tts'),
  text: 'The sensor temperature is twenty four degrees.',
  voice: 'alloy',
  outputFormat: 'wav'
})

const transcript = await transcribe({
  model: qvac.transcriptionModel('stt'),
  audio: speech.audio.uint8Array,
  providerOptions: {
    qvac: { prompt: 'Sensor terminology' }
  }
})

Per-request transcription prompt is supported. Language is selected when the QVAC model is loaded, and speech speed, instructions, and language are not consumed by the current local engines; the provider returns structured warnings when callers supply those options.

---

Managed mode…