ai-gateway-proxy weekly downloads statistics

ai-gateway-proxy

For apps whose AI SDK code runs where a Vercel AI Gateway key must not live: a browser, a mobile or desktop app, or code a third party runs. The client calls a route in your app, and ai-gateway-proxy forwards the call to the gateway from your server.

• The gateway credential stays on the server: the proxy authenticates with an API key or Vercel OIDC and never sends it to the client (Authentication).
• You decide who may call: your route can check the caller's session or token before handing the call to the proxy (Protecting the route).
• Each call can be capped, metered, rewritten or refused: beforeRequest, afterResponse and onError handle language model calls, and beforeRawRequest and afterRawResponse the other calls (embedding, image, video, speech, transcription, reranking, decision, batch start) (Example: cap, meter and mask, Refusing a call, Raw hooks).
• Streams are forwarded as they arrive: each SSE event reaches the client as the gateway sends it, and afterResponse gets the aggregated result when the stream finishes (afterResponse).

The handler takes a Fetch Request and returns a Response, so it plugs into Next.js, Hono, SvelteKit, Astro and React Router directly, into Nuxt through h3's toWebRequest, and into Express or node:http through a small adapter (Quick Start).

When you do not need it

If every model call already runs on your own server, such as in a route handler, a server action or a background job, call the gateway there with the AI SDK's AI Gateway provider and skip the proxy. That provider reads AI_GATEWAY_API_KEY or the Vercel OIDC token on the server itself, so the key never leaves it, and any check or metering is your own code around generateText or streamText.

The proxy is for the other cases: a client that must not hold the key, or calls from several clients that you want to check, cap or meter in one place.

It does not carry every gateway call either. Streaming speech, streaming transcription and realtime open WebSockets, which the HTTP-only handler cannot carry, and the credits, spend report, generation info and getToken calls go to your app's origin instead of the route (Protecting the route lists each one).

Installation

npm install ai-gateway-proxy

To have a coding agent set it up, install the agent skill with npx skills add sferarc/ai-gateway-proxy. It covers when the proxy is needed, the route per framework, the client, protecting the route and metering, without loading this README.

Quick Start

Next.js App Router

Create a catch-all route at app/api/ai/[...segments]/route.ts:

import { createGatewayProxy } from "ai-gateway-proxy";

export const { GET, POST, PUT, DELETE, PATCH } = createGatewayProxy();

The handler reads the path from the route's segments param. For a differently named catch-all, such as app/api/ai/[...path]/route.ts, pass createGatewayProxy({ segmentsParam: "path" }).

examples/nextjs is a runnable App Router app built on this route: it checks the caller as in Protecting the route, caps each caller's output tokens with beforeRequest, charging the cap until afterResponse reports the usage so a cancelled stream still counts, and streams from the browser with streamText.

Hono

For frameworks without catch-all route params, use the extractPath option:

import { Hono } from "hono";
import { createGatewayProxy } from "ai-gateway-proxy";

const app = new Hono();

const proxy = createGatewayProxy({
  extractPath: (request) => {
    const url = new URL(request.url);
    return url.pathname.replace(/^\/api\/ai\//, "");
  },
});

app.all("/api/ai/*", (c) => proxy(c.req.raw));

SvelteKit

Create a rest route at src/routes/api/ai/[...path]/+server.ts. SvelteKit passes params.path as one string such as language-model, which segmentsParam reads once the params are wrapped in a promise:

import { createGatewayProxy } from "ai-gateway-proxy";
import type { RequestHandler } from "./$types";

const proxy = createGatewayProxy({ segmentsParam: "path" });

export const fallback: RequestHandler = ({ request, params }) =>
  proxy(request, { params: Promise.resolve(params) });

fallback answers every method the file does not export a handler for, so the POST model calls and the GET reads such as getAvailableModels both reach the proxy.

Astro

Create a rest route at src/pages/api/ai/[...path].ts. Astro also passes params.path as one string:

import type { APIRoute } from "astro";
import { createGatewayProxy } from "ai-gateway-proxy";

export const prerender = false;

const proxy = createGatewayProxy({ segmentsParam: "path" });

export const ALL: APIRoute = ({ request, params }) =>
  proxy(request, { params: Promise.resolve(params) });

The route runs on demand, so the project needs a server adapter. prerender = false keeps it from being built as a static file under the default static output; with output: "server" it can go.

React Router

In React Router framework mode, a route module that exports loader and action and no component answers requests directly. Create app/routes/ai-proxy.ts:

import { createGatewayProxy } from "ai-gateway-proxy";
import type { Route } from "./+types/ai-proxy";

const proxy = createGatewayProxy({ segmentsParam: "*" });

export const loader = ({ request, params }: Route.LoaderArgs) =>
  proxy(request, { params: Promise.resolve(params) });…