Skip to content
GATEWAY & CACHING

Save up to 40% with smart caching

OpenAI-compatible proxy with semantic response caching. Multi-provider routing with automatic failover. Low latency overhead.

Start for FreeView documentation
THE ORDER A LOOKUP RUNS INfig. 01
cache · isolated per projectpromptexact match · sha-256ttl 1 hmisshitsemantic · cosine ≥ 0.92ttl 24 hmisshitprovider callbilledresponsesame response shapea provider response is written back to both stores
The hash lookup runs first; the embedding is the miss path, not the default. A prompt only reaches the provider — and only costs you a token — after both stores miss.

What the gateway gives you

01
Semantic Caching
Turn caching on per request, by exact match or by meaning. A cached answer only ever goes back to the end user it was generated for.
02
Up to 40% Cost Reduction
Dramatically reduce LLM costs by serving cached responses for similar requests.
03
Multi-Provider Routing
Route requests to OpenAI, Anthropic, Google, Mistral, Azure, and more from a single endpoint.
04
Automatic Failover
If one provider is down or slow, automatically route to alternatives. Zero downtime.
05
Low Latency Overhead
Security and caching add minimal latency. Full streaming support maintained.
06
Per-Key Rate Limits
Set different rate limits for different API keys. Control costs and prevent abuse.

How the gateway works

  1. 01

    Route

    Your request comes in. If it asked for caching, we check the cache, then select the provider based on your configuration.

  2. 02

    Secure

    All security checks run in parallel. Threats are blocked, PII is redacted, policies are enforced.

  3. 03

    Respond

    The response is returned from cache or from your provider. A request that asked for caching stores it for that end user's next matching request.

Drop-in replacement

typescript
import OpenAI from 'openai';

// Just change the base URL - works with any provider
const client = new OpenAI({
  baseURL: 'https://api.promptguard.co/api/v1',
  apiKey: process.env.OPENAI_API_KEY,
  defaultHeaders: {
    'X-API-Key': process.env.PROMPTGUARD_API_KEY,
    'X-End-User': currentUser.id,
    'X-PromptGuard-Cache': 'semantic' // optional: caching is off without it
  }
});

// All your existing code works unchanged
const response = await client.chat.completions.create({
  model="gpt-5-nano",
  messages: [{ role: 'user', content: 'Hello!' }]
});

// Security and routing on every request, caching where you asked for it

Why PromptGuard Gateway?

  • Opt-in semantic caching, scoped to each end user
  • Up to 40% cost reduction
  • Multi-provider with failover
  • Low latency overhead
  • Full streaming support

Other controls on the same path

Start saving on LLM costs

Drop-in replacement for OpenAI. Same code, significant cost savings, enterprise security included.

Get started freeContact sales