Skip to content
GATEWAY & CACHING

Save up to 40% with smart caching

OpenAI-compatible proxy with semantic response caching. Multi-provider routing with automatic failover. Low latency overhead.

Start for FreeView documentation
THE ORDER A LOOKUP RUNS INfig. 01
cache · isolated per projectpromptexact match · sha-256ttl 1 hmisshitsemantic · cosine ≥ 0.92ttl 24 hmisshitprovider callbilledresponsesame response shapea provider response is written back to both stores
The hash lookup runs first; the embedding is the miss path, not the default. A prompt only reaches the provider — and only costs you a token — after both stores miss.

What the gateway gives you

01
Semantic Caching
Cache responses based on meaning, not just exact matches. Similar prompts hit the cache even with different wording.
02
Up to 40% Cost Reduction
Dramatically reduce LLM costs by serving cached responses for similar requests.
03
Multi-Provider Routing
Route requests to OpenAI, Anthropic, Google, Mistral, Azure, and more from a single endpoint.
04
Automatic Failover
If one provider is down or slow, automatically route to alternatives. Zero downtime.
05
Low Latency Overhead
Security and caching add minimal latency. Full streaming support maintained.
06
Per-Key Rate Limits
Set different rate limits for different API keys. Control costs and prevent abuse.

How the gateway works

  1. 01

    Route

    Your request comes in. We check the cache and select the optimal provider based on your configuration.

  2. 02

    Secure

    All security checks run in parallel. Threats are blocked, PII is redacted, policies are enforced.

  3. 03

    Respond

    Response is returned (from cache or provider) and stored for future similar requests.

Drop-in replacement

typescript
import OpenAI from 'openai';

// Just change the base URL - works with any provider
const client = new OpenAI({
  baseURL: 'https://api.promptguard.co/api/v1',
  apiKey: process.env.OPENAI_API_KEY,
  defaultHeaders: {
    'X-API-Key': process.env.PROMPTGUARD_API_KEY
  }
});

// All your existing code works unchanged
const response = await client.chat.completions.create({
  model="gpt-5-nano",
  messages: [{ role: 'user', content: 'Hello!' }]
});

// Automatic caching, security, and routing!

Why PromptGuard Gateway?

  • Semantic caching (meaning-based)
  • Up to 40% cost reduction
  • Multi-provider with failover
  • Low latency overhead
  • Full streaming support

Other controls on the same path

Start saving on LLM costs

Drop-in replacement for OpenAI. Same code, significant cost savings, enterprise security included.

Get started freeContact sales