POSTPOST /calculator/llm-gateway
LLM gateway cost
LLM-aware gateway TCO. Tokens-in, tokens-out, cache-hit percentage, and guardrail overhead.
// LLM gateway model 1M req/mo, 60% semantic cache hit{"portkey_production": "$99.00", // Production tier base"kong_ai_on_konnect_plus": "$250.00", // Konnect Plus floor + AI plugin bundle"cloudflare_ai_gateway": "$5.00", // Workers paid + AI Gateway free"provider_tokens_saved_by_cache": "$3,000.00", // 60% of $5,000 token bill"net_position_vs_no_gateway": "savings $2,750+" // after deducting gateway fee}
When the cache pays for itself
Semantic caching becomes the dominant economic lever on chat-style workloads with repeating context. On RAG workloads with high context variance, hit ratios tend to land below 20%, and the gateway becomes a routing and observability play rather than a cost-reduction play. Kong AI Gateway, Portkey, and Cloudflare AI Gateway all publish semantic-cache features.
See /guides/llm-gateway-pricing-2026 for the long-form field guide.