Prompt Caching
A vendor feature that caches the prefix of a prompt so repeated calls reuse it at a discount.
What is Prompt Caching?
Many Gen AI workflows reuse the same long prefix (a system prompt, retrieved context, examples) across many requests. Prompt caching lets the vendor remember that prefix and skip recomputing it — charging a fraction of the normal token rate on cache hits.
Claude offers up to 90% discount on cached input. OpenAI offers 50%. The cache typically lasts 5–10 minutes. For workloads with a stable prefix (long system prompt + per-user query), savings of 50–80% on input tokens are realistic.
For Indian startups serving high-volume traffic on the same prompt template, prompt caching is the single biggest cost lever after model selection. Implementing it correctly (cache breakpoints, key ordering) requires understanding what the vendor caches and when.
Prompt caching is the cheapest possible inference optimisation — pure config win. Senior Gen AI engineers reach for it before any other cost optimisation.
A Pune document-analysis startup runs 10,000 queries per day against a stable 8,000-token instruction prompt. Enabling Claude prompt caching dropped their input-token cost 87% — saving ₹1.4 Lakh per month with no code changes beyond cache headers.
Want to master this?
Learn Prompt Caching in a structured cohort
3-month live program with mentors, real projects, and 50+ partner placement support.