Prompt caching

How the cache works

Grokified caches from the start of your messages or input array. When a request arrives, the server checks how many messages at the beginning match a previous request exactly. That matching prefix is served from cache: faster time-to-first-token, and cached tokens bill at a reduced rate. Only new, appended messages get computed fresh.

Any edit to an earlier message breaks the match. Editing, removing, or reordering a message you already sent starts the prefix over from scratch. Only ever append.

Pinning a conversation to one server

Cache entries live on whichever server handled the request, so two turns of the same conversation need to land on the same server to hit. On /chat/completions and /completions, set the x-conversation-id header to a stable id per conversation. On /responses, set prompt_cache_key in the body instead, same idea.

curl https://api.grokified.cyou/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $GROKIFIED_API_KEY" \
  -d '{
    "model": "grok-4.6",
    "input": "What is prompt caching?",
    "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366"
  }'

Seeing it happen: a cold call, then a warm one

The first call on a new x-conversation-id has nothing to cache — cached_tokens comes back 0.

curl https://api.grokified.cyou/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $GROKIFIED_API_KEY" \
  -H "x-conversation-id: conv_abc123" \
  -d '{
    "model": "grok-4.6",
    "messages": [
      { "role": "system", "content": "You are a helpful assistant." },
      { "role": "user", "content": "What is prompt caching?" }
    ]
  }'

Append the assistant’s reply and send the next turn on the same x-conversation-id. The system prompt, the first user turn, and the first assistant reply all match the previous request’s prefix exactly, so they’re served from cache — cached_tokens now covers all of it, and only the new user message is computed fresh.

curl https://api.grokified.cyou/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $GROKIFIED_API_KEY" \
  -H "x-conversation-id: conv_abc123" \
  -d '{
    "model": "grok-4.6",
    "messages": [
      { "role": "system", "content": "You are a helpful assistant." },
      { "role": "user", "content": "What is prompt caching?" },
      { "role": "assistant", "content": "Prompt caching reuses the compute from an earlier request when a new request starts with the exact same messages, so repeated context is billed at a reduced rate instead of full price." },
      { "role": "user", "content": "Does it work across different servers?" }
    ]
  }'

Grokified sets this for you

If you don’t set prompt_cache_key and don’t send x-conversation-id, Grokified sets prompt_cache_key for you on /chat/completions, /completions, and /responses. It derives a stable key from your account plus the conversation, so your prompt prefix keeps hitting the same cache without you wiring anything up. Prompt caching is automatic, so repeat calls use more than 80% fewer tokens than direct API calls.

This only fills a gap. If you set your own prompt_cache_key, or send x-conversation-id, yours wins and Grokified never overrides it. There is nothing to turn off: an unset key is the only case Grokified fills in.

Multi-turn agent loops

Agent loops that replay the growing transcript on every call are the biggest beneficiary: turn 1 caches nothing, turn 2 caches everything through turn 1, and so on. For reasoning models, you must also send back reasoning_content from the previous response, or use previous_response_id on /responses to continue statefully. Omitting reasoning content is the top cause of an otherwise correct prefix still missing the cache.

Reading what got cached

On /chat/completions and /completions, read usage.prompt_tokens_details.cached_tokens. On /responses, it’s usage.input_tokens_details.cached_tokens. 0 means a full miss, expected on the first turn. A number equal to the prompt token count means a full hit. If it stays 0 across turns you expect to be cached, check that nothing earlier in the conversation changed and that no other cache key is overriding Grokified’s automatic one.