Skip to main content
ENES
Bring a workflow →

The session exceeds the context window

Symptom​

With curl or the SDK, the request fails immediately with HTTP 400 and this body:

{"error":{"type":"invalid_request_error","code":"provider_invalid_request","message":"This model's maximum context length is 131072 tokens. However, you requested 32000 output tokens and your prompt contains at least 133000 input tokens, for a total of at least 165000 tokens. Please reduce the length of the input prompt or the number of requested output tokens.","param":"input_tokens","request_id":"req_…"}}

In OpenCode there are two different behaviors, depending on when it happens:

  • If it happens on the first message, OpenCode prints that full text and exits.
  • If it happens mid-session, after the model has already read files, the session goes silent: it prints nothing, writes nothing to its log and keeps retrying silently until you close it. One such case went 39 minutes without saying anything.

The typical trigger for the second case is asking for something like "read all the files in this folder". The model requests several reads in the same turn and the context jumps all at once: seven 50 KB files add up to about 125,000 tokens.

Cause​

The model's window is shared between the question and the answer. The Catalog's context_window isn't an input cap: it's the total of the prompt plus the max_tokens you request. A 131,072 model with max_tokens: 32000 accepts at most 99,072 prompt tokens, and requesting max_tokens equal to the full window always fails, even with a four-token prompt.

When the context jump happens in a single turn, no setting prevents it: the turn has already gone past the window before the tool could summarize the history.

How to confirm it​

  1. Repeat the call with a small max_tokens and the same prompt. If it passes with max_tokens: 100 and fails with the original value, this is the cause and not a problem with the model:

    curl -s https://api.<tu-dominio>/infernode/v1/chat/completions \
      -H "Authorization: Bearer $INFERNODE_API_KEY" -H 'Content-Type: application/json' \
      -d '{"model":"Saptiva Cortex","messages":[{"role":"user","content":"OK"}],"max_tokens":100}'

  2. Look up the model's real window and compare it with what you're sending:

    curl -s https://api.<tu-dominio>/infernode/v1/models/Saptiva%20Cortex \
      -H "Authorization: Bearer $INFERNODE_API_KEY"

    The 400 message already does the math: the window, the output tokens you requested and the input tokens you sent.

  3. In OpenCode, if the session has gone more than a minute without printing anything after several reads, it's the second case. There's no error to look for in the log because it doesn't write one.

Solution​

  1. Lower max_tokens. Request 8,192 instead of the Catalog's max_output_tokens: the node aborts any request that runs past about 300 seconds, so requesting more doesn't help (Measured performance).
  2. In OpenCode, declare limit.input in addition to limit.context and limit.output, so it summarizes the history before it hits the window (Choose the limits).
  3. Close the silent session with Ctrl+C and start again. Whatever was already written to disk isn't lost.
  4. Change the instruction: ask for the files one at a time, or use a search (grep) instead of reading whole files. A turn that reads seven large files goes past any model's window.
  5. If the prompt is long by necessity, split the task into several calls and pass only the relevant fragment in each one.

Next step​

Last updated on