Fix Ollama 2048 Context Truncation: num_ctx Modelfile & API Guide

Direct Answer: Ollama silently truncates any prompt exceeding 2,048 tokens down to 2,048 tokens without warning because num_ctx defaults to 2048. To fix it permanently, create a custom Modelfile with PARAMETER num_ctx 32768 and build it with ollama create my-model -f Modelfile. In Python or REST API calls, pass "options": {"num_ctx": 32768} in the JSON payload, or configure the Context Length slider under Open WebUI Model Settings.

On our developer workbench, our team was building an internal document audit pipeline using a local instance of Llama 3.3. We fed a 45-page system architecture specification (~18,000 tokens) into Open WebUI and asked a simple verification question: “What is the fallback timeout specified in Section 4.2?”

The model confidently replied: “The provided document does not mention any fallback timeout or Section 4.2.”

We knew Section 4.2 was in the file. We tested the exact same prompt against the cloud API, and it answered instantly. We checked GPU temperatures, model quantization, and prompt formatting. Everything looked normal.

Then we monitored the raw Ollama server logs:

# logs/ollama_server.log
[2026-09-09 08:14:22] [GIN] 200 | 14.21s | 127.0.0.1 | POST "/api/chat"
[2026-09-09 08:14:22] [LLAMACPP] llama_tokenize: input tokens: 17,840
[2026-09-09 08:14:23] [LLAMACPP] warning: prompt length exceeds context window (2048)
[2026-09-09 08:14:23] [LLAMACPP] sliding window: truncated 15,792 tokens from prompt head
[2026-09-09 08:14:23] [LLAMACPP] effective prompt context: 2,048 tokens

Ollama had silently thrown 15,792 tokens into the trash before inference even began. The model did not hallucinate because it was unintelligent; it hallucinated because it literally never received 88% of our text.

If you use Ollama for RAG, coding agents, or legal document review, you have almost certainly suffered from this invisible context trap. Below is why Ollama enforces this clamp, the memory physics behind it, and our verified 3-tier fix for Modelfiles, Open WebUI, and Python APIs.

The Architecture: Why Ollama Defaults to 2048 Tokens

Ollama hardcodes a 2,048 token limit to prevent massive KV-cache pre-allocations from crashing consumer GPUs with CUDA Out-of-Memory.

Modern frontier models advertise massive native context windows:

• Llama 3.1 & 3.3: 128,000 tokens
• Qwen 2.5: 128,000 tokens
• Mistral Large: 128,000 tokens
• DeepSeek-R1 Distill: 64,000–128,000 tokens

Because these models advertise 128k context, developers naturally assume running ollama run llama3.3 provides a 128k context window.

It does not. Ollama uses llama.cpp under the hood. Unlike cloud APIs that allocate server clusters dynamically, llama.cpp statically pre-allocates the entire Key-Value (KV) cache in GPU VRAM upon model initialization.

+-----------------------------------------------------------------------------------+
|               THE SILENT OLLAMA CONTEXT TRUNCATION PIPELINE                       |
+-----------------------------------------------------------------------------------+
| [User / RAG Pipeline]                                                             |
|   Ingests 16,000-Token Technical Document + User Question                         |
|         |                                                                         |
|         v                                                                         |
| [Ollama HTTP Daemon (:11434)]                                                     |
|   Probes Model Parameter: num_ctx (Default = 2,048 Tokens)                        |
|         |                                                                         |
|         +-----------------------+-------------------------+                       |
|                                 |                         |                       |
|    [First 13,952 Tokens]        |          [Last 2,048 Tokens]                    |
|    >>> SILENTLY DROPPED! <<<    |          Passed to llama.cpp Engine             |
|    (Zero Errors / Zero Logs)    |          (Only sees tail of document)           |
|                                 |                         |                       |
|                                 +------------+------------+                       |
|                                              |                                    |
|                                              v                                    |
|                                    [LLM Model Generates]                          |
|                                    "I cannot find Section 4.2"                    |
|                                    (Catastrophic Hallucination)                   |
+-----------------------------------------------------------------------------------+

The VRAM Math That Forced Ollama’s Hand

As derived in our benchmark guide on why 32k context crashes your local LLM, KV cache memory scales linearly with sequence length:

KV Cache VRAM = 2 × Layers × KV_Heads × Head_Dim × Context × Precision

For an 8B model (e.g., Llama 3.1 8B):

• At 2,048 context (FP16): ~0.26 GB VRAM
• At 8,192 context (FP16): ~1.05 GB VRAM
• At 32,768 context (FP16): ~4.20 GB VRAM
• At 131,072 context (FP16): ~16.78 GB VRAM

If the Ollama team defaulted num_ctx to the model’s native 128k, an 8B model would require 5 GB (weights) + 16.8 GB (cache) = 21.8 GB of VRAM. It would immediately crash on every consumer RTX 3060 12GB, RTX 4070 12GB, and standard 16GB laptop in the world.

To ensure models load out-of-the-box on low-spec hardware without crashing, Ollama opted for a conservative 2,048-token clamp. But by failing to display an explicit warning when text is truncated, it turned a safety feature into an invisible trap.

Tier 1: The Permanent CLI & Modelfile Fix

Creating a customized Modelfile is the cleanest and most permanent way to override num_ctx across your local system.

If you run models via the command line or want your local model to always start with a large context window, define a Modelfile.

Step 1: Create a Custom Modelfile

Create a text file named Modelfile on your machine:

# Modelfile for Llama 3.3 with 32k context
FROM llama3.3:latest

# Set the context window to 32,768 tokens (32k)
PARAMETER num_ctx 32768

# Set temperature and reasoning parameters
PARAMETER temperature 0.7
PARAMETER top_p 0.9

# System prompt
SYSTEM You are an expert systems engineer. You analyze technical documentation thoroughly without truncation.

Step 2: Build the New Model Profile

Run ollama create to compile your customized model:

# Build the model into your local Ollama library
ollama create llama3.3-32k -f ./Modelfile

# Verify the model parameters
ollama show llama3.3-32k --parameters

Step 3: Run and Test

Now launch your new model:

ollama run llama3.3-32k

This model will permanently reserve a 32,768 token context window every time it is summoned.

[!TIP] Before setting num_ctx 65536 or 131072, check our interactive VRAM calculator to ensure your GPU has sufficient memory headroom. If memory overflows, check our runbook on preventing Ollama GPU-to-CPU offload slowdowns.

Tier 2: The Open WebUI Frontend Fix

If you use Open WebUI, you must update both model-level settings and pipeline defaults to prevent UI-level context clipping.

Even if your underlying Ollama model supports 32k, Open WebUI can inadvertently force a 2,048-token limit if its internal connection defaults are not adjusted.

+-----------------------------------------------------------------------------------+
|               OPEN WEBUI 3-STEP CONTEXT EXPANSION FLOW                            |
+-----------------------------------------------------------------------------------+
| 1. Model Specific: Workspace > Models > [Select Model] > Advanced Parameters      |
|    Set: Context Length (num_ctx) = 32768                                          |
|                                                                                   |
| 2. Global Connection: Admin Panel > Settings > Connections > Ollama               |
|    Set: Default Context Length = 32768                                            |
|                                                                                   |
| 3. Chat Session: Active Chat > Controls (Top Right) > System Parameters           |
|    Verify: Context Length slider reflects 32k (Not locked to 2048)                |
+-----------------------------------------------------------------------------------+

Step-by-Step Instructions:

1. Open Open WebUI and click your profile icon in the bottom-left, then select Workspace.
2. Click Models and find the model you use (e.g., llama3.3:latest).
3. Click the pencil icon to Edit.
4. Scroll down and expand Advanced Parameters.
5. Locate the Context Length (num_ctx) field. By default, this is often empty (which falls back to 2,048) or explicitly set to 2048.
6. Enter 32768 (or your desired token count).
7. Scroll to the bottom and click Save & Update.

Now, when you upload 30-page PDFs into the chat interface, Open WebUI will forward the expanded num_ctx payload to the Ollama backend.

Tier 3: The Python REST API & LangChain Fix

In code pipelines, you must explicitly inject num_ctx into the options payload of your API request.

If you interact with Ollama via Python, curl, LangChain, or LlamaIndex, omitting the options block causes the Ollama REST engine to silently revert to 2,048 tokens, even if your prompt has 20,000 tokens.

Native Python ollama Library

import ollama

response = ollama.chat(
    model="llama3.3",
    messages=[
        {"role": "system", "content": "You are a code auditor."},
        {"role": "user", "content": large_document_prompt}
    ],
    options={
        "num_ctx": 32768,       # Explicitly forces 32k context
        "temperature": 0.2,
        "num_predict": 2048     # Maximum generation output tokens
    }
)

print(response["message"]["content"])

Raw HTTP / cURL Payload

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.3",
  "messages": [
    {
      "role": "user",
      "content": "Summarize this 10,000 token text: ..."
    }
  ],
  "stream": false,
  "options": {
    "num_ctx": 32768
  }
}'

LangChain Integration

from langchain_community.llms import Ollama

llm = Ollama(
    model="llama3.3",
    num_ctx=32768,             # Injected directly into LangChain constructor
    temperature=0.1
)

If you do not pass num_ctx=32768, LangChain will silently send requests without the option key, and Ollama will silently discard your retrieved RAG context.

Production Diagnostic Script: Test-OllamaContextLimit.py

Run this automated diagnostic script to test your Ollama instance with a needle-in-a-haystack verification at 1k, 2k, 4k, 8k, and 16k tokens.

Save the following script as Test-OllamaContextLimit.py and run it with python Test-OllamaContextLimit.py. It inserts a secret code word deep inside a synthetic document, queries Ollama at increasing context depths, and flags the exact token threshold where Ollama begins silently discarding data:

#!/usr/bin/env python3
"""
Test-OllamaContextLimit.py
Performs an automated Needle-in-a-Haystack test against a local Ollama daemon
to detect the exact token boundary where prompt truncation occurs.
"""

import sys
import json
import urllib.request
import time

OLLAMA_URL = "http://localhost:11434/api/generate"
MODEL_NAME = "llama3.3" # Change to your target model

def generate_haystack(num_words, secret_code):
    # Standard filler text (~1.3 tokens per word)
    filler = "The quick brown fox jumps over the lazy dog and explores the server infrastructure. "
    repeat_count = num_words // 13
    text_segment = filler * repeat_count
    
    # Place the needle right in the middle (50% depth)
    halfway = len(text_segment) // 2
    haystack = text_segment[:halfway] + f"\n\nCRITICAL_SECRET_KEY IS: [{secret_code}]\n\n" + text_segment[halfway:]
    return haystack

def query_ollama(prompt, override_num_ctx=None):
    payload = {
        "model": MODEL_NAME,
        "prompt": prompt,
        "stream": False
    }
    if override_num_ctx:
        payload["options"] = {"num_ctx": override_num_ctx}

    data = json.dumps(payload).encode("utf-8")
    req = urllib.request.Request(OLLAMA_URL, data=data, headers={"Content-Type": "application/json"})
    
    try:
        start = time.perf_counter()
        with urllib.request.urlopen(req, timeout=120) as response:
            result = json.loads(response.read().decode("utf-8"))
            elapsed = time.perf_counter() - start
            return result.get("response", ""), elapsed, result.get("prompt_eval_count", 0)
    except Exception as e:
        return f"ERROR: {e}", 0, 0

def run_test_suite():
    print("=" * 70)
    print("    OLLAMA SILENT CONTEXT TRUNCATION DETECTOR")
    print("=" * 70)
    print(f"[*] Testing Endpoint: {OLLAMA_URL}")
    print(f"[*] Target Model:     {MODEL_NAME}")

    test_stages = [
        ("Baseline 1,000 Tokens", 750, None),
        ("Boundary 2,000 Tokens (Default)", 1500, None),
        ("Critical 4,000 Tokens (Unset)", 3000, None),
        ("Critical 4,000 Tokens (Fixed: num_ctx 8192)", 3000, 8192),
        ("Deep Context 12,000 Tokens (Fixed: num_ctx 16384)", 9000, 16384)
    ]

    for label, word_count, ctx_override in test_stages:
        secret = f"TOKEN-ALPHA-{word_count}"
        print(f"\n[+] Running Stage: {label}...")
        prompt_body = generate_haystack(word_count, secret)
        full_prompt = f"{prompt_body}\n\nQuestion: What is the CRITICAL_SECRET_KEY listed above? Answer ONLY with the code."
        
        reply, duration, eval_tokens = query_ollama(full_prompt, ctx_override)
        
        found = secret in reply
        status = "PASSED (Retained)" if found else "FAILED (TRUNCATED / SILENTLY DISCARDED)"
        
        print(f"    - Evaluated Tokens: {eval_tokens}")
        print(f"    - Time Taken:       {duration:.2f}s")
        print(f"    - Needle Retrieved: {found}")
        print(f"    - Verdict:          {status}")
        if not found:
            print(f"    - Model Output:     {reply.strip()[:100]}...")

    print("\n" + "=" * 70)
    print("    TEST SUITE COMPLETE")
    print("=" * 70)

if __name__ == "__main__":
    run_test_suite()

Summary & Best Practices

By understanding that Ollama’s 2,048-token limit is an intentional VRAM guardrail rather than an unchangeable ceiling, you can scale your local RAG pipelines and coding agents to their full 32k, 64k, or 128k capacity without suffering from silent document truncation.