OpenWebUI Context Compaction Pipeline

A 3-level context compaction system for OpenWebUI, inspired by Claude Code's compaction architecture. Deployed as a Filter Pipeline on the official open-webui/pipelines framework.

Note

What is context compaction? When conversations grow too long for the LLM's context window, this pipeline automatically summarizes older messages into a structured summary — preserving key information (intent, decisions, files, errors) while freeing tokens. The user can continue the conversation seamlessly.

---

Features

Feature	Description
Micro-compaction	Token-budget quota for tool outputs. Oldest get compacted first. Base64 images cached too.
Auto-compaction	Triggers when token usage exceeds a configurable threshold (e.g. 80% of context window)
Manual compaction	/compact command with optional focus instructions (e.g. /compact focus on API changes)
Context stats	/context command displays token usage, breakdown by role, and compaction history
Structured summaries	Summaries follow a fixed format: user intent, technical decisions, files, errors, current state, next steps
Tree-merge	Large conversations are chunked → summarized independently → merged recursively
Post-compact rehydration	Injects recent file references and a continuation message into the system prompt
Anti-recursion	BYPASS_MARKER prevents infinite loops when the pipeline calls the LLM for summarization
Per-chat locking	asyncio locks prevent race conditions when multiple messages arrive simultaneously
Configurable tokenizer	Supports per-model tokenizer mapping (e.g. cl100k_base for GPT-4, o200k_base for newer models)

---

How It Works — The 3 Compaction Levels

flowchart TD
    A["📨 User message arrives"] --> B{"Command?<br>/context · /compact · BYPASS"}

    B -- "/context" --> C["📊 Return stats<br><i>raise Exception — no LLM call</i>"]
    B -- "/compact" --> D["🔧 Force compaction + merge<br><i>raise Exception — no LLM call</i>"]
    B -- "BYPASS_MARKER" --> E["⏭️ Strip marker, skip<br><i>internal summarization call</i>"]
    B -- "normal message" --> F

    F["🔢 Count tokens on<br>ORIGINAL messages"] --> I{"≥ threshold?<br>(e.g. 80% of context)"}

    subgraph L2["<b>1. Auto-compaction</b> — if ≥ threshold"]
        I -- "YES ≥ 80%" --> J["✂️ Chunk → Summarize each<br>→ Tree-merge summaries"]
        J --> K["💾 Store in SQLite<br>Inject summary into system message"]
        K --> MC1["📦 Micro-compact result<br>(tool outputs + images)"]
    end

    I -- "NO < 80%" --> MC2["📦 Micro-compact<br>(tool outputs + images)"]
    MC2 --> L["📄 Inject existing summary<br>from previous run (if any)"]

    MC1 --> M["🚀 Modified messages → LLM"]
    L --> M
    E --> M

Example: A conversation reaches 105,000 tokens on a 128K-context model (82% > 80% threshold):

1. Micro-compaction caches old tool outputs, freeing ~8K tokens
2. Auto-compaction chunks messages 2-90 → summarizes each chunk → tree-merges → injects a structured summary into the system message
3. Result: 105,000 → ~12,000 tokens — conversation continues with full context awareness

---

Quick Start

Option 1: Docker Compose (recommended)

# Clone and start
git clone https://github.com/your-user/openwebui-compaction-pipeline.git
cd openwebui-compaction-pipeline
docker compose up -d --build

# View logs
docker compose logs -f

The Dockerfile uses a multi-stage build:

1. Stage 1: Builds the single-file pipeline from source modules (python build.py)
2. Stage 2: Clones the official open-webui/pipelines framework and drops the pipeline into its PIPELINES_DIR

Then in OpenWebUI, configure the pipeline connection:

• Settings → Admin Settings → Connections → OpenAI API
• URL: http://compaction-pipeline:9099 (Docker network) or http://localhost:9099 (host)
• API Key: 0p3n-w3bu! (default, or whatever you set via PIPELINES_API_KEY)

Option 2: Manual installation on an existing Pipelines server

# Build the single-file pipeline
python build.py

# Copy it to your pipelines server's PIPELINES_DIR
cp dist/context_compactor_pipeline.py /path/to/pipelines/dir/

# Reload pipelines via API
curl -X POST http://your-pipelines-server:9099/pipelines/reload \
  -H "Authorization: Bearer YOUR_API_KEY"

Option 3: PIPELINES_URLS (download at startup)

Host the generated file on a URL accessible from the pipelines server, then:

docker run -d \
  -p 9099:9099 \
  -e PIPELINES_URLS="https://your-host/context_compactor_pipeline.py" \
  ghcr.io/open-webui/pipelines:main

---

Configuration

All parameters are configurable from the OpenWebUI Admin UI (Valves panel on the pipeline):

Compaction Thresholds

Valve	Default	Description
compression_threshold_pct	0.80	Auto-compaction triggers above this % of max context
max_context_tokens	128000	Maximum context window size for the target model

Micro-compaction

Valve	Default	Description
tool_output_budget_tokens	20000	Total token budget for all tool outputs. Oldest compacted first.
image_compact_enabled	true	Compact inline base64 images outside the hot tail
hot_tail_size	10	Number of recent messages protected from micro-compaction

Chunking & Summary

Valve	Default	Description
chunk_size_tokens	32000	Max tokens per summarization chunk
keep_first_messages	1	Leading messages preserved (typically the system prompt)
keep_last_messages	6	Trailing messages preserved (hot tail for context continuity)
file_references_count	5	Number of recent file references injected post-compaction

Summary Model

Valve	Default	Description
summary_model	""	Model for summarization (empty = same as conversation model)
openwebui_url	http://localhost:8080	OpenWebUI base URL for summary API calls
openwebui_api_key	""	API key for OpenWebUI (Settings → Account → API Keys)

Advanced

Valve	Default	Description
tokenizer_mapping	*:cl100k_base	Tokenizer encoding per model pattern (comma-separated, e.g. gpt-4o:o200k_base,*:cl100k_base)
debug_logging	false	Log detailed compaction decisions

Framework valves — pipelines: ["*"] (which models to filter) and priority: 0 (execution order) are standard filter pipeline valves from the framework. They appear in the UI but are not specific to this pipeline.

---

Commands

Command	Description
/context	Display token usage stats, breakdown by role, compaction history, and auto-compaction threshold
/compact	Force immediate compaction of the conversation
/compact focus on X	Force compaction with focus instructions for the summary

Example /context output:

Context Stats — "Help me build a REST API with Fast..."

Messages: 47
Tokens:   94,230 / 128,000 (73.6%)
          [██████████████████████░░░░░░░░] 73.6%

Breakdown by role:
  System:      1 msgs  (~1,240 tokens)
  User:       15 msgs  (~12,430 tokens)
  Assistant:  14 msgs  (~18,920 tokens)
  Tool:       17 pairs (~61,640 tokens)

Compaction history:
  Compactions done:    0
  Total tokens saved:  0
  Tool outputs cached: 0

Auto-compaction triggers at 80% (~102,400 tokens)

---

Project Structure

openwebui-compaction-pipeline/
├── build.py                       # Build script → generates dist/
├── Dockerfile                     # Multi-stage: build + official framework
├── docker-compose.yml
├── requirements.txt               # tiktoken, aiohttp, pydantic
├── LICENSE                        # MIT
├── compactor/                     # Source modules (editable, tested individually)
│   ├── __init__.py
│   ├── prompts.py                 # Templates + BYPASS_MARKER
│   ├── token_estimator.py         # tiktoken multi-encoding wrapper
│   ├── db.py                      # SQLite CRUD (summaries, cache, logs)
│   ├── micro_compact.py           # Tool output quota + image compaction
│   ├── chunker.py                 # Chunking with tool_call pair preservation
│   ├── rehydration.py             # Summary + file refs injection
│   ├── summarizer.py              # LLM calls + tree merge
│   └── commands.py                # /context and /compact handlers
├── pipelines/
│   └── main.py                    # Self-contained Pipeline class + Valves
├── dist/                          # (generated, gitignored)
│   └── context_compactor_pipeline.py  # Single deployable file
└── tests/                         # 62 tests (32 unit + 30 integration)
    ├── test_chunker.py            # 5 tests
    ├── test_micro_compact.py      # 9 tests
    ├── test_pipeline_integration.py  # 30 integration tests (full Pipeline flow)
    ├── test_rehydration.py        # 11 tests
    └── test_token_estimator.py    # 7 tests

Development

# Install dependencies
pip install -r requirements.txt pytest pytest-asyncio

# Run all tests (62 tests)
pytest tests/ -v

# Run only unit tests (compactor modules)
pytest tests/ -v --ignore=tests/test_pipeline_integration.py

# Run only integration tests (full Pipeline flow)
pytest tests/test_pipeline_integration.py -v

# Build the single-file pipeline
python build.py

# Verify it's valid Python
python -c "import py_compile; py_compile.compile('dist/context_compactor_pipeline.py', doraise=True)"

# Check the Pipeline class loads correctly
python -c "
import importlib.util
spec = importlib.util.spec_from_file_location('p', 'dist/context_compactor_pipeline.py')
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
p = mod.Pipeline()
print(f'type={p.type} name={p.name} valves={list(p.valves.model_dump().keys())}')
"

---

Architecture

flowchart LR
    subgraph Server["Pipeline Server — open-webui/pipelines"]
        direction TB
        subgraph Filter["Context Compactor Filter"]
            direction TB
            IN["inlet(body, user)"]
            OUT["outlet(body, user)"]
            DB[("SQLite<br>/app/data/compaction.db")]
        end
    end

    OUI["OpenWebUI"] -- "user message<br>+ chat history" --> IN
    IN -- "modified body" --> LLM["LLM Provider"]
    LLM -- "response" --> OUT
    OUT -- "logged response" --> OUI
    IN <--> DB

    subgraph InletFlow["inlet() processing pipeline"]
        direction TB
        S1["1. Acquire lock(chat_id)"] --> S2["2. Detect /compact, /context, BYPASS"]
        S2 --> S3["3. Count tokens on originals (tiktoken)"]
        S3 --> S4{"4. ≥ threshold?"}
        S4 -- "YES" --> S5["5. Chunk → Summarize → Tree-merge"]
        S5 --> S6["6. Micro-compact result + inject summary"]
        S4 -- "NO" --> S7["5. Micro-compact tool outputs + images"]
        S7 --> S8["6. Inject existing summary (if any)"]
    end

This is a Filter Pipeline — a middleware that intercepts requests before they reach the LLM and responses after. The official pipelines framework discovers and loads it automatically via PIPELINES_DIR.

---

Design Notes

Why a Filter Pipeline (not a Filter Function)?

OpenWebUI has two plugin systems: Functions (built into OpenWebUI backend) and Pipelines (external server). We chose Pipelines because:

• Isolation — Compaction logic runs in a separate process/container, no risk of crashing OpenWebUI
• Scalability — The pipeline server can be scaled independently
• Separation of concerns — Compaction is a cross-cutting concern that applies to any model, not a per-model function

Anti-Recursion Design

The summarizer calls {openwebui_url}/api/chat/completions to generate summaries. This request goes through OpenWebUI's normal pipeline — which would trigger the compactor again, creating an infinite loop. The solution:

1. Every summarization prompt contains <!--COMPACT_INTERNAL:SKIP--> (BYPASS_MARKER)
2. The inlet detects the marker in the last message and short-circuits (strips the marker and returns body unchanged)
3. The summarization request reaches the LLM without further compaction

Build System

The project uses a build pipeline to produce a single deployable file:

• compactor/*.py — Source modules, editable and tested individually
• pipelines/main.py — Self-contained Pipeline class with all code inlined (the source of truth)
• build.py — Generates dist/context_compactor_pipeline.py by concatenating modules in dependency order
• dist/ — Output directory with the deployable single-file pipeline (gitignored)

The generated file includes a frontmatter block (title, requirements) that the pipelines framework uses for auto-installation of dependencies.

Known Limitations

• No progress feedback — Pipelines don't have __event_emitter__. Compaction is invisible to the user until it completes. For large conversations, the first response after compaction may take 10-30 seconds.
• raise Exception() for commands — /context and /compact return results by raising an exception in the inlet, which the framework converts to an HTTP response. OpenWebUI displays the exception message to the user. This is the official pattern for filter-based request blocking.
• Summarizer API key required — The openwebui_api_key valve must be set to a valid OpenWebUI API key (Settings → Account → API Keys) for the summarizer to work. Without it, auto-compaction will fail silently (falls back to micro-compaction only).

---

References

• OpenWebUI Pipelines Framework — official pipeline server
• Filter Pipeline Examples — reference implementations
• Filter Pipeline Docs (DeepWiki) — lifecycle, body structure, and valve configuration
• Filter Lifecycle & Execution (DeepWiki) — inlet/outlet execution flow
• Claude Code Compaction Deep Dive
• Async Context Compression (Fu-Jie)

License

MIT