Skip to main content

Token Budget Management and Graceful Stop

UPDATED 2026-10-08 5 min read EDIT ON GITHUB ↗

This page deep-dives into Layer D — Budget defense — of the 4-layer tokenomics structure. It covers the graceful-abort mechanism that ensures the session stops without data loss when an agent reaches the context-window limit, preserving progress so the next session can continue.

Platform basics
Background on the platform layer is in Context Window. This page is the MoAI-ADK account of it.

Why Budget Defense

Anthropic SSE streams intermittently stall (stream_idle_partial) near the context-window ceiling. This is probabilistic but predictable above the model-specific threshold. When a stall occurs, an agent call may fail mid-stream, potentially losing progress.

Budget defense addresses this proactively. Before context usage reaches the threshold, the system performs a graceful abort, ensuring the session transitions to the next step without loss.

Per-Model Context Thresholds

The operational threshold is model-specific. Larger windows tolerate higher percentage utilization; smaller windows hit the operational ceiling later in percentage terms but with less absolute headroom.

Model classWindowHandoff thresholdAbsolute ceiling
Opus 5.5 (1M)1,000,000 tokens50%~500,000 tokens
GLM-5.3 (1M)1,000,000 tokens50%~500,000 tokens
Fable / Sonnet 5.5 (1M)1,000,000 tokens50%~500,000 tokens
Sonnet 4.5 and earlier (200K)200,000 tokens90%~180,000 tokens
Haiku 5.5 (1M)1,000,000 tokens50%~500,000 tokens
Haiku (200K)200,000 tokens90%~180,000 tokens

Read the Haiku 5.5 row (Claude Code v2.1.293+, Anthropic API) together with the rule above — when a session matches both a 1M row and the 200K-sessions row, the 200K row wins. On AWS Bedrock / GCP Agent Platform / Microsoft Foundry the haiku alias still resolves to Haiku 4.5 (200K), so an AWS-lineage Haiku session keeps the 200K / 90% thresholds.

GLM-5.3 (via moai glm) is a 1M-context model and is operated at the 50% threshold. Claude Code reports context_window_size based on the Claude slot (Opus=1M, Sonnet/Haiku=200K), so raw telemetry may show ~180K under GLM; MoAI corrects this to 1M. Trust the statusline CW% gauge.

Two-Stage Handoff Marker

The statusline appends a /clear hint to the context bar in two stages.

  • Soft marker — a /clear hint is displayed in the statusline context bar with a warning color. It appears at the band’s soft threshold as an advisory signal allowing the user to decide whether to run /clear.
  • Hard marker — a /clear hint is displayed in the statusline context bar with a strong warning color. It appears at the auto-compact-aware ceiling; the next action MUST be /clear.

The hard ceiling is set near the auto-compact threshold, so runtime auto-compact frequently preempts it and the hard marker rarely fires in practice. This is an intentional tradeoff of the auto-compact-aware formula.

Graceful Abort Procedure

The graceful-abort mechanism implemented by SPEC-TOKEN-BUDGET-STOP-001 works as follows.

  1. Detect — Tracker.IsAtHardLimit(agentName) returns true (cumulative usage ≥ hard_clear_threshold, default 0.90)
  2. Persist state — in-flight work state is persisted to progress.md
  3. Emit handoff — a paste-ready resume message is generated (6-block structure)
  4. Recommend turn end — the user is advised to /clear (HARD: auto-/clear is NEVER performed)
  5. Carry the evidence into the verdict — the lines of verification evidence that will be cited are written into the tracked verdict file .moai/reports/<card-id>/verdict.md

/clear is never executed automatically. The system only recommends that the user run /clear; the user decides.

Paste-Ready Resume 6-Block Structure

The session-handoff message follows the 6-block structure below. Each block is designed so the next session can continue the work with minimal information.

text
✂──── Copy from here ────✂

ultrathink. <SPEC-ID> <phase> entering.
applied lessons: <memory-file-1>, <memory-file-2>

Preconditions:
1) <verifiable precondition 1>
2) <verifiable precondition 2>

Run: <command or action>

After merge: <next action or SPEC>

✂──── Copy to here ────✂

Role of each block:

  • Block 1 — ultrathink. opener sets effort:xhigh, declares the entering phase and SPEC-ID
  • Block 2 — applied lessons: references memory files learned from prior sessions (max 4)
  • Block 3 — separator + the Preconditions: header
  • Block 4 — numbered verifiable precondition items (max 4, each ≤200 chars)
  • Block 5 — Run: single primary action (typically /moai <subcommand>)
  • Block 6 — After merge: next action or SPEC ID

Verify-Diet

The file-redirect contract that redirects long verification-command output to disk and leaves only a summary in context.

Rule: when a verification command’s verbatim output exceeds the bounded-tail ceiling (default 50 lines or 2KB, whichever is smaller), the output is redirected to a file and only the exit code + bounded tail are surfaced in context.

bash
go test ./... > /tmp/moai-verify/1-go-test.log 2>&1; echo "exit=$?"; tail -50 /tmp/moai-verify/1-go-test.log

This contract keeps verbatim evidence on disk while the context carries only exit code + bounded tail. It removes the double-burn (inline output + banner re-quote), not the evidence itself.

Evidence Export Obligation

Evidence written by the file-redirect contract to /tmp is periodically cleared by the OS (macOS reboot, Linux tmpfs remount, systemd-tmpfiles). When the cited path no longer resolves to a file, the evidence is unreachable at audit time.

Verification output is therefore captured under .moai/state/verify/<session>/ first. That directory is machine-local scratch: it outlives /tmp clearance, but it is gitignored — the same runtime-state family as context-usage/ and active-sessions.json — so it reaches no clone, no CI runner, and no other machine. Surviving /tmp and being reachable at audit time are not the same thing.

What the contract states is therefore the carry-the-deciding-evidence-into-the-verdict obligation: the lines that decided a claim are written into the tracked verdict file .moai/reports/<card-id>/verdict.md before the claim is made, and the citation names that single file, never a directory. That file is the only tracked name under a card directory, so a sibling artifact written beside it stays ignored and a citation naming it resolves nowhere off this machine. Only the lines that decided the verdict move — the exit code, the failure summary, the figure being quoted — because relocating the scratch directory wholesale merely moves the bulk trimmed from the context into the repository. Whatever stays behind is named under Residual-risk as a known loss, and material left in scratch is never offered as a verdict basis.

Next Steps