Token-Cost Observability — What's Possible per Tool

Claude Code · Codex · Cursor — the sourcing feasibility layer under the Tokenscope blueprint.
Every claim below was verified against real files on this machine on 2026-07-11, not from docs.

TL;DR verdict

One-line takeaway: Claude Code and Codex are self-instrumenting — they write cache-aware, per-turn cost data to local disk with zero infra. Cursor is opaque by design; its local artifacts cannot yield cost, so Cursor requires the Admin API (post-hoc) or a BYO gateway (pre-send). Build the pipeline around that asymmetry.

The framing: two questions, three vantage points

Cost work splits into two questions that need different sources:

1. Post-hoc truth — "what did this turn/session/user actually cost?" Needs the provider usage object (with the cache split).
2. Pre-send estimate — "before we dispatch, what will this cost?" Needs the assembled prompt tokenized + a cache-hit prediction.

Three vantage points can answer them, in descending fidelity:

• Provider boundary (gateway / local usage logs) — exact request + exact usage. Highest fidelity. Claude Code & Codex hand you the usage side for free on disk; Cursor requires a BYO gateway.
• Server billing APIs (Anthropic Admin, OpenAI usage, Cursor Admin) — server truth, cache-aware, org-wide, but post-hoc and coarser than per-turn.
• Local editor state — varies wildly by tool (see below). Great for Claude/Codex, useless-for-cost for Cursor.

Cost is not token count — the formula

The number to compute is cost, and cache dominates it:

cost = fresh_input      × base_in
     + cache_write_5m   × 1.25 × base_in
     + cache_write_1h   × 2.00 × base_in
     + cache_read       × 0.10 × base_in
     + output           × base_out

Estimating from raw input_tokens alone over-/under-counts a mature agent turn by 5–10×, because most input on a long turn is cache_read at 0.1×. Any source that lacks the cache split (i.e. Cursor local) cannot produce a real cost number.

Current pricing (per 1M tokens) & cache multipliers

Cache multipliers (relative to base input price): cache read = 0.10×; cache write, 5-min TTL = 1.25×; cache write, 1-hour TTL = 2.00×. (Break-even: 5-min cache pays off after 2 reuses; 1-hour after 3.)

For OpenAI/Codex, map model→price from the OpenAI pricing table; the same formula shape applies (OpenAI exposes cached_tokens for the read discount).

Claude Code — cost-complete on local disk ✅

Path: ~/.claude/projects/<mangled-cwd>/<sessionId>.jsonl (one JSONL event per line).

Verified envelope (top-level keys on an assistant event):
cwd, gitBranch, sessionId, requestId, uuid, parentUuid, timestamp, type, userType, isSidechain, version, message.

The gold — message.usage (verified, real values):

{
  "input_tokens": 8021,
  "cache_creation_input_tokens": 3616,
  "cache_read_input_tokens": 15853,
  "output_tokens": 3686,
  "cache_creation": { "ephemeral_1h_input_tokens": 3616, "ephemeral_5m_input_tokens": 0 },
  "service_tier": "standard"
}

Plus message.model (e.g. claude-opus-4-8). Note the ephemeral_1h vs 5m split is present — you can price cache writes at the correct 2× vs 1.25× multiplier, not a blended guess.

What you get, per turn, with zero infra: user attribution (cwd + userType), repo/branch (cwd + gitBranch), model, exact cache-split tokens, timestamp, sub-agent flag (isSidechain), and requestId to join to the Anthropic Admin API if you want to reconcile.

Extraction recipe:

find ~/.claude/projects -name '*.jsonl' | while read f; do
  python3 -c "
import json,sys
for l in open('$f'):
    d=json.loads(l); m=d.get('message',{})
    u=m.get('usage')
    if u: print(d['timestamp'], m.get('model'), u['input_tokens'],
                u['cache_creation_input_tokens'], u['cache_read_input_tokens'],
                u['output_tokens'], d.get('gitBranch'))
  "
done

Org rollup / reconciliation: the Anthropic Admin API (usage & cost reporting) gives per-workspace/API-key token + cost with the cache split server-side — use it to validate the local sum and to cover machines you don't scrape.

Codex — cost-complete on local disk ✅

Path: ~/.codex/sessions/YYYY/MM/DD/rollout-<ts>-<uuid>.jsonl. Also: ~/.codex/session_index.jsonl (index only) and a 47 MB ~/.codex/logs_2.sqlite (richer local store worth mining).

Verified event mix in one 859-line session:
session_meta, turn_context, event_msg/user_message, event_msg/agent_message, response_item/reasoning (84), response_item/function_call (200) + function_call_output (200), event_msg/mcp_tool_call_end (11), event_msg/web_search_end, event_msg/token_count (107), event_msg/turn_aborted (2), compacted/context_compacted (1).

The gold — event_msg/token_count events carry per-turn usage including reasoning tokens and the model context window (model_context_window). Codex therefore gives you everything Claude Code does plus native reasoning-token accounting and explicit compaction/abort events (rework signals).

Extraction recipe (sketch):

CF=$(find ~/.codex/sessions -name 'rollout-*.jsonl' | tail -1)
python3 -c "
import json
for l in open('$CF'):
    d=json.loads(l)
    if d.get('type')=='event_msg' and d.get('payload',{}).get('type')=='token_count':
        print(d['timestamp'], d['payload'])  # inspect for input/output/reasoning/cache + context window
"

Then map model→OpenAI price and sum. Reconcile org-wide with the OpenAI usage/costs Admin API (and note: ChatGPT-plan Codex bills against the plan, API-key Codex against the API dashboard — attribution differs by auth mode).

Also mine logs_2.sqlite — 47 MB of structured local logs likely holds richer per-request rows than the rollout JSONL; worth a schema dump before finalizing the collector.

Cursor — cost-blind locally; server or gateway only ❌

Three local artifacts, none cost-complete (all verified):

1. ~/Library/Application Support/Cursor/User/globalStorage/state.vscdb (SQLite) → table cursorDiskKV:
   • bubbleId:* messages carry tokenCount.{inputTokens,outputTokens} and a surface flag (isChat/isAgentic) — but no cache split, no cost, no model on the bubble, no per-bubble timestamp. Model lives on composerData:* / the ai-tracking DB.
   • Empty usageData; zero billing/cache/cost keys anywhere in the DB (confirmed).
2. ~/.cursor/projects/**/agent-transcripts/**/*.jsonl — content + tool trajectory only. Envelope is just {role, message:{content:[...]}}; no usage/model/timestamp, and assistant reasoning appears [REDACTED] natively in newer agentic transcripts (older Ask-mode transcripts kept content). So even content-based output-token estimation is off the table on current Cursor.
3. ~/.cursor/ai-tracking/ai-code-tracking.db — tracks AI-authored code % (scored_commits, conversation_summaries.model, tracked_file_content). Useful for "how much code is AI-written," not for token cost.

Non-Composer surfaces are the real trap. Inline Cmd+K, Apply, and Tab are largely not persisted locally at all (the old aiService.generations/prompts keys are gone), and Tab/Apply run on Cursor-hosted fast models you cannot BYO — so even a gateway never sees Tab. The only source that captures every surface is the server.

The two Cursor paths that actually work

• Admin API usage-events (Teams/Business + admin key) — post-hoc, cache-aware, per-request, org-wide, tagged with a kind/feature dimension (chat, cmd-k, composer, apply, tab, bugbot) + model + tokens + cost. This is the only complete cost source for Cursor, including the otherwise-invisible Tab/Apply share. → This is the Cursor system-of-record.
• BYO gateway (local pass-through proxy at the provider boundary via Cursor's "Override OpenAI Base URL" + your key) — the only way to get pre-send, cache-exact on Cursor. Covers Ask/Cmd+K/Composer on frontier models; blind to Tab/Apply. Use as a supplement, not a replacement.

Version note: this snapshot is Cursor 3.2.21 (last active 2026-05-06). The SQLite-in-state.vscdb architecture is stable across versions; re-validate the JSONL redaction/keys against a current install before hard-coding a scraper.

Consolidated capability matrix

Pre-send estimation — how, and where it's feasible

A stateful gateway (or local estimator) can produce a pre-send number where the assembled prompt is visible:

1. Tokenize the assembled payload — Anthropic /v1/messages/count_tokens (exact, counts tools+system+images) for Claude; tiktoken/tokenizer for OpenAI/Codex.
2. Predict the cache read — diff the current request's prefix against the previous request in the same conversation up to the last cache_control breakpoint; longest stable common prefix ≈ cache_read, the remainder is a fresh write. Cache reads are deterministic from the prefix, so this is accurate before the response.
3. Price with the formula above.
4. Reconcile against the real usage on the response to tighten the prefix-diff heuristic.

Feasible for Claude Code and Codex (their local logs + a count-tokens call are enough to model the next turn). For Cursor, true pre-send requires a BYO gateway — default-mode Cursor assembles the prompt server-side, so the cache-annotated request never exists locally.

Recommended architecture (three lanes, no source pretending to be another)

COLLECT ─────────────► NORMALIZE ────► STORE (DuckDB/Parquet) ────► SURFACE
Claude  ~/.claude/**.jsonl            one event schema:            price table → $      CLI / dashboard
Codex   ~/.codex/**/rollout-*.jsonl   ts·user·repo·model·          waste-rule tags      VS Code meter
Cursor  Admin API usage-events        tokens{in,out,cr,cw}·        cache metrics        Slack digest
        (+ JSONL trajectory,          tool·kind·parent
         SQLite context-growth)

Per-tool lane assignment:

• Claude Code / Codex → local JSONL is the cost system-of-record. Parse usage/token_count, join model→price, sum. Zero infra. Use the provider Admin APIs only for cross-machine reconciliation.
• Cursor → Admin API usage-events is the cost system-of-record (only complete, cache-aware, per-surface source). Treat local SQLite as context-growth enrichment and JSONL as trajectory/behavioral enrichment — never as cost.
• Pre-send / cache-economics → a BYO gateway where you can route it (Claude Code via ANTHROPIC_BASE_URL, Codex via OPENAI_BASE_URL, Cursor only in BYO-key mode). This is the one lane that adds pre-send on top of the post-hoc truth.

The one number to always surface: the Tab/Apply share for Cursor — it's invisible to every non-server approach, so if you skip the Admin API you won't even know how much you're missing.

Open decisions before building the collectors

• Passive vs inline gateway — inline enables pre-send blocking/warnings but adds a latency + reliability dependency on every call.
• Cursor Admin API access — confirm Teams/Business admin key + the exact usage-events endpoint fields (kind, model, cache-token keys, pagination) before committing the Cursor lane.
• Codex logs_2.sqlite — dump its schema; it may replace the rollout-JSONL parser with something richer.