CORE LORE / WIKI
SYS TOKEN SENTINEL
Updated 3 weeks ago
╔══════════════════════════════════════════════════════════════════════════════╗
║ ║
║ T O K E N S E N T I N E L ║
║ ℓ C L A U D E + G E M I N I S P E N D T R A C K E R ║
║ ║
╠══════════════════════════════════════════════════════════════════════════════╣
║ STATUS: LIVE VERIFIED: 2026-03-13 (S186) ║
╚══════════════════════════════════════════════════════════════════════════════╝
⫷✦🜛❂⛬🜞Ω🜚⛬❂🜛✦⫸───────────────────────────────────────────⫷✦🜛❂⛬🜞Ω🜚⛬❂🜛✦⫸
WHAT IT DOES: Tracks Claude and Gemini token usage and estimated cost across
all sessions. Two miners run every 5 minutes via launchd,
feeding `sentinel.db`. `token-hud.sh` renders a live spend
dashboard in the Aeris cockpit.
────────────────────────────────────────────────────────────────────────────────
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
SYSTEM GLYPH
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
GLYPH: ℓ
UNICODE: U+2113 · SCRIPT SMALL L
MEANING: The bookkeeper's mark — token ledger, renamed LEDGER from
SENTINEL (SENTINEL reserved for future cybersecurity system)
WHEN TO USE: SYS docs, cockpit token HUD headers, KID tags for
sentinel/ledger artifacts
TAGGING: KID:FORGE:LEDGER:[artifact]|V:STATUS:DATE:OWNER
> **NOTE (Session 123):** This system is canonically named **LEDGER** (glyph: ℓ). The filename `SYS_TOKEN_SENTINEL.md` is a legacy name pending rename. SENTINEL is reserved for a future security system.
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
[ ⚡ ] A R C H I T E C T U R E
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
```
claude-jsonl-miner.sh ──┐
(reads ~/.claude/projects/**/*.jsonl) │
├──► sentinel.db (token_events) ──► token-hud.sh
gemini-session-miner.sh ─┘
(reads ~/.gemini/tmp/*/chats/*.json)
```
Both miners run every 300 seconds via launchd. Each miner tracks incremental state so it only processes new data on each run. `sentinel-write.sh` is the shared atomic SQLite writer used by the Claude miner; the Gemini miner writes directly via Python sqlite3.
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
[ ❖ ] K E Y P A T H S
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
| Component | Path |
|-----------|------|
| Root directory | `~/.forge-sentinel/` |
| Database | `~/.forge-sentinel/sentinel.db` |
| Claude miner | `~/.forge-sentinel/claude-jsonl-miner.sh` |
| Gemini miner | `~/.forge-sentinel/gemini-session-miner.sh` |
| Atomic writer | `~/.forge-sentinel/sentinel-write.sh` |
| HUD renderer | `~/.forge-sentinel/token-hud.sh` |
| Token pulse script | `~/.forge-sentinel/token-pulse.sh` |
| Token pulse log | `~/.forge-sentinel/token-pulse.log` |
| Token pulse doc | `~/.forge-sentinel/TOKEN_PULSE.md` |
| Claude miner state | `~/.forge-sentinel/miner-state.json` |
| Gemini miner state | `~/.forge-sentinel/gemini-miner-state.json` |
| Claude source files | `~/.claude/projects/**/*.jsonl` |
| Gemini source files | `~/.gemini/tmp/*/chats/*.json` |
| Claude miner log | `/tmp/sentinel-claude-miner.{out,err}` |
| Gemini miner log | `/tmp/sentinel-gemini-miner.{out,err}` |
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
[ ⚙ ] H O W E A C H M I N E R W O R K S
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
┌── Claude Miner (`claude-jsonl-miner.sh`) ───────────────────────────────────┐
│ - Scans all `.jsonl` files under `~/.claude/projects/` │
│ - State file maps each file path → last line number processed; reads only │
│ new lines via `tail -n +N` │
│ - Extracts: `message.usage` or `usage`, message UUID for dedup, model │
│ string, timestamp │
│ - Token fields: `input_tokens`, `output_tokens`, `cache_read_input_tokens`, │
│ `cache_creation_input_tokens` │
│ - Calls `sentinel-write.sh` per event; `bot` = `claude_code` │
└────────────────────────────────────────────────────────────────────────────┘
┌── Gemini Miner (`gemini-session-miner.sh`) ─────────────────────────────────┐
│ - Written in Python (embedded heredoc in bash) │
│ - State file maps each file path → `lastUpdated` timestamp; skips unchanged │
│ files │
│ - Processes messages where `type` is `gemini`/`model` and `tokens` block │
│ present │
│ - Token fields: `input`, `output`, `cached` (→ `cache_read_tok`); │
│ `cache_write` always 0 │
│ - `bot` = `aeris`; writes directly to `sentinel.db` via Python sqlite3 │
└────────────────────────────────────────────────────────────────────────────┘
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
[ 🜄 ] D A T A B A S E S C H E M A
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
**Table: `token_events`**
| Column | Type | Notes |
|--------|------|-------|
| `id` | INTEGER PK | Auto-increment |
| `timestamp` | TEXT | ISO8601 |
| `bot` | TEXT | `claude_code`, `aeris`, `delegate_flash`, `delegate_pro` |
| `model` | TEXT | Full model string |
| `input_tok` | INTEGER | |
| `output_tok` | INTEGER | |
| `cache_read_tok` | INTEGER | |
| `cache_write_tok` | INTEGER | Claude only; Gemini always 0 |
| `total_tok` | INTEGER | Generated: sum of all four |
| `cost_usd` | REAL | Computed at write time |
| `session_id` | TEXT | UUID or session file ID; used for dedup |
| `source` | TEXT | `jsonl`, `gemini_session_files`, `aeris_delegate_py`, `delegate_wrap` |
| `mission` | TEXT | Mission tag (optional) |
| `project` | TEXT | Project tag (optional) |
**Unique constraint:** `(session_id, input_tok, output_tok, cache_read_tok, cache_write_tok, model)` — silently ignores duplicates.
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
[ 🜂 ] P R I C I N G R A T E S ( s e n t i n e l - w r i t e . s h )
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
| Model pattern | Input ($/tok) | Output ($/tok) |
|--------------|--------------|---------------|
| `*sonnet*` | 0.000003 | 0.000015 |
| `*opus*` | 0.000005 | 0.000025 |
| `*haiku*` | 0.000001 | 0.000005 |
| `*flash*` | 0.0000001 | 0.0000004 |
| `*pro*` | 0.00000125 | 0.000005 |
⫷ Rates are duplicated between `sentinel-write.sh` and the Python block in `gemini-session-miner.sh`. Keep in sync manually. ⫸
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
[ 📶 ] L I V E S T A T S ( 2 0 2 6 - 0 3 - 1 3 )
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
| Metric | Value |
|--------|-------|
| Total events | ~92,000+ |
| Lifetime tokens | ~9.46B |
| Lifetime cost | ~$4,069 |
| Token rate | ~3,217/s |
> **NOTE (S186):** As of S186, Aeris Gemini sessions route through cloudcode-pa PROVISIONED_THROUGHPUT (gemini-oauth-proxy on port 4891). These sessions record $0 cost_usd in sentinel.db. Only Claude Code Sonnet sessions and any direct API callers appear with real costs. Lifetime cost figure above reflects pre-proxy history; current daily Aeris cost shows near-zero.
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
[ ⛬ ] L A U N C H D D A E M O N S
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
| Label | Trigger | Script | Notes |
|-------|---------|--------|-------|
| `com.forge.sentinel.claude-miner` | Every 300s | `claude-jsonl-miner.sh` | No RunAtLoad |
| `com.forge.sentinel.gemini-miner` | Every 300s | `gemini-session-miner.sh` | No RunAtLoad |
| `com.forge.sentinel.token-pulse` | Every 600s | `token-pulse.sh` | RunAtLoad=true |
All in `~/Library/LaunchAgents/`. Logs → `/tmp/sentinel-*.{out,err}` (miners); `~/.forge-sentinel/token-pulse.log` (pulse).
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
[ 🜁 ] T O K E N P U L S E ( S 1 8 6 )
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
Token Pulse exports a live snapshot from `sentinel.db` to `kingdom_state.json` so the Tower site can display real-time spend without querying SQLite directly.
**Script:** `~/.forge-sentinel/token-pulse.sh`
**LaunchAgent:** `com.forge.sentinel.token-pulse` — 600s interval, `RunAtLoad=true`
**Writes to:** `~/Desktop/THE_TOWER/SCRYER_FEEDS/kingdom_state.json`
**Log:** `~/.forge-sentinel/token-pulse.log`
**Block written (`token_pulse`):**
```json
{
"token_pulse": {
"snapshot_at": "ISO8601",
"lifetime_tokens": 9460000000,
"lifetime_cost_usd": 4069.00,
"today_tokens": 0,
"today_cost_usd": 0.00,
"rate_per_sec": 3217,
"broadcast_label": "...",
"broadcast_cost": "..."
}
}
```
**Interval rationale:** 600s matches the Overmind Pulse cadence. The Tower site interpolates live between snapshots — no need for sub-minute polling.
**Gemini OAuth note:** Post-S186, Aeris Gemini sessions route through gemini-oauth-proxy (port 4891, PROVISIONED_THROUGHPUT). These appear as $0 cost_usd in sentinel.db. `today_cost_usd` will show near-zero during OAuth-routed sessions; `lifetime_cost_usd` reflects true historical spend from before the proxy.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
[ 🜂 ] C O M M O N C O M M A N D S
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
```bash
# View live HUD
~/.forge-sentinel/token-hud.sh
# Today's spend
sqlite3 ~/.forge-sentinel/sentinel.db \
"SELECT SUM(cost_usd) FROM token_events WHERE date(timestamp)=date('now');"
# Spend by model
sqlite3 ~/.forge-sentinel/sentinel.db \
"SELECT model, COUNT(*), SUM(total_tok), SUM(cost_usd) FROM token_events
GROUP BY model ORDER BY SUM(cost_usd) DESC;"
# Spend by day (last 7 days)
sqlite3 ~/.forge-sentinel/sentinel.db \
"SELECT date(timestamp), SUM(cost_usd) FROM token_events
WHERE timestamp >= date('now','-7 days')
GROUP BY date(timestamp) ORDER BY 1 DESC;"
# Run miners manually
bash ~/.forge-sentinel/claude-jsonl-miner.sh
bash ~/.forge-sentinel/gemini-session-miner.sh
# Check errors
cat /tmp/sentinel-claude-miner.err
cat /tmp/sentinel-gemini-miner.err
# Reload launchd
launchctl unload ~/Library/LaunchAgents/com.forge.sentinel.claude-miner.plist
launchctl load ~/Library/LaunchAgents/com.forge.sentinel.claude-miner.plist
```
════════════════════════════════════════════════════════════════════════════════
[ 🝓 ] G O T C H A S
════════════════════════════════════════════════════════════════════════════════
- **Gemini `cache_write` always 0.** Gemini CLI does not expose cache write tokens.
- **Claude miner uses line-count state, not timestamps.** If a JSONL file is truncated/rewritten, state becomes stale. DB unique constraint is secondary dedup layer.
- **Gemini miner state keyed on `lastUpdated`.** Old completed sessions with unchanged `lastUpdated` are permanently skipped.
- **Pricing rates duplicated.** `sentinel-write.sh` (Claude) and Python block in `gemini-session-miner.sh` (Gemini) must be kept in sync manually.
- **No `RunAtLoad`.** First run is 5 minutes after launchctl load.
- **DB WAL mode.** `.db-shm` and `.db-wal` are normal — do not delete while miners are running.
- **`<synthetic>` bot entries.** Resolved — 0 synthetic entries remain. Early miner artifact, cleaned up.
- **AExGO floating pane.** `token-hud.sh` runs via `viddy -d -i5` in a Zellij floating pane. Trigger: `Alt+T` in AExGO cockpit.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
🜚 SYS_TOKEN_SENTINEL // THE FORGE // ⛬⚚⛬ THE LAW STANDS.
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀