WIKI/SYS TOKEN SENTINEL

SYS TOKEN SENTINEL

Updated 3 weeks ago
╔══════════════════════════════════════════════════════════════════════════════╗
║                                                                              ║
║   T O K E N   S E N T I N E L                                                ║
║   ℓ C L A U D E   +   G E M I N I   S P E N D   T R A C K E R               ║
║                                                                              ║
╠══════════════════════════════════════════════════════════════════════════════╣
║  STATUS: LIVE                    VERIFIED: 2026-03-13 (S186)                 ║
╚══════════════════════════════════════════════════════════════════════════════╝
⫷✦🜛❂⛬🜞Ω🜚⛬❂🜛✦⫸───────────────────────────────────────────⫷✦🜛❂⛬🜞Ω🜚⛬❂🜛✦⫸
 WHAT IT DOES: Tracks Claude and Gemini token usage and estimated cost across
               all sessions. Two miners run every 5 minutes via launchd,
               feeding `sentinel.db`. `token-hud.sh` renders a live spend
               dashboard in the Aeris cockpit.
────────────────────────────────────────────────────────────────────────────────

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  SYSTEM GLYPH
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  GLYPH:        ℓ
  UNICODE:      U+2113 · SCRIPT SMALL L
  MEANING:      The bookkeeper's mark — token ledger, renamed LEDGER from
                SENTINEL (SENTINEL reserved for future cybersecurity system)
  WHEN TO USE:  SYS docs, cockpit token HUD headers, KID tags for
                sentinel/ledger artifacts
  TAGGING:      KID:FORGE:LEDGER:[artifact]|V:STATUS:DATE:OWNER

> **NOTE (Session 123):** This system is canonically named **LEDGER** (glyph: ℓ). The filename `SYS_TOKEN_SENTINEL.md` is a legacy name pending rename. SENTINEL is reserved for a future security system.


▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
[ ⚡ ] A R C H I T E C T U R E
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

```
claude-jsonl-miner.sh  ──┐
  (reads ~/.claude/projects/**/*.jsonl)  │
                          ├──► sentinel.db (token_events) ──► token-hud.sh
gemini-session-miner.sh ─┘
  (reads ~/.gemini/tmp/*/chats/*.json)
```

Both miners run every 300 seconds via launchd. Each miner tracks incremental state so it only processes new data on each run. `sentinel-write.sh` is the shared atomic SQLite writer used by the Claude miner; the Gemini miner writes directly via Python sqlite3.


░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
[ ❖ ] K E Y   P A T H S
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░

| Component | Path |
|-----------|------|
| Root directory | `~/.forge-sentinel/` |
| Database | `~/.forge-sentinel/sentinel.db` |
| Claude miner | `~/.forge-sentinel/claude-jsonl-miner.sh` |
| Gemini miner | `~/.forge-sentinel/gemini-session-miner.sh` |
| Atomic writer | `~/.forge-sentinel/sentinel-write.sh` |
| HUD renderer | `~/.forge-sentinel/token-hud.sh` |
| Token pulse script | `~/.forge-sentinel/token-pulse.sh` |
| Token pulse log | `~/.forge-sentinel/token-pulse.log` |
| Token pulse doc | `~/.forge-sentinel/TOKEN_PULSE.md` |
| Claude miner state | `~/.forge-sentinel/miner-state.json` |
| Gemini miner state | `~/.forge-sentinel/gemini-miner-state.json` |
| Claude source files | `~/.claude/projects/**/*.jsonl` |
| Gemini source files | `~/.gemini/tmp/*/chats/*.json` |
| Claude miner log | `/tmp/sentinel-claude-miner.{out,err}` |
| Gemini miner log | `/tmp/sentinel-gemini-miner.{out,err}` |


▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
[ ⚙ ] H O W   E A C H   M I N E R   W O R K S
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

┌── Claude Miner (`claude-jsonl-miner.sh`) ───────────────────────────────────┐
│ - Scans all `.jsonl` files under `~/.claude/projects/`                      │
│ - State file maps each file path → last line number processed; reads only   │
│   new lines via `tail -n +N`                                                │
│ - Extracts: `message.usage` or `usage`, message UUID for dedup, model       │
│   string, timestamp                                                          │
│ - Token fields: `input_tokens`, `output_tokens`, `cache_read_input_tokens`, │
│   `cache_creation_input_tokens`                                              │
│ - Calls `sentinel-write.sh` per event; `bot` = `claude_code`                │
└────────────────────────────────────────────────────────────────────────────┘

┌── Gemini Miner (`gemini-session-miner.sh`) ─────────────────────────────────┐
│ - Written in Python (embedded heredoc in bash)                               │
│ - State file maps each file path → `lastUpdated` timestamp; skips unchanged │
│   files                                                                      │
│ - Processes messages where `type` is `gemini`/`model` and `tokens` block    │
│   present                                                                    │
│ - Token fields: `input`, `output`, `cached` (→ `cache_read_tok`);           │
│   `cache_write` always 0                                                     │
│ - `bot` = `aeris`; writes directly to `sentinel.db` via Python sqlite3      │
└────────────────────────────────────────────────────────────────────────────┘


▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
[ 🜄 ] D A T A B A S E   S C H E M A
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄

**Table: `token_events`**

| Column | Type | Notes |
|--------|------|-------|
| `id` | INTEGER PK | Auto-increment |
| `timestamp` | TEXT | ISO8601 |
| `bot` | TEXT | `claude_code`, `aeris`, `delegate_flash`, `delegate_pro` |
| `model` | TEXT | Full model string |
| `input_tok` | INTEGER | |
| `output_tok` | INTEGER | |
| `cache_read_tok` | INTEGER | |
| `cache_write_tok` | INTEGER | Claude only; Gemini always 0 |
| `total_tok` | INTEGER | Generated: sum of all four |
| `cost_usd` | REAL | Computed at write time |
| `session_id` | TEXT | UUID or session file ID; used for dedup |
| `source` | TEXT | `jsonl`, `gemini_session_files`, `aeris_delegate_py`, `delegate_wrap` |
| `mission` | TEXT | Mission tag (optional) |
| `project` | TEXT | Project tag (optional) |

**Unique constraint:** `(session_id, input_tok, output_tok, cache_read_tok, cache_write_tok, model)` — silently ignores duplicates.


▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
[ 🜂 ] P R I C I N G   R A T E S   ( s e n t i n e l - w r i t e . s h )
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

| Model pattern | Input ($/tok) | Output ($/tok) |
|--------------|--------------|---------------|
| `*sonnet*` | 0.000003 | 0.000015 |
| `*opus*` | 0.000005 | 0.000025 |
| `*haiku*` | 0.000001 | 0.000005 |
| `*flash*` | 0.0000001 | 0.0000004 |
| `*pro*` | 0.00000125 | 0.000005 |

⫷ Rates are duplicated between `sentinel-write.sh` and the Python block in `gemini-session-miner.sh`. Keep in sync manually. ⫸


░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
[ 📶 ] L I V E   S T A T S   ( 2 0 2 6 - 0 3 - 1 3 )
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░

| Metric | Value |
|--------|-------|
| Total events | ~92,000+ |
| Lifetime tokens | ~9.46B |
| Lifetime cost | ~$4,069 |
| Token rate | ~3,217/s |

> **NOTE (S186):** As of S186, Aeris Gemini sessions route through cloudcode-pa PROVISIONED_THROUGHPUT (gemini-oauth-proxy on port 4891). These sessions record $0 cost_usd in sentinel.db. Only Claude Code Sonnet sessions and any direct API callers appear with real costs. Lifetime cost figure above reflects pre-proxy history; current daily Aeris cost shows near-zero.


▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
[ ⛬ ] L A U N C H D   D A E M O N S
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

| Label | Trigger | Script | Notes |
|-------|---------|--------|-------|
| `com.forge.sentinel.claude-miner` | Every 300s | `claude-jsonl-miner.sh` | No RunAtLoad |
| `com.forge.sentinel.gemini-miner` | Every 300s | `gemini-session-miner.sh` | No RunAtLoad |
| `com.forge.sentinel.token-pulse` | Every 600s | `token-pulse.sh` | RunAtLoad=true |

All in `~/Library/LaunchAgents/`. Logs → `/tmp/sentinel-*.{out,err}` (miners); `~/.forge-sentinel/token-pulse.log` (pulse).


▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
[ 🜁 ] T O K E N   P U L S E   ( S 1 8 6 )
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄

Token Pulse exports a live snapshot from `sentinel.db` to `kingdom_state.json` so the Tower site can display real-time spend without querying SQLite directly.

**Script:** `~/.forge-sentinel/token-pulse.sh`
**LaunchAgent:** `com.forge.sentinel.token-pulse` — 600s interval, `RunAtLoad=true`
**Writes to:** `~/Desktop/THE_TOWER/SCRYER_FEEDS/kingdom_state.json`
**Log:** `~/.forge-sentinel/token-pulse.log`

**Block written (`token_pulse`):**

```json
{
  "token_pulse": {
    "snapshot_at": "ISO8601",
    "lifetime_tokens": 9460000000,
    "lifetime_cost_usd": 4069.00,
    "today_tokens": 0,
    "today_cost_usd": 0.00,
    "rate_per_sec": 3217,
    "broadcast_label": "...",
    "broadcast_cost": "..."
  }
}
```

**Interval rationale:** 600s matches the Overmind Pulse cadence. The Tower site interpolates live between snapshots — no need for sub-minute polling.

**Gemini OAuth note:** Post-S186, Aeris Gemini sessions route through gemini-oauth-proxy (port 4891, PROVISIONED_THROUGHPUT). These appear as $0 cost_usd in sentinel.db. `today_cost_usd` will show near-zero during OAuth-routed sessions; `lifetime_cost_usd` reflects true historical spend from before the proxy.


▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
[ 🜂 ] C O M M O N   C O M M A N D S
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄

```bash
# View live HUD
~/.forge-sentinel/token-hud.sh

# Today's spend
sqlite3 ~/.forge-sentinel/sentinel.db \
  "SELECT SUM(cost_usd) FROM token_events WHERE date(timestamp)=date('now');"

# Spend by model
sqlite3 ~/.forge-sentinel/sentinel.db \
  "SELECT model, COUNT(*), SUM(total_tok), SUM(cost_usd) FROM token_events
   GROUP BY model ORDER BY SUM(cost_usd) DESC;"

# Spend by day (last 7 days)
sqlite3 ~/.forge-sentinel/sentinel.db \
  "SELECT date(timestamp), SUM(cost_usd) FROM token_events
   WHERE timestamp >= date('now','-7 days')
   GROUP BY date(timestamp) ORDER BY 1 DESC;"

# Run miners manually
bash ~/.forge-sentinel/claude-jsonl-miner.sh
bash ~/.forge-sentinel/gemini-session-miner.sh

# Check errors
cat /tmp/sentinel-claude-miner.err
cat /tmp/sentinel-gemini-miner.err

# Reload launchd
launchctl unload ~/Library/LaunchAgents/com.forge.sentinel.claude-miner.plist
launchctl load ~/Library/LaunchAgents/com.forge.sentinel.claude-miner.plist
```


════════════════════════════════════════════════════════════════════════════════
[ 🝓 ] G O T C H A S
════════════════════════════════════════════════════════════════════════════════

- **Gemini `cache_write` always 0.** Gemini CLI does not expose cache write tokens.
- **Claude miner uses line-count state, not timestamps.** If a JSONL file is truncated/rewritten, state becomes stale. DB unique constraint is secondary dedup layer.
- **Gemini miner state keyed on `lastUpdated`.** Old completed sessions with unchanged `lastUpdated` are permanently skipped.
- **Pricing rates duplicated.** `sentinel-write.sh` (Claude) and Python block in `gemini-session-miner.sh` (Gemini) must be kept in sync manually.
- **No `RunAtLoad`.** First run is 5 minutes after launchctl load.
- **DB WAL mode.** `.db-shm` and `.db-wal` are normal — do not delete while miners are running.
- **`<synthetic>` bot entries.** Resolved — 0 synthetic entries remain. Early miner artifact, cleaned up.
- **AExGO floating pane.** `token-hud.sh` runs via `viddy -d -i5` in a Zellij floating pane. Trigger: `Alt+T` in AExGO cockpit.

▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
🜚 SYS_TOKEN_SENTINEL // THE FORGE // ⛬⚚⛬ THE LAW STANDS.
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀