← All apps

sherpa-hub

Hub to track my app development

README

# sherpa-hub

A read-only aggregator dashboard for Ken's app portfolio.

- See `docs/superpowers/specs/2026-05-06-sherpa-hub-rebuild-design.md` for the design.
- See `docs/superpowers/plans/2026-05-06-sherpa-hub-rebuild.md` for the implementation plan.
- `web/` is the deployed FastAPI hub (read-only; reads `hub-cache.json`).
- `sync/` is the local Windows sync script that builds the cache and POSTs it.
- `legacy/` is the original sherpa-hub repo, kept as reference only (gitignored).

## Local development

### Hub (web)

```sh
cd web
poetry install
npm install
npm run build:css
poetry run uvicorn app.main:app --reload
```

Open http://localhost:8000/healthz to confirm it's running. UI routes require `HUB_USERNAME`/`HUB_PASSWORD` env vars.

### Sync (local)

```sh
cd sync
poetry install
cp .env.example .env  # then fill in real values
poetry run python -m sync
```

This pulls from GitHub + Render, builds `hub-cache.json`, and POSTs it to the deployed hub at `HUB_URL`.

## Tests

```sh
cd web && poetry run pytest -v   # 35 tests
cd ../sync && poetry run pytest -v   # 50 tests
```

STATUS

# sherpa-hub — Status

## Done
- Brainstormed and locked design: aggregator-only, no DB, two-piece split (sync + web)
- Wrote design spec at `docs/superpowers/specs/2026-05-06-sherpa-hub-rebuild-design.md`
- Wrote implementation plan at `docs/superpowers/plans/2026-05-06-sherpa-hub-rebuild.md`
- Implemented all 20 plan tasks on branch `rebuild-v1` via TDD (subagent-driven)
  - Phase 0: scaffolding (sync/ + web/ Poetry projects)
  - Phase 1: sync side — schema, derive, GitHub/Render/filesystem/diary sources, builder, uploader, CLI entry point
  - Phase 2: hub side — config, cache reader, auth, /healthz, /admin/refresh, /, /apps, /apps/{slug}, /diary, /diary/{date}, /health
  - Phase 3: deployment — render.yaml + build.sh + README
- 85 tests pass (50 sync + 35 web)
- Merged `rebuild-v1` to `main` and deleted the feature branch
- **v1 is live and serving real data** at `https://sherpa-hub-6psj.onrender.com`
- First successful sync (2026-05-20): 41 GitHub repos, 20 Render services, 19 local folders matched, 10 orphans, 3 diary entries

## Post-deploy fixes landed during first-run debugging
- `fix: rename /admin/refresh → /sync/upload` — Render's edge WAF 403s any POST to a URL containing "admin"
- `fix(sync): truncate READMEs + memory files to 2000 chars each` — keeps the cache lean
- `fix: gzip the upload body` — Cloudflare DLP was scanning the cache and flagging env-var-name patterns (`GITHUB_TOKEN`, etc.) as "leaked credentials." Gzipping turns the body into a binary blob the scanner can't parse.

## UI v2 redesign — DONE (2026-06-08)
Reskinned the hub to a dark dashboard matching a mockup Ken liked, while keeping
everything auto-derived (Path 3 hybrid — status/buckets/counts stay automatic;
only icon glyph/color and GitHub topics are set-once).
- Spec: `docs/superpowers/specs/2026-05-20-ui-redesign-v2-design.md`
- Plan: `docs/superpowers/plans/2026-05-20-ui-redesign-v2.md` (10 TDD tasks, subagent-driven)
- Shipped: dark theme; left sidebar with bucket filters + live

…(truncated for upload size)

DECISIONS

# sherpa-hub — Decisions

> Architectural choices made during the rebuild brainstorm (2026-05-06). Not to be
> re-litigated without explicit reason.

## D-001: Aggregator-only architecture (no DB, no owned data)
**Date:** 2026-05-06
**Decision:** The hub stores no app metadata of its own. Every rendered field comes
from GitHub, Render, a memory file, or the diary. The only thing the hub owns is a
JSON cache of the last successful sync.

**Why:** The legacy hub had its own copies of name/description/status/tags/priority/
notes/next_action — all of which already lived elsewhere. That's why it went stale.
Removing ownership eliminates the failure mode entirely.

## D-002: Two-piece split — local sync + cloud hub
**Date:** 2026-05-06
**Decision:** Local Windows script does all credentialed work (GitHub API, Render API,
filesystem reads, diary reads). Builds `hub-cache.json`. POSTs it to the deployed hub.
Hub on Render only reads the JSON.

**Why:** Smaller attack surface (no API tokens on Render), simpler hub, naturally pairs
with the SessionEnd diary write. Trade-off: requires Ken's PC to be running for fresh
data — acceptable because he doesn't turn it off and refresh only matters when he's
been coding.

## D-003: Stack — FastAPI + Jinja2 + Tailwind Plus + JSON cache + Poetry + Render
**Date:** 2026-05-06
**Decision:** Same family as the legacy stack but cleaner. No SQLite, no DB at all.
Server-rendered Jinja templates, not a SPA.

**Why:** Data is small and derived. JSON cache is the right shape. Per global
CLAUDE.md, "small tools don't need to follow the full suite stack" — this is firmly
in that bucket. SPA is unnecessary complexity.

## D-004: Move diary from Obsidian/Dropbox to Google Drive plain Markdown
**Date:** 2026-05-06
**Decision:** Diary lives at `G:\My Drive\kens-personal-life\diary\YYYY-MM-DD.md`
with YAML front-matter (`projects: [slug, slug]`) plus per-project H2 sections in body.
One file per coding day. Plain Markdown, no Obsidian dependency.

**Why

…(truncated for upload size)

MEMORY

# sherpa-hub — Memory

## Origin
Ken had a previous sherpa-hub deployed on Render (suspended). It was a FastAPI + Jinja
+ SQLite app that hand-curated a registry of apps and a tasks list. The legacy DB had
12 apps and 0 tasks ever created — the registry went stale because it duplicated facts
(name, description, status, priority, area, tags) that already lived in GitHub repos,
STATUS.md files, and Render. The task feature was never adopted because Ken already
tracks tasks in Claude Code, STATUS.md, and other tools.

Legacy code is preserved at `legacy/` as reference (its own git history, gitignored).

## Why This Rebuild Looks Different
The legacy was option-C in shape (registry + dashboard) but failed on input quality.
The rebuild flips it: the hub stores nothing and aggregates from authoritative sources.
Source-of-truth ownership:
- **Repo identity / description / topics / archived** ← GitHub
- **Deploy status / URL** ← Render API
- **What's done / next / blocked** ← per-project STATUS.md
- **Architecture / decisions / context** ← per-project DECISIONS.md, MEMORY.md, CLAUDE.md
- **Daily activity & ideas** ← the diary at `G:\My Drive\kens-personal-life\diary\`

## Standing Facts
- GitHub user: `klill6506`
- Apps live in `D:\dev\` (work) and `D:\Personal\` (personal)
- Memory files mirror to `G:\My Drive\kens-personal-life\apps\<project>\` via background sync
- Diary is being moved from Obsidian/Dropbox to Google Drive plain Markdown as part of
  the same effort. Old location: `C:\Users\Ken2\Tax Shelter Dropbox\Ken Lill\KenVault\Claude Diary`
- Diary structure (post-move): one file per day at `G:\My Drive\kens-personal-life\diary\YYYY-MM-DD.md`
  with YAML front-matter (`projects: [slug, slug]`) plus per-project H2 sections
- Diary writing is automated via a SessionEnd hook that fires a headless Claude Code
  run to summarize the just-ended session and write the entry

## Deployment Learnings (2026-05-20)
Stuff we figured out the hard way during the first real deploy

…(truncated for upload size)

CLAUDE.md

# sherpa-hub — Project Conventions

> Read-only aggregator dashboard for Ken's app portfolio. Pulls from GitHub, Render,
> per-project memory files, and the daily diary. Owns no app data — every field is
> rendered from a source of truth elsewhere.

## Architecture in One Paragraph
Two pieces split by a JSON contract. `sync/` is a local Windows script that runs after
every Claude Code session (via SessionEnd hook): it calls GitHub + Render APIs, reads
local memory files from `D:\dev\` and `D:\Personal\`, reads the diary from
`G:\My Drive\kens-personal-life\diary\`, and POSTs a `hub-cache.json` to the deployed
hub. `web/` is a FastAPI + Jinja2 app on Render that reads that JSON and renders pages.
The hub itself has no API credentials and no database.

## Stack
- **Python:** 3.13 (per global CLAUDE.md, 3.12+ minimum)
- **Backend:** FastAPI + Jinja2 + Tailwind Plus
- **Cache:** JSON file on Render persistent disk — no database
- **Packaging:** Poetry
- **Hosting:** Render (Virginia), single web service for `web/`
- **Auth:** HTTP Basic (single user) for the public UI; bearer token on `/admin/refresh`
- **Sync runtime:** Python 3.13 on local Windows, invoked by Claude Code SessionEnd hook

## Directory Layout
- `web/` — the deployed FastAPI hub (reads JSON, renders pages)
- `sync/` — local Windows script (writes JSON, never deployed)
- `docs/superpowers/specs/` — design specs (this brainstorm's output lives here)
- `legacy/` — old FastAPI+SQLite hub, reference only, gitignored (separate repo)

## Hard Rules (Do Not Drift)
- **The hub stores nothing about apps.** Every visible field is sourced from GitHub,
  Render, a memory file, or the diary. If a field needs to be added, find a source for
  it — do not introduce a "hub-owned" field.
- **No database.** The whole point of v1 is no DB. JSON cache only.
- **No credentials in `web/`.** Render env vars on the hub are limited to:
  `HUB_USERNAME`, `HUB_PASSWORD`, `HUB_UPLOAD_SECRET`. GitHub/Render/Google Drive
  tokens belon

…(truncated for upload size)

Diary mentions

2026-06-08
# 2026-06-08

## sherpa-hub

Shipped the UI v2 redesign — the hub went from plain light-Tailwind to a dark,
polished dashboard matching the mockup I liked, without giving up the
"stays-current-by-itself" guarantee that's the whole point of the rebuild.

### What we did

Full brainstorm → spec → plan → subagent-driven build cycle:
- Brainstormed the look using the visual companion. Looked at three directions
  first (minimal / dense-dark / playful), then I dropped in a reference mockup
  (the sherpa-dashboard-style dark dashboard with sidebar + summary cards + icon
  tiles + two-button cards) and we built around that.
- Key call: **Path 3 (hybrid).** Adopt the mockup's look, but keep status,
  activity buckets, and all counts 100% auto-derived from GitHub + Render. The
  only hand-set data is set-once and non-rotting: per-app icon (glyph + color)
  in `icons.yaml`, and category tags from GitHub repo topics. This preserves the
  thing that matters most to me — it never goes stale on its own.
- Wrote the spec (`docs/superpowers/specs/2026-05-20-ui-redesign-v2-design.md`,
  decisions UI-1..UI-15) and a 10-task TDD plan, then executed subagent-driven
  with spec + code-quality review on the load-bearing tasks.

### What shipped

- Dark theme, violet accent, left sidebar (bucket filters with live counts +
  Diary/Health links)
- 4 summary stat cards: Total / Live / Active / Needs Attention (the last links
  to /health and shows a green ✓ when nothing's wrong)
- App cards: colored Lucide icon tile, name, description, category tag pills,
  status pill, and two buttons — "Open App ↗" (jumps to the app's URL) and
  "⌁ Status" (drills into the detail page)
- Client-side search + grid/list toggle (no page reload, no backend)
- `/apps` list view dropped — home IS the apps view now (301 redirect)
- Detail, diary, and health pages all restyled dark

### How the icons work (for future me)

`web/app/icons.yaml` maps each slug to a Lucide icon name + a color. The SVGs are
self-hosted via the `lucide-static` npm package and inlined server-side — no CDN,
no icon flash, glyphs color from the tile via currentColor. Lookup is
case-insensitive (matters for repos like `Pips`/`Tanks` that GitHub stores with
capitals). Add a line per new app; anything missing falls back to a slate box.
`reload_icons()` fires on every sync upload so icon edits show up without a restart.

### Decisions recorded

- D-012: UI v2 dark dashboard, Path 3 hybrid (auto-derived + set-once icons)
- D-013: icons.yaml is a deliberate, bounded exception to D-001 (hub-owns-nothing)
  — an icon is a display preference with no source of truth elsewhere, and it
  doesn't rot, so it doesn't reintroduce the legacy failure mode
- D-014: self-host Lucide via lucide-static, inlined (no CDN, no runtime JS)

### Numbers

- 10 plan tasks, all subagent-driven with review gates
- 99 tests pass (49 web + 50 sync)
- Local full-page smoke render confirmed real Lucide SVGs inline on every page
  (17 on the dashboard) — proved the icon pipeline end-to-end, which the unit
  tests alone couldn't
- Pushed to main; Render auto-deploy in flight

## ideas

- Once the deploy lands, fill in `icons.yaml` for any app still showing the slate
  box, and sprinkle GitHub topics on repos I want category tags for. Both
  set-once.
- Favorites was in the mockup but I cut it — no data source and it'd need upkeep.
  If I ever want it, the cheap version is a `favorites: [slugs]` list in a hub
  config file + a star toggle that's really just a filter. Only if I'll actually
  use it.
- The "Needs Attention" card is a nice nudge — when it's >0 it's red with a count,
  when clean it's a green check. Worth watching whether it actually drives me to
  fix orphans/missing-memory, or whether I just ignore it like the legacy hub.

## notes

- Reviewer caught a good one on the icon module: it suggested lowercasing the
  YAML keys to fix `Pips`/`Tanks`, but that would've BROKEN the match (GitHub
  slugs preserve case, so the cache has `Pips` with a capital). Right fix was
  case-insensitive lookup on both sides. Good reminder to verify review feedback
  against the actual data rather than implementing it blindly.
- Subagent-driven dev keeps paying off: the model-vs-data reasoning above, the
  CVE catch last session, the schema-datetime tightening — all came from the
  review gates, not the first-pass implementation.
- Hub URL unchanged: https://sherpa-hub-6psj.onrender.com (dark UI live after the
  deploy finishes). Still need to eyeball it + run a fresh sync.

---

## sherpa-hub — UI v2.1 polish (later same day)

Same-day follow-up after I looked at the live v2 dashboard. Three quick
refinements, brainstormed with the visual companion then built subagent-driven
(5 tasks, 103 tests green).

- **Buttons toned down.** The big filled-purple "Open App" bar was too much. Now
  both buttons are small outlined; "Open ↗" keeps a faint violet tint as the
  primary, "Status" is a quiet outline.
- **Light mode.** Added a sidebar Light/Dark toggle (sun-moon). Default dark,
  remembered in the browser, applied before paint so there's no flash on reload.
  The light palette is a token-override block plus explicit light variants for the
  status pills and the open button (their dark translucent colors were invisible
  on white).
- **The Yeti.** Went down a fun rabbit hole on the app icon — the plain Lucide
  "mountain" looked like a pencil. Generated a few yetis; landed on a flat-style
  "Coding Yeti" logo (shades + hoodie + laptop with `</>`). It's now sherpa-hub's
  icon on its card, the sidebar brand, and the favicon. Built a small extension:
  icons.yaml entries can now carry an `image` field that renders a standalone PNG
  instead of a recolored Lucide glyph — bounded one-app exception, every other app
  keeps the clean Lucide tiles.

### Notes / lessons
- The icon detour was a good reminder that "give me an icon" and "here's a
  gorgeous AI illustration" are different things. Detailed full-color mascots are
  hero images, not 24px tiles. The flat two-tone logo (which even shipped with its
  own small-size preview) was the right kind of asset.
- Verified review feedback against real data again: a reviewer wanted me to
  lowercase the icons.yaml keys to "fix" Pips/Tanks, but GitHub slugs preserve
  case so the cache really has `Pips` — case-insensitive lookup was the correct
  fix, not lowercasing the keys.
- Source yeti images saved in D:\dev\sherpa-server\Images\ (yeti-laptop.png is the
  keeper). Crop recipe is in the v2.1 plan.

### Still to do
- Eyeball the live site after deploy: yeti everywhere, smaller buttons, theme
  toggle. Run a fresh sync so the sherpa-hub *card* picks up the yeti.
- If the yeti's dark hoodie is low-contrast on a dark card, there's a ready
  one-line `.icon-tile:has(.icon-img)` plate fix noted in the plan.

---

## sherpa-hub — end of day: two bugs on the live v2.1 site (paused, resume tomorrow)

Looked at the deployed v2.1 site. Light/dark mode works. Two issues; both
root-caused (systematic-debugging), no fixes applied yet — stopped for the day.

**Bug 1 — yeti not showing.** Server side is verified correct: the image returns
200 image/png live, and the render test proves the sidebar HTML includes the
`<img>`. So it's a browser cache (a stale failed-load from the brief window before
the asset deployed). To do: hard-refresh (Ctrl+Shift+R) and confirm. Likely no code
change.

**Bug 2 — "Open" links go to GitHub, not the live app.** Real root cause: only 13
of 43 apps match a Render service. The matcher requires the Render service name to
EXACTLY equal the repo slug, and many don't — georgianna-mahjong=games-mahjong,
ken-half=half-training-app, four CFB* services=cfb-picks, plus ~27 repos not on
Render (some at kenlill.com). All of those fall back to GitHub. The hub can't guess
these — only I know the mappings/custom domains. Plan: a manual per-app `url:`
override in hub config (like icons.yaml), precedence url > render > github. Need to
decide tomorrow: (A) manual override only, or (B) also auto-pull Render custom
domains. Leaning A. Then I supply the real URLs and re-sync.

Also: the deployed cache is stale (last sync Jun 5) — a fresh sync is needed after
the fix regardless.


---

## games-mahjong + sherpa-hub — evening wrap

**Hub link fix shipped.** Built the per-app `url:` override in icons.yaml
(precedence url > Render > GitHub), read at render time so no re-sync needed.
First mapping: games-mahjong → https://mahjong.kenlill.com (verified live there —
"Georgianna's Mahjong Tutorial"; the old Render service is suspended). 55 web
tests green, pushed. Remaining GitHub-linking apps just need their real URLs
added one line each.

**kenlill.com apex finding:** the root domain is serving a *different, empty*
sherpa-hub instance — no data, and NO auth (open to the world, though only the
empty state shows). The real hub (sherpa-hub-6psj on Render) has the data and
Basic auth. Recommended fix: move the kenlill.com domain to the real hub service.
Decide tomorrow.

**KenBot → mahjong: investigated, correctly STOPPED at pre-flight.** Ken dropped
`D:\Personal\sherpa-kenbot\INTEGRATION.md` (animated talking help character for
the mahjong help screen). Ran the doc's pre-flight gates; both fail:
1. KenBot is React-19-only; the mahjong app is vanilla JS + Vite (no React).
   Recommended path: a tiny React island just for the bot (package ships built
   ESM, so no JSX tooling needed in the host).
2. The doc requires an existing AI help backend; mahjong has none (Supabase Edge
   Functions for game moves only). Natural fit: a new `ask_kenbot` Edge Function
   calling an LLM with keys in Supabase secrets; the ElevenLabs voice proxy can
   be an Edge Function too (doc's Django proxy doesn't apply).
Two decisions for Ken tomorrow: React-island vs vanilla-build, and which
LLM/knowledge should power the bot's answers.

Also: re-cloned games-mahjong to D:\Personal\games-mahjong (local copy had been
lost in the D: cleanup — it was a repo-no-local orphan on the hub Health page),
and created its missing STATUS.md + MEMORY.md with all of the above. Tomorrow's
agenda: hub URL list + kenlill.com domain move + KenBot decisions.
2026-05-20
# 2026-05-20

## sherpa-hub

Took the new sherpa-hub from "deployed but empty" to "deployed and serving real
data." Two stacked bugs blocked the first sync end-to-end — peeled them off in
order.

### What we did

- Provisioned the Render service via the Blueprint config. Hub came up clean,
  Basic Auth worked, `/healthz` returned OK with `cache_age_seconds: null`.
- Generated a GitHub fine-grained PAT for `klill6506`. First attempt had no
  permissions assigned — GitHub silently issued a token with zero scope. Fixed
  by editing the token to add Contents + Metadata read-only on All Repositories.
  Eventually swapped to a classic PAT (`ghp_…`) because the fine-grained flow
  has too many UI gotchas. Both formats work since the sync uploader just sends
  `Authorization: Bearer <token>`.
- Filled out `sync/.env` with GitHub token, Render API key, Hub URL, upload
  secret, dev/personal roots, and diary dir.

### The hard bug: Cloudflare DLP

First sync ran the whole pipeline successfully — GitHub, Render, filesystem,
diary — then choked on the upload with a `403 Forbidden` and an HTML "Blocked"
page from Render's edge layer.

Traced through several false leads:
- User-Agent change (no fix — the WAF wasn't checking that)
- Renamed `/admin/refresh` → `/sync/upload` (helped a known rule but not the
  real blocker)
- Truncated READMEs and memory files from full content to 2000 chars each
  (415 KB → 178 KB cache, but still blocked)
- New `HUB_UPLOAD_SECRET` with no special characters (still blocked because
  this wasn't about the secret value — 403 fires before auth)

The actual fix was **gzip the body**. Render's edge runs Cloudflare-style DLP
scanning that looks for "leaked credential" patterns in POST bodies. Our cache
contains memory files that *mention* env var names like `GITHUB_TOKEN` and
`HUB_UPLOAD_SECRET` — those substrings trip the leaked-credentials scanner.
Gzipping the body turns it into a binary blob the scanner can't parse, so it
passes through. As a bonus the wire size drops 4x (178 KB → 46 KB).

After the gzip fix landed, the request reached our FastAPI app for the first
time — and got a clean `401 Invalid bearer token`. Different problem, but
visible only once the WAF stopped swallowing requests. The 401 was
self-inflicted: I had typed `HUB_UPLOAD_SECRET=$secret=` into `.env`. The
file is read literally by python-dotenv — `$secret` is not a variable
reference, it's just a literal six-character string. Replaced with a clean
40-char alphanumeric value on both sides (Render env + local `.env`).

Next sync run: `200 OK`, `Done.`

### Numbers from first successful sync

- 41 GitHub repos pulled
- 20 Render services matched by name
- 19 local folders matched (across `D:\dev` + `D:\Personal`)
- 10 orphan folders (local-only, no GitHub repo)
- 3 diary entries loaded
- Cache: 178 KB raw → 46 KB gzipped
- Total runtime: ~12 seconds end-to-end

The deployed dashboard now shows everything. Health page surfaces real items
(the 10 orphans + apps missing memory files). Big payoff for the rebuild —
exactly the "wrap your arms around it all" view that was the original goal.

### Decisions recorded today

- **D-009:** Upload route is `/sync/upload`, not `/admin/refresh`
- **D-010:** Truncate READMEs + memory files to 2000 chars in the cache
- **D-011:** Gzip the upload body to bypass Cloudflare DLP

## ideas

- **UI redesign (queued for tomorrow).** Functionality is in; appearance is
  basic. Asks: an icon per app, separate tabs by activity bucket (active /
  dormant / stale / abandoned) instead of one mixed grid, generally prettier.
  Brainstorming skill was invoked but paused at the visual-companion offer —
  resume there.
- **Lesson worth keeping in mind for other apps deployed on Render:** if you
  ever POST a body that contains things that *look like* credentials (env var
  names, API key prefixes, even keywords like "secret" or "token"), Cloudflare
  will block it at the edge regardless of whether they're actual credentials.
  Gzip the body or accept that you'll need to keep stripping keyword content.
- **Lesson on PATs:** Fine-grained PATs in GitHub start with **zero** scope by
  default. The UI lets you generate one without selecting any access. Verify
  the token detail page shows actual permissions before debugging deeper.

## notes

- Force-pushed `rebuild-v1` branch to `main` and deleted the branch. New
  preference recorded: for solo projects with no prod risk, work directly on
  `main` — branches in GitHub feel like clutter. Captured as
  `feedback_branches.md` in the per-project memory dir.
- Hub URL is `https://sherpa-hub-6psj.onrender.com` (the `-6psj` is Render's
  random suffix for the auto-generated name). HTTP Basic Auth in front of all
  UI routes; Bearer on `/sync/upload`.
- Still on the "Phase 1 only" milestone — diary auto-write hook and Obsidian
  diary migration remain queued as adjacent tasks. Neither is blocking the
  next UI work.

Render

  • Service: sherpa-hub
  • Status: suspended
  • Last deploy: 2026-06-11T21:58:16.076761Z