Hermes Wiki
AIDigest/ai-digest-scheduler

AI Digest Scheduler

Documents the recurring cloud routine that grows AIDigest automatically. Like weekend-vocab-scheduler, this is a Claude Code cloud routine (Anthropic-hosted CCR session) created via the /schedule skill — a separate, isolated sandbox with its own git checkout of this repo, not part of the local Hermes nightly cron chain.

This folder is included in both Hermes synthesis passes (AgentStack/Hermes/synthesis-incremental.md, every ~3 days, and AgentStack/Hermes/synthesis-fullsweep.md, monthly) alongside Tools/, TechResearch/, BlogPosts/, CoursesDocs/, and LocalzDocs/.

2026-07-11: cadence cut from 5x/day to 1x/day (12am, 5am, 10am, 3pm, 8pm EDT fixed clock times → a single fixed 6am EST run) after usage-limit errors from the 5x/day cadence burning too many tokens. New articles from this run onward use AIDigest/YYYY/MM/DD/YYYY-MM-DD-06-slug.md (06 = fixed literal for 6am EST, not the actual UTC hour); pre-existing articles keep their original UTC-hour names, untouched.

2026-07-24: folder switched from flat to AIDigest/YYYY/MM/DD/ date-sharded directories — the flat folder had grown too long to browse. The date component of the subfolder path is the same UTC calendar date used in the filename itself (see step 7), so YYYY/MM/DD and the YYYY-MM-DD prefix in the filename always agree. All pre-existing articles were moved into this structure with their original filenames untouched; only the folder location changed.

2026-07-24: _sources.md capped to a rolling 60-day window. The dedup log itself (not the articles) was growing unbounded and getting read in full by every run and every synthesis pass. Since step 5's hard recency rule only needs the last 14 days to actually prevent a duplicate, 60 days is a ~4x safety margin against edge cases like a source's publish date drifting on revision (an arXiv paper's v2, a blog post's "last updated" stamp). This is a conscious tradeoff, not a fully airtight guarantee — see step 8.

2026-07-24: dedup check switched from a full read to a targeted grep. Steps 1, 2, and 6 previously read _sources.md in full to check for an already-covered source. Now every check is a single grep -F '<source-url>' AIDigest/_sources.md for that one URL — the file is never loaded whole into the run's context, even bounded at 60 days.

What it does

Once a day, at a fixed 6am EST clock time (0 11 * * * UTC — see Configuration), the routine:

  1. Reads writelist for the current writer/source list and topic scope and _queue for any topics you've queued up since the last run. Does not read _sources in full — every check against it is a targeted grep for one specific URL (see step 2), never a full read of the file.

  2. Dedup check, up front: every candidate source URL gets checked against _sources.md via grep -F '<source-url>' _sources.md (a single targeted lookup, not a full read) before any research effort is spent on it, with existing AIDigest/*.md source: frontmatter as a backstop if the two ever disagree. Never writes a second article for a source already logged.

  3. Processes the queue first: researches every pending topic in _queue.md not already covered per step 2, writes an article for each, then moves those lines from Pending to Done (with date + link).

  4. Spends any remaining effort on autonomous discovery, in two parts:

    • Writer-list sweep: the writers/blogs and arXiv listings in writelist.md.
    • Open web discovery: broad web search (not limited to the fixed writer list) for genuinely new, notable articles/papers/posts on any in-scope Topic — core AI/agents/RAG/models/MCP/etc. included, not just the Verticals section. The writer list is a floor (always checked), never a ceiling — a great new post from a writer/publication not on the list is still fair game.

  5. Hard recency rule: source must be published within the last 14 days of the run date, verified against the actual publish date. Skips anything older or unverifiable.

  6. Re-confirms the source isn't already in _sources.md immediately before writing, via the same targeted grep as step 2 (covers anything found via open web discovery that step 2 didn't already catch).

  7. Writes one markdown file per article: AIDigest/YYYY/MM/DD/YYYY-MM-DD-06-slug.md — the 06 is a fixed literal for 6am EST (the run's one-and-only fire time), not the actual UTC hour, since this now fires once a day. The date component (both the subfolder and the filename prefix) is the UTC calendar date at run time, which is the same as the EST calendar date at 6am EST. The 06 filename-hour convention applies only to articles written from 2026-07-11 onward — pre-existing files keep their original YYYY-MM-DD-HH-slug.md UTC-hour naming, untouched, just relocated under the matching YYYY/MM/DD/ subfolder. Each file gets frontmatter (title, tags, source, created), a summary, key takeaways, a compulsory ## Reel Script section (see below), and an empty ## Discussion section for later Q&A.

    2026-07-13: ## Reel Script made compulsory on every scheduled article, with an eligibility gate. Mihir records these as 2-5 minute Instagram Reels; his angle is a practicing Lead SWE cutting through AI marketing fluff to explain the real mechanism. Not every digest item has a genuine hands-on artifact — before writing, the routine checks for real code, a verified metric, or a diagrammable flow. If none exists, it sets reel_eligible: false in frontmatter and skips the section entirely (no separate log entry; AgentStack/Hermes/dispatcher.py's nightly structural lint already flags any AIDigest/*.md article missing the reel_eligible field via check_missing_reel_eligible(), the same mechanism as its missing-frontmatter/missing-Related checks). If eligible, reel_eligible: true and the section is written as an actual spoken script (sentences meant to be read aloud, not bullet fragments), always exactly 4 labeled beats with a ~140wpm word budget: Hook (~15-20s, 35-45 words, stakes/consequence first, never a self-intro), Core Concept (~45-90s, 105-200 words, the "why" and mechanism, jargon defined via a plain analogy), Hands-On (~45-150s, 105-350 words, the single most concrete screen-worthy artifact the topic genuinely offers — code, a system-design flow, or a real before/after metric, never invented, never lifted verbatim from the source), Takeaway (~20-30s, 45-70 words, committed ROI verdict + a one-line CTA, not a hedge). The word budgets sum to ~290 words (~2 min) at the low end and ~665 words (~4.75 min, i.e. "up to 5 min") at the high end — default to the tight end, only stretching toward the long end when the Hands-On beat has real substance. Thin topics stay short rather than padded. This is compulsory for the scheduled routine (updated via /schedule); interactive articles only get this section when explicitly requested in-session — see AGENTS.md.

  8. Appends to _sources.md: one line per new article (- [Title](slug.md) — <source-url> — YYYY-MM-DD), so the next run's dedup check sees it immediately. Then prunes: removes any log line whose trailing date is more than 60 days before the run date, keeping the file capped to a rolling 60-day window. This only ever trims lines from _sources.md itself — it never touches or deletes anything under AIDigest/YYYY/MM/DD/; the articles are permanent, only their dedup-log entry ages out. A pruned entry loses its fast dedup check; the AIDigest/*.md frontmatter backstop mentioned in step 2 is a manual reconciliation aid, not an automatic re-check, so this relies on the 14-day recency rule alone to keep duplicates from re-appearing that far out.

  9. Logs the run: prepends a ## [YYYY-MM-DD HH:MM UTC] AI Digest Scraper block to Logs/2026.md with an [AIDIGEST] bullet — either the article slugs written, or no qualifying content this run. This happens every run, unconditionally, with a fresh timestamped header each time.

  10. Commits — any new articles, _sources.md (if it changed), _queue.md (if topics were processed), and Logs/2026.md (always) together — and pushes directly to master — no PR gate, same as the vocab scraper. Commit message is aidigest: N new articles (HH:MM UTC) when articles were written, or aidigest: log entry, no new content (HH:MM UTC) on a quiet run.

There's always a commit, even on a quiet run — the log entry is real progress even when no article gets written. Quality over a fixed quota; typically expect a handful of articles a day, not one every single run. See Dashboard's "Scheduler Activity" section for a running count of [AIDIGEST] log entries.

Asking from mobile

Deliberately not a custom in-app chat UI. An earlier version of this feature called the Claude API directly from viewer/'s server, but that needs its own ANTHROPIC_API_KEY billed per-token — separate from, and not covered by, a claude.ai subscription. Removed in favor of making the existing claude.ai-chat workflow effortless instead:

  • Ask about an article — every AIDigest article page has a "Copy prompt for Claude" button (components/CopyClaudePrompt.tsx) that copies a ready-made instruction to your clipboard: it names the exact file, asks your question to be appended under that file's ## Discussion section only, and to push. Paste it into a fresh claude.ai (or Claude Code) chat pointed at the fullstackfusions/PKM repo, then type your actual question — covered by your subscription, zero extra API cost.
  • Queue a topic for the next scheduled run — hand-edit _queue.md's ## Pending list, or ask the same kind of claude.ai chat session to do it for you.
  • Research something right now (not queued) — same claude.ai chat session; per AGENTS.md's Immediate Research trigger, it researches and writes the article in that session rather than deferring to the queue.

Configuration

Field Value
Routine name AI Digest Scraper
Routine ID trig_01SUbiwDdUyXrgKQvAGSBZZv
Cadence 0 11 * * * — 11:00 UTC = 6:00am EST, once a day. This UTC value is permanent, not re-adjusted for DST — during EDT (summer) it'll read as 7:00am local instead of 6:00am. That drift is accepted on purpose rather than hand-shifted twice a year. Changed from the original 0 0,4,9,14,19 * * * (5x/day) on 2026-07-11 after usage-limit errors from the token cost of 5 runs/day.
Model claude-sonnet-5
Repo https://github.com/fullstackfusions/PKM (master branch, direct push)
Environment Anthropic cloud default (env_01S8gzx6qMrwgKjVjq6ygeiD)
Tools Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch
Scope guard Only ever creates files under AIDigest/ and edits AIDigest/_queue.md and AIDigest/_sources.md — never touches AIDigest/writelist.md (that's yours to edit) or anything outside this folder
Manage / view runs https://claude.ai/code/routines/trig_01SUbiwDdUyXrgKQvAGSBZZv

To change cadence, model, writer list, or prompt behavior: edit writelist directly for sources/topics (no routine update needed), or use the /schedule skill (list → update) for cadence/model/prompt changes. Routines can't be deleted via the API — only through https://claude.ai/code/routines.

Note: since this runs in an isolated cloud sandbox, it has no access to the local machine — it operates purely on its own clone of the GitHub repo, so its changes only reach this vault once pushed and pulled locally (or picked up by whatever already syncs master here, e.g. viewer/deploy/rebuild.sh's pull-on-rebuild).

Hermes Wiki