Commit Graph
28 Commits
Author SHA1 Message Date
Julien HerrandClaude Opus 4.7 b534ce5bf8 feat(monitoring): add stats counters API and public status page
Add GET /api/stats exposing cumulative counters (feeds created/deleted,
emails received/rejected, recent date-times) plus live values (active
feeds, active WebSub subscriptions). Counters persist in a stats:counters
KV singleton and are incremented at the email-processing chokepoint and
feed create/delete paths. Replace the / → /admin redirect with a public
status page rendering these figures with a link to the admin.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-23 09:50:51 +02:00
Julien HerrandClaude Sonnet 4.6 7b2b98d693 refactor: extract url helpers, add EMAIL_DOMAIN support
- Add src/utils/urls.ts with baseUrl, feedRssUrl, feedAtomUrl, feedUrl,
  feedEmailAddress, feedTopicPattern
- Add optional EMAIL_DOMAIN env var so web domain and email domain can
  differ (e.g. demo.kill-the.news serves feeds, @kill-the.news receives mail)
- Replace all inline domain template literals with the new helpers
- Remove unused site_url/feed_url fields from FeedConfig
- Remove unused feedPath param from fetchFeedData
- Extract verifyCallback() to deduplicate verifyAndStoreSubscription /
  verifyAndDeleteSubscription

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 22:38:29 +02:00
Julien HerrandClaude Sonnet 4.5 a29e9ab372 feat: WebSub Atom support, HTML processing via linkedom, W3C badges
WebSub / PubSubHubbub:
- Hub now accepts both /rss/:id and /atom/:id topic URLs
- WebSubSubscription stores format ("rss" | "atom")
- notifySubscribers sends RSS or Atom XML with correct Content-Type
- verifyAndStoreSubscription sends correct topic URL per format
- CI paths-ignore docs/** to skip deploy on docs-only changes

HTML processing (linkedom + escape-html):
- New html-processor.ts: body extraction, script/iframe/object removal,
  event handler + javascript: URL stripping, mso-* style cleanup,
  plain text → <pre> with HTML escaping via escape-html
- feed-generator.ts and entries.ts use processEmailContent

Admin UI:
- W3C validation badges (Atom + RSS) on feed detail page

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
2026-05-22 21:12:22 +02:00
Julien HerrandClaude Sonnet 4.6 afed4464cf fix(feed): use permalink URL as Atom entry id, strip mso-* inline styles
- Entry <id> was a non-URL string (timestamp + base64 snippet), which
  is invalid per the Atom spec; now uses the entry permalink URL which
  is both valid and stable across feed regeneration
- Strip mso-* properties from inline style attributes in extracted body
  content to eliminate the feed validator DangerousStyleAttr warning
  caused by Microsoft Office HTML in newsletter emails

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 18:43:06 +02:00
Julien HerrandClaude Sonnet 4.6 4428f35dd4 fix(feed): add Atom link in emails page, fix HTML stripping, use request URL for self-link
- Add Atom Feed URL to the Feed Details card in the emails page
- Fix extractBodyContent to handle emails without a closing </body> tag
  (regex now falls back to capturing everything after the opening <body>)
- Use the actual request URL origin for atom:link rel="self" in RSS/Atom
  feeds, guaranteeing it always matches the document location regardless
  of how DOMAIN is configured

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 18:41:21 +02:00
Julien HerrandClaude Sonnet 4.6 bcc9640591 fix(feed): correct feed link, canonical id, and strip html wrapper from content
- link: computed as /admin/feeds/:id/emails instead of stale site_url from KV
- id: computed dynamically from baseUrl instead of stale feed_url from KV
- item description/content: strip <html><head><body> wrapper via extractBodyContent()
  so feed readers receive a body fragment, not a full HTML document

Fixes RSS validator warnings: SelfDoesntMatchLocation (stale KV domain) and
InvalidHTML (full HTML document inside <description>/<content:encoded>).
Adds 8 tests covering extractBodyContent and the new feed/atom link assertions.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 18:28:13 +02:00
Julien HerrandClaude Sonnet 4.6 a9501d6e44 feat: add structured JSON logger and worker utility (P1-4)
Introduces src/lib/logger.ts emitting JSON lines (level, message, data)
compatible with Cloudflare Logpush. Replaces all console.log/warn/error
calls in email-processor.ts, index.ts, and hub.ts with structured logger
calls. Extracts waitUntilSafe into src/utils/worker.ts to avoid duplicating
the executionCtx guard across routes.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 10:55:15 +02:00
Julien HerrandClaude Sonnet 4.6 0c0669c473 feat: extract constants module (P2-10)
Centralises magic numbers and string literals into src/config/constants.ts
(FEED_MAX_BYTES, MAX_FEED_ITEMS, MAX_METADATA_EMAILS, ADMIN_COOKIE_MAX_AGE,
FORWARD_EMAIL_IPS_CACHE_TTL_MS, DEFAULT_LEASE_SECONDS, MAX_LEASE_SECONDS,
FEEDS_LIST_KEY). Consumers updated in storage.ts and feed-fetcher.ts.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 10:55:01 +02:00
Julien HerrandClaude Sonnet 4.6 a0415cdc41 refactor: replace custom HMAC CSRF with hono/csrf middleware
Removes 38-line hand-rolled HMAC-SHA256 implementation in favour of
the built-in hono/csrf, which validates the Origin header natively.

- Delete src/utils/csrf.ts
- Replace custom CSRF middleware with hono/csrf (Origin-header check)
- Remove csrfToken from ContextVariableMap, layout(), forms, and JS fetch() calls
- Update admin tests: swap X-CSRF-Token for Origin header

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 10:28:26 +02:00
Julien HerrandClaude Sonnet 4.6 7d375693b9 feat: complete Phase 2 tech debt remediation
- Extract shared RSS/Atom fetch logic into feed-fetcher utility (P1-3)
- Split email-processor into validateEmail/storeEmail functions (P1-6)
- Add stateless HMAC-SHA256 CSRF protection to admin forms (P2-8)
- Fix Hono<{ Bindings: Env }> type safety across all routes (P3-13)
- Add entries.test.ts and files.test.ts with full coverage (P1-7)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-22 09:46:55 +02:00
Julien Herr 10404efffa fix: delete orphaned KV entries when trimming feed metadata 2026-05-22 07:39:37 +02:00
Julien Herr e8f5af8b87 test(websub): add missing error-path tests for verify functions 2026-05-21 23:46:49 +02:00
Julien Herr c6785554d4 test(websub): add unit tests for WebSub utilities 2026-05-21 23:46:49 +02:00
Julien Herr 0b4ed0fbec fix(websub): fix signature header, add delivery logging, parallelize KV reads 2026-05-21 23:46:49 +02:00
Julien Herr 8810109e6f feat(websub): add WebSub utilities (HMAC, KV helpers, notify, verify) 2026-05-21 23:46:49 +02:00
Julien HerrandClaude Sonnet 4.6 b24ee969d1 style: fix Prettier formatting on 11 files
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-21 11:35:37 +02:00
Julien HerrandClaude Sonnet 4.6 3aea41f862 feat: add ESLint, lint-staged, and update pre-commit hook + CI
- Add ESLint 9 flat config (eslint.config.mjs) with typescript-eslint
  recommended rules and eslint-config-prettier
- Add lint-staged to run eslint+prettier only on staged files
- Update pre-commit hook to use lint-staged instead of full prettier check
- Add `lint` and `format:check` scripts to package.json
- Add Lint step to CI workflow
- Fix resulting lint errors: unused vars (_ctx, _options, catch binding),
  any→unknown in type declarations, stale eslint-disable comments

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-21 09:49:20 +02:00
Julien HerrandClaude Sonnet 4.6 e93bbb8d3e feat: store email attachments in R2 and expose as RSS enclosures
Attachments from incoming emails are uploaded to an optional Cloudflare R2
bucket and exposed as <enclosure> elements in RSS and <link rel="enclosure">
in Atom feeds, served at /files/{id}/{filename} with immutable caching.

R2 is opt-in: if ATTACHMENT_BUCKET is not bound the feature is a no-op.
Attachments are cleaned up from R2 on email/feed deletion and during
size-based feed trimming. Adds MockR2 to the test setup.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-21 09:09:37 +02:00
Julien Herr 4d8d1fdc82 refactor: extract buildFeed helper, add generateAtomFeed 2026-05-21 07:34:29 +02:00
Julien HerrandClaude Sonnet 4.6 2e9cdd4364 test: add unit tests for EmailParser
Covers extractFeedId, decodeEncodedWords, and parseForwardEmailPayload
including edge cases: missing fields, structured vs text from, headerLines
vs raw headers string, RFC 2047 subject decoding.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-21 00:06:13 +02:00
Julien HerrandClaude Sonnet 4.6 41efee44ca refactor: replace custom escapeHtml with Hono's html template
Hono's `html` tagged template auto-escapes all interpolated values;
`raw()` is used for the email body which must render as HTML.
This removes the ad-hoc utility and aligns entries.ts with the
same pattern already used in admin.ts.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-21 00:05:35 +02:00
Julien HerrandClaude Sonnet 4.6 5308544672 refactor: simplify quick-win code after review
- Make feedId required in generateRssFeed (removes dead /emails/ fallback)
- Hoist loop-invariant conditional and remove intermediate variable
- Extract normalizeAllowedSenders() so JSON and form paths share same logic
- Move escapeHtml to src/utils/html.ts for reuse by admin.ts
- Parallelize the two independent KV puts in feed creation

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-20 23:55:17 +02:00
Julien HerrandClaude Sonnet 4.6 54e7a1bfa0 feat: parse <author> from From header in RSS items
Parse the From header into name + email parts so the feed library
renders proper RFC 2822 format (email (Name)) in <author> elements.
Also passes feedId to the generator so item links can point to the
upcoming /entries/:feedId/:receivedAt route.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-20 23:50:54 +02:00
Julien HerrandClaude Sonnet 4.6 71a5c20e62 refactor: parallelize KV writes, cap metadata, remove redundant normalization
- email-processor: run email put + metadata get in parallel (saves one KV round-trip per email)
- email-processor: add missing 50-email metadata cap (was unbounded unlike storage.ts)
- email-processor: remove redundant normalizeEmail call on already-normalized allowedSender
- email-parser: strip WHAT-comments and boilerplate JSDoc throughout

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-05-20 23:05:26 +02:00
Julien HerrandClaude Sonnet 4.5 3ed9d2ee22 chore: apply Prettier formatting to entire codebase
Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
2026-05-20 22:01:53 +02:00
Young Lee 022c188873 fix(admin): truncate spam titles + speed up table view 2026-02-06 00:11:32 -08:00
Young Lee 56a8263f33 Enhance admin interface, security, and feed management with improved UX and authentication 2025-02-27 18:04:01 -08:00
Young Lee 8839aac24b Set up initial project and files 2025-02-27 14:51:38 -08:00