feat(cache,worker): overhaul cache layer and add generation task lease recovery
Deployment Config CI / Deployment Config (push) Successful in 24s
Backend CI / Backend (push) Successful in 14m33s

- shared/cache: add Options/L1/async/metrics/prefix decorators, multi-key ops, Redis pool tuning, and JSON readthrough metrics
- worker-generate: claim tasks via DB lease + heartbeat, requeue stale queued tasks, expire dead leases with refund/cache invalidation
- tenant: version article cache keys so worker recovery invalidations propagate cleanly
- shared/config: expand Redis (pool/timeouts/TLS) and Generation (lease/recovery) configs with defaults

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-03 02:02:39 +08:00
parent bbfeabdaa5
commit c1ef717d17
37 changed files with 2502 additions and 93 deletions
@@ -0,0 +1,9 @@
DROP INDEX IF EXISTS idx_generation_tasks_queued_stale;
DROP INDEX IF EXISTS idx_generation_tasks_running_lease;
ALTER TABLE generation_tasks
DROP COLUMN IF EXISTS last_heartbeat_at,
DROP COLUMN IF EXISTS attempt_count,
DROP COLUMN IF EXISTS lease_expires_at,
DROP COLUMN IF EXISTS lease_owner,
DROP COLUMN IF EXISTS lease_token;
@@ -0,0 +1,14 @@
ALTER TABLE generation_tasks
ADD COLUMN lease_token VARCHAR(64),
ADD COLUMN lease_owner VARCHAR(128),
ADD COLUMN lease_expires_at TIMESTAMPTZ,
ADD COLUMN attempt_count INT NOT NULL DEFAULT 0,
ADD COLUMN last_heartbeat_at TIMESTAMPTZ;
CREATE INDEX idx_generation_tasks_running_lease
ON generation_tasks(lease_expires_at, id)
WHERE status = 'running';
CREATE INDEX idx_generation_tasks_queued_stale
ON generation_tasks(updated_at, id)
WHERE status = 'queued';