root/geo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

harden article generation reliability
Deployment Config CI / Deployment Config (push) Successful in 24s
Details
Backend CI / Backend (push) Successful in 14m12s
Details

Browse Source
This commit is contained in:
liangxu
2026-05-05 23:43:10 +08:00
parent 3c912949e4
commit 65e9b7e293
24 changed files with 1259 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
progress.md
+21
View File
@@ -769,3 +769,24 @@
- Verification passed:
- `pnpm --filter ops-web typecheck`
- `go test ./internal/ops/app ./internal/ops/transport ./internal/tenant/compliance ./cmd/ops-api`
## 2026-05-05T21:20:00+08:00 - Article generation reliability hardening
- Started Phase 44 after the user requested large-company style safeguards: structured logs, verifiable states, no swallowed errors, and automatic alerts.
- Confirmed the remaining weak points are in article body generation worker paths, not title/outline assist paths:
- Initial task `running` updates are ignored in template/custom/imitation generation execution.
- Failure cleanup logs status update errors but still swallows quota ledger/refund errors.
- Recovery requeue/finalization logs do not carry a consistent event/stage/anomaly schema yet.
- Scope locked to generation worker/app observability and state checks, avoiding unrelated deployment/compliance dirty files.
## 2026-05-05T23:45:00+08:00 - Article generation reliability verification
- Added structured article-generation task log context and counters for transitions, failures, cleanup failures, queue failures, recovery actions, state anomalies, and state-check failures.
- Added bounded scheduler-owned generation state consistency checks with configurable interval/timeout/batch/lookback and partial DB indexes; this is a low-frequency safety net, not a full-table polling loop.
- Added internal JSON/Prometheus metrics endpoints and Prometheus alert rules for cleanup failures, state anomalies, state-check failures, queue failures, failure spikes, and recovery finalization.
- Fixed compile/test follow-ups: matched the state-check DB interface to `pgx.Rows` and made the Prometheus label assertion order-independent.
- Verification passed:
- `cd server && go test ./internal/shared/config ./internal/tenant/app ./internal/tenant/transport ./internal/worker/generate ./cmd/scheduler ./cmd/worker-generate`
- `cd server && go test ./...`
- `git diff --check`
- `ruby -e 'require "yaml"; YAML.load_file("deploy/monitoring/article-generation-alerts.yml"); puts "ok"'`