Commit Graph

17 Commits

Author SHA1 Message Date
root 842782b3dd feat(scheduler): random run-window scheduling and deduped manual triggers
Support a random daily run window (random_window_start/end) where the
next auto-run time is derived from a stable per-job hash, avoiding
thundering-herd starts while keeping one run per day. Honors the last
auto-run on initial scheduling so restarts don't double-fire.

Manual triggers can opt into dedupe (dedupe_manual_triggers): an
advisory-locked path collapses concurrent requests onto an existing
pending/running job instead of queueing duplicates. Scheduler error
messages are localized to Chinese.

Register the media_supply_cache_sync job (dry-run aware) and add the
ops migration that seeds it.
2026-06-02 14:50:25 +08:00
root fa51a3455f fix(publish): prevent stuck publish queue and duplicate posting
Desktop Client Build / Resolve Build Metadata (push) Successful in 41s
Backend CI / Backend (push) Failing after 7m14s
Desktop Client Build / Build Desktop Client (push) Successful in 23m46s
Desktop Client Build / Publish Client Artifacts to NAS (push) Successful in 56s
The desktop publish queue could stall for a long time and end users assumed
the software was broken. Root cause: a hung adapter held its execution slot
with no wall-clock timeout while auto-renewing its lease forever, so the
server never reclaimed it and every queued task behind it stayed 等待发布.
Auto-recovery also risked silently re-posting a non-idempotent article.

Client (Electron):
- per-task wall-clock deadline + abort; progress-gated lease renewal that
  stops and aborts a stalled task instead of renewing it forever
- decouple the concurrency cap from CDP-induced CPU/memory pressure
  (admission gate instead of self-throttling collapse); 15s watchdog pump
- all adapter network I/O now has fetch timeouts and honors context.signal;
  bounded image-upload concurrency with per-image timeout
- surface live adapter progress, elapsed time, queue position and a
  working-vs-queued distinction in the publish view

Server (tenant-api):
- publish lease-recovery worker (every 3m) + supporting index: reclaim
  expired in_progress publish leases, requeue (<3 attempts) or terminal-fail
- 3-minute lease TTL with client-presence-gated extension; max 3 attempts

Idempotency (production-grade core):
- durable publish_submit_started_at marker, set before the irreversible
  platform submit POST; recovery and abort route a maybe-submitted task to
  unknown (manual reconcile, kept in the dedup set) instead of re-posting
- desktop UI requires explicit confirmation before retrying a possibly-
  already-published task

Verified: go build/vet/test (incl. resolvePublishRecoveryOutcome), vue-tsc,
vitest 141/141, gofmt all clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 19:52:17 +08:00
root c28c4f1419 feat(schedule): support KOL subscription targets and random time windows
Frontend CI / Frontend (push) Successful in 3m37s
Backend CI / Backend (push) Failing after 6m47s
Extend schedule tasks to dispatch KOL subscription prompts in addition
to prompt rules, add daily/weekly scheduling with fixed or random time
windows, and track dispatch outcomes (last run, status, consecutive
failures).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:51:45 +08:00
root 49b1ccc961 Reduce scheduler polling noise
Backend CI / Backend (push) Successful in 14m44s
2026-05-21 00:56:43 +08:00
root 509f9e0c71 Add AI point usage cycle cleanup
Frontend CI / Frontend (push) Successful in 2m52s
Backend CI / Backend (push) Successful in 14m11s
2026-05-20 21:59:36 +08:00
root dd082e2ed1 feat: scope articles by brand 2026-05-20 15:37:25 +08:00
root 5fb9d0b0dd perf(scheduler): harden jobs for scalable execution 2026-05-20 12:19:27 +08:00
root 4b09a34748 refactor(knowledge): migrate deleted-item cleanup from polling scheduler to event-driven worker
Replaces the scheduler-polled KnowledgeDeletedCleanupWorker with an
event-driven KnowledgeDeletedCleanupEventWorker in tenant-api; cleanup
events are now enqueued transactionally at delete time instead of swept
periodically. Also fixes statement_timeout parameterization in
MonitoringRetentionWorker and propagates errors from cleanupDeletedKnowledgeItem.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-20 11:22:37 +08:00
root e82ae56236 feat: add ops scheduler control center 2026-05-20 10:46:49 +08:00
root cbbed4b42c fix(generation): repair articles stuck in generating when latest task is terminal
The state-consistency worker now detects articles still flagged as
generating whose latest generation_task is already failed or completed
(with a version) and aligns the article status to the terminal task,
invalidating article caches on the way out. Adds supporting partial
indexes and switches UpdateGenerationTaskStatus to COALESCE started_at
and completed_at so partial status updates don't wipe the timestamps.
2026-05-11 11:11:31 +08:00
root 65e9b7e293 harden article generation reliability
Deployment Config CI / Deployment Config (push) Successful in 24s
Backend CI / Backend (push) Successful in 14m12s
2026-05-05 23:43:10 +08:00
root 618399f86d feat(server): hot-reload config store and reloadable infra clients
- Replace single Load() with a watching Store that re-reads config files
  (and config.local.yaml) and fans out a ReloadEvent with a per-field diff
  so consumers can decide whether the change is hot-applicable or requires
  a process restart.
- Wrap llm, retrieval, vector store, and object storage clients in
  Reloadable* shells so the bootstrap can swap their underlying impls when
  config changes without re-instantiating handlers.
- Make jwt.Manager and ops TokenIssuer mutable under a lock so secrets and
  TTLs can be rotated live; thread default plan code through a setter on
  the ops AdminUserService.
- Wire ConfigStore through bootstrap and every cmd/main.go, scheduler /
  worker / tenant-api / ops-api start the watcher; services and handlers
  take a config.Provider so they always read current values for things
  like generation.stream_enabled, scheduler dispatch, retrieval, etc.
- Switch shared/config decoding off viper to a Kratos-derived runtime
  package so env placeholders (\${VAR:default}) resolve consistently and
  the same source machinery powers both the loader and the watcher.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:01:23 +08:00
root 6e6e19cccb feat(schedule-tasks): support auto-publish via media accounts
Replace the legacy target_platform string on schedule tasks with a
workspace-aware auto-publish payload (auto_publish + publish_account_ids
JSONB + cover_asset_url + cover_image_asset_id) and migrate the schema,
sqlc queries, generated models, domain struct, ScheduleTaskService DTOs,
and dispatch worker to round-trip the new fields. PromptRuleGeneration
gains a WithPublishJobService hook so executeGeneration can enqueue an
auto-publish job once the article is ready, and worker-generate wires
the publish-job service in. On the admin-web side, extract
PublishArticleModal's account-card builders into a shared
publish-account-cards module, rebuild GenerateTaskDrawer's schedule
mode around account selection plus a CoverPickerModal, and surface the
new "auto publish" column on ScheduleTaskTab. Shared types and i18n
strings cover the new fields.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 01:32:19 +08:00
root 7605890a14 refactor(server/monitoring): route monitor desktop tasks by account presence
Daily monitoring tasks were pinned to a single primary client. If that
client went offline the task wedged even when another desktop client of
the same workspace was online and bound to the same account. Drop the
primary-client constraint on the materialized collect rows, and instead
pick a target per-platform from live account/client presence in Redis,
falling back to the DB client_id when the desktop client is still flagged
online. Desktop task lease for kind=monitor now matches by account
ownership in addition to target_client_id, so any client owning the
account can drain the queue.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 09:14:24 +08:00
root c0253c98f9 feat(scheduler): expose metrics HTTP and add graceful shutdown
Scheduler now runs a token-protected HTTP server on scheduler.http_port
(default 8081, -1 disables) that exposes Prometheus /metrics and the
daily-task JSON snapshot endpoint. Workers gain a blocking Run(ctx) and
are supervised by a WaitGroup: on SIGINT/SIGTERM the metrics server
shuts down first, worker ticks stop scheduling but ongoing runOnce
calls keep their own context and finish, and the process waits up to
60s before exiting. Adds scheduler.http_host/http_port/internal_metrics_token
to config with SCHEDULER_* env overrides and exposes port 8081 in the
Docker image.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 22:20:43 +08:00
root 1538a12042 feat(cache): add read-through cache layer across all app services
Introduce a generic read-through caching infrastructure and wire it into
all major tenant app services to reduce database load on hot read paths.

Key changes:
- Add `DeletePrefix` to Cache interface with memory (prefix scan) and
  Redis (SCAN + DEL) implementations
- New `readthrough.go`: generic `LoadJSON` / `LoadJSONWithEmpty` helpers
  backed by singleflight to prevent cache stampedes; supports jittered TTL
- New `cache_support.go`: centralized cache key builders and invalidation
  helpers for all entities (workspace, brand, prompt rules, schedule tasks,
  articles)
- Wire optional cache into ArticleService, BrandService, WorkspaceService,
  PromptRuleService, ScheduleTaskService, TemplateService, MediaService,
  PromptGenerateService via `WithCache()` builder pattern
- ScheduleDispatchWorker invalidates schedule task cache after dispatching
- ArticleService gains a new `Detail` endpoint with empty-result caching
- Update cmd entrypoints and transport handlers to propagate cache
2026-04-15 16:11:05 +08:00
root 3f5b5c1e01 feat(infra): add scheduler and worker-generate standalone processes
Extract monitoring recovery/inspection workers and schedule dispatch
into a dedicated scheduler process. Add worker-generate process for
article generation and template-assist queue consumption. Introduce
shared/schedule runtime for cron-based worker lifecycle management.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:19:21 +08:00