Commit Graph

167 Commits

Author SHA1 Message Date
root f6e553dcc3 feat(monitoring): add DeepSeek monitoring end-to-end
- desktop: new DeepSeek page adapter and monitor registry
- desktop: extract generic AI auth helpers into shared module and
  skip binding when challenge signals are present
- admin-web: render DeepSeek HTML answers with inline citation anchors
  and surface reference-number badges on source cards
- server: fall back to inline_links/source_panel_links and read
  text/siteName fields when extracting DeepSeek citations

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 22:54:56 +08:00
root 4f4f063200 refactor(monitoring): simplify brand dashboard and harden citation rendering
- drop brand_time_buckets and sparkline/coverage UI; dashboard now returns a single day
- compute mention rate from total actual samples instead of averaging per-platform rates
- canonicalize ai_platform_id across services and aggregate platform-overview snapshots
- strip mojibake and non-answer noise from callback payloads and question detail answers
- keep Kimi citations panel-only; still store raw search_results for audit
- render linked [n] inline citations for qwen/yuanbao answers with scroll-to-source behavior
- restrict立即采集 to today's business date with a clear i18n hint
- refresh content-citations into an ant-design table with truncation and hover styling

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 09:11:53 +08:00
root ce028c56bf feat(desktop): auto-recover in-progress desktop tasks on client lifecycle events
- reset stale in_progress tasks owned by a client on startup, offline, and lease expiry
- monitor tasks re-queue for another pick; publish tasks move to unknown for manual reconcile
- send startup=true flag on first heartbeat so the server can trigger recovery
- finalize the linked desktop task when a phase2 monitor callback arrives via desktop_tasks path
- short-circuit the desktop monitor attempt write after the callback already finalized it

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 09:11:23 +08:00
root 4142c53fa6 feat(monitoring): dispatch monitoring tasks to desktop via AMQP outbox
Replace SSE /desktop/events with priority AMQP dispatch for monitoring
runs, and add phase1/phase2 infrastructure so desktop workers can lease,
resume, report, skip, and cancel monitoring tasks over the existing
dispatch WebSocket.

- Add monitoring collect outbox worker and phase2 desktop task fields
- Add /desktop/monitoring/tasks/{lease,resume,result,skip,cancel} routes
- Introduce execution-devtools and network-observer on desktop runtime
- Refactor runtime-controller, LoginView, and doubao adapter for the new flow
- Add channelName column to tracking views

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 00:24:21 +08:00
root 9fa091c150 feat(desktop): push publish tasks via AMQP topic dispatch WebSocket
Introduce a desktop.task.dispatch topic exchange with per-client routing
keys. Tenant-api publishes a task_available frame keyed by target client
ID when a publish job is created; every instance binds its own transient
queue and forwards to the matching WebSocket (/api/desktop/dispatch) it
owns. Desktop-client now prefers this push channel and only falls back
to HTTP /lease polling when the socket is down. Also drop admin-web
monitoring-plugin remnants and scope the publish modal account list to
publish platforms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 19:46:59 +08:00
root 7abac1e9c4 feat: Implement Qwen adapter and enhance AI platform detection
- Added a new Qwen adapter to facilitate monitoring through Playwright, leveraging internal text/chat managers.
- Updated account detection logic to recognize Qwen sessions based on persisted cookies, improving binding reliability.
- Relaxed authentication requirements for monitor tasks, allowing anonymous execution for certain AI platforms.
- Enhanced the generic AI platform detection to include Qwen-specific logic, ensuring accurate session identification.
- Modified the monitoring dashboard to include runtime state indicating if the current user's desktop client is online.
- Updated various files including `account-binder.ts`, `runtime-controller.ts`, and `monitoring_service.go` to support new features and improvements.
2026-04-20 19:10:35 +08:00
root 55e1c2e74b feat(desktop): add monitor scheduler, Playwright CDP layer, and account health
Move monitor-task scheduling authority onto the client with a durable
file-backed queue that survives restart, drops stale cross-day tasks,
enforces per-platform serialism, and adapts global concurrency from
Electron process metrics. Publish tasks keep their existing FIFO.

Add a hidden Playwright CDP manager that attaches to Electron Chromium
on account session partitions, lets adapters opt into `executionMode:
"playwright"`, and leaves the existing hidden WebContentsView path in
place for current adapters.

Introduce an account-health subsystem with silent probes, projected
health/auth states, and IPC invalidation events so the renderer can
show accurate auth/probe status and verification timestamps.

Server-side, derive and forward title/business_date/scheduler_group_key/
question_text metadata on desktop task events so the local scheduler
can defer same-question fan-out before leasing.
2026-04-20 17:16:15 +08:00
root 09295d11a1 feat(monitoring): decouple AI platforms from media_platforms and expand catalog to 6
Add ai_platforms table + shared catalog (yuanbao/kimi/wenxin/deepseek/doubao/qwen),
drop FKs from platform_accounts and desktop_tasks to media_platforms, and wire
target_account_id + platform into DesktopTaskEvent so the desktop runtime no
longer has to infer them from local state. Desktop client gains generic AI page
detection for binding, drops stale-business-date monitor tasks at the edge, and
refactors AccountsView/AiPlatformsView around the shared catalog. Admin tracking
view surfaces per-platform sampling status cards.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 15:40:18 +08:00
root 5ff2e2e74c refactor(tenant): drop legacy plugin_installations, migrate monitoring to desktop_clients
Hard cutover from the browser-extension plugin flow to desktop clients:
remove plugin_installations/plugin_sessions tables and related service,
handler, router, and generated model code; migrate monitoring quotas
and collector types to desktop_clients (UUID primary_client_id);
recreate platform_access_snapshots keyed by client_id; update dev-seed
and callback types accordingly; mark legacy design docs as historical.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 13:52:35 +08:00
root dc68ad044c refactor(publish): collapse pending_review/unknown task states into failed
Treat ambiguous terminal states as failures end-to-end: server
normalizes unknown desktop task completions and views to failed, drops
the pending_review aggregate branch, and remaps pending_review to
failed in publish record conversion. admin-web removes the
pending_review status option from article list filters, pushes
pending_review through the failed tone, and folds lingering
publishing/pending entries under the publishing bucket. desktop-client
relabels unknown as 发送失败 in PublishManagement/TasksView, rewrites
the scaffold adapter result to a proper failed error envelope, and
sanitises legacy scaffold-only error messages to a user-friendly note.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 11:44:47 +08:00
root a46ed4b9f6 fix(tenant): resolve desktop retry article id via publish record/batch
Fall back to looking up article_id from publish_records or publish_batches
when the desktop task payload lacks article_id at the top level. Covers
payloads that only carry content_ref.{article_id,id}, publish_record_id,
or publish_batch_id, preventing spurious 40096 errors when retrying
older desktop tasks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 11:07:48 +08:00
root a617d39a4a feat(desktop): drop parked review flow, add publish management
SaaS 侧人工审核后才创建 publish job,desktop 不再做二次审核:移除
manual/waiting_user/parked/from_parked 状态机与 LeaseFromParked 查询,
desktop client 只执行发布并新增"发布管理"页(待发布队列 / 历史 / 再次
发送)。同时抽离 @geo/publisher-platforms 共享适配器包、新增 Redis-based
desktop_presence 与 publish_record_support,刷新 admin-web 发布弹窗与
媒体库;plan A / spec 文档同步口径。
2026-04-20 09:52:48 +08:00
root b16e9f0bd1 feat(desktop): implement workspace foundation + desktop-client skeleton
Plan 0 (workspaces): add workspaces + workspace_memberships schema, extend
JWT/Actor/claims with primary_workspace_id, seed default workspace per tenant,
thread workspace_id through tenant monitoring quota.

Plan A (desktop skeleton): new Electron app (apps/desktop-client) with main/
preload/renderer, shared Vue component package (packages/ui-shared), and server
surface — desktop client registration + token rotation + heartbeat, SSE task
event stream, desktop accounts/tasks/content handlers, publish job endpoint,
and supporting repositories, services, sqlc queries, and migrations.

Hard cutover per plan: remove browser-extension monitoring callback endpoints,
stub legacy media API in admin-web, and delete monitoring_callback_handler.go.
2026-04-19 14:18:20 +08:00
root 63667ed2d1 feat(membership): enforce tenant subscription plans with blocked UI
Add configurable membership plans (free/plus/pro) synced to DB on boot, a
subscription guard middleware that blocks tenant endpoints on expired or
missing plans, and a MembershipBlockedView that surfaces the reason so
the admin can contact the tenant owner. Quota and brand-library reads now
honor the active plan's policy JSON and expired subscriptions.
2026-04-18 20:56:05 +08:00
root 6e77f72c53 feat(kol): return to source page after generation and poll while busy
- Add a ?source=workspace|templates query param when opening the
  generate page so the back button and the post-submit redirect return
  to the originating page; invalidate articles + workspace caches on
  success so the new article shows up immediately.
- Poll recent articles every 10s in WorkspaceView and TemplatesView
  while any article is in queued/pending/generating/running, and stop
  the timer once nothing is active.
- Mark the new article as generating inside the generation Submit
  transaction so the client sees the right status as soon as the task
  is queued.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:17:17 +08:00
root fe4a4ffeaa feat(kol): enrich assist generate with URL references and template prompt
- Add kolAssistURLResolver that fetches reference articles via the Ark
  URL extractor when the user's description contains http(s) links, and
  merges up to kolAssistReferenceContentMaxRunes of extracted content
  into the generate description.
- Rewrite the generate user prompt into a strict template-authoring
  brief: output only the final prompt template in Chinese, parameterize
  variable info as {{placeholder}}, require at least 3 placeholders
  when references exist, and cover goal/topic/title/structure/style/
  interaction/constraints.
- Wire LLMConfig and logger through NewKolAssistService and the
  handler.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:17:07 +08:00
root 052bf38dc4 feat(observability): enrich request logs with auth and knowledge context
- Collect per-request auth diagnostics (header presence, scheme, token
  fingerprint, failure stage/reason, subject, expiry) in the auth
  middleware and expose them through RequestLogContextFromGin so the
  shared logger middleware can emit them.
- Add route, query, user-agent, and referer fields to the request log.
- Log knowledge resolve outcome per stage (no_search_points,
  no_candidate_text, no_active_candidates, rerank_failed_fallback,
  resolved) with candidate/selected snippet previews and precise facts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:16:57 +08:00
root 4bbce5f083 feat(kol): add SSE streaming for prompt assist
- Extract assist runtime (prompt building, response parsing, finalize)
  from the worker into kol_assist_runtime so the sync streaming path
  and the async task path share the same logic.
- Add POST /kol/manage/assist/stream as a Server-Sent Events endpoint
  emitting start/delta/completed/error events, and wire it to a
  streamAssist helper on kolManageApi that reports ApiClientError with
  the server's error envelope.
- Stream generated content live in KolPromptEditor and KolGenerateView,
  switching KolPromptEditArea to a boolean aiBusy flag and refining the
  generator page layout.
- Add KolAssistStreamEvent to shared types.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 16:17:11 +08:00
root 79c65c1da7 feat(kol): expand variable schema and polish KOL UI
- Add length/range limits to KOL variable definitions: max_length and
  default_value for input/textarea, min_value/max_value/default_number
  for number. Backend validates and normalizes; frontend renders limit
  inputs in KolVariableConfig and applies limits at runtime in
  KolDynamicForm.
- Sort workspace cards by most recent prompt/package update, falling
  back to granted_at. Adds updated_at to KolWorkspaceCard.
- Polish TemplatesView, WorkspaceView, and KOL package management UI;
  route create/update/publish/archive/delete toasts through i18n and
  expand package form copy and status labels.
- Remove stale KnowledgeView.vue.patch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 15:01:40 +08:00
root d2922478ed feat(kol): expose PUT /kol/manage/profile and POST /kol/manage/profile/avatar 2026-04-18 14:06:20 +08:00
root 1393b25419 feat(kol): add avatar upload to KolProfileService 2026-04-18 14:05:09 +08:00
root 0e3515530f feat(kol): add KolProfileService.UpdateProfile 2026-04-18 14:03:49 +08:00
root 4bfa4242ef feat(kol): add KOL profile field validators 2026-04-18 14:03:07 +08:00
root b2605abd6a feat: Enhance Kol Generation Service with web search and knowledge group support
- Added `EnableWebSearch` and `KnowledgeGroupIDs` fields to `KolGenerationSubmitRequest`.
- Updated `Submit` method in `KolGenerationService` to handle new request fields.
- Integrated web search and knowledge group validation based on card configuration.
- Introduced caching mechanisms in `KolPackageService`, `KolPromptService`, and `KolSubscriptionAdminService` to improve performance.
- Implemented knowledge context resolution in `KolGenerationWorker` to enrich prompts with relevant knowledge snippets.
- Added utility functions for handling card configuration in both backend and frontend.
- Created tests for knowledge extraction and rendering to ensure accuracy and reliability.
2026-04-18 13:47:32 +08:00
root 3ef0807456 fix: Enhance Kol Variable Rendering and Management
- Updated the regex for placeholder matching to support more flexible key formats.
- Refactored variable storage to allow both ID and key lookups in rendering.
- Improved schema validation to check for duplicate keys and enforce non-empty keys.
- Added new utility functions for variable value lookup and display name generation.
- Enhanced tests to cover new key-based variable scenarios.
- Refactored KolPrompt repository to simplify prompt storage and management, including the removal of obsolete revision handling.
- Introduced new API endpoints for saving, activating, and archiving prompts.
- Added new utility functions for handling Kol placeholders and platform options in the admin web.
- Implemented database migrations to simplify prompt storage structure and ensure data integrity.
2026-04-18 00:47:57 +08:00
root b6bd4f02fb feat(kol): workspace KOL cards + article source type 2026-04-17 15:19:11 +08:00
root 6e95e00907 feat(kol): dashboard service + API 2026-04-17 15:05:09 +08:00
root 97c18cdbef feat(kol): generation + schema HTTP endpoints 2026-04-17 14:52:16 +08:00
root 019410fffe feat(kol): generation service + worker branch 2026-04-17 14:52:08 +08:00
root 26bc8a3749 feat(kol): repo support for cross-tenant revision lookup 2026-04-17 14:51:54 +08:00
root 2a2489ce5c feat(kol): marketplace + admin subscription HTTP endpoints 2026-04-17 14:23:39 +08:00
root 99fc6c1af4 feat(kol): marketplace + subscription admin services + publish fan-out 2026-04-17 14:23:32 +08:00
root df334c1e26 feat(kol): marketplace and admin subscription repo queries 2026-04-17 14:23:25 +08:00
root d5f7cdfc5f feat(kol): AI assist HTTP endpoints 2026-04-17 13:52:08 +08:00
root d55f1276f3 feat(kol): AI assist service + worker 2026-04-17 13:52:01 +08:00
root a6ded27c1d feat(kol): KOL workspace HTTP handlers + route registration 2026-04-17 09:19:37 +08:00
root 6029b0b169 feat(kol): KOL workspace services (profile/package/prompt/asset/renderer) 2026-04-17 09:19:30 +08:00
root 40519545c6 feat(kol): expose kol_profile on /api/auth/me and add seed script 2026-04-17 09:02:33 +08:00
root 70725193eb fix(kol): Phase 1 review — ACL lineage, expiry filters, tighter tenant scope
Addresses findings from consolidated Phase 1 code review:

- Add composite unique indexes + composite FKs so subscription_prompts and
  packages cannot reference mismatched (tenant_id, package_id, prompt_id)
  tuples, enforcing ACL lineage at the schema level.
- Add status CHECK constraints on all KOL tables as defense-in-depth.
- ListActiveSubscribersForPackage: drop meaningless tenant_id IS NOT NULL,
  add end_at expiry filter so fan-out skips expired subscribers.
- ApproveKolSubscription: add status='pending' guard to prevent double-approval.
- ListSubscriptionPromptsByTenant: join subscription/prompt/package/profile
  lifecycle so hidden/archived content no longer leaks; surface
  kol_display_name for UI.
- ListPublishedKolPackages: exclude expired subscribers from marketplace count.
- KolDashboardTrend: rewrite as CTE with day-series, returning both daily
  usage and new subscription counts.
- Remove cosmetic tenant_id IS NOT NULL from dashboard overview queries
  (the column is NOT NULL by schema).
- check_tenant_scope.sh: require tenant_id = sqlc.(n?)arg filter, reject
  tenant_id IS NOT NULL as a fake scope, maintain explicit exemption list.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 08:34:12 +08:00
root 3d4842e983 feat(kol): repository layer for all KOL tables 2026-04-17 08:17:22 +08:00
root b78664c7d0 feat(kol): add sqlc queries for all KOL tables 2026-04-17 08:10:45 +08:00
root ed10fd82e6 fix: add image asset ID extraction and filtering functions for markdown content 2026-04-16 21:12:35 +08:00
root 41f3060e00 feat: add brand library management features
- Introduced BrandLibrarySummary type to encapsulate brand library limits and usage.
- Updated Brand interface to include keyword_count and question_count fields.
- Implemented brand library limits in the backend, including max brands, keywords, and questions per keyword.
- Added API endpoint to retrieve brand library summary.
- Enhanced BrandsView.vue to display brand library usage and limits.
- Implemented computed properties to manage brand, keyword, and question limits in the UI.
- Updated mutation handlers to invalidate relevant queries upon creating/updating brands, keywords, and questions.
- Added visual indicators for brand, keyword, and question limits in the UI.
- Enhanced error handling for exceeding brand and keyword limits during creation.
2026-04-16 21:01:40 +08:00
root 27389164b0 feat: add image management functionality
- Implemented image folder repository with CRUD operations.
- Added image reference repository for managing image references to articles.
- Created image repository for handling image assets, including listing, inserting, updating, and deleting images.
- Introduced image usage repository to track storage usage and quotas for tenants.
- Added SQL queries for image assets, folders, references, and usage.
- Developed image handler for HTTP endpoints to manage images and folders.
- Created database migration scripts for image-related tables and structures.
2026-04-16 20:40:41 +08:00
root 1538a12042 feat(cache): add read-through cache layer across all app services
Introduce a generic read-through caching infrastructure and wire it into
all major tenant app services to reduce database load on hot read paths.

Key changes:
- Add `DeletePrefix` to Cache interface with memory (prefix scan) and
  Redis (SCAN + DEL) implementations
- New `readthrough.go`: generic `LoadJSON` / `LoadJSONWithEmpty` helpers
  backed by singleflight to prevent cache stampedes; supports jittered TTL
- New `cache_support.go`: centralized cache key builders and invalidation
  helpers for all entities (workspace, brand, prompt rules, schedule tasks,
  articles)
- Wire optional cache into ArticleService, BrandService, WorkspaceService,
  PromptRuleService, ScheduleTaskService, TemplateService, MediaService,
  PromptGenerateService via `WithCache()` builder pattern
- ScheduleDispatchWorker invalidates schedule task cache after dispatching
- ArticleService gains a new `Detail` endpoint with empty-result caching
- Update cmd entrypoints and transport handlers to propagate cache
2026-04-15 16:11:05 +08:00
root 6869dc2fa6 feat(schedule): enhance schedule tasks with web search and batch generation support
Add enable_web_search and generate_count columns to schedule_tasks table.
Refactor schedule task service to publish generation jobs to RabbitMQ
instead of inline execution, supporting batch dispatch and backpressure
awareness via scheduler process.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:20:54 +08:00
root ce13331e26 feat(generation): migrate article generation and template-assist to RabbitMQ queues
Replace in-process generation with queue-based async execution via
RabbitMQ. Add generation task runtime for lifecycle management, article
generation payload/queue abstractions, and template-assist queue publisher.
Refactor prompt generation service to support batch generation with
per-item error tracking. Update stream hub to bridge queue events to SSE.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:20:26 +08:00
root e8f48c6d37 refactor(monitoring): consolidate workers and extract to scheduler process
Remove standalone lease-recovery, received-inspection, and result-recovery
workers from tenant-api (now handled by scheduler process). Add configurable
concurrency to ingest and projection-rebuild workers via MonitoringWorkers
runtime config. Extract shared runtime types for worker configuration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:19:57 +08:00
root 3f5b5c1e01 feat(infra): add scheduler and worker-generate standalone processes
Extract monitoring recovery/inspection workers and schedule dispatch
into a dedicated scheduler process. Add worker-generate process for
article generation and template-assist queue consumption. Introduce
shared/schedule runtime for cron-based worker lifecycle management.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:19:21 +08:00
root b46cfaaa61 feat(infra): add RabbitMQ generation/template-assist queue configs and scheduler settings
Introduce generation task and template-assist queue declarations with
DLX/DLQ support in RabbitMQ client. Add scheduler dispatch and
monitoring worker concurrency configs. Include publish channel pool
for high-throughput message publishing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:18:55 +08:00