Files
geo/task_plan.md
T

353 lines
26 KiB
Markdown
Raw Normal View History

# Task Plan: GEO admin-web backend completion and frontend foundation
## Goal
Continue the desktop AI monitoring implementation by first adding a client-side local scheduler with durable queueing and adaptive execution policy, then adding a reusable hidden Playwright CDP execution layer that attaches to Electron Chromium and reuses existing account partitions.
## Current Phase
Phase 37
## Phases
### Phase 1: Progress Verification
- [x] Understand user intent
- [x] Identify constraints and requirements
- [x] Compare repository state with the 9-step plan
- [x] Document findings in findings.md
- **Status:** complete
### Phase 2: Gap Analysis and Scope Lock
- [x] Confirm the earliest incomplete step
- [x] Read the relevant sections of the design doc
- [x] Define the exact implementation slice for this turn
- **Status:** complete
### Phase 3: Implementation
- [x] Implement the missing backend code/config/schema/tests
- [x] Keep changes aligned with the design doc and existing structure
- [x] Update findings/progress after meaningful milestones
- **Status:** complete
### Phase 4: Testing & Verification
- [x] Run targeted commands and tests
- [x] Confirm the step acceptance criteria or record gaps
- [x] Fix issues found during verification
- **Status:** complete
### Phase 5: Delivery
- [x] Summarize step completion status with evidence
- [x] Highlight remaining gaps and risks
- [x] Deliver next-step outcome to the user
- **Status:** complete
### Phase 6: Frontend Scope Recovery
- [x] Recover the interrupted task context from planning files and Claude artifacts
- [x] Confirm whether frontend work already exists in the repository
- [x] Lock the smallest useful frontend slice for this turn
- **Status:** complete
### Phase 7: Frontend Foundation
- [x] Scaffold the pnpm workspace and shared frontend packages
- [x] Create the `apps/admin-web` Vite + Vue 3 application skeleton
- [x] Wire environment/config conventions for local API access
- **Status:** complete
### Phase 8: Admin-Web Initial Features
- [x] Implement auth state, login flow, and route guarding
- [x] Implement the main application shell and navigation
- [x] Implement the workspace dashboard with live backend data
- **Status:** complete
### Phase 9: Frontend Verification & Delivery
- [x] Run install/build checks for the new frontend workspace
- [x] Record remaining frontend gaps and follow-up slices
- [x] Deliver the resumed-task status to the user
- **Status:** complete
### Phase 10: Interface Coverage Audit & Visual Mapping
- [x] Inventory all currently exposed tenant-facing backend interfaces
- [x] Map interface groups to real admin-web pages
- [x] Recover the reference layout structure from `docs/refer` screenshots, including positional relationships
- **Status:** complete
### Phase 11: Admin-Web Full Interface Delivery
- [ ] Implement templates/articles frontend coverage against current backend APIs
- [ ] Implement brands/keywords/questions/competitors frontend coverage against current backend APIs
- [ ] Realign the shell and page layouts to the reference screenshot structure
- [x] Restore `pnpm dev:admin` and `pnpm typecheck:admin` after the frontend refactor introduced compile/runtime regressions
- [x] Establish the i18n and style hygiene baseline for the pages touched in this turn
- **Status:** in_progress
### Phase 12: Verification & Delivery
- [ ] Run frontend type/build verification
- [ ] Spot-check the main interaction flows against the live backend
- [ ] Update planning files and deliver the completed scope
- **Status:** pending
### Phase 13: Prompt Centralization
- [x] Inventory all currently used hard-coded prompts in runtime and seed paths
- [x] Extract prompt definitions into a dedicated shared location
- [x] Refactor existing callers to read from the centralized prompt layer
- **Status:** complete
### Phase 14: Prompt Verification & Delivery
- [x] Run targeted compile/test verification for the prompt refactor
- [x] Update planning files with the final prompt inventory and changed files
- [x] Deliver the extraction result and any remaining gaps
- **Status:** complete
### Phase 15: Desktop Scheduler Design & Integration
- [x] Audit the current desktop runtime queue/task execution path and lock the scheduler constraints
- [x] Implement a durable local monitor-task queue/cache with stale-task cleanup
- [x] Implement platform-level mutual exclusion and adaptive global concurrency policy
- [x] Integrate the scheduler into runtime leasing/execution without regressing publish tasks
- **Status:** complete
### Phase 16: Hidden Playwright CDP Execution Layer
- [x] Add a reusable hidden-browser manager that attaches Playwright to Electron Chromium over CDP
- [x] Reuse desktop account partition/session state for hidden pages
- [x] Expose the hidden Playwright context/page lifecycle to future monitor adapters
- [x] Keep the current hidden-view path working for existing adapters during the transition
- **Status:** complete
### Phase 17: Verification & Delivery
- [x] Run targeted desktop type/build verification
- [ ] Inspect runtime snapshot output for new scheduler/CDP state
- [ ] Summarize what is production-ready versus still scaffolded
- **Status:** in_progress
### Phase 18: Qwen Adapter & Auth Relaxation
- [x] Port the browser-extension `qwen` monitor logic into a desktop Playwright adapter
- [x] Register the new adapter in the desktop runtime monitor execution path
- [x] Relax Qwen binding completion so the auth window can close once a real session footprint is present
- [x] Allow anonymous-capable AI monitor platforms to execute without preflight auth blocking
- **Status:** complete
### Phase 19: Qwen Verification & Delivery
- [x] Run targeted desktop type/build verification after the Qwen adapter and auth updates
- [x] Update planning files with the Qwen implementation details and access-policy decision
- [x] Deliver the completed slice and remaining validation gaps
- **Status:** complete
### Phase 20: Tracking Collect-Now Repair
- [x] Audit the `admin-web` tracking collect-now flow against the current desktop monitoring architecture
- [x] Remove the obsolete browser-plugin kickoff dependency from `TrackingView`
- [x] Enforce the current logged-in user's desktop client online requirement in both frontend gating and backend collect-now validation
- [x] Run targeted frontend/backend verification for the repaired flow
- **Status:** complete
### Phase 21: Foreground Publish Admission
- [x] Add a lightweight in-memory publish scheduler with per-platform locks and cooldowns
- [x] Refactor desktop runtime admission so publish is foreground and monitor is background
- [x] Add hardware/runtime adaptive total concurrency with a publish-reserved slot
- [x] Run targeted desktop verification
- **Status:** complete
### Phase 22: Monitoring Daily Collection Review
- [x] Trace the scheduler entrypoints that could generate monitoring collect tasks
- [x] Inspect current database evidence for automatic vs manual monitoring tasks
- [x] Verify desktop dispatch/result/detail query behavior for existing monitoring data
- [x] Run targeted compile/tests where practical
- [x] Deliver findings on whether the daily scheduled collection path is working
- **Status:** complete
### Phase 23: Monitoring Daily Task Generation Repair
- [x] Add a scheduler-owned daily monitoring task worker
- [x] Materialize daily tasks after midnight with stable jitter and per-run/per-plan/per-brand caps
- [x] Keep desktop-offline tasks pending instead of falling back to backend legacy execution
- [x] Claim due desktop dispatch batches with `FOR UPDATE SKIP LOCKED` and retry cooldown
- [x] Run targeted Go tests and SQL syntax verification
- **Status:** complete
### Phase 24: Monitoring Daily Observability Hardening
- [x] Keep transient tenant/brand errors from aborting the entire worker tick
- [x] Add daily worker counters/gauges for runs, failures, skips, task stages, and last-run state
- [x] Expose scheduler-local metrics over HTTP for Prometheus scraping and JSON inspection
- [x] Align daily hard-cap accounting with durable materialized task counts and actual inserts
- [x] Restore product-policy enforcement for subscription limits, brand-library question limits, and user-authorized AI platforms
- [x] Run targeted Go verification
- **Status:** complete
### Phase 25: Heartbeat Primary Stability Hardening
- [x] Replace heartbeat primary selection by latest `last_seen_at` with a sticky per-workspace lease
- [x] Keep the incumbent desktop client primary while its lease is live, and switch only after expiry/revocation
- [x] Make daily monitoring plans prefer the sticky primary client before fallback selection
- [x] Add desktop-client indexes for tenant/workspace heartbeat and online-client lookup paths
- [x] Run targeted and full Go verification
- **Status:** complete
### Phase 26: Scheduler Metrics HTTP Hardening
- [x] Stop binding scheduler metrics to all interfaces by default
- [x] Require an internal metrics token for `/metrics` and internal JSON metrics
- [x] Preserve explicit HTTP disablement with negative `http_port`
- [x] Replace `Engine.Run`/`Fatalf` with `http.Server` and graceful shutdown
- [x] Add targeted config and scheduler HTTP tests
- **Status:** complete
### Phase 27: Scheduler Worker Graceful Shutdown
- [x] Add blocking `Run(ctx)` entrypoints for scheduler workers while preserving `Start(ctx)`
- [x] Start scheduler workers under a process-level `sync.WaitGroup`
- [x] Stop new ticks on shutdown but allow the current tick to finish under its own timeout
- [x] Wait up to a bounded grace period before process exit and DB/MQ cleanup
- [x] Add targeted scheduler lifecycle tests
- **Status:** complete
### Phase 28: Monitoring Platform Authorization UI Contract
- [x] Expose explicit monitoring platform authorization status in dashboard and question-detail responses
- [x] Update admin-web dashboard and question detail pages to render no-desktop/no-authorized-platform states instead of blank platform surfaces
- [x] Disable collect-now for no-authorized and selected-unauthorized platform states
- [x] Run backend, frontend build, diff, and browser no-authorized-platform regression checks
- **Status:** complete
### Phase 29: Monitoring Six-Platform Display Contract
- [x] Restore projection platform rows to the canonical 6-platform display catalog
- [x] Make dashboard platform breakdown use the 6-platform display catalog instead of authorized execution platforms
- [x] Keep collect-now authorization checks based on explicit authorized platform IDs
- [x] Verify dashboard still displays all 6 platforms when no platform is authorized
- **Status:** complete
### Phase 30: Workspace Scope And Daily Config Visibility
- [x] Add request-scoped current workspace context with validated `X-Workspace-ID` support
- [x] Update monitoring quota/runtime/access-state/collect-now paths to use current workspace instead of actor primary workspace
- [x] Document latest-registered client ordering semantics
- [x] Make daily monitoring platform JSON decode errors visible through logs and plan-failure metrics
- [x] Run targeted backend tests, full `go test ./...`, and `git diff --check`
- **Status:** complete
### Phase 31: Heartbeat Primary Lease Scalability
- [x] Remove per-heartbeat primary lease write transaction and row-lock read
- [x] Convert primary lease handling to read-mostly plus conditional insert/update paths
- [x] Add primary-hit, non-primary-miss, renew/claim, contention, error, and duration metrics
- [x] Expose heartbeat primary lease metrics through token-protected tenant-api internal endpoints
- [x] Run targeted backend tests, full `go test ./...`, and `git diff --check`
- **Status:** complete
### Phase 32: Monitoring Heartbeat Snapshot Scalability
- [x] Avoid opening a monitoring DB transaction for stable heartbeat snapshot reports
- [x] Refresh unchanged platform access snapshots at a bounded low frequency
- [x] Reconcile inaccessible-platform pending tasks only when pending rows exist
- [x] Add heartbeat snapshot write/skip/reconcile/projection metrics
- [x] Run targeted backend tests, full `go test ./...`, and `git diff --check`
- **Status:** complete
### Phase 33: Prometheus Metrics Exposition Hardening
- [x] Replace hand-written Prometheus text generation with `prometheus/client_golang`
- [x] Register Go runtime and process collectors on scheduler `/metrics`
- [x] Keep internal JSON metrics endpoints unchanged while serving Prometheus through `promhttp`
- [x] Normalize carriage returns in label values before collector emission
- [x] Run targeted backend tests, full `go test ./...`, and `git diff --check`
- **Status:** complete
### Phase 34: Daily Task Batch Materialization
- [x] Replace per-candidate daily monitoring task INSERT calls with one batch INSERT
- [x] Preserve conflict de-duplication, dispatch timing, target client, shard key, and execution-owner semantics
- [x] Run targeted backend tests, full `go test ./...`, and `git diff --check`
- **Status:** complete
### Phase 35: Daily Plan SQL Decomposition
- [x] Split `loadDailyMonitoringPlans` into primary-client selection and platform-list loading
- [x] Keep lease preference, subscription-derived brand limits, workspace scoping, and platform authorization semantics
- [x] Add a focused helper test for plan assembly when platform snapshots disappear between queries
- [x] Run targeted backend tests, full `go test ./...`, and `git diff --check`
- **Status:** complete
### Phase 36: Monitoring Authorization And Detail Display Bug Fix
- [x] Treat bound platform accounts as authorization without requiring `health = 'live'`
- [x] Keep daily scheduled task generation independent of desktop-client online state
- [x] Keep immediate collection gated by the authorized target desktop client being online
- [x] Stop question detail from hiding historical data behind current authorization status
- [x] Run targeted backend tests, full `go test ./...`, admin-web typecheck/build, and `git diff --check`
- **Status:** complete
### Phase 37: k3s Deployment Foundation
- [x] Inventory compose/runtime services and dependency graph
- [x] Add kustomize-based k3s manifests for app services, infra stateful services, migration jobs, config, secrets, and ingress
- [x] Include deployment documentation and image/package notes for online and offline clusters
- **Status:** complete
### Phase 38: k3s Verification
- [x] Render manifests locally with kubectl/kustomize
- [x] Run YAML/static checks available in this workspace
- [x] Update findings/progress and summarize deployment commands plus required edits
- **Status:** complete
### Phase 39: Gitea Registry CI/CD Boundary
- [x] Keep image publishing owned by frontend/backend CI only
- [x] Make CD/offline workflows verify commit8 images instead of building fallback images
- [x] Keep deploy-config CI scoped to deployment config validation only
- [x] Run workflow YAML and shell syntax checks
- **Status:** complete
## Key Questions
1. How should the desktop client defer and locally optimize monitor-task execution without violating one-task-per-platform serialism?
2. What is the smallest durable local cache that allows same-day resume while safely dropping stale next-day monitor tasks?
3. How can Playwright CDP attach cleanly to Electron Chromium while reusing existing account session partitions?
## Decisions Made
| Decision | Rationale |
|----------|-----------|
| Use file-based planning for this task | Verification plus implementation will span multiple reads, edits, and test runs |
| Treat the earliest incomplete step as the current development target | Keeps the roadmap sequential and avoids skipping acceptance gaps |
| Lock the current implementation target to Step 4 | `sqlc` generation is broken and repository wrappers are absent, making it the earliest failed acceptance gate |
| Continue into the Step 5 refresh-rotation fix after Step 4 passed locally | The next acceptance gap was small, isolated, and directly adjacent to the auth module already being modified |
| Resume from the completed backend chain by building the missing `admin-web` foundation | The repository has no frontend apps, so UI integration cannot start without workspace scaffolding |
| Keep this turn scoped to `admin-web` rather than also creating `ops-admin-web` | The available backend is `tenant-api`; platform-side APIs and pages are not yet present in the repo |
| Use a pnpm workspace with shared packages from day one | Matches the architecture doc and avoids painting the repo into a single-app corner |
| Stop using Gemini for this turn after the user changed direction | Avoids spending more time on an unavailable external model path and keeps momentum inside the repo |
| Verify the first frontend slice with a real browser pass once Playwright became available again | Browser-level validation catches interaction bugs that install/build checks miss |
| Use the current backend route graph as the implementation boundary for this turn | The user asked for frontend coverage of existing interfaces, so pages without APIs stay non-primary |
| Reconstruct the screenshot layouts from visual position data, not OCR text alone | The user explicitly asked to follow image design including element placement |
| Move the quota card into the left sidebar footer and use page-level hero sections | This matches the reference screenshots more closely than the previous generic topbar layout |
| Centralize prompt text in a dedicated package instead of leaving it inside business logic functions | The user explicitly asked to extract hard-coded prompts for easier future optimization |
| Move scheduling authority fully into `desktop-client` | The user explicitly wants the server to dispatch tasks while the client decides when to execute them |
| Allow stale cross-day monitor tasks to be dropped on the client | The user explicitly allows漏采 and does not want next-day catch-up for unfinished tasks |
| Implement hidden browser infrastructure before expanding more adapters | Qwen and similar platforms cannot reliably use direct API calls and need browser-native execution |
| Only require active login for monitor tasks on `yuanbao` / `kimi` / `deepseek` | The user clarified that other AI platforms can still query and collect anonymously |
| Treat media publish as foreground and AI monitoring as background | The user explicitly wants publish to have the highest priority and never be starved by monitor execution |
| Reserve runtime capacity for publish instead of relying on queue priority alone | Running monitor tasks cannot be preempted by queue ordering, so production-grade priority needs admission control and reserved capacity |
| Keep publish scheduling in memory with per-platform locks/cooldowns | Publish needs foreground platform admission but not monitor-style durable recovery, cross-day pruning, or question cooldown semantics |
| Generate daily monitoring tasks after midnight with stable jitter and hard caps | The backend needs a durable daily plan without creating a single scheduler spike |
| Keep daily automatic monitoring pending when desktop is offline | Offline clients should defer desktop work, not force backend legacy execution |
| Claim due desktop dispatch batches in the database | `FOR UPDATE SKIP LOCKED` plus dispatch cooldown prevents multi-instance duplicate dispatch storms |
| Skip daily generation when `desktop_tasks` rollout is disabled or no primary client is configured | Creating `automatic/legacy` tasks would be an orphan path and hide a disabled execution channel as successful scheduling |
| Do not use `tenant_monitoring_quotas` in current monitoring runtime | Current product scope can derive execution plans from registered desktop clients and use canonical default platform coverage |
| Bound daily worker RabbitMQ publish with the worker context plus a per-publish timeout | Scheduler ticks must not leave unbounded goroutines when the broker stalls |
| Expose daily-worker metrics from the scheduler process itself | The worker runs in `cmd/scheduler`, so observing only `tenant-api` would still miss cron stalls and generation drops |
| Count daily cap against durable materialized tasks, not candidate history | Existing candidates should not consume later brands' new-task budget, while already-pending tasks must still be dispatchable after the cap is full |
| Source daily monitoring policy from subscriptions, brand_library config, and bound platform_accounts | Hard-coded `1 brand / 6 platforms / 36 tasks` bypassed SaaS entitlement and user authorization; account `health` is execution state, not authorization state |
| Use `desktop_client_primary_leases` for monitoring heartbeat leadership | A TTL lease gives sticky primary semantics without reusing `tenant_monitoring_quotas`, and prevents multiple desktop clients from alternating primary on every heartbeat |
| Use negative `scheduler.http_port` as the explicit metrics-server disable sentinel | `0` remains the default-port shorthand while `-1` can intentionally disable scheduler HTTP without being normalized back to `8081` |
| Require a scheduler internal metrics token before starting HTTP metrics | Failing closed avoids exposing tenant/brand counts and worker error state through unauthenticated endpoints |
| Separate worker lifecycle cancellation from per-tick operation contexts | Signal handling should stop the next tick, but an in-flight DB transaction or RabbitMQ publish should finish or hit its own timeout before the process exits |
| Treat no authorized monitoring platforms as an explicit API/UI state | Empty platform arrays are a weak contract; dashboard and detail pages should render a stable state and block collection actions |
| Separate monitoring display platforms from executable platforms | Product expects today's 6 AI platform columns to remain visible; task generation and collect-now use bound platform accounts, while collect-now still requires the target desktop client to be online |
| Resolve monitoring desktop state from the request-scoped workspace | Multi-workspace SaaS views must not borrow desktop clients or platform account state from the actor's primary workspace |
| Keep heartbeat primary lease resolution read-mostly | Stable heartbeat traffic should not create a row-lock/write-transaction stream; writes are reserved for lease lifecycle transitions and throttled renewals |
| Treat monitoring access snapshots as semantic state, not heartbeat liveness | Stable platform reports skip snapshot writes; unchanged rows refresh at most every 10 minutes, and pending-task reconciliation only opens a transaction when pending rows exist |
| Use `prometheus/client_golang` for Prometheus text exposition | Avoid hand-written text-format escaping/typing drift; scheduler `/metrics` now also registers Go runtime and process collectors while JSON metric snapshots remain unchanged |
| Materialize daily monitoring tasks with batched array INSERT | The per-brand cap is currently small, but one `INSERT ... SELECT FROM unnest(...) ON CONFLICT DO NOTHING` keeps DB round trips fixed if caps or retry batches grow |
| Split daily monitoring plan loading into selection and aggregation steps | Primary-client election and platform authorization aggregation have different explainability and testability concerns; two smaller queries are easier to inspect and later move to sqlc |
| Treat platform-account health as execution state, not authorization state | `health != live` should not suppress daily task planning, dashboard authorization, or detail history; only desktop-client online state gates immediate collection |
| Start k3s with a self-contained stack that mirrors the existing compose deployment | The repo already has working Docker images and compose topology; k3s should first preserve service names and runtime assumptions before introducing managed cloud replacements |
| Publish deployable images only from frontend/backend CI | Deployment config has no runtime image ownership; `deploy-config-ci` stays a validation workflow while frontend/backend CI publish Gitea Registry images tagged by commit8 |
| Use commit hash first 8 chars as the canonical deploy image tag | CD/offline packaging can compare the pushed commit to Registry tags and fail fast when CI has not produced the matching image |
## Errors Encountered
| Error | Attempt | Resolution |
|-------|---------|------------|
| None yet | 1 | N/A |
| `make sqlc-generate` failed because `sqlc.yaml` points to `../../migrations/`, which resolves to `server/internal/migrations` | 1 | Fix schema path to repository-relative `../../../migrations/` and then generate code |
| `make lint` initially failed on several `errcheck` and one `ineffassign` warning | 1 | Fixed transaction defers, the middleware test Redis set call, and the dead increment in `article_service.go` |
| Gemini CLI `gemini-3.1-pro-preview` returned `429 MODEL_CAPACITY_EXHAUSTED` repeatedly | 1 | Stopped the Gemini path after the user requested not to use Gemini anymore |
| Playwright MCP navigation failed locally because it attempted to create `/.playwright-mcp` | 1 | Fell back to preview HTTP checks instead of blocking the turn on Playwright environment setup |
| Login button click did not trigger the sign-in flow in browser testing even though the backend endpoint worked | 1 | Bound the primary login button directly to `handleSubmit` and re-ran browser verification until the route change succeeded |
| Desktop monitor execution is currently global-serial and lacks durable local queueing | 1 | Replace the ad-hoc in-memory FIFO with a scheduler module that owns persistence, concurrency, and stale-task cleanup |
| `connectOverCDP()` is not wired yet even though `playwright-core` is installed | 1 | Add a dedicated hidden-browser manager instead of pushing CDP code directly into adapters or the runtime controller |
| `pnpm --filter @geo/desktop-client typecheck` fails in `electron.vite.config.ts` due to a pre-existing Vite 5/8 plugin type mismatch | 1 | Verified this change with `pnpm --filter @geo/desktop-client build` and `pnpm --filter @geo/desktop-client test`; leave config dependency alignment as a separate fix |
| `kubectl apply --dry-run=client` still attempted to contact the local kube-apiserver, which is not running at `127.0.0.1:26443` | 1 | Verified with `kubectl kustomize`, parsed the rendered YAML locally, and recorded that live API validation should run on the target k3s cluster |
## Notes
- Re-check task_plan.md before major implementation decisions.
- Record concrete evidence for each claimed completed step.
- Frontend scope for this turn is limited to the tenant-facing `admin-web` shell, not the platform ops console.
- Visual references for this turn must be treated as layout guides, not just copy decks.
- Desktop AI monitoring scope for this turn is infrastructure-first: scheduler plus hidden browser layer, not all six adapters.