root
b6858d2605
Add task_plan/findings/progress notes for knowledge fact fidelity, brand description context, media supply favorites, and the workspace sync-to-39 deployment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
55 lines
4.8 KiB
Markdown
55 lines
4.8 KiB
Markdown
# Progress: RAG Numeric Fact Fidelity
|
|
|
|
## 2026-07-10
|
|
- Read diagnosis and file-planning workflows.
|
|
- Preserved unrelated dirty favorite-media changes.
|
|
- Traced knowledge resolution, precise-fact rendering, worker dispatch, generation, sanitization, and persistence.
|
|
- Confirmed founding time is missing from precise-fact extraction.
|
|
- Confirmed generated output has no deterministic fact validation before persistence.
|
|
- Connected read-only to the supplied production k3s host and confirmed workload health.
|
|
- Corrected the worker selector and inspected 24 hours of generation/knowledge logs.
|
|
- Confirmed the precise-fact fallback is exercised in production and moved exact article lookup to PostgreSQL.
|
|
- Opened a read-only production database inspection session and limited planned queries to IDs, metadata, and narrow founding-date excerpts.
|
|
- Located the exact article/version/task metadata and identified the affected generation path as KOL.
|
|
- Confirmed the exact wrong founding-year/service-duration sentence in the stored article and captured the six selected knowledge group IDs.
|
|
- Found conflicting source values (newest 2020, older 2019) and identified missing KOL `brand_id` as the primary retrieval-scope defect.
|
|
- Published four ranked, falsifiable hypotheses before implementing probes/fixes.
|
|
- Confirmed the KOL scope defect in both producer and consumer code and found no existing regression coverage for it.
|
|
- Selected a three-layer fix: KOL scope propagation, typed/canonical fact extraction, and pre-persistence validate-repair-reject handling across generation paths.
|
|
- Verified the brand scope exists before payload serialization and the LLM interface supports a shared guarded-generation implementation.
|
|
- Found a reusable fake-LLM testing pattern; no existing generation-payload snapshot tests cover the KOL scope contract.
|
|
- Added focused payload, fact extraction/canonicalization, contradiction, encoded-content, and repair/fail regression tests.
|
|
- Ran the focused tests and observed the expected red compile failure because the new fact-guard contract is not implemented yet.
|
|
- Implemented the core typed fact model, canonical conflict handling, subject-aware validation, percent-decoding, and one-repair guard.
|
|
- Wired fact constraints into knowledge prompt rendering and added KOL brand/group scope serialization.
|
|
- Integrated guarded generation into template, imitation, and KOL execution paths.
|
|
- Identified the custom prompt-rule snapshot seam for serializing and restoring fact constraints alongside the existing knowledge prompt.
|
|
- Reviewed the scoped diff separately from unrelated favorite-media/Swagger changes and flagged canonical source ordering for an additional regression check.
|
|
- Hardened canonical source ordering so `updated_at DESC` precedes retrieval priority, and added nil-result guards around both LLM calls.
|
|
- Package-level tests passed for tenant app, generation worker, and prompt loader after all four generation paths were wired.
|
|
- Full `go test ./...` passed across the server repository.
|
|
- A later full vet attempt was blocked by a new syntax error in an unrelated concurrently edited favorite-media test; no changes were made to that file.
|
|
- `go vet -tests=false` passed for the affected production packages.
|
|
- `git diff --check` passed and no `[DEBUG-*]` instrumentation was present.
|
|
- Closed the production PostgreSQL and SSH sessions cleanly.
|
|
- Did not deploy because the shared worktree contains unrelated uncommitted feature work.
|
|
- Captured final implementation references and rechecked worktree ownership before handoff.
|
|
- First green run stopped at compile time because text/payload decoders shared a name and the payload test referenced worker-private helpers; both issues were isolated without changing test intent.
|
|
- Corrected the two compile issues and re-ran the focused regression suite successfully.
|
|
|
|
## Test Results
|
|
| Test | Expected | Actual | Status |
|
|
|------|----------|--------|--------|
|
|
| Production worker selector probe | Matching worker logs | Selector matched no resources | Needs corrected selector |
|
|
| Corrected worker log probe | Knowledge-generation events | Found active precise-fact fallback and task IDs | Pass |
|
|
| RAG numeric fidelity red tests | Fail before implementation | Undefined fact guard types/functions at compile time | Expected fail |
|
|
| RAG numeric fidelity focused green tests | Pass after implementation | `ok`, including repair and reject behavior | Pass |
|
|
| Generation package verification | `tenant/app`, `worker/generate`, and `tenant/prompts` pass | All packages `ok` | Pass |
|
|
| Full server test suite | All Go packages pass | `go test ./...` exit 0 | Pass |
|
|
| Production-package vet | Affected non-test code passes vet | Exit 0 | Pass |
|
|
| Diff/instrumentation cleanup | No whitespace errors or debug markers | Clean | Pass |
|
|
|
|
## Next
|
|
- Implement the fact guard and wire it into all generation paths.
|
|
- Re-run focused and package-level tests.
|