geo/progress.rag-numeric-fidelity.md

Progress: RAG Numeric Fact Fidelity

2026-07-10

• Read diagnosis and file-planning workflows.
• Preserved unrelated dirty favorite-media changes.
• Traced knowledge resolution, precise-fact rendering, worker dispatch, generation, sanitization, and persistence.
• Confirmed founding time is missing from precise-fact extraction.
• Confirmed generated output has no deterministic fact validation before persistence.
• Connected read-only to the supplied production k3s host and confirmed workload health.
• Corrected the worker selector and inspected 24 hours of generation/knowledge logs.
• Confirmed the precise-fact fallback is exercised in production and moved exact article lookup to PostgreSQL.
• Opened a read-only production database inspection session and limited planned queries to IDs, metadata, and narrow founding-date excerpts.
• Located the exact article/version/task metadata and identified the affected generation path as KOL.
• Confirmed the exact wrong founding-year/service-duration sentence in the stored article and captured the six selected knowledge group IDs.
• Found conflicting source values (newest 2020, older 2019) and identified missing KOL brand_id as the primary retrieval-scope defect.
• Published four ranked, falsifiable hypotheses before implementing probes/fixes.
• Confirmed the KOL scope defect in both producer and consumer code and found no existing regression coverage for it.
• Selected a three-layer fix: KOL scope propagation, typed/canonical fact extraction, and pre-persistence validate-repair-reject handling across generation paths.
• Verified the brand scope exists before payload serialization and the LLM interface supports a shared guarded-generation implementation.
• Found a reusable fake-LLM testing pattern; no existing generation-payload snapshot tests cover the KOL scope contract.
• Added focused payload, fact extraction/canonicalization, contradiction, encoded-content, and repair/fail regression tests.
• Ran the focused tests and observed the expected red compile failure because the new fact-guard contract is not implemented yet.
• Implemented the core typed fact model, canonical conflict handling, subject-aware validation, percent-decoding, and one-repair guard.
• Wired fact constraints into knowledge prompt rendering and added KOL brand/group scope serialization.
• Integrated guarded generation into template, imitation, and KOL execution paths.
• Identified the custom prompt-rule snapshot seam for serializing and restoring fact constraints alongside the existing knowledge prompt.
• Reviewed the scoped diff separately from unrelated favorite-media/Swagger changes and flagged canonical source ordering for an additional regression check.
• Hardened canonical source ordering so updated_at DESC precedes retrieval priority, and added nil-result guards around both LLM calls.
• Package-level tests passed for tenant app, generation worker, and prompt loader after all four generation paths were wired.
• Full go test ./... passed across the server repository.
• A later full vet attempt was blocked by a new syntax error in an unrelated concurrently edited favorite-media test; no changes were made to that file.
• go vet -tests=false passed for the affected production packages.
• git diff --check passed and no [DEBUG-*] instrumentation was present.
• Closed the production PostgreSQL and SSH sessions cleanly.
• Did not deploy because the shared worktree contains unrelated uncommitted feature work.
• Captured final implementation references and rechecked worktree ownership before handoff.
• First green run stopped at compile time because text/payload decoders shared a name and the payload test referenced worker-private helpers; both issues were isolated without changing test intent.
• Corrected the two compile issues and re-ran the focused regression suite successfully.

Test Results

Test	Expected	Actual	Status
Production worker selector probe	Matching worker logs	Selector matched no resources	Needs corrected selector
Corrected worker log probe	Knowledge-generation events	Found active precise-fact fallback and task IDs	Pass
RAG numeric fidelity red tests	Fail before implementation	Undefined fact guard types/functions at compile time	Expected fail
RAG numeric fidelity focused green tests	Pass after implementation	ok, including repair and reject behavior	Pass
Generation package verification	tenant/app, worker/generate, and tenant/prompts pass	All packages ok	Pass
Full server test suite	All Go packages pass	go test ./... exit 0	Pass
Production-package vet	Affected non-test code passes vet	Exit 0	Pass
Diff/instrumentation cleanup	No whitespace errors or debug markers	Clean	Pass

Next

• Implement the fact guard and wire it into all generation paths.
• Re-run focused and package-level tests.