feat(research): hydrate web-search results with cleaned page content

Fetch and clean each search result's page body (bounded by size,
concurrency, and a configurable MaxPageContentRunes), store it on the
new ResearchResult.Content field, and surface those excerpts in agent
memory so the model reasons over real page text rather than snippets
alone. Promotes golang.org/x/net to a direct dependency for HTML
parsing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-08 10:56:31 +08:00
parent 222e5a0219
commit 836de5b597
8 changed files with 252 additions and 17 deletions
+3 -2
View File
@@ -256,8 +256,9 @@ func newResearcher(c config.Config) design.Researcher {
switch opts.Driver {
case "", "duckduckgo":
return websearch.NewDuckDuckGoResearcher(websearch.DuckDuckGoOptions{
TimeoutSeconds: opts.TimeoutSeconds,
MaxResults: opts.MaxResults,
TimeoutSeconds: opts.TimeoutSeconds,
MaxResults: opts.MaxResults,
MaxPageContentRunes: opts.MaxPageContentRunes,
})
case "none", "disabled":
return nil