feat(research): hydrate web-search results with cleaned page content
Fetch and clean each search result's page body (bounded by size, concurrency, and a configurable MaxPageContentRunes), store it on the new ResearchResult.Content field, and surface those excerpts in agent memory so the model reasons over real page text rather than snippets alone. Promotes golang.org/x/net to a direct dependency for HTML parsing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -256,8 +256,9 @@ func newResearcher(c config.Config) design.Researcher {
|
||||
switch opts.Driver {
|
||||
case "", "duckduckgo":
|
||||
return websearch.NewDuckDuckGoResearcher(websearch.DuckDuckGoOptions{
|
||||
TimeoutSeconds: opts.TimeoutSeconds,
|
||||
MaxResults: opts.MaxResults,
|
||||
TimeoutSeconds: opts.TimeoutSeconds,
|
||||
MaxResults: opts.MaxResults,
|
||||
MaxPageContentRunes: opts.MaxPageContentRunes,
|
||||
})
|
||||
case "none", "disabled":
|
||||
return nil
|
||||
|
||||
Reference in New Issue
Block a user