feat(browser-fetch): add lightpanda-backed fetch service and knowledge URL fallback
Adds a new browser-fetch microservice that wraps Lightpanda for rendering JS-heavy pages, and wires it into the knowledge URL parser as a fallback when the upstream link reader returns 403/429/5xx for an allow-listed domain (e.g. zhuanlan.zhihu.com). Includes config/env plumbing, hot-reload diff support, and full deploy assets (Dockerfile target, docker-compose, k3s manifests, image-build/package scripts).
This commit is contained in:
@@ -4,7 +4,7 @@
|
||||
|
||||
- infra: `postgres`, `monitoring-postgres`, `pgbouncer`, `monitoring-pgbouncer`, `rabbitmq`, `redis`, `qdrant`, `minio`
|
||||
- jobs: `migrate`, `minio-init`
|
||||
- apps: `tenant-api`, `worker-generate`, `kol-assist-worker`, `scheduler`, `ops-api`, `frontend`, `ops-web`
|
||||
- apps: `tenant-api`, `browser-fetch`, `worker-generate`, `kol-assist-worker`, `scheduler`, `ops-api`, `frontend`, `ops-web`
|
||||
- ingress: `geo-rankly.local` -> `frontend`, `ops.geo-rankly.local` -> `ops-web`
|
||||
|
||||
## 1. 准备镜像
|
||||
@@ -34,6 +34,7 @@ sudo k3s ctr images import images.tar
|
||||
- `OPS_JWT_SECRET`
|
||||
- `OPS_DEFAULT_ADMIN_PASSWORD`
|
||||
- `SCHEDULER_INTERNAL_METRICS_TOKEN`
|
||||
- `BROWSER_FETCH_TOKEN`
|
||||
- `POSTGRES_PASSWORD` / `RABBITMQ_DEFAULT_PASS` / `MINIO_ROOT_*`
|
||||
- `config/config.yaml` 里的 LLM / SiliconFlow API key
|
||||
- `config.local.yaml` 里对应的数据库、RabbitMQ、MinIO、JWT 配置
|
||||
@@ -131,6 +132,7 @@ kubectl -n geo-rankly port-forward svc/minio 9001:9001
|
||||
kubectl -n geo-rankly get all
|
||||
kubectl -n geo-rankly describe pod <pod-name>
|
||||
kubectl -n geo-rankly logs deploy/tenant-api -f
|
||||
kubectl -n geo-rankly logs deploy/browser-fetch -f
|
||||
kubectl -n geo-rankly logs deploy/scheduler -f
|
||||
kubectl -n geo-rankly logs deploy/worker-generate -f
|
||||
kubectl -n geo-rankly logs deploy/ops-api -f
|
||||
|
||||
Reference in New Issue
Block a user