root/geo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(objectstorage): add Aliyun OSS support and auto-detect provider in deploy workflows

Browse Source 
Introduces Aliyun OSS as an alternative to MinIO; deploy scripts and CI
workflows now read object_storage.provider from config and conditionally
include or skip MinIO resources in both Docker Compose and k3s paths.
Also adds ops scheduler domain and its migration tables.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
liangxu
2026-05-20 10:25:02 +08:00
parent 00eb514efc
commit 98f73c5bea
18 changed files with 901 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deploy/OFFLINE_PACKAGE.md
+5
View File
@@ -20,6 +20,11 @@ bash deploy.sh
docker compose -f docker-compose.yaml -f docker-compose.offline.yaml --env-file .env up -d
```
Object storage is selected from `config.yaml` / `config.yml`, with `config.local.yaml` / `config.local.yml` as an override:
- `object_storage.provider: minio` or `mino` deploys MinIO and `minio-init`
- `object_storage.provider: aliyun`, `aliyun_oss`, `aliyun-oss`, or `oss` skips MinIO and uses the Aliyun OSS config from the same config files
Runtime services connect to Postgres through PgBouncer in `session` pooling mode. The one-shot `migrate` job still connects directly to `postgres` and `monitoring-postgres` so DDL and seed work do not go through the pooler.
On a k3s host:
deploy/docker-compose.aliyun-oss.yaml
+38
View File
@@ -0,0 +1,38 @@
# docker-compose.aliyun-oss.yaml
# Overlay used when object_storage.provider is aliyun/oss.
# MinIO is moved behind an inactive profile and app services no longer wait for it.
services:
minio:
profiles: ["local-minio"]
minio-init:
profiles: ["local-minio"]
tenant-api:
environment:
OBJECT_STORAGE_ACCESS_KEY: !reset null
OBJECT_STORAGE_SECRET_KEY: !reset null
depends_on:
minio: !reset null
worker-generate:
environment:
OBJECT_STORAGE_ACCESS_KEY: !reset null
OBJECT_STORAGE_SECRET_KEY: !reset null
depends_on:
minio: !reset null
kol-assist-worker:
environment:
OBJECT_STORAGE_ACCESS_KEY: !reset null
OBJECT_STORAGE_SECRET_KEY: !reset null
depends_on:
minio: !reset null
scheduler:
environment:
OBJECT_STORAGE_ACCESS_KEY: !reset null
OBJECT_STORAGE_SECRET_KEY: !reset null
depends_on:
minio: !reset null
deploy/docker-compose.yaml
+2 -2
View File
@@ -17,8 +17,8 @@ x-app-env: &app-env
JWT_REFRESH_TTL: ${JWT_REFRESH_TTL:-720h}
SCHEDULER_INTERNAL_METRICS_TOKEN: ${SCHEDULER_INTERNAL_METRICS_TOKEN:-}
BROWSER_FETCH_TOKEN: ${BROWSER_FETCH_TOKEN:-}
OBJECT_STORAGE_ACCESS_KEY: ${MINIO_ROOT_USER:-minioadmin}
OBJECT_STORAGE_SECRET_KEY: ${MINIO_ROOT_PASSWORD:-minioadmin}
OBJECT_STORAGE_ACCESS_KEY: ${OBJECT_STORAGE_ACCESS_KEY:-${MINIO_ROOT_USER:-minioadmin}}
OBJECT_STORAGE_SECRET_KEY: ${OBJECT_STORAGE_SECRET_KEY:-${MINIO_ROOT_PASSWORD:-minioadmin}}
x-app-volumes: &app-volumes
- ./config.yaml:/app/configs/config.yaml:ro
deploy/k3s-aliyun-oss/kustomization.yaml
+40
View File
@@ -0,0 +1,40 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../k3s
patches:
- target:
version: v1
kind: Service
name: minio
patch: |-
$patch: delete
apiVersion: v1
kind: Service
metadata:
name: minio
- target:
group: apps
version: v1
kind: StatefulSet
name: minio
patch: |-
$patch: delete
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: minio
- target:
group: batch
version: v1
kind: Job
name: minio-init
patch: |-
$patch: delete
apiVersion: batch/v1
kind: Job
metadata:
name: minio-init
- path: patches/object-storage-env.yaml
deploy/k3s-aliyun-oss/patches/object-storage-env.yaml
+59
View File
@@ -0,0 +1,59 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: tenant-api
spec:
template:
spec:
containers:
- name: tenant-api
env:
- name: OBJECT_STORAGE_ACCESS_KEY
$patch: delete
- name: OBJECT_STORAGE_SECRET_KEY
$patch: delete
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: worker-generate
spec:
template:
spec:
containers:
- name: worker-generate
env:
- name: OBJECT_STORAGE_ACCESS_KEY
$patch: delete
- name: OBJECT_STORAGE_SECRET_KEY
$patch: delete
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kol-assist-worker
spec:
template:
spec:
containers:
- name: kol-assist-worker
env:
- name: OBJECT_STORAGE_ACCESS_KEY
$patch: delete
- name: OBJECT_STORAGE_SECRET_KEY
$patch: delete
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: scheduler
spec:
template:
spec:
containers:
- name: scheduler
env:
- name: OBJECT_STORAGE_ACCESS_KEY
$patch: delete
- name: OBJECT_STORAGE_SECRET_KEY
$patch: delete
deploy/k3s/README.md
+14
View File
@@ -7,6 +7,11 @@
- apps: `tenant-api`, `browser-fetch`, `worker-generate`, `kol-assist-worker`, `scheduler`, `ops-api`, `frontend`, `ops-web`
- ingress: `geo-rankly.local` -> `frontend`, `ops.geo-rankly.local` -> `ops-web`
对象存储部署跟随 `deploy/config.yaml` / `deploy/config.yml` 里的 `object_storage.provider`，并允许 `deploy/config.local.yaml` / `deploy/config.local.yml` 覆盖：
- `provider: minio` 或 `provider: mino`：部署 MinIO 和 `minio-init`
- `provider: aliyun` / `aliyun_oss` / `aliyun-oss` / `oss`：使用 `deploy/k3s-aliyun-oss` overlay，不部署 MinIO 和 `minio-init`
## 1. 准备镜像
在线环境建议推到镜像仓库后修改 `kustomization.yaml` 里的 `images`：
@@ -59,6 +64,15 @@ kubectl apply -k deploy/k3s
kubectl -n geo-rankly get pods -w
```
如果使用阿里云 OSS：
```bash
kubectl apply -k deploy/k3s-aliyun-oss
kubectl -n geo-rankly get pods -w
```
离线包里的 `bash deploy.sh k3s` 会自动读取根目录 `config.yaml/config.yml` 和 `config.local.yaml/config.local.yml`，并选择 `k3s` 或 `k3s-aliyun-oss`。
查看一次性任务：
```bash
deploy/k3s/config/config.yaml
+7 -7
View File
@@ -156,14 +156,14 @@ qdrant:
timeout: 15s
object_storage:
provider: minio
endpoint: minio:9000
access_key: minioadmin
secret_key: minioadmin
bucket: geo-private
use_ssl: false
provider: aliyun
endpoint: https://oss-cn-shanghai.aliyuncs.com
access_key: LTAI5tAEj8euR8B1gXzMoG84
secret_key: lJPrKo9ViyHJE8UWIMmMCY0B6Je8v2
bucket: shengxintui
use_ssl: true
public_base_url: ""
region: ""
region: cn-shanghai
cache:
driver: redis
deploy/scripts/load-and-start.sh
+175 -10
View File
@@ -37,6 +37,21 @@ ensure_local_config() {
fi
}
ensure_config_aliases() {
local yaml yml
for yaml in config.yaml config.local.yaml ops-config.yaml ops-config.local.yaml; do
yml="${yaml%.yaml}.yml"
if [[ ! -f "${SCRIPT_DIR}/${yaml}" && -f "${SCRIPT_DIR}/${yml}" ]]; then
cp "${SCRIPT_DIR}/${yml}" "${SCRIPT_DIR}/${yaml}"
echo "==> 已从 ${yml} 创建 ${yaml}，供部署清单挂载使用。"
fi
done
if [[ ! -f "${SCRIPT_DIR}/prompts.yml" && -f "${SCRIPT_DIR}/prompts.yaml" ]]; then
cp "${SCRIPT_DIR}/prompts.yaml" "${SCRIPT_DIR}/prompts.yml"
echo "==> 已从 prompts.yaml 创建 prompts.yml，供部署清单挂载使用。"
fi
}
env_value() {
local key="$1" default_value="$2" value
value="$(grep -E "^${key}=" "${SCRIPT_DIR}/.env" | tail -1 | cut -d= -f2- || true)"
@@ -45,9 +60,130 @@ env_value() {
printf '%s' "${value:-${default_value}}"
}
config_file_path() {
local base="$1"
for file in "${SCRIPT_DIR}/${base}.yaml" "${SCRIPT_DIR}/${base}.yml"; do
if [[ -f "${file}" ]]; then
printf '%s' "${file}"
return
fi
done
}
read_object_storage_provider() {
local provider="minio" file
for file in "$(config_file_path config)" "$(config_file_path config.local)"; do
if [[ -z "${file}" || ! -f "${file}" ]]; then
continue
fi
local next
next="$(
awk '
/^[[:space:]]*object_storage:[[:space:]]*$/ {
in_section=1
base=match($0, /[^[:space:]]/) - 1
next
}
in_section {
indent=match($0, /[^[:space:]]/) - 1
if ($0 ~ /^[[:space:]]*$/ || $0 ~ /^[[:space:]]*#/) {
next
}
if (indent <= base) {
in_section=0
next
}
if ($0 ~ /^[[:space:]]*provider[[:space:]]*:/) {
value=$0
sub(/^[[:space:]]*provider[[:space:]]*:[[:space:]]*/, "", value)
sub(/[[:space:]]+#.*$/, "", value)
gsub(/^["'\'']|["'\'']$/, "", value)
print value
}
}
' "${file}" | tail -1
)"
if [[ -n "${next}" ]]; then
provider="${next}"
fi
done
printf '%s' "$(printf '%s' "${provider}" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')"
}
use_minio_for_object_storage() {
case "$(read_object_storage_provider)" in
""|minio|mino)
return 0
;;
aliyun|aliyun_oss|aliyun-oss|oss)
return 1
;;
*)
return 0
;;
esac
}
sync_k3s_config() {
mkdir -p "${SCRIPT_DIR}/k3s/config"
for name in config ops-config prompts; do
local output="${name}.yaml"
if [[ "${name}" == "prompts" ]]; then
output="prompts.yml"
fi
local source=""
for candidate in "${SCRIPT_DIR}/${name}.yaml" "${SCRIPT_DIR}/${name}.yml"; do
if [[ -f "${candidate}" ]]; then
source="${candidate}"
break
fi
done
if [[ -n "${source}" ]]; then
cp "${source}" "${SCRIPT_DIR}/k3s/config/${output}"
fi
done
}
prepare_k3s_runtime_overlay() {
local base_dir="$1"
local runtime_dir="${SCRIPT_DIR}/.k3s-runtime"
rm -rf "${runtime_dir}"
mkdir -p "${runtime_dir}"
cp "$(config_file_path config.local)" "${runtime_dir}/config.local.yaml"
cp "$(config_file_path ops-config.local)" "${runtime_dir}/ops-config.local.yaml"
cat > "${runtime_dir}/kustomization.yaml" <<EOF
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../$(basename "${base_dir}")
secretGenerator:
- name: geo-rankly-app-secret
behavior: merge
files:
- config.local.yaml=config.local.yaml
- ops-config.local.yaml=ops-config.local.yaml
generatorOptions:
disableNameSuffixHash: true
EOF
printf '%s' "${runtime_dir}"
}
deploy_compose() {
ensure_config_aliases
ensure_local_config "config.local.yaml" "config.local.yaml.example"
ensure_local_config "ops-config.local.yaml" "ops-config.local.yaml.example"
local compose_files=("-f" "docker-compose.yaml" "-f" "docker-compose.offline.yaml")
local minio_enabled="true"
if ! use_minio_for_object_storage; then
compose_files+=("-f" "docker-compose.aliyun-oss.yaml")
minio_enabled="false"
fi
# ─── Step 1: 加载镜像 ───────────────────────────────────────────────────────
echo "==> [1/3] 加载 Docker 镜像（可能需要几分钟）..."
@@ -62,30 +198,46 @@ deploy_compose() {
# ─── Step 3: 启动服务 ───────────────────────────────────────────────────────
echo "==> [3/3] 启动所有服务..."
if [[ "${minio_enabled}" == "true" ]]; then
echo " object_storage.provider=$(read_object_storage_provider)，将部署 MinIO。"
else
echo " object_storage.provider=$(read_object_storage_provider)，将跳过 MinIO。"
export COMPOSE_PROFILES=""
docker rm -f geo-minio geo-minio-init >/dev/null 2>&1 || true
fi
cd "${SCRIPT_DIR}"
docker compose \
-f docker-compose.yaml \
-f docker-compose.offline.yaml \
--env-file .env \
up -d
docker compose "${compose_files[@]}" --env-file .env up -d
echo ""
echo "✅ Docker Compose 部署完成！"
echo ""
echo "服务状态："
docker compose -f docker-compose.yaml -f docker-compose.offline.yaml ps
docker compose "${compose_files[@]}" ps
echo ""
echo "查看日志： docker compose -f docker-compose.yaml logs -f tenant-api"
echo "停止服务： docker compose -f docker-compose.yaml down"
echo "查看日志： docker compose ${compose_files[*]} logs -f tenant-api"
echo "停止服务： docker compose ${compose_files[*]} down"
echo "管理界面："
echo " Tenant Web: http://<server-ip>:$(env_value FRONTEND_PORT 18080)"
echo " Ops Web: http://<server-ip>:$(env_value OPS_WEB_PORT 18082)"
echo " RabbitMQ: http://<server-ip>:$(env_value RABBITMQ_MGMT_PORT 15673) (geo / geo_dev)"
echo " MinIO: http://<server-ip>:$(env_value MINIO_CONSOLE_PORT 19001) ($(env_value MINIO_ROOT_USER minioadmin) / $(env_value MINIO_ROOT_PASSWORD minioadmin))"
if [[ "${minio_enabled}" == "true" ]]; then
echo " MinIO: http://<server-ip>:$(env_value MINIO_CONSOLE_PORT 19001) ($(env_value MINIO_ROOT_USER minioadmin) / $(env_value MINIO_ROOT_PASSWORD minioadmin))"
fi
}
deploy_k3s() {
local kubectl_cmd="kubectl"
local kustomize_dir="${SCRIPT_DIR}/k3s"
local minio_enabled="true"
ensure_config_aliases
ensure_local_config "config.local.yaml" "config.local.yaml.example"
ensure_local_config "ops-config.local.yaml" "ops-config.local.yaml.example"
if ! use_minio_for_object_storage; then
kustomize_dir="${SCRIPT_DIR}/k3s-aliyun-oss"
minio_enabled="false"
fi
if command -v k3s >/dev/null 2>&1; then
echo "==> [1/3] 导入镜像到 k3s containerd..."
@@ -100,7 +252,20 @@ deploy_k3s() {
fi
echo "==> [2/3] 应用 k3s 清单..."
${kubectl_cmd} apply -k "${SCRIPT_DIR}/k3s"
sync_k3s_config
kustomize_dir="$(prepare_k3s_runtime_overlay "${kustomize_dir}")"
if [[ "${minio_enabled}" == "true" ]]; then
echo " object_storage.provider=$(read_object_storage_provider)，将部署 MinIO。"
else
echo " object_storage.provider=$(read_object_storage_provider)，将跳过 MinIO。"
${kubectl_cmd} -n geo-rankly delete statefulset minio --ignore-not-found
${kubectl_cmd} -n geo-rankly delete service minio --ignore-not-found
${kubectl_cmd} -n geo-rankly delete job minio-init --ignore-not-found
for svc in tenant-api worker-generate kol-assist-worker scheduler; do
${kubectl_cmd} -n geo-rankly set env "deployment/${svc}" OBJECT_STORAGE_ACCESS_KEY- OBJECT_STORAGE_SECRET_KEY- >/dev/null 2>&1 || true
done
fi
${kubectl_cmd} apply -k "${kustomize_dir}"
echo "==> [3/3] 查看服务状态..."
${kubectl_cmd} -n geo-rankly get pods,svc
deploy/scripts/package.sh
+7
View File
@@ -159,12 +159,16 @@ echo " 镜像包大小: $(du -sh "${WORK_DIR}/images.tar" | cut -f1)"
echo "==> [4/5] 打包部署文件"
cp "${DEPLOY_DIR}/docker-compose.yaml" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/docker-compose.offline.yaml" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/docker-compose.aliyun-oss.yaml" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/config.yaml" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/ops-config.yaml" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/prompts.yml" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/.env.example" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/config.local.yaml.example" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/ops-config.local.yaml.example" "${WORK_DIR}/"
cp "${DEPLOY_DIR}/scripts/load-and-start.sh" "${WORK_DIR}/deploy.sh"
cp -R "${DEPLOY_DIR}/k3s" "${WORK_DIR}/k3s"
cp -R "${DEPLOY_DIR}/k3s-aliyun-oss" "${WORK_DIR}/k3s-aliyun-oss"
if [[ -f "${DEPLOY_DIR}/OFFLINE_PACKAGE.md" ]]; then
cp "${DEPLOY_DIR}/OFFLINE_PACKAGE.md" "${WORK_DIR}/README.md"
fi
@@ -201,7 +205,10 @@ Files:
images.tar
docker-compose.yaml
docker-compose.offline.yaml
docker-compose.aliyun-oss.yaml
.env.example
config.local.yaml.example
ops-config.local.yaml.example
deploy.sh
EOF