项目初始化-首次提交
This commit is contained in:
commit
2c1f06c84c
27
.env.example
Normal file
27
.env.example
Normal file
@ -0,0 +1,27 @@
|
||||
# Redis 连接
|
||||
REDIS_URL=redis://localhost:6379/0
|
||||
|
||||
# 网关服务
|
||||
HOST=0.0.0.0
|
||||
PORT=8000
|
||||
|
||||
# 后端运行目录(用于前端代理,预留)
|
||||
BACKEND_ROOT=backend
|
||||
|
||||
# 心跳保活参数(秒)
|
||||
HEARTBEAT_SCAN_INTERVAL=60
|
||||
AGENT_HEARTBEAT_TIMEOUT=120
|
||||
AGENT_HEARTBEAT_EXPECTED=10
|
||||
|
||||
# 调度参数(秒)
|
||||
DISPATCH_INTERVAL=2
|
||||
TASK_TIMEOUT_CHECK_INTERVAL=5
|
||||
|
||||
# 任务默认超时(秒)
|
||||
DEFAULT_TASK_TIMEOUT=3600
|
||||
|
||||
# 任务数据保留 TTL(秒),默认 24 小时
|
||||
TASK_TTL=86400
|
||||
|
||||
# 认证:CLI 提交任务 / Agent 注册时必须携带的密码
|
||||
GATEWAY_AUTH=change-me
|
||||
71
.gitignore
vendored
Normal file
71
.gitignore
vendored
Normal file
@ -0,0 +1,71 @@
|
||||
# ===== Environment / Secrets =====
|
||||
# 环境变量(含密钥,切勿提交)
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
|
||||
# ===== Python (backend) =====
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
*.egg-info/
|
||||
.eggs/
|
||||
build/
|
||||
dist/
|
||||
htmlcov/
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
ENV/
|
||||
.python-version
|
||||
.mypy_cache/
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
.coverage
|
||||
.tox/
|
||||
*.egg
|
||||
|
||||
# ===== Node (frontend) =====
|
||||
node_modules/
|
||||
frontend/node_modules/
|
||||
frontend/dist/
|
||||
*.tsbuildinfo
|
||||
frontend/tsconfig.tsbuildinfo
|
||||
|
||||
# ===== Logs =====
|
||||
*.log
|
||||
logs/
|
||||
*.log.*
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
|
||||
# ===== 运行时 / 生成文件 =====
|
||||
gateway_restart.log
|
||||
gateway_restart_err.log
|
||||
frontend/frontend.err.log
|
||||
frontend/frontend.log
|
||||
frontend/vite.log
|
||||
backend/backend.err.log
|
||||
backend/backend.log
|
||||
backend/gateway_restart.err.log
|
||||
backend/gateway_restart.log
|
||||
|
||||
# ===== IDE / 编辑器 =====
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# ===== 项目工具数据 =====
|
||||
.codebuddy/
|
||||
|
||||
# ===== 构建产物与压缩包 =====
|
||||
*.tar.gz
|
||||
*.zip
|
||||
*.tgz
|
||||
231
A2A需求分析-网关-K8s全链路架构详细设计方案.md
Normal file
231
A2A需求分析-网关-K8s全链路架构详细设计方案.md
Normal file
@ -0,0 +1,231 @@
|
||||
# A2A需求分析\-网关\-K8s全链路架构详细设计方案
|
||||
|
||||
## 一、架构整体概述
|
||||
|
||||
### 1\.1 核心架构分层
|
||||
|
||||
整体采用**三层解耦架构:需求分析层 → 智能网关层 → K8s Agent算力层**,完全贴合你的设计思路,兼顾本地IDE/CLI异步任务、云端Agent弹性调度、全链路任务管控。
|
||||
|
||||
- **需求分析层(前端调度端)**:CLI终端、VSCode IDE、Claude Code/CodeBuddy等AI终端,负责自然语言需求解析、复杂任务拆解、本地轻量任务异步执行、结构化任务上报网关。
|
||||
|
||||
- **智能网关层(核心中枢)**:基于FastAPI\+ElementUI\+Redis搭建,核心能力为任务池管理、Agent池管理、心跳保活、任务\-Agent规则匹配、全链路通信中转、任务生命周期管控、可视化运维。
|
||||
|
||||
- **K8s算力层(执行端)**:全权负责Agent容器的生命周期管理(创建、扩容、销毁、重启),Agent启动/销毁主动通知网关,实现集群弹性伸缩。
|
||||
|
||||
### 1\.2 核心设计理念
|
||||
|
||||
- 分层解耦:需求拆解、任务调度、算力执行完全隔离,支持独立迭代扩容;
|
||||
|
||||
- 异步协同:本地CLI可自主处理轻量编程任务,复杂任务上云调度K8s Agent,轻重任务分流;
|
||||
|
||||
- 动态感知:网关实时维护在线Agent池,通过定时心跳剔除异常节点,适配K8s动态扩缩容;
|
||||
|
||||
- 精准匹配:基于自定义规则实现任务与Agent能力精准绑定,支持负载均衡、优先级调度;
|
||||
|
||||
- 统一通信:网关作为唯一通信中枢,承接CLI与云端Agent的双向交互,通过AgentID精准关联任务链路。
|
||||
|
||||
## 二、各层级详细能力设计
|
||||
|
||||
### 2\.1 需求分析层(CLI/IDE/AI终端)
|
||||
|
||||
#### 2\.1\.1 核心角色
|
||||
|
||||
包含自研CLI、VSCode插件、Claude Code、CodeBuddy等所有前端需求入口,作为**顶层任务调度者**,具备LLM自主拆解能力。
|
||||
|
||||
#### 2\.1\.2 核心能力
|
||||
|
||||
1. **需求智能拆解**:接收用户自然语言需求,自主拆分出可独立执行的原子任务,区分轻量本地任务、重型云端任务;
|
||||
|
||||
2. **本地异步执行**:简单代码补全、语法校验、本地文件修改等轻量任务,由CLI/IDE本地异步处理,无需上云;
|
||||
|
||||
3. **云端任务上报**:编译构建、批量测试、数据分析、流媒体处理等复杂任务,结构化封装后提交至网关;
|
||||
|
||||
4. **结果接收与聚合**:接收网关转发的Agent执行结果,LLM聚合输出最终答案,反馈给用户;
|
||||
|
||||
5. **全链路追踪**:每个任务生成全局唯一RequestID,贯穿网关、Agent全链路,用于结果精准匹配、日志溯源。
|
||||
|
||||
#### 2\.1\.3 任务上报协议
|
||||
|
||||
采用标准HTTP JSON协议适配所有AI终端工具调用规范,请求体核心字段:RequestID、任务类型、任务标签、任务载荷、超时时间、本地工作目录信息。
|
||||
|
||||
### 2\.2 智能网关层(核心中枢)
|
||||
|
||||
技术栈:**FastAPI(高性能接口服务)\+ ElementUI(可视化后台)\+ Redis(内存数据存储)**,无状态服务设计,支持水平扩容,核心维护两大核心资源池:Task任务池、Agent算力池。
|
||||
|
||||
#### 2\.2\.1 核心模块拆分
|
||||
|
||||
- **API接入模块**:接收CLI/IDE任务提交、Agent注册/注销、心跳上报、任务结果回传;
|
||||
|
||||
- **任务池管理模块**:缓存所有待执行/执行中/已完成任务,维护任务全生命周期状态;
|
||||
|
||||
- **Agent池管理模块**:维护所有K8s在线Agent信息,基于定时心跳保活、剔除异常节点;
|
||||
|
||||
- **规则调度模块**:实现任务与Agent的标签匹配、负载均衡、优先级调度;
|
||||
|
||||
- **通信中转模块**:基于AgentID关联CLI与Agent,双向转发任务指令、执行日志、最终结果;
|
||||
|
||||
- **可视化运维模块**:ElementUI实现任务列表、Agent在线状态、执行日志、异常告警可视化展示。
|
||||
|
||||
#### 2\.2\.2 Task任务池设计(Redis存储)
|
||||
|
||||
网关接收CLI任务后,立即写入Redis任务池,同步返回「任务受理成功」响应,不阻塞CLI,实现异步调度。
|
||||
|
||||
**任务数据结构**:RequestID、TaskType、TaskTags、Payload、Status(pending/running/success/failed)、AgentID、CliSessionID、CreateTime、Timeout、Progress、Result、ErrorInfo。
|
||||
|
||||
**任务状态流转**:待受理 → 待调度 → 执行中 → 执行成功/执行失败 → 任务归档(TTL自动清理)。
|
||||
|
||||
#### 2\.2\.3 Agent算力池设计(Redis存储)
|
||||
|
||||
全权管理K8s集群所有Agent实例,适配K8s动态生命周期,实现动态注册、失效剔除。
|
||||
|
||||
**Agent数据结构**:AgentID、Endpoint(K8s内部访问地址)、AgentTags(能力标签)、MaxConcurrent(最大并发)、CurrentLoad(当前负载)、LastHeartbeat(最后心跳时间)、Status(online/offline)、CreateTime。
|
||||
|
||||
#### 2\.2\.4 心跳保活机制(核心设计)
|
||||
|
||||
严格遵循你的设计:**网关定时每分钟轮询维持心跳**,补充工程化容错逻辑:
|
||||
|
||||
1. 网关后台启动定时任务,每60s扫描一次Redis Agent池;
|
||||
|
||||
2. 判定规则:连续2个周期(120s)未上报心跳 → 标记Agent为offline,从可用算力池剔除;
|
||||
|
||||
3. 被动更新:Agent启动主动注册、销毁主动注销,实时更新Agent池状态,优先覆盖定时轮询结果;
|
||||
|
||||
4. 负载同步:心跳同步上报当前任务并发数,为调度算法提供依据。
|
||||
|
||||
#### 2\.2\.5 任务\-Agent匹配调度规则
|
||||
|
||||
网关核心调度逻辑,实现精准任务分发,优先级从高到低:
|
||||
|
||||
1. **标签精准匹配**:根据任务TaskTags筛选具备对应能力标签的Agent(如代码编译、日志分析、流媒体处理);
|
||||
|
||||
2. **负载过滤**:剔除当前负载已满(CurrentLoad ≥ MaxConcurrent)的Agent;
|
||||
|
||||
3. **最优算力选择**:优先选择负载最低、在线时间最稳定的Agent;
|
||||
|
||||
4. **任务绑定**:匹配成功后,将AgentID写入任务池对应任务,标记任务为running,下发任务至目标Agent。
|
||||
|
||||
#### 2\.2\.6 双向通信机制(核心闭环)
|
||||
|
||||
完全贴合你的设计:**网关作为唯一中转,携带AgentID完成CLI与Agent双向通信**
|
||||
|
||||
1. 正向链路(CLI → Agent):CLI提交任务 → 网关入库匹配Agent → 携带AgentID下发任务指令;
|
||||
|
||||
2. 反向链路(Agent → CLI):Agent执行完成/产生日志 → 结果回传网关 → 网关通过RequestID\+AgentID关联对应CLI会话 → 精准返回结果;
|
||||
|
||||
3. 链路隔离:多任务、多Agent、多CLI会话通过RequestID\+AgentID双维度隔离,杜绝消息错乱。
|
||||
|
||||
### 2\.3 K8s算力层
|
||||
|
||||
#### 2\.3\.1 核心职责
|
||||
|
||||
专注Agent容器生命周期管理,不参与业务调度,职责单一:Agent镜像部署、弹性扩缩容、故障重启、资源配额管控、节点销毁。
|
||||
|
||||
#### 2\.3\.2 Agent生命周期联动网关机制
|
||||
|
||||
1. **Agent创建启动**:Pod启动后自动读取K8s环境变量(网关地址、Pod信息),主动调用网关`/agent/register`接口完成注册,上报能力标签、并发上限、访问地址;网关更新Agent池,纳入算力调度。
|
||||
|
||||
2. **Agent运行中**:每10s主动上报心跳,同步当前负载,网关定时轮询兜底校验;
|
||||
|
||||
3. **Agent销毁/缩容**:Pod收到SIGTERM信号后,主动调用网关`/agent/unregister`接口优雅注销;网关立即剔除该Agent,终止其未完成任务,避免任务调度失效;
|
||||
|
||||
4. **Agent异常崩溃**:无主动注销时,依赖网关120s心跳超时机制自动剔除,实现故障自愈。
|
||||
|
||||
## 三、全链路完整执行流程
|
||||
|
||||
整合所有模块,端到端闭环流程如下:
|
||||
|
||||
1. **需求拆解上报**:用户在CLI/IDE输入需求,AI终端自主拆解任务,轻量任务本地异步执行,复杂任务结构化后提交网关;
|
||||
|
||||
2. **网关任务受理**:网关接收任务,生成/复用RequestID,写入Redis任务池,立即返回受理响应,避免CLI阻塞;
|
||||
|
||||
3. **算力匹配调度**:网关遍历在线Agent池,通过标签\+负载规则匹配最优Agent,绑定AgentID,下发任务;
|
||||
|
||||
4. **Agent执行任务**:K8s Agent接收任务,开始执行,定时上报执行进度、日志至网关;
|
||||
|
||||
5. **结果回传中转**:Agent执行完成后,将结果/错误信息回传网关;网关通过AgentID\+RequestID匹配原CLI会话;
|
||||
|
||||
6. **结果聚合反馈**:网关推送结果至CLI,AI终端聚合结果、整理答案,最终反馈给用户;
|
||||
|
||||
7. **状态归档清理**:网关更新任务为成功/失败状态,留存日志,到期自动清理冗余数据。
|
||||
|
||||
## 四、核心数据存储设计(Redis)
|
||||
|
||||
### 4\.1 任务池存储结构
|
||||
|
||||
- Hash结构:`task:info:{RequestID}`存储任务全量信息;
|
||||
|
||||
- Set结构:`task:pending` 待调度任务集合、`task:running` 执行中任务集合;
|
||||
|
||||
- TTL策略:任务数据默认保留24小时,自动过期清理,释放资源。
|
||||
|
||||
### 4\.2 Agent池存储结构
|
||||
|
||||
- Hash结构:`agent:info:{AgentID}` 存储Agent全量信息;
|
||||
|
||||
- ZSet结构:`agent:heartbeat` score=最后心跳时间,用于网关定时清理失效节点;
|
||||
|
||||
- Set索引:`agent:tag:{tag}` 按能力标签索引Agent,加速任务匹配效率。
|
||||
|
||||
## 五、容错与异常处理机制
|
||||
|
||||
### 5\.1 任务异常容错
|
||||
|
||||
- 任务超时:网关自定义任务超时时间,超时自动标记失败,释放Agent算力,反馈CLI;
|
||||
|
||||
- Agent离线:执行中Agent失联,网关立即终止任务,标记异常,支持CLI重提任务;
|
||||
|
||||
- 幂等性:基于RequestID实现任务幂等,避免CLI重试导致重复调度。
|
||||
|
||||
### 5\.2 Agent容错
|
||||
|
||||
- 心跳兜底:主动心跳\+网关定时轮询双机制,杜绝僵尸节点;
|
||||
|
||||
- 重启自愈:K8s负责Agent故障重启,重启后自动重新注册,纳入算力池。
|
||||
|
||||
### 5\.3 通信容错
|
||||
|
||||
- 断连不丢任务:任务数据持久化Redis,网关重启、网络波动不丢失任务状态;
|
||||
|
||||
- 精准重连匹配:重连后通过RequestID\+AgentID恢复任务链路,继续推送结果。
|
||||
|
||||
## 六、可视化运维能力(ElementUI)
|
||||
|
||||
基于ElementUI搭建后台,提供全维度可视化管控:
|
||||
|
||||
- 任务管理:查看所有任务状态、进度、关联AgentID、执行日志、结果详情;
|
||||
|
||||
- Agent管理:在线Agent列表、能力标签、实时负载、心跳状态、异常节点告警;
|
||||
|
||||
- 日志审计:全链路任务操作日志、Agent注册注销日志、通信记录溯源;
|
||||
|
||||
- 手动管控:支持手动取消任务、下线异常Agent、重置任务状态。
|
||||
|
||||
## 七、架构核心优势
|
||||
|
||||
1. **极简解耦**:三层架构职责清晰,K8s只管生命周期,网关只管调度通信,CLI只管需求拆解;
|
||||
|
||||
2. **弹性适配**:完美适配K8s扩缩容,Agent动态上下线自动感知,无需人工干预;
|
||||
|
||||
3. **高低速任务分流**:本地轻量异步处理,云端重型算力调度,资源利用率最大化;
|
||||
|
||||
4. **精准链路管控**:AgentID\+RequestID双维度绑定,通信不错乱、任务可溯源;
|
||||
|
||||
5. **轻量易落地**:FastAPI\+Redis技术栈轻量化,部署简单、开发成本低,配套可视化运维;
|
||||
|
||||
6. **可扩展性强**:后续可扩展多租户、权限管控、负载均衡优化、大文件中转、本地文件远程读写等能力。
|
||||
|
||||
## 八、落地实施步骤
|
||||
|
||||
1. 搭建网关基础服务:FastAPI接口、Redis数据结构、ElementUI可视化基础页面;
|
||||
|
||||
2. 实现Agent注册、注销、心跳上报、网关定时轮询保活能力;
|
||||
|
||||
3. 开发任务池管理、任务\-Agent匹配调度核心逻辑;
|
||||
|
||||
4. 打通CLI任务上报、网关调度、Agent执行、结果回传全链路;
|
||||
|
||||
5. 完善异常容错、日志审计、可视化运维功能;
|
||||
|
||||
6. 压测优化,适配K8s大规模Agent集群调度。
|
||||
|
||||
> (注:部分内容可能由 AI 生成)
|
||||
139
README.md
Normal file
139
README.md
Normal file
@ -0,0 +1,139 @@
|
||||
# A2A 智能网关(gateway)
|
||||
|
||||
基于设计文档《A2A需求分析-网关-K8s全链路架构详细设计方案》实现的智能网关核心服务。
|
||||
|
||||
本期范围:**网关核心 + 可视化运维后台**。Agent/CLI 为预留接口,K8s 算力层不实际部署。
|
||||
|
||||
## 技术栈
|
||||
|
||||
- 后端:Python + FastAPI + Uvicorn + redis-py(asyncio)
|
||||
- 存储:Redis(任务池 / Agent 池 / 调度索引 / 心跳 ZSet / 审计日志)
|
||||
- 前端:Vue 3 + Vite + Element Plus(完整运维后台)
|
||||
- 后台任务:asyncio 循环(心跳扫描 / 调度 / 超时检查),配 Redis 分布式锁防多实例重复执行
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
gateway/
|
||||
├── docker-compose.yml # 本地一键启动 Redis
|
||||
├── requirements.txt # Python 依赖
|
||||
├── .env.example # 环境变量示例
|
||||
├── backend/
|
||||
│ ├── app/
|
||||
│ │ ├── main.py # FastAPI 入口
|
||||
│ │ ├── config.py # 配置
|
||||
│ │ ├── constants.py # Redis 键名/TTL/状态常量
|
||||
│ │ ├── models/schemas.py # Pydantic 模型
|
||||
│ │ ├── repository/ # Redis 存储层(task/agent/log)
|
||||
│ │ ├── services/ # 业务服务(任务/Agent/调度/心跳/通信/超时/锁)
|
||||
│ │ ├── api/ # 路由(cli/agent/admin)
|
||||
│ │ ├── middleware.py # 日志中间件
|
||||
│ │ └── scheduler_loop.py # 后台任务循环
|
||||
│ └── tests/ # 单元与集成测试
|
||||
└── frontend/ # Vue3 + Element Plus 运维后台
|
||||
```
|
||||
|
||||
## 快速启动
|
||||
|
||||
### 1. 启动 Redis
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
无 Docker 时需本机提供 Redis 实例,并设置 `REDIS_URL`。
|
||||
|
||||
### 2. 启动后端
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
pip install -r ../requirements.txt
|
||||
# 复制 .env.example 为 .env 并按需修改
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
接口文档(Swagger):http://localhost:8000/docs
|
||||
|
||||
### 3. 启动前端运维后台
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
# 使用 Node >= 18(推荐 18/20/22)
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
访问 http://localhost:5173 ,Vite 已将 `/api` 代理到后端 `:8000`。
|
||||
|
||||
### 4. 运行后端测试
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
python -m pytest -q
|
||||
```
|
||||
|
||||
## 核心机制
|
||||
|
||||
### 任务池(Redis)
|
||||
|
||||
- `task:info:{request_id}`:任务全量信息(Hash)
|
||||
- `task:pending` / `task:running`:调度状态索引(Set)
|
||||
- 状态流转:pending → running → success/failed
|
||||
- TTL 默认 24h 自动归档;RequestID 幂等去重
|
||||
|
||||
### Agent 池(Redis)
|
||||
|
||||
- `agent:info:{agent_id}`:Agent 信息(Hash)
|
||||
- `agent:heartbeat`:心跳时间戳(ZSet,score=最后心跳)
|
||||
- `agent:tag:{tag}`:能力标签索引(Set)
|
||||
- 心跳保活:网关每 60s 扫描,连续 120s 未心跳标记 offline
|
||||
|
||||
### 规则调度
|
||||
|
||||
标签精准匹配 → 负载过滤(当前负载 < 并发上限)→ 最低负载(停留时间久者优先)→ 绑定 Agent 并下发。
|
||||
|
||||
### 认证
|
||||
|
||||
CLI 提交任务与 Agent 注册时,请求体必须携带 `auth` 字段,值须与网关配置的 `GATEWAY_AUTH`(`.env` 中设置)一致,否则返回 `401`。示例:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/cli/tasks \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"auth": "<GATEWAY_AUTH>", "task_type": "compile", "task_tags": ["build"], "payload": {"cmd": "python -m build"}}'
|
||||
```
|
||||
|
||||
### 预留 Agent 协议接口
|
||||
|
||||
> 所有接口中 `register` 必须携带 `auth` 字段,其余接口以已注册的 `agent_id` 关联身份。
|
||||
|
||||
| 接口 | 说明 |
|
||||
| --- | --- |
|
||||
| `POST /api/agent/register` | Agent 启动注册(`auth` + 能力标签、并发上限、地址) |
|
||||
| `POST /api/agent/unregister` | 优雅注销 |
|
||||
| `POST /api/agent/heartbeat` | 心跳(约每 10s,同步负载) |
|
||||
| `POST /api/agent/result` | 任务结果回传 |
|
||||
|
||||
### 通信中转
|
||||
|
||||
以 `RequestID + AgentID` 双维度关联 CLI 会话与 Agent,正向下发任务指令、反向回传进度与结果。本期为状态机闭环 + 日志记录,实际网络下发由 Agent 接入时扩展。
|
||||
|
||||
## 运维后台
|
||||
|
||||
- 任务管理:列表 / 筛选 / 详情 / 进度 / 取消
|
||||
- Agent 管理:卡片网格 / 标签 / 负载 / 心跳 / 离线高亮
|
||||
- 日志审计:全链路时间线,按 RequestID / AgentID 筛选
|
||||
- 手动管控:取消任务 / 重置任务 / 下线 Agent
|
||||
|
||||
## 环境变量
|
||||
|
||||
见 `.env.example`,关键参数:
|
||||
|
||||
| 变量 | 默认 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `REDIS_URL` | `redis://localhost:6379/0` | Redis 连接 |
|
||||
| `HEARTBEAT_SCAN_INTERVAL` | `60` | 心跳扫描间隔(秒) |
|
||||
| `AGENT_HEARTBEAT_TIMEOUT` | `120` | 心跳超时剔除阈值(秒) |
|
||||
| `DISPATCH_INTERVAL` | `2` | 调度循环间隔(秒) |
|
||||
| `TASK_TIMEOUT_CHECK_INTERVAL` | `5` | 任务超时检查间隔(秒) |
|
||||
| `DEFAULT_TASK_TIMEOUT` | `3600` | 任务默认超时(秒) |
|
||||
| `TASK_TTL` | `86400` | 任务数据保留 TTL(秒) |
|
||||
81
_test_sse.py
Normal file
81
_test_sse.py
Normal file
@ -0,0 +1,81 @@
|
||||
"""端到端验证:SSE 订阅 + 任务提交,确认网关主动推送 task_done。"""
|
||||
import asyncio
|
||||
import json
|
||||
import uuid
|
||||
|
||||
import httpx
|
||||
|
||||
BASE = "http://127.0.0.1:8000"
|
||||
AUTH = "gw_Hz8Qp3Km9f"
|
||||
SESSION = f"e2e-{uuid.uuid4().hex[:8]}"
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
async with httpx.AsyncClient(timeout=30) as client:
|
||||
# 1) 提交任务(携带 cli_session_id),任务类型用 agent 已注册的能力
|
||||
resp = await client.post(
|
||||
f"{BASE}/api/cli/tasks",
|
||||
json={
|
||||
"auth": AUTH,
|
||||
"task_type": "dev",
|
||||
"task_tags": ["code", "dev"],
|
||||
"description": "SSE 通知验证:创建临时文件",
|
||||
"cli_session_id": SESSION,
|
||||
"payload": {
|
||||
"type": "create_files",
|
||||
"files": [{"path": "_sse_probe.txt", "content": "hello-sse"}],
|
||||
},
|
||||
"timeout": 120,
|
||||
},
|
||||
)
|
||||
print("submit status:", resp.status_code)
|
||||
task = resp.json()
|
||||
request_id = task["request_id"]
|
||||
print("request_id:", request_id, "initial status:", task["status"])
|
||||
|
||||
# 2) 订阅 SSE,等待 task_done
|
||||
got = []
|
||||
|
||||
async def subscribe():
|
||||
try:
|
||||
async with client.stream(
|
||||
"GET",
|
||||
f"{BASE}/api/cli/events",
|
||||
params={"cli_session_id": SESSION, "auth": AUTH},
|
||||
) as stream:
|
||||
async for line in stream.aiter_lines():
|
||||
if line.startswith("data: "):
|
||||
evt = json.loads(line[6:])
|
||||
got.append(evt)
|
||||
print("SSE EVENT:", json.dumps(evt, ensure_ascii=False))
|
||||
if evt.get("request_id") == request_id:
|
||||
return
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print("SSE error:", exc)
|
||||
|
||||
# 3) 并行:订阅 + 轮询兜底打印(仅观察,不作为结论)
|
||||
await asyncio.gather(
|
||||
subscribe(),
|
||||
_poll_until_done(client, request_id),
|
||||
)
|
||||
if any(e.get("request_id") == request_id for e in got):
|
||||
print("\nRESULT: PASS - SSE task_done received without polling")
|
||||
else:
|
||||
print("\nRESULT: FAIL - no SSE event for this request_id")
|
||||
|
||||
|
||||
async def _poll_until_done(client, request_id: str) -> None:
|
||||
for _ in range(60):
|
||||
r = await client.get(f"{BASE}/api/cli/tasks/{request_id}")
|
||||
if r.status_code == 404:
|
||||
await asyncio.sleep(2)
|
||||
continue
|
||||
t = r.json()
|
||||
print("poll status:", t["status"])
|
||||
if t["status"] in ("success", "failed"):
|
||||
return
|
||||
await asyncio.sleep(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
0
backend/app/__init__.py
Normal file
0
backend/app/__init__.py
Normal file
0
backend/app/api/__init__.py
Normal file
0
backend/app/api/__init__.py
Normal file
103
backend/app/api/admin.py
Normal file
103
backend/app/api/admin.py
Normal file
@ -0,0 +1,103 @@
|
||||
"""后台管理接口:任务/Agent/日志/手动管控。"""
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
|
||||
from app.constants import TaskStatus
|
||||
from app.models.schemas import AgentInfo, LogEntry, TaskInfo
|
||||
from app.repository.log_repo import LogRepo
|
||||
from app.services.agent_service import AgentService
|
||||
from app.services.task_service import TaskService
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def get_task_service(request: Request) -> TaskService:
|
||||
return TaskService(request.app.state.redis)
|
||||
|
||||
|
||||
def get_agent_service(request: Request) -> AgentService:
|
||||
return AgentService(request.app.state.redis)
|
||||
|
||||
|
||||
def get_log_repo(request: Request) -> LogRepo:
|
||||
return LogRepo(request.app.state.redis)
|
||||
|
||||
|
||||
# ---------- 任务管理 ----------
|
||||
@router.get("/tasks", response_model=list[TaskInfo], summary="任务列表")
|
||||
async def list_tasks(
|
||||
status: Optional[TaskStatus] = None,
|
||||
limit: int = 100,
|
||||
svc: TaskService = Depends(get_task_service),
|
||||
):
|
||||
return await svc.list(status=status, limit=limit)
|
||||
|
||||
|
||||
@router.get("/tasks/{request_id}", response_model=TaskInfo, summary="任务详情")
|
||||
async def task_detail(request_id: str, svc: TaskService = Depends(get_task_service)):
|
||||
task = await svc.get(request_id)
|
||||
if not task:
|
||||
raise HTTPException(status_code=404, detail="task not found")
|
||||
return task
|
||||
|
||||
|
||||
@router.post("/tasks/{request_id}/cancel", summary="取消任务")
|
||||
async def admin_cancel(request_id: str, svc: TaskService = Depends(get_task_service)):
|
||||
if not await svc.cancel(request_id):
|
||||
raise HTTPException(status_code=400, detail="cannot cancel task")
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.post("/tasks/{request_id}/reset", summary="重置任务状态")
|
||||
async def admin_reset(request_id: str, svc: TaskService = Depends(get_task_service)):
|
||||
if not await svc.reset(request_id):
|
||||
raise HTTPException(status_code=400, detail="cannot reset task")
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
# ---------- Agent 管理 ----------
|
||||
@router.get("/agents", response_model=list[AgentInfo], summary="Agent 列表")
|
||||
async def list_agents(svc: AgentService = Depends(get_agent_service)):
|
||||
return await svc.all()
|
||||
|
||||
|
||||
@router.post("/agents/{agent_id}/offline", summary="下线 Agent")
|
||||
async def agent_offline(agent_id: str, svc: AgentService = Depends(get_agent_service)):
|
||||
if not await svc.take_offline(agent_id):
|
||||
raise HTTPException(status_code=404, detail="agent not found")
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
# ---------- 日志审计 ----------
|
||||
@router.get("/logs", response_model=list[LogEntry], summary="日志审计")
|
||||
async def logs(
|
||||
limit: int = 200,
|
||||
request_id: Optional[str] = None,
|
||||
agent_id: Optional[str] = None,
|
||||
repo: LogRepo = Depends(get_log_repo),
|
||||
):
|
||||
return await repo.list(limit=limit, request_id=request_id, agent_id=agent_id)
|
||||
|
||||
|
||||
# ---------- 概览 ----------
|
||||
@router.get("/overview", summary="概览统计")
|
||||
async def overview(
|
||||
task_svc: TaskService = Depends(get_task_service),
|
||||
agent_svc: AgentService = Depends(get_agent_service),
|
||||
):
|
||||
tasks = await task_svc.list(limit=1000)
|
||||
agents = await agent_svc.all()
|
||||
from collections import Counter
|
||||
|
||||
status_count = Counter(t.status.value for t in tasks)
|
||||
return {
|
||||
"task_total": len(tasks),
|
||||
"task_pending": status_count.get(TaskStatus.PENDING.value, 0),
|
||||
"task_running": status_count.get(TaskStatus.RUNNING.value, 0),
|
||||
"task_success": status_count.get(TaskStatus.SUCCESS.value, 0),
|
||||
"task_failed": status_count.get(TaskStatus.FAILED.value, 0),
|
||||
"agent_total": len(agents),
|
||||
"agent_online": sum(1 for a in agents if a.status.value == "online"),
|
||||
"agent_offline": sum(1 for a in agents if a.status.value == "offline"),
|
||||
}
|
||||
53
backend/app/api/agent.py
Normal file
53
backend/app/api/agent.py
Normal file
@ -0,0 +1,53 @@
|
||||
"""预留 Agent 协议接口:register / unregister / heartbeat / result。"""
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
|
||||
from app.models.schemas import AgentHeartbeat, AgentInfo, AgentRegister, TaskResult
|
||||
from app.services.agent_service import AgentService
|
||||
from app.services.relay import RelayService
|
||||
from app.services.security import require_auth
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def get_agent_service(request: Request) -> AgentService:
|
||||
return AgentService(request.app.state.redis)
|
||||
|
||||
|
||||
def get_relay_service(request: Request) -> RelayService:
|
||||
return RelayService(request.app.state.redis)
|
||||
|
||||
|
||||
@router.post("/register", response_model=AgentInfo, summary="Agent 注册")
|
||||
async def register(body: AgentRegister, svc: AgentService = Depends(get_agent_service)):
|
||||
require_auth(body.auth)
|
||||
return await svc.register(body)
|
||||
|
||||
|
||||
@router.post("/unregister", summary="Agent 注销")
|
||||
async def unregister(body: dict, svc: AgentService = Depends(get_agent_service)):
|
||||
from fastapi import HTTPException
|
||||
|
||||
agent_id = body.get("agent_id")
|
||||
if not agent_id:
|
||||
raise HTTPException(status_code=400, detail="agent_id required")
|
||||
if not await svc.unregister(agent_id):
|
||||
raise HTTPException(status_code=404, detail="agent not found")
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.post("/heartbeat", summary="Agent 心跳")
|
||||
async def heartbeat(body: AgentHeartbeat, svc: AgentService = Depends(get_agent_service)):
|
||||
from fastapi import HTTPException
|
||||
|
||||
if not await svc.heartbeat(body):
|
||||
raise HTTPException(status_code=404, detail="agent not registered")
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.post("/result", summary="Agent 结果回传")
|
||||
async def result(body: TaskResult, relay: RelayService = Depends(get_relay_service)):
|
||||
from fastapi import HTTPException
|
||||
|
||||
if not await relay.on_result(body):
|
||||
raise HTTPException(status_code=400, detail="result rejected")
|
||||
return {"ok": True}
|
||||
88
backend/app/api/cli.py
Normal file
88
backend/app/api/cli.py
Normal file
@ -0,0 +1,88 @@
|
||||
"""CLI 接口:任务提交/查询/取消/完成事件订阅。"""
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi.responses import StreamingResponse
|
||||
|
||||
from app.models.schemas import TaskInfo, TaskSubmit
|
||||
from app.services.notifier import TaskNotifier
|
||||
from app.services.security import require_auth
|
||||
from app.services.task_service import TaskService
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def get_task_service(request: Request) -> TaskService:
|
||||
return TaskService(request.app.state.redis)
|
||||
|
||||
|
||||
def get_notifier(request: Request) -> TaskNotifier:
|
||||
return TaskNotifier(request.app.state.redis)
|
||||
|
||||
|
||||
@router.get("/events", summary="订阅任务完成事件(SSE 长连接)")
|
||||
async def subscribe_events(
|
||||
request: Request,
|
||||
cli_session_id: str,
|
||||
auth: str = "",
|
||||
notifier: TaskNotifier = Depends(get_notifier),
|
||||
):
|
||||
"""任务到达终态时网关主动推送完成事件,无需轮询。
|
||||
|
||||
返回 SSE 流(text/event-stream),每帧形如:
|
||||
data: {"event": "task_done", "request_id": "...", "status": "success", "result": {...}}
|
||||
"""
|
||||
require_auth(auth)
|
||||
|
||||
channel = notifier.channel(cli_session_id)
|
||||
pubsub = request.app.state.redis.pubsub()
|
||||
await pubsub.subscribe(channel)
|
||||
replay = await notifier.replay(cli_session_id)
|
||||
|
||||
async def event_gen():
|
||||
try:
|
||||
# 先回放已完成的(连接晚于任务完成的场景)
|
||||
for evt in replay:
|
||||
yield f"data: {evt}\n\n"
|
||||
# 再实时监听
|
||||
async for message in pubsub.listen():
|
||||
if await request.is_disconnected():
|
||||
break
|
||||
if message["type"] != "message":
|
||||
continue
|
||||
data = message["data"]
|
||||
if isinstance(data, bytes):
|
||||
data = data.decode()
|
||||
yield f"data: {data}\n\n"
|
||||
finally:
|
||||
await pubsub.unsubscribe(channel)
|
||||
await pubsub.aclose()
|
||||
|
||||
return StreamingResponse(
|
||||
event_gen(),
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
|
||||
|
||||
@router.post("/tasks", response_model=TaskInfo, summary="提交任务")
|
||||
async def submit_task(body: TaskSubmit, svc: TaskService = Depends(get_task_service)):
|
||||
require_auth(body.auth)
|
||||
return await svc.submit(body)
|
||||
|
||||
|
||||
@router.get("/tasks/{request_id}", response_model=TaskInfo, summary="查询任务")
|
||||
async def get_task(request_id: str, svc: TaskService = Depends(get_task_service)):
|
||||
from fastapi import HTTPException
|
||||
|
||||
task = await svc.get(request_id)
|
||||
if not task:
|
||||
raise HTTPException(status_code=404, detail="task not found")
|
||||
return task
|
||||
|
||||
|
||||
@router.post("/tasks/{request_id}/cancel", summary="取消任务")
|
||||
async def cancel_task(request_id: str, svc: TaskService = Depends(get_task_service)):
|
||||
from fastapi import HTTPException
|
||||
|
||||
if not await svc.cancel(request_id):
|
||||
raise HTTPException(status_code=400, detail="cannot cancel task")
|
||||
return {"ok": True, "request_id": request_id}
|
||||
52
backend/app/config.py
Normal file
52
backend/app/config.py
Normal file
@ -0,0 +1,52 @@
|
||||
"""应用配置:通过环境变量 / .env 读取。"""
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
# 项目根目录(backend/ 的上一级)
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(
|
||||
env_file=str(PROJECT_ROOT / ".env"),
|
||||
env_file_encoding="utf-8",
|
||||
extra="ignore",
|
||||
)
|
||||
|
||||
# Redis
|
||||
redis_url: str = "redis://localhost:6379/0"
|
||||
|
||||
# 网关服务
|
||||
host: str = "0.0.0.0"
|
||||
port: int = 8000
|
||||
|
||||
# 心跳保活参数(秒)
|
||||
heartbeat_scan_interval: int = 60
|
||||
agent_heartbeat_timeout: int = 120
|
||||
agent_heartbeat_expected: int = 10
|
||||
|
||||
# 调度参数(秒)
|
||||
dispatch_interval: int = 2
|
||||
task_timeout_check_interval: int = 5
|
||||
|
||||
# 任务默认超时(秒)
|
||||
default_task_timeout: int = 3600
|
||||
|
||||
# 任务数据保留 TTL(秒)
|
||||
task_ttl: int = 86400
|
||||
|
||||
# 日志级别
|
||||
log_level: str = "INFO"
|
||||
|
||||
# 认证:CLI 提交任务 / Agent 注册时必须携带的密码
|
||||
gateway_auth: str = "dev-gateway-auth"
|
||||
|
||||
# 分布式锁相关
|
||||
lock_timeout: int = 30
|
||||
|
||||
# 最大并发调度轮次
|
||||
dispatch_batch_size: int = 20
|
||||
|
||||
|
||||
settings = Settings()
|
||||
38
backend/app/constants.py
Normal file
38
backend/app/constants.py
Normal file
@ -0,0 +1,38 @@
|
||||
"""Redis 键名与 TTL 常量、状态枚举。"""
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class TaskStatus(str, Enum):
|
||||
PENDING = "pending"
|
||||
RUNNING = "running"
|
||||
SUCCESS = "success"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
class AgentStatus(str, Enum):
|
||||
ONLINE = "online"
|
||||
OFFLINE = "offline"
|
||||
|
||||
|
||||
# ---- 任务池键 ----
|
||||
# 任务全量信息:hash task:info:{request_id}
|
||||
TASK_INFO_KEY = "task:info:{request_id}"
|
||||
# 待调度任务集合
|
||||
TASK_PENDING_SET = "task:pending"
|
||||
# 执行中任务集合
|
||||
TASK_RUNNING_SET = "task:running"
|
||||
|
||||
# ---- Agent 池键 ----
|
||||
# Agent 全量信息:hash agent:info:{agent_id}
|
||||
AGENT_INFO_KEY = "agent:info:{agent_id}"
|
||||
# 心跳时间戳 ZSet:score = 最后心跳时间
|
||||
AGENT_HEARTBEAT_ZSET = "agent:heartbeat"
|
||||
# 按能力标签索引 Agent:set agent:tag:{tag}
|
||||
AGENT_TAG_KEY = "agent:tag:{tag}"
|
||||
# 全部在线 Agent 索引
|
||||
AGENT_ALL_SET = "agent:all"
|
||||
|
||||
# ---- 分布式锁键 ----
|
||||
LOCK_SCHEDULER = "lock:scheduler"
|
||||
LOCK_HEARTBEAT = "lock:heartbeat"
|
||||
LOCK_TIMEOUT = "lock:timeout"
|
||||
62
backend/app/main.py
Normal file
62
backend/app/main.py
Normal file
@ -0,0 +1,62 @@
|
||||
"""FastAPI 应用入口:注册路由、CORS、启动后台任务循环。"""
|
||||
import logging
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from app.config import settings
|
||||
from app.api import admin, agent, cli
|
||||
from app.middleware import RequestLogMiddleware
|
||||
from app.scheduler_loop import SchedulerLoop
|
||||
from app.repository.redis_client import get_redis_pool
|
||||
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, settings.log_level.upper(), logging.INFO),
|
||||
format="%(asctime)s %(levelname)s [%(name)s] %(message)s",
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
# 启动时会话
|
||||
redis = await get_redis_pool()
|
||||
loop = SchedulerLoop(redis)
|
||||
app.state.redis = redis
|
||||
app.state.scheduler_loop = loop
|
||||
await loop.start()
|
||||
logger.info("Gateway started, host=%s port=%s", settings.host, settings.port)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
await loop.stop()
|
||||
await redis.aclose()
|
||||
logger.info("Gateway stopped")
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title="A2A Gateway",
|
||||
description="A2A 智能网关:任务池 / Agent 池 / 规则调度 / 通信中转 / 可视化运维",
|
||||
version="1.0.0",
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
# CORS:允许前端开发服务器访问
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
app.add_middleware(RequestLogMiddleware)
|
||||
|
||||
app.include_router(cli.router, prefix="/api/cli", tags=["CLI"])
|
||||
app.include_router(agent.router, prefix="/api/agent", tags=["Agent"])
|
||||
app.include_router(admin.router, prefix="/api/admin", tags=["Admin"])
|
||||
|
||||
|
||||
@app.get("/health", tags=["health"])
|
||||
async def health():
|
||||
return {"status": "ok"}
|
||||
33
backend/app/middleware.py
Normal file
33
backend/app/middleware.py
Normal file
@ -0,0 +1,33 @@
|
||||
"""日志中间件:记录 RequestID/AgentID 链路与访问日志,避免敏感载荷与日志刷屏。"""
|
||||
import logging
|
||||
import time
|
||||
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.requests import Request
|
||||
|
||||
logger = logging.getLogger("access")
|
||||
|
||||
|
||||
class RequestLogMiddleware(BaseHTTPMiddleware):
|
||||
async def dispatch(self, request: Request, call_next):
|
||||
start = time.perf_counter()
|
||||
# 心跳类接口降级为 debug,避免刷屏
|
||||
is_heartbeat = request.url.path.endswith("/heartbeat")
|
||||
if logger.isEnabledFor(logging.DEBUG) or not is_heartbeat:
|
||||
logger.info(
|
||||
"%s %s path=%s",
|
||||
request.method,
|
||||
request.client.host if request.client else "-",
|
||||
request.url.path,
|
||||
)
|
||||
response = await call_next(request)
|
||||
cost_ms = (time.perf_counter() - start) * 1000
|
||||
if not is_heartbeat or logger.isEnabledFor(logging.DEBUG):
|
||||
logger.info(
|
||||
"%s %s -> %s (%dms)",
|
||||
request.method,
|
||||
request.url.path,
|
||||
response.status_code,
|
||||
round(cost_ms, 2),
|
||||
)
|
||||
return response
|
||||
0
backend/app/models/__init__.py
Normal file
0
backend/app/models/__init__.py
Normal file
92
backend/app/models/schemas.py
Normal file
92
backend/app/models/schemas.py
Normal file
@ -0,0 +1,92 @@
|
||||
"""Pydantic 数据模型。"""
|
||||
import time
|
||||
import uuid
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from app.constants import AgentStatus, TaskStatus
|
||||
|
||||
|
||||
def new_request_id() -> str:
|
||||
return uuid.uuid4().hex
|
||||
|
||||
|
||||
# ---------- 任务 ----------
|
||||
class TaskSubmit(BaseModel):
|
||||
"""CLI 任务上报请求体。"""
|
||||
|
||||
auth: str = Field(..., description="网关认证密码,需与 GATEWAY_AUTH 一致")
|
||||
request_id: str | None = Field(default=None, description="幂等键,不传则自动生成")
|
||||
task_type: str = Field(..., description="任务类型,如 compile/build/test")
|
||||
task_tags: list[str] = Field(default_factory=list, description="任务能力标签")
|
||||
description: str | None = Field(default=None, description="任务描述")
|
||||
payload: dict[str, Any] = Field(default_factory=dict, description="任务载荷")
|
||||
cli_session_id: str | None = Field(default=None, description="CLI 会话标识")
|
||||
timeout: int = Field(default=0, description="超时秒数,0 使用默认值")
|
||||
local_workdir: str | None = Field(default=None, description="本地工作目录信息")
|
||||
|
||||
|
||||
class TaskInfo(BaseModel):
|
||||
request_id: str
|
||||
task_type: str
|
||||
task_tags: list[str] = Field(default_factory=list)
|
||||
description: str | None = None
|
||||
payload: dict[str, Any] = Field(default_factory=dict)
|
||||
status: TaskStatus = TaskStatus.PENDING
|
||||
agent_id: str | None = None
|
||||
cli_session_id: str | None = None
|
||||
create_time: float = Field(default_factory=time.time)
|
||||
timeout: int = 0
|
||||
progress: int = 0
|
||||
result: dict[str, Any] | None = None
|
||||
error_info: str | None = None
|
||||
|
||||
|
||||
# ---------- Agent ----------
|
||||
class AgentRegister(BaseModel):
|
||||
"""Agent 启动注册请求体。"""
|
||||
|
||||
auth: str = Field(..., description="网关认证密码,需与 GATEWAY_AUTH 一致")
|
||||
agent_id: str
|
||||
endpoint: str
|
||||
agent_tags: list[str] = Field(default_factory=list)
|
||||
max_concurrent: int = Field(default=1, ge=1)
|
||||
current_load: int = Field(default=0, ge=0)
|
||||
|
||||
|
||||
class AgentHeartbeat(BaseModel):
|
||||
agent_id: str
|
||||
current_load: int = Field(default=0, ge=0)
|
||||
|
||||
|
||||
class AgentInfo(BaseModel):
|
||||
agent_id: str
|
||||
endpoint: str
|
||||
agent_tags: list[str] = Field(default_factory=list)
|
||||
max_concurrent: int = 1
|
||||
current_load: int = 0
|
||||
last_heartbeat: float = Field(default_factory=time.time)
|
||||
status: AgentStatus = AgentStatus.ONLINE
|
||||
create_time: float = Field(default_factory=time.time)
|
||||
|
||||
|
||||
# ---------- 结果回传 ----------
|
||||
class TaskResult(BaseModel):
|
||||
request_id: str
|
||||
agent_id: str
|
||||
status: TaskStatus = TaskStatus.SUCCESS
|
||||
progress: int = Field(default=100, ge=0, le=100)
|
||||
result: dict[str, Any] | None = None
|
||||
error_info: str | None = None
|
||||
|
||||
|
||||
# ---------- 日志 ----------
|
||||
class LogEntry(BaseModel):
|
||||
ts: float = Field(default_factory=time.time)
|
||||
level: str = "info"
|
||||
source: str = "gateway" # gateway / agent / cli / admin
|
||||
scope: str = "task" # task / agent / system
|
||||
request_id: str | None = None
|
||||
agent_id: str | None = None
|
||||
message: str
|
||||
0
backend/app/repository/__init__.py
Normal file
0
backend/app/repository/__init__.py
Normal file
146
backend/app/repository/agent_repo.py
Normal file
146
backend/app/repository/agent_repo.py
Normal file
@ -0,0 +1,146 @@
|
||||
"""Agent 池 Redis 存储层。"""
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app import constants as C
|
||||
from app.constants import AgentStatus
|
||||
from app.models.schemas import AgentInfo
|
||||
|
||||
|
||||
class AgentRepo:
|
||||
def __init__(self, redis: aioredis.Redis):
|
||||
self.redis = redis
|
||||
|
||||
@staticmethod
|
||||
def _info_key(agent_id: str) -> str:
|
||||
return C.AGENT_INFO_KEY.format(agent_id=agent_id)
|
||||
|
||||
@staticmethod
|
||||
def _tag_key(tag: str) -> str:
|
||||
return C.AGENT_TAG_KEY.format(tag=tag)
|
||||
|
||||
# ---------- 写入 ----------
|
||||
async def upsert(self, agent: AgentInfo, *, heartbeat: bool = False) -> bool:
|
||||
"""写入 Agent 信息,返回是否新建。非心跳时为全量注册。"""
|
||||
key = self._info_key(agent.agent_id)
|
||||
existed = await self.redis.exists(key)
|
||||
if not existed:
|
||||
await self.redis.sadd(C.AGENT_ALL_SET, agent.agent_id)
|
||||
# 建立标签索引
|
||||
for tag in agent.agent_tags:
|
||||
await self.redis.sadd(self._tag_key(tag), agent.agent_id)
|
||||
mapping: dict[str, Any] = {
|
||||
"agent_id": agent.agent_id,
|
||||
"endpoint": agent.endpoint,
|
||||
"agent_tags": ",".join(agent.agent_tags),
|
||||
"max_concurrent": str(agent.max_concurrent),
|
||||
"current_load": str(agent.current_load),
|
||||
"last_heartbeat": str(agent.last_heartbeat),
|
||||
"status": agent.status.value,
|
||||
"create_time": str(agent.create_time),
|
||||
}
|
||||
await self.redis.hset(key, mapping=mapping)
|
||||
# 心跳 ZSet 记录时间戳
|
||||
await self.redis.zadd(C.AGENT_HEARTBEAT_ZSET, {agent.agent_id: agent.last_heartbeat})
|
||||
return not existed
|
||||
|
||||
async def update_status(self, agent_id: str, status: AgentStatus) -> bool:
|
||||
key = self._info_key(agent_id)
|
||||
if not await self.redis.exists(key):
|
||||
return False
|
||||
await self.redis.hset(key, "status", status.value)
|
||||
if status == AgentStatus.OFFLINE:
|
||||
await self.redis.zrem(C.AGENT_HEARTBEAT_ZSET, agent_id)
|
||||
return True
|
||||
|
||||
async def beat(self, agent_id: str, current_load: int) -> bool:
|
||||
"""更新心跳时间与负载。"""
|
||||
key = self._info_key(agent_id)
|
||||
if not await self.redis.exists(key):
|
||||
return False
|
||||
now = time.time()
|
||||
await self.redis.hset(
|
||||
key,
|
||||
mapping={"last_heartbeat": str(now), "current_load": str(current_load), "status": AgentStatus.ONLINE.value},
|
||||
)
|
||||
await self.redis.zadd(C.AGENT_HEARTBEAT_ZSET, {agent_id: now})
|
||||
return True
|
||||
|
||||
async def adjust_load(self, agent_id: str, delta: int) -> bool:
|
||||
"""原子调整 Agent 负载(delta 可为正/负),负载下限为 0。"""
|
||||
key = self._info_key(agent_id)
|
||||
if not await self.redis.exists(key):
|
||||
return False
|
||||
# hincrby 为原子增量操作,兼容 fakeredis(不支持 Lua eval)
|
||||
new = await self.redis.hincrby(key, "current_load", delta)
|
||||
if new < 0:
|
||||
await self.redis.hset(key, "current_load", "0")
|
||||
return True
|
||||
|
||||
# ---------- 读取 ----------
|
||||
async def get(self, agent_id: str) -> AgentInfo | None:
|
||||
raw = await self.redis.hgetall(self._info_key(agent_id))
|
||||
if not raw:
|
||||
return None
|
||||
return AgentInfo(
|
||||
agent_id=raw.get("agent_id", ""),
|
||||
endpoint=raw.get("endpoint", ""),
|
||||
agent_tags=[t for t in raw.get("agent_tags", "").split(",") if t],
|
||||
max_concurrent=int(raw.get("max_concurrent", 1) or 1),
|
||||
current_load=int(raw.get("current_load", 0) or 0),
|
||||
last_heartbeat=float(raw.get("last_heartbeat", 0) or 0),
|
||||
status=AgentStatus(raw.get("status", AgentStatus.ONLINE.value)),
|
||||
create_time=float(raw.get("create_time", 0) or 0),
|
||||
)
|
||||
|
||||
async def online(self) -> list[AgentInfo]:
|
||||
return [a for a in await self.all() if a.status == AgentStatus.ONLINE]
|
||||
|
||||
async def all(self) -> list[AgentInfo]:
|
||||
ids = list(await self.redis.smembers(C.AGENT_ALL_SET))
|
||||
out = []
|
||||
for aid in ids:
|
||||
a = await self.get(aid)
|
||||
if a:
|
||||
out.append(a)
|
||||
return out
|
||||
|
||||
async def by_tags(self, tags: list[str]) -> list[AgentInfo]:
|
||||
"""按标签索引取 Agent 池(取所有标签的交集,无标签则返回全部在线)。"""
|
||||
if not tags:
|
||||
return await self.online()
|
||||
keys = [self._tag_key(t) for t in tags]
|
||||
if len(keys) == 1:
|
||||
ids = list(await self.redis.smembers(keys[0]))
|
||||
else:
|
||||
await self.redis.sinterstore("agent:tmp:intersect", keys)
|
||||
ids = list(await self.redis.smembers("agent:tmp:intersect"))
|
||||
await self.redis.delete("agent:tmp:intersect")
|
||||
out = []
|
||||
for aid in ids:
|
||||
a = await self.get(aid)
|
||||
if a and a.status == AgentStatus.ONLINE:
|
||||
out.append(a)
|
||||
return out
|
||||
|
||||
async def remove(self, agent_id: str) -> None:
|
||||
"""注销:删除信息、心跳 ZSet、标签索引、全量索引。"""
|
||||
a = await self.get(agent_id)
|
||||
if a:
|
||||
for tag in a.agent_tags:
|
||||
await self.redis.srem(self._tag_key(tag), agent_id)
|
||||
await self.redis.delete(self._info_key(agent_id))
|
||||
await self.redis.zrem(C.AGENT_HEARTBEAT_ZSET, agent_id)
|
||||
await self.redis.srem(C.AGENT_ALL_SET, agent_id)
|
||||
|
||||
async def stale_agents(self, timeout: float) -> list[str]:
|
||||
"""返回超过 timeout 秒未心跳的 Agent ID(基于 ZSet score)。"""
|
||||
cutoff = time.time() - timeout
|
||||
scored = await self.redis.zrangebyscore(C.AGENT_HEARTBEAT_ZSET, 0, cutoff)
|
||||
return list(scored)
|
||||
|
||||
async def heartbeat_map(self) -> dict[str, float]:
|
||||
"""返回 {agent_id: last_heartbeat}。"""
|
||||
return {k: float(v) for k, v in await self.redis.zrange(C.AGENT_HEARTBEAT_ZSET, 0, -1, withscores=True)}
|
||||
50
backend/app/repository/log_repo.py
Normal file
50
backend/app/repository/log_repo.py
Normal file
@ -0,0 +1,50 @@
|
||||
"""日志审计存储层:全链路操作/注册注销/通信记录。"""
|
||||
import time
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.models.schemas import LogEntry
|
||||
|
||||
LOG_KEY = "log:audit"
|
||||
LOG_MAX = 2000
|
||||
|
||||
|
||||
class LogRepo:
|
||||
def __init__(self, redis: aioredis.Redis):
|
||||
self.redis = redis
|
||||
|
||||
async def append(
|
||||
self,
|
||||
message: str,
|
||||
*,
|
||||
level: str = "info",
|
||||
source: str = "gateway",
|
||||
scope: str = "system",
|
||||
request_id: str | None = None,
|
||||
agent_id: str | None = None,
|
||||
) -> None:
|
||||
entry = LogEntry(
|
||||
level=level,
|
||||
source=source,
|
||||
scope=scope,
|
||||
request_id=request_id,
|
||||
agent_id=agent_id,
|
||||
message=message,
|
||||
)
|
||||
await self.redis.lpush(LOG_KEY, entry.model_dump_json())
|
||||
await self.redis.ltrim(LOG_KEY, 0, LOG_MAX - 1)
|
||||
|
||||
async def list(self, limit: int = 200, request_id: str | None = None, agent_id: str | None = None) -> list[LogEntry]:
|
||||
raw = await self.redis.lrange(LOG_KEY, 0, limit - 1)
|
||||
out = []
|
||||
for item in raw:
|
||||
try:
|
||||
entry = LogEntry.model_validate_json(item)
|
||||
except Exception:
|
||||
continue
|
||||
if request_id and entry.request_id != request_id:
|
||||
continue
|
||||
if agent_id and entry.agent_id != agent_id:
|
||||
continue
|
||||
out.append(entry)
|
||||
return out
|
||||
14
backend/app/repository/redis_client.py
Normal file
14
backend/app/repository/redis_client.py
Normal file
@ -0,0 +1,14 @@
|
||||
"""Redis 连接池与共享访问封装。"""
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.config import settings
|
||||
|
||||
_pool: aioredis.Redis | None = None
|
||||
|
||||
|
||||
async def get_redis_pool() -> aioredis.Redis:
|
||||
"""返回全局共享的 Redis 客户端(懒初始化)。"""
|
||||
global _pool
|
||||
if _pool is None:
|
||||
_pool = aioredis.from_url(settings.redis_url, decode_responses=True)
|
||||
return _pool
|
||||
166
backend/app/repository/task_repo.py
Normal file
166
backend/app/repository/task_repo.py
Normal file
@ -0,0 +1,166 @@
|
||||
"""任务池 Redis 存储层。"""
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app import constants as C
|
||||
from app.config import settings
|
||||
from app.constants import TaskStatus
|
||||
from app.models.schemas import TaskInfo
|
||||
|
||||
|
||||
class TaskRepo:
|
||||
def __init__(self, redis: aioredis.Redis):
|
||||
self.redis = redis
|
||||
|
||||
@staticmethod
|
||||
def _info_key(request_id: str) -> str:
|
||||
return C.TASK_INFO_KEY.format(request_id=request_id)
|
||||
|
||||
# ---------- 写入 ----------
|
||||
async def create(self, task: TaskInfo) -> bool:
|
||||
"""录入任务(幂等),返回是否新建。"""
|
||||
key = self._info_key(task.request_id)
|
||||
existed = await self.redis.exists(key)
|
||||
if existed:
|
||||
return False
|
||||
await self.redis.hset(
|
||||
key,
|
||||
mapping={
|
||||
"request_id": task.request_id,
|
||||
"task_type": task.task_type,
|
||||
"task_tags": ",".join(task.task_tags),
|
||||
"description": task.description or "",
|
||||
"payload": _json(task.payload),
|
||||
"status": task.status.value,
|
||||
"agent_id": task.agent_id or "",
|
||||
"cli_session_id": task.cli_session_id or "",
|
||||
"create_time": str(task.create_time),
|
||||
"timeout": str(task.timeout),
|
||||
"progress": str(task.progress),
|
||||
"result": _json(task.result),
|
||||
"error_info": task.error_info or "",
|
||||
},
|
||||
)
|
||||
await self.redis.expire(key, settings.task_ttl)
|
||||
await self.redis.sadd(C.TASK_PENDING_SET, task.request_id)
|
||||
return True
|
||||
|
||||
async def update(
|
||||
self,
|
||||
request_id: str,
|
||||
*,
|
||||
status: TaskStatus | None = None,
|
||||
agent_id: str | None = None,
|
||||
progress: int | None = None,
|
||||
result: dict | None = None,
|
||||
error_info: str | None = None,
|
||||
) -> bool:
|
||||
key = self._info_key(request_id)
|
||||
if not await self.redis.exists(key):
|
||||
return False
|
||||
mapping: dict[str, Any] = {}
|
||||
if status is not None:
|
||||
mapping["status"] = status.value
|
||||
if agent_id is not None:
|
||||
mapping["agent_id"] = agent_id
|
||||
if progress is not None:
|
||||
mapping["progress"] = str(progress)
|
||||
if result is not None:
|
||||
mapping["result"] = _json(result)
|
||||
if error_info is not None:
|
||||
mapping["error_info"] = error_info
|
||||
if mapping:
|
||||
await self.redis.hset(key, mapping=mapping)
|
||||
return True
|
||||
|
||||
# ---------- 状态集合维护 ----------
|
||||
async def mark_pending(self, request_id: str) -> None:
|
||||
await self.redis.sadd(C.TASK_PENDING_SET, request_id)
|
||||
await self.redis.srem(C.TASK_RUNNING_SET, request_id)
|
||||
|
||||
async def mark_running(self, request_id: str) -> None:
|
||||
await self.redis.sadd(C.TASK_RUNNING_SET, request_id)
|
||||
await self.redis.srem(C.TASK_PENDING_SET, request_id)
|
||||
|
||||
async def mark_finished(self, request_id: str) -> None:
|
||||
await self.redis.srem(C.TASK_PENDING_SET, request_id)
|
||||
await self.redis.srem(C.TASK_RUNNING_SET, request_id)
|
||||
|
||||
async def pending_tasks(self) -> list[str]:
|
||||
return list(await self.redis.smembers(C.TASK_PENDING_SET))
|
||||
|
||||
async def running_tasks(self) -> list[str]:
|
||||
return list(await self.redis.smembers(C.TASK_RUNNING_SET))
|
||||
|
||||
async def remove_pending(self, request_id: str) -> None:
|
||||
await self.redis.srem(C.TASK_PENDING_SET, request_id)
|
||||
|
||||
# ---------- 读取 ----------
|
||||
async def get(self, request_id: str) -> TaskInfo | None:
|
||||
raw = await self.redis.hgetall(self._info_key(request_id))
|
||||
if not raw:
|
||||
return None
|
||||
return await self._to_task(raw)
|
||||
|
||||
async def _to_task(self, raw: dict) -> TaskInfo:
|
||||
return TaskInfo(
|
||||
request_id=raw.get("request_id", ""),
|
||||
task_type=raw.get("task_type", ""),
|
||||
task_tags=[t for t in raw.get("task_tags", "").split(",") if t],
|
||||
description=raw.get("description") or None,
|
||||
payload=_unjson(raw.get("payload")),
|
||||
status=TaskStatus(raw.get("status", TaskStatus.PENDING.value)),
|
||||
agent_id=raw.get("agent_id") or None,
|
||||
cli_session_id=raw.get("cli_session_id") or None,
|
||||
create_time=float(raw.get("create_time", 0) or 0),
|
||||
timeout=int(raw.get("timeout", 0) or 0),
|
||||
progress=int(raw.get("progress", 0) or 0),
|
||||
result=_unjson(raw.get("result")),
|
||||
error_info=raw.get("error_info") or None,
|
||||
)
|
||||
|
||||
async def list(self, status: TaskStatus | None = None, limit: int = 100) -> list[TaskInfo]:
|
||||
"""按状态过滤返回任务列表(含终态任务,通过 scan 全量扫描 task:info:*)。"""
|
||||
ids = await self.scan_ids()
|
||||
out = []
|
||||
for rid in ids:
|
||||
t = await self.get(rid)
|
||||
if not t:
|
||||
continue
|
||||
if status is not None and t.status != status:
|
||||
continue
|
||||
out.append(t)
|
||||
# 按创建时间倒序
|
||||
out.sort(key=lambda x: x.create_time, reverse=True)
|
||||
return out[:limit]
|
||||
|
||||
async def scan_ids(self) -> list[str]:
|
||||
"""扫描所有 task:info:* 键,返回 RequestID 列表。"""
|
||||
prefix = C.TASK_INFO_KEY.replace("{request_id}", "")
|
||||
ids = []
|
||||
async for key in self.redis.scan_iter(match=f"{prefix}*", count=1000):
|
||||
ids.append(key[len(prefix):])
|
||||
return ids
|
||||
|
||||
async def delete(self, request_id: str) -> None:
|
||||
await self.redis.delete(self._info_key(request_id))
|
||||
await self.mark_finished(request_id)
|
||||
|
||||
|
||||
def _json(obj: Any) -> str:
|
||||
import json
|
||||
|
||||
return json.dumps(obj, ensure_ascii=False) if obj is not None else ""
|
||||
|
||||
|
||||
def _unjson(s: str | None) -> Any:
|
||||
import json
|
||||
|
||||
if not s:
|
||||
return None
|
||||
try:
|
||||
return json.loads(s)
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
70
backend/app/scheduler_loop.py
Normal file
70
backend/app/scheduler_loop.py
Normal file
@ -0,0 +1,70 @@
|
||||
"""后台任务循环启动器:心跳扫描 / 任务调度 / 任务超时,带分布式锁防多实例重复执行。"""
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.config import settings
|
||||
from app.constants import LOCK_HEARTBEAT, LOCK_SCHEDULER, LOCK_TIMEOUT
|
||||
from app.services.heartbeat import HeartbeatService
|
||||
from app.services.lock import Lock
|
||||
from app.services.relay import RelayService
|
||||
from app.services.scheduler import Scheduler
|
||||
from app.services.timeout import TimeoutService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SchedulerLoop:
|
||||
def __init__(self, redis: aioredis.Redis):
|
||||
self.redis = redis
|
||||
self._tasks: list[asyncio.Task] = []
|
||||
|
||||
async def start(self) -> None:
|
||||
self._tasks.append(asyncio.create_task(self._heartbeat_loop()))
|
||||
self._tasks.append(asyncio.create_task(self._dispatch_loop()))
|
||||
self._tasks.append(asyncio.create_task(self._timeout_loop()))
|
||||
logger.info("scheduler loops started")
|
||||
|
||||
async def stop(self) -> None:
|
||||
for t in self._tasks:
|
||||
t.cancel()
|
||||
for t in self._tasks:
|
||||
try:
|
||||
await t
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._tasks.clear()
|
||||
|
||||
async def _heartbeat_loop(self) -> None:
|
||||
hb = HeartbeatService(self.redis)
|
||||
while True:
|
||||
await asyncio.sleep(settings.heartbeat_scan_interval)
|
||||
lock = Lock(self.redis, LOCK_HEARTBEAT)
|
||||
if await lock.acquire():
|
||||
try:
|
||||
await hb.scan()
|
||||
finally:
|
||||
await lock.release()
|
||||
|
||||
async def _dispatch_loop(self) -> None:
|
||||
sched = Scheduler(self.redis, relay=RelayService(self.redis))
|
||||
while True:
|
||||
await asyncio.sleep(settings.dispatch_interval)
|
||||
lock = Lock(self.redis, LOCK_SCHEDULER)
|
||||
if await lock.acquire():
|
||||
try:
|
||||
await sched.dispatch_pending(settings.dispatch_batch_size)
|
||||
finally:
|
||||
await lock.release()
|
||||
|
||||
async def _timeout_loop(self) -> None:
|
||||
to = TimeoutService(self.redis)
|
||||
while True:
|
||||
await asyncio.sleep(settings.task_timeout_check_interval)
|
||||
lock = Lock(self.redis, LOCK_TIMEOUT)
|
||||
if await lock.acquire():
|
||||
try:
|
||||
await to.check()
|
||||
finally:
|
||||
await lock.release()
|
||||
0
backend/app/services/__init__.py
Normal file
0
backend/app/services/__init__.py
Normal file
72
backend/app/services/agent_service.py
Normal file
72
backend/app/services/agent_service.py
Normal file
@ -0,0 +1,72 @@
|
||||
"""Agent 池服务:注册/注销/心跳/负载/离线剔除。"""
|
||||
import logging
|
||||
import time
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.constants import AgentStatus
|
||||
from app.models.schemas import AgentHeartbeat, AgentInfo, AgentRegister
|
||||
from app.repository.agent_repo import AgentRepo
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AgentService:
|
||||
def __init__(self, redis: aioredis.Redis, repo: AgentRepo | None = None):
|
||||
self.redis = redis
|
||||
self.repo = repo or AgentRepo(redis)
|
||||
|
||||
async def register(self, reg: AgentRegister) -> AgentInfo:
|
||||
agent = AgentInfo(
|
||||
agent_id=reg.agent_id,
|
||||
endpoint=reg.endpoint,
|
||||
agent_tags=reg.agent_tags,
|
||||
max_concurrent=reg.max_concurrent,
|
||||
current_load=reg.current_load,
|
||||
status=AgentStatus.ONLINE,
|
||||
create_time=time.time(),
|
||||
last_heartbeat=time.time(),
|
||||
)
|
||||
created = await self.repo.upsert(agent)
|
||||
logger.info("agent registered agent_id=%s created=%s", agent.agent_id, created)
|
||||
return agent
|
||||
|
||||
async def unregister(self, agent_id: str) -> bool:
|
||||
existed = await self.repo.get(agent_id)
|
||||
if not existed:
|
||||
return False
|
||||
await self.repo.remove(agent_id)
|
||||
logger.info("agent unregistered agent_id=%s", agent_id)
|
||||
return True
|
||||
|
||||
async def heartbeat(self, hb: AgentHeartbeat) -> bool:
|
||||
ok = await self.repo.beat(hb.agent_id, hb.current_load)
|
||||
if not ok:
|
||||
logger.warning("heartbeat from unknown agent agent_id=%s", hb.agent_id)
|
||||
return ok
|
||||
|
||||
async def online(self) -> list[AgentInfo]:
|
||||
return await self.repo.online()
|
||||
|
||||
async def all(self) -> list[AgentInfo]:
|
||||
return await self.repo.all()
|
||||
|
||||
async def get(self, agent_id: str) -> AgentInfo | None:
|
||||
return await self.repo.get(agent_id)
|
||||
|
||||
async def take_offline(self, agent_id: str) -> bool:
|
||||
"""手动下线 Agent。"""
|
||||
a = await self.repo.get(agent_id)
|
||||
if not a:
|
||||
return False
|
||||
await self.repo.update_status(agent_id, AgentStatus.OFFLINE)
|
||||
logger.info("agent taken offline agent_id=%s", agent_id)
|
||||
return True
|
||||
|
||||
async def purge_stale(self, timeout: float) -> list[str]:
|
||||
"""剔除超时未心跳的 Agent。"""
|
||||
stale = await self.repo.stale_agents(timeout)
|
||||
for aid in stale:
|
||||
await self.repo.update_status(aid, AgentStatus.OFFLINE)
|
||||
logger.info("agent purged (stale) agent_id=%s", aid)
|
||||
return stale
|
||||
24
backend/app/services/heartbeat.py
Normal file
24
backend/app/services/heartbeat.py
Normal file
@ -0,0 +1,24 @@
|
||||
"""心跳保活服务:扫描 Agent 池,剔除超时未心跳节点。"""
|
||||
import logging
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.config import settings
|
||||
from app.constants import AgentStatus
|
||||
from app.repository.agent_repo import AgentRepo
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class HeartbeatService:
|
||||
def __init__(self, redis: aioredis.Redis, agent_repo: AgentRepo | None = None):
|
||||
self.redis = redis
|
||||
self.agent_repo = agent_repo or AgentRepo(redis)
|
||||
|
||||
async def scan(self) -> int:
|
||||
"""扫描并剔除超时 Agent,返回剔除数量。"""
|
||||
stale = await self.agent_repo.stale_agents(settings.agent_heartbeat_timeout)
|
||||
for aid in stale:
|
||||
await self.agent_repo.update_status(aid, AgentStatus.OFFLINE)
|
||||
logger.info("heartbeat: agent offline (stale) agent_id=%s", aid)
|
||||
return len(stale)
|
||||
35
backend/app/services/lock.py
Normal file
35
backend/app/services/lock.py
Normal file
@ -0,0 +1,35 @@
|
||||
"""Redis 分布式锁(SET NX EX)。"""
|
||||
import secrets
|
||||
import time
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.config import settings
|
||||
|
||||
|
||||
class Lock:
|
||||
def __init__(self, redis: aioredis.Redis, name: str, timeout: int = settings.lock_timeout):
|
||||
self.redis = redis
|
||||
self.name = name
|
||||
self.timeout = timeout
|
||||
self._token = secrets.token_hex(8)
|
||||
|
||||
async def acquire(self) -> bool:
|
||||
ok = await self.redis.set(self.name, self._token, nx=True, ex=self.timeout)
|
||||
return bool(ok)
|
||||
|
||||
async def release(self) -> None:
|
||||
# 仅在持有相同 token 时释放,避免误删他人锁
|
||||
val = await self.redis.get(self.name)
|
||||
if val == self._token:
|
||||
await self.redis.delete(self.name)
|
||||
|
||||
async def guarded(self, fn):
|
||||
"""上下文封装:获取锁执行,释放锁。"""
|
||||
if not await self.acquire():
|
||||
return False
|
||||
try:
|
||||
await fn()
|
||||
return True
|
||||
finally:
|
||||
await self.release()
|
||||
78
backend/app/services/notifier.py
Normal file
78
backend/app/services/notifier.py
Normal file
@ -0,0 +1,78 @@
|
||||
"""任务完成事件通知服务。
|
||||
|
||||
设计:CLI 提交任务时可携带 cli_session_id,并通过 SSE 长连接订阅
|
||||
GET /api/cli/events?cli_session_id=xxx 事件流。任务到达终态
|
||||
(success / failed,含取消、超时)时,网关通过 Redis Pub/Sub 向
|
||||
对应会话推送事件,避免 CLI 轮询。
|
||||
|
||||
为保证订阅晚于任务完成也不丢事件,publish 时同时写入一份
|
||||
回放缓存(Redis List,保留最近 N 条),SSE 连接建立后先回放再实时推送。
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.models.schemas import TaskInfo
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
CHANNEL_PREFIX = "task:event:cli:" # Redis Pub/Sub channel
|
||||
REPLAY_KEY_PREFIX = "task:event:replay:" # 回放缓存 List key
|
||||
REPLAY_TTL = 600
|
||||
REPLAY_MAX = 200
|
||||
|
||||
|
||||
class TaskNotifier:
|
||||
def __init__(self, redis: aioredis.Redis):
|
||||
self.redis = redis
|
||||
|
||||
@staticmethod
|
||||
def channel(cli_session_id: str) -> str:
|
||||
return f"{CHANNEL_PREFIX}{cli_session_id}"
|
||||
|
||||
@staticmethod
|
||||
def _replay_key(cli_session_id: str) -> str:
|
||||
return f"{REPLAY_KEY_PREFIX}{cli_session_id}"
|
||||
|
||||
# ---------- 发布 ----------
|
||||
async def publish_task_done(self, task: TaskInfo) -> None:
|
||||
"""任务到达终态时调用:写回放缓存并广播到该 CLI 会话。"""
|
||||
if not task.cli_session_id:
|
||||
return
|
||||
payload = json.dumps(
|
||||
{
|
||||
"event": "task_done",
|
||||
"request_id": task.request_id,
|
||||
"status": task.status.value,
|
||||
"result": task.result,
|
||||
"error_info": task.error_info,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
key = self._replay_key(task.cli_session_id)
|
||||
channel = self.channel(task.cli_session_id)
|
||||
await self.redis.lpush(key, payload)
|
||||
await self.redis.ltrim(key, 0, REPLAY_MAX - 1)
|
||||
await self.redis.expire(key, REPLAY_TTL)
|
||||
await self.redis.publish(channel, payload)
|
||||
logger.info("notify task done session=%s request=%s status=%s",
|
||||
task.cli_session_id, task.request_id, task.status.value)
|
||||
|
||||
# ---------- 消费 ----------
|
||||
async def replay(self, cli_session_id: str) -> list[str]:
|
||||
"""返回该会话的历史完成事件(新 → 旧),供 SSE 连接回放。"""
|
||||
items = await self.redis.lrange(self._replay_key(cli_session_id), 0, -1)
|
||||
return [s.decode() if isinstance(s, bytes) else s for s in items]
|
||||
|
||||
async def drain_replay(self, cli_session_id: str) -> None:
|
||||
"""SSE 连接回放结束后清空回放缓存(已消费)。"""
|
||||
await self.redis.delete(self._replay_key(cli_session_id))
|
||||
|
||||
|
||||
def format_sse(data: str | dict[str, Any]) -> str:
|
||||
"""格式化为 SSE 帧。"""
|
||||
if isinstance(data, dict):
|
||||
data = json.dumps(data, ensure_ascii=False)
|
||||
return f"data: {data}\n\n"
|
||||
101
backend/app/services/relay.py
Normal file
101
backend/app/services/relay.py
Normal file
@ -0,0 +1,101 @@
|
||||
"""通信中转服务:正向下发任务指令、反向回传结果/日志,RequestID+AgentID 关联。"""
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.config import settings
|
||||
from app.constants import TaskStatus
|
||||
from app.models.schemas import TaskResult
|
||||
from app.repository.agent_repo import AgentRepo
|
||||
from app.repository.task_repo import TaskRepo
|
||||
from app.services.notifier import TaskNotifier
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class RelayService:
|
||||
"""网关作为唯一通信中枢:正向把任务指令推送到 Agent 端点,反向接收结果回传。"""
|
||||
|
||||
def __init__(self, redis: aioredis.Redis, task_repo: TaskRepo | None = None):
|
||||
self.redis = redis
|
||||
self.task_repo = task_repo or TaskRepo(redis)
|
||||
self.agent_repo = AgentRepo(redis)
|
||||
|
||||
async def dispatch_command(self, agent_id: str, request_id: str, payload: dict) -> bool:
|
||||
"""正向:向 Agent 真实 HTTP 推送任务指令(POST {agent.endpoint}/tasks/{request_id})。
|
||||
|
||||
成功返回 True;Agent 不存在 / 端点缺失 / 网络失败 / 非 202 均返回 False,
|
||||
由调度器回退任务状态并释放负载。
|
||||
"""
|
||||
agent = await self.agent_repo.get(agent_id)
|
||||
if not agent:
|
||||
logger.warning("relay dispatch failed: agent not found agent=%s request=%s", agent_id, request_id)
|
||||
return False
|
||||
if not agent.endpoint:
|
||||
logger.warning("relay dispatch failed: endpoint empty agent=%s request=%s", agent_id, request_id)
|
||||
return False
|
||||
url = f"{agent.endpoint.rstrip('/')}/tasks/{request_id}"
|
||||
body = {
|
||||
"auth": settings.gateway_auth,
|
||||
"request_id": request_id,
|
||||
"payload": payload,
|
||||
}
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
resp = await client.post(url, json=body)
|
||||
except httpx.HTTPError as e:
|
||||
logger.error("relay dispatch network error agent=%s request=%s err=%s", agent_id, request_id, e)
|
||||
await self._log(agent_id, request_id, "dispatch", f"push failed: {e}")
|
||||
return False
|
||||
if resp.status_code != 202:
|
||||
logger.warning("relay dispatch http %s agent=%s request=%s body=%s",
|
||||
resp.status_code, agent_id, request_id, resp.text[:200])
|
||||
await self._log(agent_id, request_id, "dispatch", f"push failed http {resp.status_code}")
|
||||
return False
|
||||
await self._log(agent_id, request_id, "dispatch", f"command pushed to {agent.endpoint}")
|
||||
logger.info("relay dispatch ok agent=%s request=%s url=%s", agent_id, request_id, url)
|
||||
return True
|
||||
|
||||
async def on_result(self, result: TaskResult) -> bool:
|
||||
"""反向:Agent 回传结果,更新任务状态并记录日志。"""
|
||||
task = await self.task_repo.get(result.request_id)
|
||||
if not task:
|
||||
logger.warning("result for unknown task request=%s", result.request_id)
|
||||
return False
|
||||
if task.agent_id and task.agent_id != result.agent_id:
|
||||
logger.warning("result agent mismatch request=%s expected=%s got=%s",
|
||||
result.request_id, task.agent_id, result.agent_id)
|
||||
return False
|
||||
await self.task_repo.update(
|
||||
result.request_id,
|
||||
status=result.status,
|
||||
progress=result.progress,
|
||||
result=result.result,
|
||||
error_info=result.error_info,
|
||||
)
|
||||
await self.task_repo.mark_finished(result.request_id)
|
||||
# 释放 Agent 算力(任务确实绑定在该 Agent 时才释放)
|
||||
if task.agent_id == result.agent_id:
|
||||
await self.agent_repo.adjust_load(result.agent_id, -1)
|
||||
await self._log(result.agent_id, result.request_id, "result", f"result received status={result.status.value}")
|
||||
# 任务到达终态,通知对应 CLI 会话(若携带了 cli_session_id)
|
||||
if task.cli_session_id:
|
||||
finished = await self.task_repo.get(result.request_id)
|
||||
if finished:
|
||||
await TaskNotifier(self.redis).publish_task_done(finished)
|
||||
logger.info("task result request=%s agent=%s status=%s", result.request_id, result.agent_id, result.status.value)
|
||||
return True
|
||||
|
||||
async def on_progress(self, request_id: str, agent_id: str, progress: int) -> bool:
|
||||
await self.task_repo.update(request_id, progress=progress)
|
||||
await self._log(agent_id, request_id, "progress", f"progress {progress}%")
|
||||
return True
|
||||
|
||||
async def _log(self, agent_id: str, request_id: str, action: str, message: str) -> None:
|
||||
from app.repository.log_repo import LogRepo
|
||||
|
||||
await LogRepo(self.redis).append(
|
||||
source="gateway", scope="task", message=message,
|
||||
request_id=request_id, agent_id=agent_id,
|
||||
)
|
||||
76
backend/app/services/scheduler.py
Normal file
76
backend/app/services/scheduler.py
Normal file
@ -0,0 +1,76 @@
|
||||
"""规则调度服务:标签匹配→负载过滤→最优选择→任务绑定→推送下发。"""
|
||||
import logging
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.constants import TaskStatus
|
||||
from app.models.schemas import AgentInfo
|
||||
from app.repository.agent_repo import AgentRepo
|
||||
from app.repository.task_repo import TaskRepo
|
||||
from app.services.relay import RelayService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Scheduler:
|
||||
def __init__(
|
||||
self,
|
||||
redis: aioredis.Redis,
|
||||
task_repo: TaskRepo | None = None,
|
||||
agent_repo: AgentRepo | None = None,
|
||||
relay: RelayService | None = None,
|
||||
):
|
||||
self.redis = redis
|
||||
self.task_repo = task_repo or TaskRepo(redis)
|
||||
self.agent_repo = agent_repo or AgentRepo(redis)
|
||||
self.relay = relay or RelayService(redis)
|
||||
|
||||
async def pick_candidate(self, task_tags: list[str]) -> AgentInfo | None:
|
||||
"""按规则挑选最优 Agent:标签匹配→负载过滤→最低负载。"""
|
||||
candidates = await self.agent_repo.by_tags(task_tags)
|
||||
# 负载过滤:剔除满载
|
||||
candidates = [a for a in candidates if a.current_load < a.max_concurrent]
|
||||
if not candidates:
|
||||
return None
|
||||
# 最优:负载最低、在线时间最长(稳定)
|
||||
candidates.sort(key=lambda a: (a.current_load, -a.create_time))
|
||||
return candidates[0]
|
||||
|
||||
async def dispatch(self, request_id: str) -> bool:
|
||||
"""将单个 pending 任务下发给匹配 Agent(绑定 → 推送 payload → 失败回退)。"""
|
||||
task = await self.task_repo.get(request_id)
|
||||
if not task or task.status != TaskStatus.PENDING:
|
||||
return False
|
||||
agent = await self.pick_candidate(task.task_tags)
|
||||
if not agent:
|
||||
return False # 暂无可用 Agent,保持 pending,等待下次调度
|
||||
# 绑定 Agent,标记 running
|
||||
await self.task_repo.update(
|
||||
request_id,
|
||||
status=TaskStatus.RUNNING,
|
||||
agent_id=agent.agent_id,
|
||||
progress=0,
|
||||
)
|
||||
await self.task_repo.mark_running(request_id)
|
||||
# 更新 Agent 负载
|
||||
await self.agent_repo.adjust_load(agent.agent_id, 1)
|
||||
# 真实推送 payload 到 Agent 端点
|
||||
pushed = await self.relay.dispatch_command(agent.agent_id, request_id, task.payload or {})
|
||||
if not pushed:
|
||||
# 推送失败:回退 pending 并释放负载,等待下次调度(避免重复扣负载)
|
||||
await self.task_repo.update(request_id, status=TaskStatus.PENDING, agent_id="", progress=0)
|
||||
await self.task_repo.mark_pending(request_id)
|
||||
await self.agent_repo.adjust_load(agent.agent_id, -1)
|
||||
logger.warning("task dispatch push failed, reverted request_id=%s agent=%s", request_id, agent.agent_id)
|
||||
return False
|
||||
logger.info("task dispatched request_id=%s -> agent=%s", request_id, agent.agent_id)
|
||||
return True
|
||||
|
||||
async def dispatch_pending(self, batch_size: int = 20) -> int:
|
||||
"""调度所有可调度的 pending 任务,返回成功下发数。"""
|
||||
pending = await self.task_repo.pending_tasks()
|
||||
dispatched = 0
|
||||
for rid in pending[:batch_size]:
|
||||
if await self.dispatch(rid):
|
||||
dispatched += 1
|
||||
return dispatched
|
||||
10
backend/app/services/security.py
Normal file
10
backend/app/services/security.py
Normal file
@ -0,0 +1,10 @@
|
||||
"""认证校验:CLI / Agent 请求必须携带与 GATEWAY_AUTH 一致的密码。"""
|
||||
from fastapi import HTTPException
|
||||
|
||||
from app.config import settings
|
||||
|
||||
|
||||
def require_auth(auth: str) -> None:
|
||||
"""auth 不匹配时抛出 401。"""
|
||||
if auth != settings.gateway_auth:
|
||||
raise HTTPException(status_code=401, detail="invalid auth")
|
||||
106
backend/app/services/task_service.py
Normal file
106
backend/app/services/task_service.py
Normal file
@ -0,0 +1,106 @@
|
||||
"""任务池服务:受理、状态流转、幂等、超时失败。"""
|
||||
import logging
|
||||
import time
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.config import settings
|
||||
from app.constants import TaskStatus
|
||||
from app.models.schemas import TaskInfo, TaskSubmit
|
||||
from app.repository.agent_repo import AgentRepo
|
||||
from app.repository.task_repo import TaskRepo
|
||||
from app.services.notifier import TaskNotifier
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TaskService:
|
||||
def __init__(self, redis: aioredis.Redis, repo: TaskRepo | None = None):
|
||||
self.redis = redis
|
||||
self.repo = repo or TaskRepo(redis)
|
||||
self.agent_repo = AgentRepo(redis)
|
||||
|
||||
async def _release_if_running(self, task: TaskInfo) -> None:
|
||||
"""若任务处于 running 且绑定 Agent,释放其算力。"""
|
||||
if task.status == TaskStatus.RUNNING and task.agent_id:
|
||||
await self.agent_repo.adjust_load(task.agent_id, -1)
|
||||
|
||||
async def submit(self, submit: TaskSubmit) -> TaskInfo:
|
||||
"""受理任务:生成/复用 RequestID,幂等写入任务池。"""
|
||||
task = TaskInfo(
|
||||
request_id=submit.request_id or submit.request_id or _new_id(),
|
||||
task_type=submit.task_type,
|
||||
task_tags=submit.task_tags,
|
||||
description=submit.description,
|
||||
payload=submit.payload,
|
||||
cli_session_id=submit.cli_session_id,
|
||||
timeout=submit.timeout or settings.default_task_timeout,
|
||||
status=TaskStatus.PENDING,
|
||||
)
|
||||
created = await self.repo.create(task)
|
||||
if not created:
|
||||
# 幂等:返回已有任务
|
||||
existing = await self.repo.get(task.request_id)
|
||||
if existing:
|
||||
return existing
|
||||
logger.info("task submitted request_id=%s type=%s", task.request_id, task.task_type)
|
||||
return task
|
||||
|
||||
async def get(self, request_id: str) -> TaskInfo | None:
|
||||
return await self.repo.get(request_id)
|
||||
|
||||
async def list(self, status: TaskStatus | None = None, limit: int = 100) -> list[TaskInfo]:
|
||||
return await self.repo.list(status=status, limit=limit)
|
||||
|
||||
async def cancel(self, request_id: str) -> bool:
|
||||
"""取消任务(仅 pending/running 可取消)。"""
|
||||
task = await self.repo.get(request_id)
|
||||
if not task:
|
||||
return False
|
||||
if task.status in (TaskStatus.SUCCESS, TaskStatus.FAILED):
|
||||
return False
|
||||
await self._release_if_running(task)
|
||||
await self.repo.update(request_id, status=TaskStatus.FAILED, error_info="cancelled by user")
|
||||
await self.repo.mark_finished(request_id)
|
||||
if task.cli_session_id:
|
||||
finished = await self.repo.get(request_id)
|
||||
if finished:
|
||||
await TaskNotifier(self.redis).publish_task_done(finished)
|
||||
logger.info("task cancelled request_id=%s", request_id)
|
||||
return True
|
||||
|
||||
async def reset(self, request_id: str) -> bool:
|
||||
"""重置任务状态为 pending(重新调度)。"""
|
||||
task = await self.repo.get(request_id)
|
||||
if not task:
|
||||
return False
|
||||
await self.repo.update(request_id, status=TaskStatus.PENDING, agent_id=None, error_info=None, progress=0)
|
||||
await self.repo.mark_pending(request_id)
|
||||
logger.info("task reset request_id=%s", request_id)
|
||||
return True
|
||||
|
||||
async def check_timeouts(self) -> int:
|
||||
"""扫描 running 任务,超时自动标记失败。"""
|
||||
now = time.time()
|
||||
failed = 0
|
||||
for rid in await self.repo.running_tasks():
|
||||
task = await self.repo.get(rid)
|
||||
if not task:
|
||||
continue
|
||||
if task.timeout and (now - task.create_time) > task.timeout:
|
||||
await self._release_if_running(task)
|
||||
await self.repo.update(rid, status=TaskStatus.FAILED, error_info="timeout")
|
||||
await self.repo.mark_finished(rid)
|
||||
if task.cli_session_id:
|
||||
finished = await self.repo.get(rid)
|
||||
if finished:
|
||||
await TaskNotifier(self.redis).publish_task_done(finished)
|
||||
failed += 1
|
||||
logger.info("task timeout request_id=%s", rid)
|
||||
return failed
|
||||
|
||||
|
||||
def _new_id() -> str:
|
||||
import uuid
|
||||
|
||||
return uuid.uuid4().hex
|
||||
19
backend/app/services/timeout.py
Normal file
19
backend/app/services/timeout.py
Normal file
@ -0,0 +1,19 @@
|
||||
"""任务超时检查服务。"""
|
||||
import logging
|
||||
|
||||
import redis.asyncio as aioredis
|
||||
|
||||
from app.repository.task_repo import TaskRepo
|
||||
from app.services.task_service import TaskService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TimeoutService:
|
||||
def __init__(self, redis: aioredis.Redis, task_repo: TaskRepo | None = None):
|
||||
self.redis = redis
|
||||
self.task_repo = task_repo or TaskRepo(redis)
|
||||
|
||||
async def check(self) -> int:
|
||||
"""扫描 running 任务,超时自动标记失败,返回失败数。"""
|
||||
return await TaskService(self.redis, self.task_repo).check_timeouts()
|
||||
0
backend/tests/__init__.py
Normal file
0
backend/tests/__init__.py
Normal file
25
backend/tests/conftest.py
Normal file
25
backend/tests/conftest.py
Normal file
@ -0,0 +1,25 @@
|
||||
"""pytest 共享 fixture。"""
|
||||
import fakeredis.aioredis
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
|
||||
from app.repository.agent_repo import AgentRepo
|
||||
from app.repository.task_repo import TaskRepo
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def redis():
|
||||
server = fakeredis.FakeServer()
|
||||
client = fakeredis.aioredis.FakeRedis(server=server, decode_responses=True)
|
||||
yield client
|
||||
await client.aclose()
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
def task_repo(redis):
|
||||
return TaskRepo(redis)
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
def agent_repo(redis):
|
||||
return AgentRepo(redis)
|
||||
55
backend/tests/test_agent_repo.py
Normal file
55
backend/tests/test_agent_repo.py
Normal file
@ -0,0 +1,55 @@
|
||||
"""Agent 池存储层单元测试。"""
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from app.constants import AgentStatus
|
||||
from app.models.schemas import AgentInfo
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upsert_and_get(agent_repo):
|
||||
a = AgentInfo(agent_id="a1", endpoint="agent-1:5000", agent_tags=["compile", "test"], max_concurrent=2)
|
||||
created = await agent_repo.upsert(a)
|
||||
assert created is True
|
||||
|
||||
got = await agent_repo.get("a1")
|
||||
assert got is not None
|
||||
assert got.agent_tags == ["compile", "test"]
|
||||
assert got.status == AgentStatus.ONLINE
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_beat_and_stale(agent_repo):
|
||||
a = AgentInfo(agent_id="a1", endpoint="agent-1:5000", agent_tags=["compile"])
|
||||
await agent_repo.upsert(a)
|
||||
assert await agent_repo.beat("a1", 1) is True
|
||||
|
||||
# 心跳为最新,不应 stale
|
||||
stale = await agent_repo.stale_agents(1)
|
||||
assert "a1" not in stale
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_by_tags(agent_repo):
|
||||
a1 = AgentInfo(agent_id="a1", endpoint="e1", agent_tags=["compile"], max_concurrent=2)
|
||||
a2 = AgentInfo(agent_id="a2", endpoint="e2", agent_tags=["compile", "test"], max_concurrent=2)
|
||||
a3 = AgentInfo(agent_id="a3", endpoint="e3", agent_tags=["test"], max_concurrent=2)
|
||||
await agent_repo.upsert(a1)
|
||||
await agent_repo.upsert(a2)
|
||||
await agent_repo.upsert(a3)
|
||||
|
||||
ids = {a.agent_id for a in await agent_repo.by_tags(["compile"])}
|
||||
assert ids == {"a1", "a2"}
|
||||
|
||||
ids = {a.agent_id for a in await agent_repo.by_tags(["compile", "test"])}
|
||||
assert ids == {"a2"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_remove(agent_repo):
|
||||
a1 = AgentInfo(agent_id="a1", endpoint="e1", agent_tags=["compile"])
|
||||
await agent_repo.upsert(a1)
|
||||
await agent_repo.remove("a1")
|
||||
assert await agent_repo.get("a1") is None
|
||||
assert await agent_repo.by_tags(["compile"]) == []
|
||||
90
backend/tests/test_api.py
Normal file
90
backend/tests/test_api.py
Normal file
@ -0,0 +1,90 @@
|
||||
"""接口集成测试。"""
|
||||
import fakeredis.aioredis
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
from app.config import settings
|
||||
from app.main import app
|
||||
|
||||
AUTH = settings.gateway_auth
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def client():
|
||||
server = fakeredis.FakeServer()
|
||||
redis = fakeredis.aioredis.FakeRedis(server=server, decode_responses=True)
|
||||
app.state.redis = redis
|
||||
transport = ASGITransport(app=app)
|
||||
async with AsyncClient(transport=transport, base_url="http://test") as c:
|
||||
yield c
|
||||
await redis.aclose()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_health(client):
|
||||
r = await client.get("/health")
|
||||
assert r.status_code == 200
|
||||
assert r.json()["status"] == "ok"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_submit_and_query_task(client):
|
||||
r = await client.post("/api/cli/tasks", json={"auth": AUTH, "task_type": "compile", "task_tags": ["build"]})
|
||||
assert r.status_code == 200
|
||||
data = r.json()
|
||||
assert data["status"] == "pending"
|
||||
rid = data["request_id"]
|
||||
|
||||
r2 = await client.get(f"/api/cli/tasks/{rid}")
|
||||
assert r2.status_code == 200
|
||||
assert r2.json()["request_id"] == rid
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_register_heartbeat_result(client):
|
||||
# 注册
|
||||
r = await client.post("/api/agent/register", json={"auth": AUTH, "agent_id": "a1", "endpoint": "e1", "agent_tags": ["compile"], "max_concurrent": 2})
|
||||
assert r.status_code == 200
|
||||
|
||||
# 心跳
|
||||
r = await client.post("/api/agent/heartbeat", json={"agent_id": "a1", "current_load": 0})
|
||||
assert r.status_code == 200
|
||||
|
||||
# 提交任务并等待调度(手动触发一次调度)
|
||||
r = await client.post("/api/cli/tasks", json={"auth": AUTH, "task_type": "compile", "task_tags": ["compile"]})
|
||||
rid = r.json()["request_id"]
|
||||
from app.services.scheduler import Scheduler
|
||||
await Scheduler(app.state.redis).dispatch_pending(10)
|
||||
|
||||
# 回传结果
|
||||
r = await client.post("/api/agent/result", json={"request_id": rid, "agent_id": "a1", "status": "success", "progress": 100, "result": {"ok": True}})
|
||||
assert r.status_code == 200
|
||||
|
||||
detail = await client.get(f"/api/cli/tasks/{rid}")
|
||||
assert detail.json()["status"] == "success"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_admin_overview(client):
|
||||
r = await client.get("/api/admin/overview")
|
||||
assert r.status_code == 200
|
||||
assert "task_total" in r.json()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_submit_task_rejects_bad_auth(client):
|
||||
r = await client.post("/api/cli/tasks", json={"auth": "wrong-password", "task_type": "compile"})
|
||||
assert r.status_code == 401
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_submit_task_requires_auth(client):
|
||||
r = await client.post("/api/cli/tasks", json={"task_type": "compile"})
|
||||
assert r.status_code == 422 # 缺 auth 字段
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_register_rejects_bad_auth(client):
|
||||
r = await client.post("/api/agent/register", json={"auth": "wrong-password", "agent_id": "a9", "endpoint": "e9"})
|
||||
assert r.status_code == 401
|
||||
67
backend/tests/test_scheduler.py
Normal file
67
backend/tests/test_scheduler.py
Normal file
@ -0,0 +1,67 @@
|
||||
"""调度服务单元测试。"""
|
||||
import pytest
|
||||
|
||||
from app.constants import TaskStatus
|
||||
from app.models.schemas import AgentInfo, TaskInfo
|
||||
from app.repository.agent_repo import AgentRepo
|
||||
from app.repository.task_repo import TaskRepo
|
||||
from app.services.scheduler import Scheduler
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dispatch_binds_min_load_agent(redis):
|
||||
task_repo = TaskRepo(redis)
|
||||
agent_repo = AgentRepo(redis)
|
||||
|
||||
# 两个 compile Agent,b2 负载更低
|
||||
await agent_repo.upsert(AgentInfo(agent_id="b1", endpoint="e1", agent_tags=["compile"], max_concurrent=2, current_load=1))
|
||||
await agent_repo.upsert(AgentInfo(agent_id="b2", endpoint="e2", agent_tags=["compile"], max_concurrent=2, current_load=0))
|
||||
|
||||
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
|
||||
|
||||
sched = Scheduler(redis, task_repo, agent_repo)
|
||||
ok = await sched.dispatch("t1")
|
||||
assert ok is True
|
||||
|
||||
task = await task_repo.get("t1")
|
||||
assert task.agent_id == "b2"
|
||||
assert task.status == TaskStatus.RUNNING
|
||||
assert "t1" in await task_repo.running_tasks()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dispatch_no_agent_stays_pending(redis):
|
||||
task_repo = TaskRepo(redis)
|
||||
agent_repo = AgentRepo(redis)
|
||||
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
|
||||
|
||||
sched = Scheduler(redis, task_repo, agent_repo)
|
||||
ok = await sched.dispatch("t1")
|
||||
assert ok is False
|
||||
task = await task_repo.get("t1")
|
||||
assert task.status == TaskStatus.PENDING
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dispatch_skips_full_agent(redis):
|
||||
task_repo = TaskRepo(redis)
|
||||
agent_repo = AgentRepo(redis)
|
||||
# 唯一 agent 已满载
|
||||
await agent_repo.upsert(AgentInfo(agent_id="b1", endpoint="e1", agent_tags=["compile"], max_concurrent=1, current_load=1))
|
||||
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
|
||||
|
||||
sched = Scheduler(redis, task_repo, agent_repo)
|
||||
assert await sched.dispatch("t1") is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dispatch_pending(redis):
|
||||
task_repo = TaskRepo(redis)
|
||||
agent_repo = AgentRepo(redis)
|
||||
await agent_repo.upsert(AgentInfo(agent_id="b1", endpoint="e1", agent_tags=["compile"], max_concurrent=2))
|
||||
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
|
||||
await task_repo.create(TaskInfo(request_id="t2", task_type="compile", task_tags=["compile"]))
|
||||
|
||||
sched = Scheduler(redis, task_repo, agent_repo)
|
||||
n = await sched.dispatch_pending(10)
|
||||
assert n == 2
|
||||
51
backend/tests/test_task_repo.py
Normal file
51
backend/tests/test_task_repo.py
Normal file
@ -0,0 +1,51 @@
|
||||
"""任务池存储层单元测试。"""
|
||||
import pytest
|
||||
|
||||
from app.constants import TaskStatus
|
||||
from app.models.schemas import TaskInfo
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_and_get(task_repo):
|
||||
t = TaskInfo(request_id="r1", task_type="compile", task_tags=["build"], payload={"cmd": "go build"})
|
||||
created = await task_repo.create(t)
|
||||
assert created is True
|
||||
|
||||
got = await task_repo.get("r1")
|
||||
assert got is not None
|
||||
assert got.task_type == "compile"
|
||||
assert got.task_tags == ["build"]
|
||||
assert got.payload == {"cmd": "go build"}
|
||||
assert got.status == TaskStatus.PENDING
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_idempotent_create(task_repo):
|
||||
t = TaskInfo(request_id="r1", task_type="compile")
|
||||
assert await task_repo.create(t) is True
|
||||
assert await task_repo.create(t) is False # 幂等
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_state_sets(task_repo):
|
||||
t = TaskInfo(request_id="r1", task_type="compile")
|
||||
await task_repo.create(t)
|
||||
assert "r1" in await task_repo.pending_tasks()
|
||||
|
||||
await task_repo.mark_running("r1")
|
||||
assert "r1" not in await task_repo.pending_tasks()
|
||||
assert "r1" in await task_repo.running_tasks()
|
||||
|
||||
await task_repo.mark_finished("r1")
|
||||
assert "r1" not in await task_repo.running_tasks()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update(task_repo):
|
||||
t = TaskInfo(request_id="r1", task_type="compile")
|
||||
await task_repo.create(t)
|
||||
await task_repo.update("r1", status=TaskStatus.RUNNING, agent_id="a1", progress=50)
|
||||
got = await task_repo.get("r1")
|
||||
assert got.status == TaskStatus.RUNNING
|
||||
assert got.agent_id == "a1"
|
||||
assert got.progress == 50
|
||||
15
docker-compose.yml
Normal file
15
docker-compose.yml
Normal file
@ -0,0 +1,15 @@
|
||||
version: "3.8"
|
||||
|
||||
services:
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
container_name: gateway-redis
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "6379:6379"
|
||||
volumes:
|
||||
- redis-data:/data
|
||||
command: ["redis-server", "--appendonly", "yes"]
|
||||
|
||||
volumes:
|
||||
redis-data:
|
||||
12
frontend/index.html
Normal file
12
frontend/index.html
Normal file
@ -0,0 +1,12 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>A2A 智能网关运维台</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="/src/main.ts"></script>
|
||||
</body>
|
||||
</html>
|
||||
2751
frontend/package-lock.json
generated
Normal file
2751
frontend/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
27
frontend/package.json
Normal file
27
frontend/package.json
Normal file
@ -0,0 +1,27 @@
|
||||
{
|
||||
"name": "gateway-admin",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "vue-tsc -b && vite build",
|
||||
"preview": "vite preview"
|
||||
},
|
||||
"dependencies": {
|
||||
"@element-plus/icons-vue": "^2.3.1",
|
||||
"element-plus": "^2.9.1",
|
||||
"lucide-vue-next": "^0.469.0",
|
||||
"vue": "^3.5.13",
|
||||
"vue-router": "^4.5.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@vitejs/plugin-vue": "^5.2.1",
|
||||
"autoprefixer": "^10.5.4",
|
||||
"postcss": "^8.5.25",
|
||||
"tailwindcss": "^3.4.0",
|
||||
"typescript": "~5.6.2",
|
||||
"vite": "^5.4.11",
|
||||
"vue-tsc": "^2.1.10"
|
||||
}
|
||||
}
|
||||
6
frontend/postcss.config.js
Normal file
6
frontend/postcss.config.js
Normal file
@ -0,0 +1,6 @@
|
||||
export default {
|
||||
plugins: {
|
||||
tailwindcss: {},
|
||||
autoprefixer: {},
|
||||
},
|
||||
}
|
||||
160
frontend/src/App.vue
Normal file
160
frontend/src/App.vue
Normal file
@ -0,0 +1,160 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, onUnmounted, ref } from 'vue'
|
||||
import { useRoute } from 'vue-router'
|
||||
import { api } from './api'
|
||||
import type { Overview } from './api'
|
||||
|
||||
const route = useRoute()
|
||||
const overview = ref<Overview | null>(null)
|
||||
const isCollapse = ref(false)
|
||||
let timer: number | undefined
|
||||
|
||||
const title = computed(() => (route.meta.title as string) || '')
|
||||
|
||||
const menus = [
|
||||
{ path: '/tasks', label: '任务管理', icon: 'List' },
|
||||
{ path: '/agents', label: 'Agent 管理', icon: 'Cpu' },
|
||||
{ path: '/logs', label: '日志审计', icon: 'Document' },
|
||||
{ path: '/control', label: '手动管控', icon: 'Operation' },
|
||||
]
|
||||
|
||||
async function refresh() {
|
||||
try {
|
||||
overview.value = await api.overview()
|
||||
} catch {
|
||||
overview.value = null
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(() => {
|
||||
refresh()
|
||||
timer = window.setInterval(refresh, 5000)
|
||||
})
|
||||
|
||||
onUnmounted(() => {
|
||||
if (timer) window.clearInterval(timer)
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="flex h-full">
|
||||
<!-- 可折叠侧边菜单 -->
|
||||
<aside
|
||||
class="flex flex-col shrink-0 h-full bg-[#0f172a] transition-all duration-200 overflow-hidden"
|
||||
:class="isCollapse ? 'w-16' : 'w-56'"
|
||||
>
|
||||
<!-- Logo 区 -->
|
||||
<div class="h-16 flex items-center gap-2.5 px-4 border-b border-white/10 overflow-hidden shrink-0">
|
||||
<div
|
||||
class="w-9 h-9 shrink-0 rounded-lg bg-gradient-to-br from-[#1e63e8] to-[#3b82f6] flex items-center justify-center font-bold text-white"
|
||||
>
|
||||
A
|
||||
</div>
|
||||
<div v-show="!isCollapse" class="min-w-0">
|
||||
<div class="font-semibold text-sm text-white leading-tight">A2A 智能网关</div>
|
||||
<div class="text-[11px] text-slate-400">运维控制台</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 菜单 -->
|
||||
<el-menu
|
||||
:default-active="route.path"
|
||||
:collapse="isCollapse"
|
||||
:collapse-transition="false"
|
||||
router
|
||||
background-color="#0f172a"
|
||||
text-color="#94a3b8"
|
||||
active-text-color="#ffffff"
|
||||
class="gw-menu flex-1 overflow-y-auto overflow-x-hidden"
|
||||
>
|
||||
<el-menu-item v-for="m in menus" :key="m.path" :index="m.path">
|
||||
<el-icon><component :is="m.icon" /></el-icon>
|
||||
<template #title>{{ m.label }}</template>
|
||||
</el-menu-item>
|
||||
</el-menu>
|
||||
|
||||
<div
|
||||
v-show="!isCollapse"
|
||||
class="px-5 py-4 text-[11px] text-slate-500 border-t border-white/10 shrink-0"
|
||||
>
|
||||
Gateway v1.0.0
|
||||
</div>
|
||||
<div
|
||||
v-show="isCollapse"
|
||||
class="py-4 text-center text-[10px] text-slate-600 border-t border-white/10 shrink-0"
|
||||
>
|
||||
v1.0
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<!-- 主区域 -->
|
||||
<div class="flex-1 flex flex-col min-w-0 h-full">
|
||||
<header class="h-16 bg-white border-b flex items-center justify-between px-5 gap-4 shrink-0">
|
||||
<div class="flex items-center gap-3 min-w-0">
|
||||
<button class="gw-toggle" title="折叠 / 展开菜单" @click="isCollapse = !isCollapse">
|
||||
<el-icon :size="18"><Expand v-if="isCollapse" /><Fold v-else /></el-icon>
|
||||
</button>
|
||||
<h1 class="text-base font-semibold text-slate-800 truncate">{{ title }}</h1>
|
||||
</div>
|
||||
<div class="flex items-center gap-5 text-sm shrink-0">
|
||||
<div class="flex items-center gap-1.5">
|
||||
<span class="badge-dot bg-[#10b981] pulse"></span>
|
||||
<span class="text-slate-600">在线 Agent</span>
|
||||
<span class="font-bold text-[#10b981]">{{ overview?.agent_online ?? '-' }}</span>
|
||||
</div>
|
||||
<div class="flex items-center gap-1.5">
|
||||
<span class="badge-dot bg-[#1e63e8]"></span>
|
||||
<span class="text-slate-600">任务总数</span>
|
||||
<span class="font-bold text-[#1e63e8]">{{ overview?.task_total ?? '-' }}</span>
|
||||
</div>
|
||||
<div class="flex items-center gap-1.5">
|
||||
<span class="badge-dot bg-[#ef4444]"></span>
|
||||
<span class="text-slate-600">告警</span>
|
||||
<span class="font-bold text-[#ef4444]">
|
||||
{{ (overview?.task_failed ?? 0) + (overview?.agent_offline ?? 0) }}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<main class="flex-1 overflow-auto p-6">
|
||||
<router-view />
|
||||
</main>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.gw-toggle {
|
||||
width: 36px;
|
||||
height: 36px;
|
||||
border-radius: 8px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
color: #64748b;
|
||||
background: transparent;
|
||||
border: none;
|
||||
cursor: pointer;
|
||||
transition: background 0.2s;
|
||||
}
|
||||
.gw-toggle:hover {
|
||||
background: #f1f5f9;
|
||||
}
|
||||
|
||||
/* 深色侧栏菜单细节修正 */
|
||||
.gw-menu {
|
||||
--el-menu-hover-bg-color: rgba(255, 255, 255, 0.06);
|
||||
--el-menu-active-color: #ffffff;
|
||||
border-right: none !important;
|
||||
}
|
||||
.gw-menu .el-menu-item {
|
||||
margin: 2px 8px;
|
||||
border-radius: 8px;
|
||||
}
|
||||
.gw-menu .el-menu-item.is-active {
|
||||
background: linear-gradient(90deg, rgba(30, 99, 232, 0.9), rgba(59, 130, 246, 0.7));
|
||||
}
|
||||
.gw-menu:not(.el-menu--collapse) .el-menu-item.is-active {
|
||||
box-shadow: 0 4px 12px rgba(30, 99, 232, 0.35);
|
||||
}
|
||||
</style>
|
||||
78
frontend/src/api/index.ts
Normal file
78
frontend/src/api/index.ts
Normal file
@ -0,0 +1,78 @@
|
||||
const BASE = '/api/admin'
|
||||
|
||||
async function request<T>(path: string, options: RequestInit = {}): Promise<T> {
|
||||
const res = await fetch(`${BASE}${path}`, {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
...options,
|
||||
})
|
||||
if (!res.ok) {
|
||||
const text = await res.text()
|
||||
throw new Error(text || `HTTP ${res.status}`)
|
||||
}
|
||||
return res.json() as Promise<T>
|
||||
}
|
||||
|
||||
export interface TaskInfo {
|
||||
request_id: string
|
||||
task_type: string
|
||||
task_tags: string[]
|
||||
description: string | null
|
||||
status: 'pending' | 'running' | 'success' | 'failed'
|
||||
agent_id: string | null
|
||||
create_time: number
|
||||
progress: number
|
||||
result: unknown
|
||||
error_info: string | null
|
||||
}
|
||||
|
||||
export interface AgentInfo {
|
||||
agent_id: string
|
||||
endpoint: string
|
||||
agent_tags: string[]
|
||||
max_concurrent: number
|
||||
current_load: number
|
||||
last_heartbeat: number
|
||||
status: 'online' | 'offline'
|
||||
create_time: number
|
||||
}
|
||||
|
||||
export interface LogEntry {
|
||||
ts: number
|
||||
level: string
|
||||
source: string
|
||||
scope: string
|
||||
request_id: string | null
|
||||
agent_id: string | null
|
||||
message: string
|
||||
}
|
||||
|
||||
export interface Overview {
|
||||
task_total: number
|
||||
task_pending: number
|
||||
task_running: number
|
||||
task_success: number
|
||||
task_failed: number
|
||||
agent_total: number
|
||||
agent_online: number
|
||||
agent_offline: number
|
||||
}
|
||||
|
||||
export const api = {
|
||||
listTasks: (status?: string) => {
|
||||
const q = status ? `?status=${status}` : ''
|
||||
return request<TaskInfo[]>(`/tasks${q}`)
|
||||
},
|
||||
taskDetail: (id: string) => request<TaskInfo>(`/tasks/${id}`),
|
||||
cancelTask: (id: string) => request<{ ok: boolean }>(`/tasks/${id}/cancel`, { method: 'POST' }),
|
||||
resetTask: (id: string) => request<{ ok: boolean }>(`/tasks/${id}/reset`, { method: 'POST' }),
|
||||
listAgents: () => request<AgentInfo[]>('/agents'),
|
||||
offlineAgent: (id: string) => request<{ ok: boolean }>(`/agents/${id}/offline`, { method: 'POST' }),
|
||||
listLogs: (params?: { request_id?: string; agent_id?: string }) => {
|
||||
const q = new URLSearchParams()
|
||||
if (params?.request_id) q.set('request_id', params.request_id)
|
||||
if (params?.agent_id) q.set('agent_id', params.agent_id)
|
||||
const s = q.toString()
|
||||
return request<LogEntry[]>(`/logs${s ? `?${s}` : ''}`)
|
||||
},
|
||||
overview: () => request<Overview>('/overview'),
|
||||
}
|
||||
7
frontend/src/env.d.ts
vendored
Normal file
7
frontend/src/env.d.ts
vendored
Normal file
@ -0,0 +1,7 @@
|
||||
/// <reference types="vite/client" />
|
||||
|
||||
declare module '*.vue' {
|
||||
import type { DefineComponent } from 'vue'
|
||||
const component: DefineComponent<{}, {}, any>
|
||||
export default component
|
||||
}
|
||||
15
frontend/src/main.ts
Normal file
15
frontend/src/main.ts
Normal file
@ -0,0 +1,15 @@
|
||||
import { createApp } from 'vue'
|
||||
import ElementPlus from 'element-plus'
|
||||
import 'element-plus/dist/index.css'
|
||||
import * as ElementPlusIconsVue from '@element-plus/icons-vue'
|
||||
import App from './App.vue'
|
||||
import router from './router'
|
||||
import './style.css'
|
||||
|
||||
const app = createApp(App)
|
||||
for (const [key, component] of Object.entries(ElementPlusIconsVue)) {
|
||||
app.component(key, component)
|
||||
}
|
||||
app.use(ElementPlus)
|
||||
app.use(router)
|
||||
app.mount('#app')
|
||||
18
frontend/src/router/index.ts
Normal file
18
frontend/src/router/index.ts
Normal file
@ -0,0 +1,18 @@
|
||||
import { createRouter, createWebHistory } from 'vue-router'
|
||||
import TaskView from '../views/TaskView.vue'
|
||||
import AgentView from '../views/AgentView.vue'
|
||||
import LogView from '../views/LogView.vue'
|
||||
import ControlView from '../views/ControlView.vue'
|
||||
|
||||
const router = createRouter({
|
||||
history: createWebHistory(),
|
||||
routes: [
|
||||
{ path: '/', redirect: '/tasks' },
|
||||
{ path: '/tasks', name: 'tasks', component: TaskView, meta: { title: '任务管理' } },
|
||||
{ path: '/agents', name: 'agents', component: AgentView, meta: { title: 'Agent 管理' } },
|
||||
{ path: '/logs', name: 'logs', component: LogView, meta: { title: '日志审计' } },
|
||||
{ path: '/control', name: 'control', component: ControlView, meta: { title: '手动管控' } },
|
||||
],
|
||||
})
|
||||
|
||||
export default router
|
||||
93
frontend/src/style.css
Normal file
93
frontend/src/style.css
Normal file
@ -0,0 +1,93 @@
|
||||
@tailwind components;
|
||||
@tailwind utilities;
|
||||
|
||||
:root {
|
||||
--gw-primary: #1e63e8;
|
||||
--gw-primary-light: #3b82f6;
|
||||
--gw-dark: #0f172a;
|
||||
--gw-bg: #f5f7fa;
|
||||
--gw-success: #10b981;
|
||||
--gw-warning: #f59e0b;
|
||||
--gw-danger: #ef4444;
|
||||
--gw-info: #3b82f6;
|
||||
--gw-text: #1e293b;
|
||||
--gw-text-sub: #64748b;
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
html,
|
||||
body,
|
||||
#app {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
height: 100%;
|
||||
font-family: 'PingFang SC', 'Microsoft YaHei', -apple-system, sans-serif;
|
||||
background: var(--gw-bg);
|
||||
color: var(--gw-text);
|
||||
}
|
||||
|
||||
.stat-card {
|
||||
border-radius: 12px;
|
||||
background: linear-gradient(135deg, #ffffff 0%, #f8fafc 100%);
|
||||
border: 1px solid #e2e8f0;
|
||||
padding: 18px 20px;
|
||||
transition: transform 0.2s ease, box-shadow 0.2s ease;
|
||||
}
|
||||
|
||||
.stat-card:hover {
|
||||
transform: translateY(-3px);
|
||||
box-shadow: 0 8px 24px rgba(30, 99, 232, 0.10);
|
||||
}
|
||||
|
||||
.stat-value {
|
||||
font-size: 28px;
|
||||
font-weight: 700;
|
||||
line-height: 1.2;
|
||||
}
|
||||
|
||||
.badge-dot {
|
||||
display: inline-block;
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
margin-right: 6px;
|
||||
}
|
||||
|
||||
.pulse {
|
||||
animation: pulse 1.6s ease-in-out infinite;
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0%,
|
||||
100% {
|
||||
opacity: 1;
|
||||
}
|
||||
50% {
|
||||
opacity: 0.35;
|
||||
}
|
||||
}
|
||||
|
||||
.page-card {
|
||||
background: #fff;
|
||||
border-radius: 12px;
|
||||
border: 1px solid #e2e8f0;
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.fade-in {
|
||||
animation: fadeIn 0.3s ease;
|
||||
}
|
||||
|
||||
@keyframes fadeIn {
|
||||
from {
|
||||
opacity: 0;
|
||||
transform: translateY(6px);
|
||||
}
|
||||
to {
|
||||
opacity: 1;
|
||||
transform: translateY(0);
|
||||
}
|
||||
}
|
||||
92
frontend/src/views/AgentView.vue
Normal file
92
frontend/src/views/AgentView.vue
Normal file
@ -0,0 +1,92 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, onUnmounted, ref } from 'vue'
|
||||
import { api, type AgentInfo } from '../api'
|
||||
|
||||
const agents = ref<AgentInfo[]>([])
|
||||
const loading = ref(false)
|
||||
let timer: number | undefined
|
||||
|
||||
async function load() {
|
||||
loading.value = true
|
||||
try {
|
||||
agents.value = await api.listAgents()
|
||||
} catch (e) {
|
||||
console.error(e)
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function fmtTime(ts: number) {
|
||||
if (!ts) return '-'
|
||||
return new Date(ts * 1000).toLocaleString()
|
||||
}
|
||||
|
||||
function loadPct(a: AgentInfo) {
|
||||
return a.max_concurrent ? Math.round((a.current_load / a.max_concurrent) * 100) : 0
|
||||
}
|
||||
|
||||
onMounted(() => {
|
||||
load()
|
||||
timer = window.setInterval(load, 5000)
|
||||
})
|
||||
onUnmounted(() => {
|
||||
if (timer) window.clearInterval(timer)
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="space-y-5 fade-in">
|
||||
<div class="flex items-center justify-between">
|
||||
<h2 class="font-semibold">Agent 池状态</h2>
|
||||
<span class="text-sm text-slate-500">在线 {{ agents.filter((a) => a.status === 'online').length }} / {{ agents.length }}</span>
|
||||
</div>
|
||||
|
||||
<div v-if="loading && agents.length === 0" class="page-card text-center text-slate-400 py-16">加载中...</div>
|
||||
<div v-else-if="agents.length === 0" class="page-card text-center text-slate-400 py-16">
|
||||
暂无 Agent 接入,请等待 K8s Agent 注册。
|
||||
</div>
|
||||
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-4">
|
||||
<div
|
||||
v-for="a in agents"
|
||||
:key="a.agent_id"
|
||||
class="stat-card"
|
||||
:class="{ 'ring-2 ring-[#ef4444]': a.status === 'offline' }"
|
||||
>
|
||||
<div class="flex items-start justify-between">
|
||||
<div class="flex items-center gap-2">
|
||||
<el-icon :size="18" :color="a.status === 'online' ? '#10b981' : '#ef4444'"><Cpu /></el-icon>
|
||||
<span class="font-mono font-semibold">{{ a.agent_id }}</span>
|
||||
</div>
|
||||
<el-tag :type="a.status === 'online' ? 'success' : 'danger'" effect="light" size="small">
|
||||
{{ a.status === 'online' ? '在线' : '离线' }}
|
||||
</el-tag>
|
||||
</div>
|
||||
|
||||
<div class="mt-3 text-sm text-slate-500 truncate">{{ a.endpoint }}</div>
|
||||
|
||||
<div class="mt-2 flex flex-wrap gap-1">
|
||||
<el-tag v-for="t in a.agent_tags" :key="t" size="small" type="info" effect="plain">{{ t }}</el-tag>
|
||||
</div>
|
||||
|
||||
<div class="mt-4">
|
||||
<div class="flex justify-between text-xs text-slate-500 mb-1">
|
||||
<span>负载 {{ a.current_load }}/{{ a.max_concurrent }}</span>
|
||||
<span>{{ loadPct(a) }}%</span>
|
||||
</div>
|
||||
<el-progress
|
||||
:percentage="loadPct(a)"
|
||||
:stroke-width="10"
|
||||
:color="loadPct(a) >= 100 ? '#ef4444' : loadPct(a) >= 70 ? '#f59e0b' : '#10b981'"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div class="mt-3 text-xs text-slate-400 flex items-center gap-1">
|
||||
<span class="badge-dot" :class="a.status === 'online' ? 'bg-[#10b981] pulse' : 'bg-[#ef4444]'"></span>
|
||||
心跳 {{ fmtTime(a.last_heartbeat) }} · 注册 {{ fmtTime(a.create_time) }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
115
frontend/src/views/ControlView.vue
Normal file
115
frontend/src/views/ControlView.vue
Normal file
@ -0,0 +1,115 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, ref } from 'vue'
|
||||
import { ElMessage, ElMessageBox } from 'element-plus'
|
||||
import { api, type AgentInfo, type TaskInfo } from '../api'
|
||||
|
||||
const tasks = ref<TaskInfo[]>([])
|
||||
const agents = ref<AgentInfo[]>([])
|
||||
const loading = ref(false)
|
||||
|
||||
async function load() {
|
||||
loading.value = true
|
||||
try {
|
||||
const [t, a] = await Promise.all([api.listTasks(), api.listAgents()])
|
||||
tasks.value = t
|
||||
agents.value = a
|
||||
} catch (e) {
|
||||
console.error(e)
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function cancelTask(id: string) {
|
||||
try {
|
||||
await ElMessageBox.confirm(`确认取消任务 ${id} ?`, '取消任务', { type: 'warning' })
|
||||
await api.cancelTask(id)
|
||||
ElMessage.success('任务已取消')
|
||||
load()
|
||||
} catch (e) {
|
||||
if (e !== 'cancel') console.error(e)
|
||||
}
|
||||
}
|
||||
|
||||
async function resetTask(id: string) {
|
||||
try {
|
||||
await ElMessageBox.confirm(`确认重置任务 ${id} 为待调度?`, '重置任务', { type: 'warning' })
|
||||
await api.resetTask(id)
|
||||
ElMessage.success('任务已重置')
|
||||
load()
|
||||
} catch (e) {
|
||||
if (e !== 'cancel') console.error(e)
|
||||
}
|
||||
}
|
||||
|
||||
async function offlineAgent(id: string) {
|
||||
try {
|
||||
await ElMessageBox.confirm(`确认下线 Agent ${id} ?`, '下线 Agent', { type: 'warning' })
|
||||
await api.offlineAgent(id)
|
||||
ElMessage.success('Agent 已下线')
|
||||
load()
|
||||
} catch (e) {
|
||||
if (e !== 'cancel') console.error(e)
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(load)
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="space-y-5 fade-in">
|
||||
<div class="page-card">
|
||||
<h2 class="font-semibold mb-4">取消任务</h2>
|
||||
<el-table :data="tasks.filter((t) => t.status === 'pending' || t.status === 'running')" v-loading="loading" stripe>
|
||||
<el-table-column label="RequestID" prop="request_id" min-width="220" show-overflow-tooltip />
|
||||
<el-table-column label="任务描述" min-width="220" show-overflow-tooltip>
|
||||
<template #default="{ row }">{{ row.description || '-' }}</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="状态" width="100">
|
||||
<template #default="{ row }">
|
||||
<el-tag :type="row.status === 'running' ? 'primary' : 'warning'" effect="light">
|
||||
{{ row.status === 'running' ? '执行中' : '待调度' }}
|
||||
</el-tag>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="操作" width="160">
|
||||
<template #default="{ row }">
|
||||
<el-button link type="danger" @click="cancelTask(row.request_id)">取消</el-button>
|
||||
<el-button link type="warning" @click="resetTask(row.request_id)">重置</el-button>
|
||||
</template>
|
||||
</el-table-column>
|
||||
</el-table>
|
||||
<div v-if="loading === false && tasks.filter((t) => t.status === 'pending' || t.status === 'running').length === 0" class="text-center text-slate-400 py-8">
|
||||
当前没有可管控的任务
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="page-card">
|
||||
<h2 class="font-semibold mb-4">下线 Agent</h2>
|
||||
<el-table :data="agents" v-loading="loading" stripe>
|
||||
<el-table-column label="AgentID" prop="agent_id" min-width="180" />
|
||||
<el-table-column label="地址" prop="endpoint" min-width="160" show-overflow-tooltip />
|
||||
<el-table-column label="状态" width="100">
|
||||
<template #default="{ row }">
|
||||
<el-tag :type="row.status === 'online' ? 'success' : 'danger'" effect="light">
|
||||
{{ row.status === 'online' ? '在线' : '离线' }}
|
||||
</el-tag>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="操作" width="160">
|
||||
<template #default="{ row }">
|
||||
<el-button
|
||||
link
|
||||
type="danger"
|
||||
:disabled="row.status !== 'online'"
|
||||
@click="offlineAgent(row.agent_id)"
|
||||
>
|
||||
下线
|
||||
</el-button>
|
||||
</template>
|
||||
</el-table-column>
|
||||
</el-table>
|
||||
<div v-if="loading === false && agents.length === 0" class="text-center text-slate-400 py-8">暂无 Agent</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
71
frontend/src/views/LogView.vue
Normal file
71
frontend/src/views/LogView.vue
Normal file
@ -0,0 +1,71 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, ref } from 'vue'
|
||||
import { api, type LogEntry } from '../api'
|
||||
|
||||
const logs = ref<LogEntry[]>([])
|
||||
const loading = ref(false)
|
||||
const requestId = ref('')
|
||||
const agentId = ref('')
|
||||
|
||||
const scopeLabel: Record<string, string> = { task: '任务', agent: 'Agent', system: '系统' }
|
||||
const levelColor: Record<string, string> = {
|
||||
info: '#10b981',
|
||||
warning: '#f59e0b',
|
||||
error: '#ef4444',
|
||||
}
|
||||
|
||||
async function load() {
|
||||
loading.value = true
|
||||
try {
|
||||
logs.value = await api.listLogs({
|
||||
request_id: requestId.value || undefined,
|
||||
agent_id: agentId.value || undefined,
|
||||
})
|
||||
} catch (e) {
|
||||
console.error(e)
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function fmtTime(ts: number) {
|
||||
return new Date(ts * 1000).toLocaleString()
|
||||
}
|
||||
|
||||
onMounted(load)
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="page-card fade-in">
|
||||
<div class="flex flex-wrap items-center gap-3 mb-5">
|
||||
<h2 class="font-semibold mr-2">全链路审计日志</h2>
|
||||
<el-input v-model="requestId" placeholder="RequestID 筛选" clearable style="width: 220px" @keyup.enter="load" />
|
||||
<el-input v-model="agentId" placeholder="AgentID 筛选" clearable style="width: 200px" @keyup.enter="load" />
|
||||
<el-button type="primary" @click="load">查询</el-button>
|
||||
</div>
|
||||
|
||||
<el-timeline v-loading="loading">
|
||||
<el-timeline-item
|
||||
v-for="(log, i) in logs"
|
||||
:key="i"
|
||||
:color="levelColor[log.level] || '#cbd5e1'"
|
||||
:timestamp="fmtTime(log.ts)"
|
||||
placement="top"
|
||||
>
|
||||
<div class="border border-slate-100 rounded-lg p-3 bg-slate-50/50">
|
||||
<div class="flex items-center gap-2 mb-1">
|
||||
<el-tag size="small" :type="log.scope === 'agent' ? 'success' : log.scope === 'system' ? 'info' : 'primary'" effect="light">
|
||||
{{ scopeLabel[log.scope] || log.scope }}
|
||||
</el-tag>
|
||||
<span class="text-xs text-slate-400">{{ log.source }}</span>
|
||||
<span v-if="log.request_id" class="text-xs font-mono text-slate-500">#{{ log.request_id }}</span>
|
||||
<span v-if="log.agent_id" class="text-xs font-mono text-slate-500">agent:{{ log.agent_id }}</span>
|
||||
</div>
|
||||
<div class="text-sm">{{ log.message }}</div>
|
||||
</div>
|
||||
</el-timeline-item>
|
||||
</el-timeline>
|
||||
|
||||
<div v-if="!loading && logs.length === 0" class="text-center text-slate-400 py-16">暂无日志</div>
|
||||
</div>
|
||||
</template>
|
||||
175
frontend/src/views/TaskView.vue
Normal file
175
frontend/src/views/TaskView.vue
Normal file
@ -0,0 +1,175 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, onUnmounted, ref } from 'vue'
|
||||
import { ElMessage } from 'element-plus'
|
||||
import { api, type TaskInfo, type Overview } from '../api'
|
||||
|
||||
const tasks = ref<TaskInfo[]>([])
|
||||
const overview = ref<Overview | null>(null)
|
||||
const statusFilter = ref('')
|
||||
const loading = ref(false)
|
||||
const detail = ref<TaskInfo | null>(null)
|
||||
const detailVisible = ref(false)
|
||||
let timer: number | undefined
|
||||
|
||||
const statusMeta: Record<string, { label: string; type: string; color: string }> = {
|
||||
pending: { label: '待调度', type: 'warning', color: '#f59e0b' },
|
||||
running: { label: '执行中', type: 'primary', color: '#1e63e8' },
|
||||
success: { label: '成功', type: 'success', color: '#10b981' },
|
||||
failed: { label: '失败', type: 'danger', color: '#ef4444' },
|
||||
}
|
||||
|
||||
async function load(silent = false) {
|
||||
if (!silent) loading.value = true
|
||||
try {
|
||||
tasks.value = await api.listTasks(statusFilter.value || undefined)
|
||||
overview.value = await api.overview()
|
||||
} catch (e) {
|
||||
console.error(e)
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function fmtTime(ts: number) {
|
||||
if (!ts) return '-'
|
||||
return new Date(ts * 1000).toLocaleString()
|
||||
}
|
||||
|
||||
// result 结构:{"reply": "接受任务后的回复", "output": "最终总结"}
|
||||
function resultReply(t: TaskInfo | null): string {
|
||||
const r = t?.result as Record<string, unknown> | null | undefined
|
||||
return r && typeof r.reply === 'string' && r.reply.trim() ? r.reply : '-'
|
||||
}
|
||||
|
||||
function resultOutput(t: TaskInfo | null): string {
|
||||
const r = t?.result as Record<string, unknown> | null | undefined
|
||||
if (r && typeof r.output === 'string' && r.output.trim()) return r.output
|
||||
return t?.result ? JSON.stringify(t.result, null, 2) : '-'
|
||||
}
|
||||
|
||||
async function openDetail(id: string) {
|
||||
try {
|
||||
detail.value = await api.taskDetail(id)
|
||||
detailVisible.value = true
|
||||
} catch (e) {
|
||||
console.error(e)
|
||||
}
|
||||
}
|
||||
|
||||
async function cancelTask(id: string) {
|
||||
try {
|
||||
await api.cancelTask(id)
|
||||
ElMessage.success('任务已取消')
|
||||
load()
|
||||
} catch (e) {
|
||||
ElMessage.error('取消失败')
|
||||
console.error(e)
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(() => {
|
||||
load()
|
||||
timer = window.setInterval(() => load(true), 5000)
|
||||
})
|
||||
onUnmounted(() => {
|
||||
if (timer) window.clearInterval(timer)
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="space-y-5 fade-in">
|
||||
<!-- 状态统计卡片 -->
|
||||
<div class="grid grid-cols-2 md:grid-cols-4 gap-4">
|
||||
<div class="stat-card">
|
||||
<div class="text-sm text-slate-500">待调度</div>
|
||||
<div class="stat-value text-[#f59e0b]">{{ overview?.task_pending ?? 0 }}</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="text-sm text-slate-500">执行中</div>
|
||||
<div class="stat-value text-[#1e63e8]">{{ overview?.task_running ?? 0 }}</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="text-sm text-slate-500">成功</div>
|
||||
<div class="stat-value text-[#10b981]">{{ overview?.task_success ?? 0 }}</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="text-sm text-slate-500">失败</div>
|
||||
<div class="stat-value text-[#ef4444]">{{ overview?.task_failed ?? 0 }}</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 任务表格 -->
|
||||
<div class="page-card">
|
||||
<div class="flex justify-between items-center mb-4">
|
||||
<h2 class="font-semibold">任务列表</h2>
|
||||
<el-select v-model="statusFilter" placeholder="全部状态" clearable style="width: 160px" @change="load">
|
||||
<el-option label="待调度" value="pending" />
|
||||
<el-option label="执行中" value="running" />
|
||||
<el-option label="成功" value="success" />
|
||||
<el-option label="失败" value="failed" />
|
||||
</el-select>
|
||||
</div>
|
||||
|
||||
<el-table :data="tasks" v-loading="loading" stripe>
|
||||
<el-table-column label="RequestID" prop="request_id" min-width="220" show-overflow-tooltip />
|
||||
<el-table-column label="任务描述" prop="description" min-width="220" show-overflow-tooltip>
|
||||
<template #default="{ row }">{{ row.description || '-' }}</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="状态" width="100">
|
||||
<template #default="{ row }">
|
||||
<el-tag :type="statusMeta[row.status]?.type" effect="light">{{ statusMeta[row.status]?.label }}</el-tag>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="Agent" width="150">
|
||||
<template #default="{ row }">{{ row.agent_id || '-' }}</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="进度" width="160">
|
||||
<template #default="{ row }">
|
||||
<el-progress :percentage="row.progress" :stroke-width="8" />
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="创建时间" width="170">
|
||||
<template #default="{ row }">{{ fmtTime(row.create_time) }}</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="操作" width="150" fixed="right">
|
||||
<template #default="{ row }">
|
||||
<el-button link type="primary" @click="openDetail(row.request_id)">详情</el-button>
|
||||
<el-button
|
||||
v-if="row.status === 'pending' || row.status === 'running'"
|
||||
link
|
||||
type="danger"
|
||||
@click="cancelTask(row.request_id)"
|
||||
>
|
||||
取消
|
||||
</el-button>
|
||||
</template>
|
||||
</el-table-column>
|
||||
</el-table>
|
||||
</div>
|
||||
|
||||
<!-- 详情抽屉 -->
|
||||
<el-drawer v-model="detailVisible" title="任务详情" size="440px">
|
||||
<div v-if="detail" class="space-y-4">
|
||||
<div><div class="text-slate-500 text-sm">RequestID</div><div class="font-mono text-sm">{{ detail.request_id }}</div></div>
|
||||
<div><div class="text-slate-500 text-sm">任务描述</div><div>{{ detail.description || '-' }}</div></div>
|
||||
<div><div class="text-slate-500 text-sm">状态</div>
|
||||
<el-tag :type="statusMeta[detail.status]?.type">{{ statusMeta[detail.status]?.label }}</el-tag>
|
||||
</div>
|
||||
<div><div class="text-slate-500 text-sm">Agent</div><div>{{ detail.agent_id || '-' }}</div></div>
|
||||
<div><div class="text-slate-500 text-sm">进度</div>
|
||||
<el-progress :percentage="detail.progress" />
|
||||
</div>
|
||||
<div><div class="text-slate-500 text-sm">创建时间</div><div>{{ fmtTime(detail.create_time) }}</div></div>
|
||||
<div><div class="text-slate-500 text-sm">Agent 接受任务回复</div>
|
||||
<pre class="bg-slate-50 rounded p-2 text-xs overflow-auto whitespace-pre-wrap">{{ resultReply(detail) }}</pre>
|
||||
</div>
|
||||
<div><div class="text-slate-500 text-sm">Agent 总结</div>
|
||||
<pre class="bg-slate-50 rounded p-2 text-xs overflow-auto whitespace-pre-wrap">{{ resultOutput(detail) }}</pre>
|
||||
</div>
|
||||
<div v-if="detail.error_info"><div class="text-slate-500 text-sm">错误信息</div>
|
||||
<div class="text-[#ef4444] text-sm">{{ detail.error_info }}</div>
|
||||
</div>
|
||||
</div>
|
||||
</el-drawer>
|
||||
</div>
|
||||
</template>
|
||||
8
frontend/tailwind.config.js
Normal file
8
frontend/tailwind.config.js
Normal file
@ -0,0 +1,8 @@
|
||||
/** @type {import('tailwindcss').Config} */
|
||||
export default {
|
||||
content: ['./index.html', './src/**/*.{vue,js,ts,jsx,tsx}'],
|
||||
theme: {
|
||||
extend: {},
|
||||
},
|
||||
plugins: [],
|
||||
}
|
||||
20
frontend/tsconfig.json
Normal file
20
frontend/tsconfig.json
Normal file
@ -0,0 +1,20 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2020",
|
||||
"useDefineForClassFields": true,
|
||||
"module": "ESNext",
|
||||
"lib": ["ES2020", "DOM", "DOM.Iterable"],
|
||||
"skipLibCheck": true,
|
||||
"moduleResolution": "bundler",
|
||||
"allowImportingTsExtensions": true,
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"noEmit": true,
|
||||
"jsx": "preserve",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"noUnusedParameters": false,
|
||||
"verbatimModuleSyntax": false
|
||||
},
|
||||
"include": ["src/**/*.ts", "src/**/*.d.ts", "src/**/*.tsx", "src/**/*.vue"]
|
||||
}
|
||||
16
frontend/vite.config.ts
Normal file
16
frontend/vite.config.ts
Normal file
@ -0,0 +1,16 @@
|
||||
import { defineConfig } from 'vite'
|
||||
import vue from '@vitejs/plugin-vue'
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [vue()],
|
||||
server: {
|
||||
host: '0.0.0.0',
|
||||
port: 5173,
|
||||
proxy: {
|
||||
'/api': {
|
||||
target: 'http://localhost:8000',
|
||||
changeOrigin: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
12
requirements.txt
Normal file
12
requirements.txt
Normal file
@ -0,0 +1,12 @@
|
||||
fastapi==0.115.12
|
||||
uvicorn[standard]==0.34.0
|
||||
redis==5.2.1
|
||||
pydantic==2.12.4
|
||||
pydantic-settings==2.10.1
|
||||
python-multipart==0.0.20
|
||||
|
||||
# 测试
|
||||
pytest==8.3.4
|
||||
pytest-asyncio==0.25.3
|
||||
httpx==0.28.1
|
||||
fakeredis==2.26.2
|
||||
236
网关架构流程图.md
Normal file
236
网关架构流程图.md
Normal file
@ -0,0 +1,236 @@
|
||||
# 网关(Gateway)项目架构流程图
|
||||
|
||||
> 基于代码现状绘制的架构图(backend + frontend + agent 联动)。
|
||||
> 端口:后端 `:8000`、前端 `:5173`、Agent `:8001`、Redis 远程实例 `45.207.192.237:56987/0`。
|
||||
|
||||
---
|
||||
|
||||
## 一、系统总体架构图(分层)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph OUT["外部调用方"]
|
||||
CLI["CLI 客户端 / 模型对话<br/>提交任务(带 auth)"]
|
||||
end
|
||||
|
||||
subgraph FE["前端运维后台(Vue3 + Vite + Element Plus):5173"]
|
||||
TV["TaskView 任务管理<br/>列表/筛选/详情/取消/重置"]
|
||||
AV["AgentView Agent 管理<br/>卡片/标签/负载/下线"]
|
||||
LV["LogView 日志审计<br/>按 request_id/agent_id 筛选"]
|
||||
CV["ControlView 总览<br/>任务/Agent 统计"]
|
||||
API["api/index.ts<br/>fetch 封装(/api/admin/*)"]
|
||||
end
|
||||
|
||||
subgraph GW["网关后端(FastAPI + Uvicorn):8000"]
|
||||
direction TB
|
||||
subgraph API_L["API 层(app/api/)"]
|
||||
A1["/api/cli/*<br/>任务提交"]
|
||||
A2["/api/agent/*<br/>注册/心跳/结果回传"]
|
||||
A3["/api/admin/*<br/>运维后台查询与管控"]
|
||||
end
|
||||
|
||||
subgraph SVC["服务层(app/services/)"]
|
||||
S1["TaskService 任务池<br/>受理/取消/重置/超时"]
|
||||
S2["AgentService Agent 池<br/>注册/心跳/下线/剔除"]
|
||||
S3["Scheduler 规则调度<br/>选人→绑定→推送→回退"]
|
||||
S4["RelayService 通信中转<br/>正向推送/反向收结果"]
|
||||
S5["HeartbeatService<br/>心跳扫描(60s)"]
|
||||
S6["TimeoutService<br/>任务超时检查(5s)"]
|
||||
end
|
||||
|
||||
subgraph REPO["存储层(app/repository/)"]
|
||||
R1["TaskRepo"]
|
||||
R2["AgentRepo"]
|
||||
R3["LogRepo"]
|
||||
end
|
||||
|
||||
subgraph LOOP["后台循环(scheduler_loop.py)"]
|
||||
L1["heartbeat_loop<br/>Lock:lock:heartbeat"]
|
||||
L2["dispatch_loop(2s)<br/>Lock:lock:scheduler"]
|
||||
L3["timeout_loop<br/>Lock:lock:timeout"]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph RD["Redis 存储(DB0)"]
|
||||
K1["task:info:{id} Hash<br/>task:pending / task:running Set"]
|
||||
K2["agent:info:{id} Hash<br/>agent:all / agent:tag:{tag} Set"]
|
||||
K3["agent:heartbeat ZSet"]
|
||||
K4["lock:* 分布式锁"]
|
||||
K5["log:audit List"]
|
||||
end
|
||||
|
||||
subgraph AG["Agent 服务(ADK ApiServer):8001"]
|
||||
direction TB
|
||||
AG0["api_server.py<br/>REST /run /run_sse<br/>SQLite 会话 / InMemory 记忆"]
|
||||
AG1["task_receiver.py<br/>POST /tasks/{id}<br/>校验 auth → 202 → 后台线程"]
|
||||
AG2["gateway_client.py<br/>注册 / 心跳 / 回传结果"]
|
||||
AG3["ADK InMemoryRunner<br/>执行 dev_app(my_agent)"]
|
||||
end
|
||||
|
||||
CLI -->|"POST /api/cli/tasks"| A1
|
||||
FE --> API
|
||||
API -->|"/api/admin/*"| A3
|
||||
|
||||
A1 --> S1
|
||||
A2 --> S2
|
||||
A3 --> S1 & S2
|
||||
|
||||
S1 & S2 & S3 & S4 & S5 & S6 --> R1 & R2 & R3
|
||||
L1 -->|"调用"| S5
|
||||
L2 -->|"调用"| S3
|
||||
L3 -->|"调用"| S6
|
||||
S5 -->|"Lock"| K4
|
||||
S3 -->|"Lock"| K4
|
||||
|
||||
R1 & R2 & R3 --> K1 & K2 & K3 & K5
|
||||
|
||||
S3 -->|"Scheduler 选中 Agent"| S4
|
||||
S4 -->|"POST {endpoint}/tasks/{request_id}<br/>期望 202"| AG1
|
||||
AG1 -->|"校验通过后后台线程"| AG3
|
||||
AG3 -->|"执行完成"| AG2
|
||||
AG2 -->|"POST /api/agent/result"| A2
|
||||
AG2 -->|"心跳 /api/agent/heartbeat(10s)"| A2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 二、任务全生命周期时序图(完整闭环)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant CLI as CLI / 对话模型
|
||||
participant GW as 网关后端(FastAPI :8000)
|
||||
participant RD as Redis
|
||||
participant LOOP as SchedulerLoop 后台循环
|
||||
participant AG as Agent 服务(:8001)
|
||||
participant ADK as ADK InMemoryRunner
|
||||
|
||||
Note over AG,ADK: Agent 启动:register → 每 10s heartbeat
|
||||
|
||||
CLI->>GW: POST /api/cli/tasks {auth, task_type, tags, payload}
|
||||
GW->>GW: TaskService.submit(幂等:RequestID 复用)
|
||||
GW->>RD: hset task:info:{id} + sadd task:pending
|
||||
GW-->>CLI: 200 {request_id}
|
||||
|
||||
loop dispatch_loop(每 2s,持 lock:scheduler)
|
||||
LOOP->>RD: 取 task:pending 成员
|
||||
LOOP->>GW: Scheduler.dispatch_pending(batch)
|
||||
GW->>RD: AgentRepo.by_tags(tags) 标签匹配
|
||||
GW->>GW: 负载过滤(load < max_concurrent)
|
||||
GW->>GW: 最低负载 + 在线最久者优先
|
||||
GW->>RD: 任务置 running + 绑定 agent_id<br/>task:pending→task:running,agent 负载 +1
|
||||
GW->>AG: RelayService.dispatch_command<br/>POST {endpoint}/tasks/{id} {auth, payload}
|
||||
alt 推送成功(202)
|
||||
AG->>AG: 校验 auth → 后台线程接收任务
|
||||
AG-->>GW: 202 {ok, request_id, status:"accepted"}
|
||||
AG->>ADK: 运行 agent(InMemoryRunner.run_async)
|
||||
ADK-->>AG: 最终输出文本
|
||||
AG->>GW: POST /api/agent/result {success, progress:100, result}
|
||||
GW->>RD: 更新任务 success + 释放 agent 负载<br/>task:running 移除
|
||||
GW-->>CLI: (前端可查)
|
||||
else 推送失败(网络错误 / 非 202)
|
||||
AG-->>GW: 4xx/5xx 或无响应
|
||||
GW->>RD: 回退:任务置 pending + 解绑 + 释放负载<br/>task:running→task:pending
|
||||
end
|
||||
end
|
||||
|
||||
Note over LOOP,RD: heartbeat_loop:60s 扫 agent:heartbeat ZSet,<br/>超 120s 未心跳 → 标记 offline<br/>timeout_loop:5s 扫 running,超时 → failed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、规则调度决策流程(Scheduler)
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
START(["dispatch_pending<br/>取 task:pending 批量"]) --> T1{"任务存在且为 pending?"}
|
||||
T1 -- 否 --> END(["跳过"])
|
||||
T1 -- 是 --> T2{"按 task_tags 标签匹配<br/>AgentRepo.by_tags(tags)"}
|
||||
T2 -- 无候选 --> END2(["保持 pending<br/>等待下次调度"])
|
||||
T2 -- 有候选 --> T3["负载过滤<br/>current_load < max_concurrent"]
|
||||
T3 -- 空 --> END2
|
||||
T3 -- 非空 --> T4["排序:最低负载优先<br/>负载相同时在线最久优先"]
|
||||
T4 --> T5["绑定 Agent<br/>task:info 置 running + agent_id<br/>task:pending→task:running"]
|
||||
T5 --> T6["agent 负载 +1<br/>adjust_load(agent, +1)"]
|
||||
T6 --> T7["RelayService.dispatch_command<br/>POST {endpoint}/tasks/{id}"]
|
||||
T7 --> T8{"HTTP 202?"}
|
||||
T8 -- 是 --> OK(["下发成功<br/>记录 dispatch 日志"])
|
||||
T8 -- 否 --> T9["回退:task 置 pending + 解绑<br/>task:running→task:pending<br/>agent 负载 -1"]
|
||||
T9 --> END2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 四、Agent 端任务处理流程
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
RECV["POST {endpoint}/tasks/{request_id}"] --> AUTH{"auth == GATEWAY_AUTH?"}
|
||||
AUTH -- 否 --> 401["401 invalid auth"]
|
||||
AUTH -- 是 --> THREAD["启动 daemon 后台线程<br/>(立即返回 202 accepted)"]
|
||||
THREAD --> PROMPT["payload → prompt<br/>(prompt / cmd / 兜底序列化)"]
|
||||
PROMPT --> RUN["InMemoryRunner.run_async<br/>user_id=gateway, session=task-{id}"]
|
||||
RUN --> OUT["收集最终文本输出"]
|
||||
OUT --> REP["gateway_client.report_result<br/>POST /api/agent/result"]
|
||||
RUN --> ERR["异常捕获"]
|
||||
ERR --> REPF["report_result(status=failed, error_info)"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 五、Redis 数据结构一览
|
||||
|
||||
| 前缀 / 键 | 类型 | 用途 |
|
||||
| --- | --- | --- |
|
||||
| `task:info:{request_id}` | Hash | 任务全量信息(状态/绑定 Agent/进度/结果),TTL 24h |
|
||||
| `task:pending` | Set | 待调度任务索引 |
|
||||
| `task:running` | Set | 执行中任务索引 |
|
||||
| `agent:info:{agent_id}` | Hash | Agent 信息(endpoint/标签/并发上限/负载/心跳时间) |
|
||||
| `agent:all` | Set | 全部 Agent 索引 |
|
||||
| `agent:tag:{tag}` | Set | 能力标签 → Agent 索引(调度匹配用) |
|
||||
| `agent:heartbeat` | ZSet | score=最后心跳时间戳(剔除离线用) |
|
||||
| `lock:heartbeat` / `lock:scheduler` / `lock:timeout` | String | 分布式锁,防多实例重复执行后台循环 |
|
||||
| `log:audit` | List | 全链路审计日志(dispatch/progress/result) |
|
||||
|
||||
---
|
||||
|
||||
## 六、技术栈与端口汇总
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph 后端
|
||||
B["Python + FastAPI + Uvicorn<br/>redis-py(asyncio) + httpx<br/>端口 8000"]
|
||||
end
|
||||
subgraph 前端
|
||||
F["Vue 3 + Vite + Element Plus<br/>端口 5173(/api 代理到 8000)"]
|
||||
end
|
||||
subgraph Agent
|
||||
A["Python + Google ADK v2.5<br/>FastAPI ApiServer + InMemoryRunner<br/>端口 8001"]
|
||||
end
|
||||
subgraph 存储
|
||||
R["Redis(远程 45.207.192.237:56987/0)<br/>任务池 + Agent 池 + 索引 + 日志 + 锁"]
|
||||
end
|
||||
B <--> R
|
||||
B <-->|"HTTP 推送 / 结果回传"| A
|
||||
F -->|"/api/admin/*"| B
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 七、任务状态机
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> pending: submit 受理(幂等)
|
||||
pending --> running: 调度器选 Agent 并推送成功
|
||||
pending --> pending: 无可用 Agent / 推送失败回退
|
||||
running --> success: Agent 回传 result(success)
|
||||
running --> failed: Agent 回传 failed
|
||||
running --> failed: 超时 / 用户取消
|
||||
pending --> failed: 用户取消
|
||||
success --> pending: 手动重置(重新调度)
|
||||
failed --> pending: 手动重置(重新调度)
|
||||
success --> [*]: TTL 归档
|
||||
failed --> [*]: TTL 归档
|
||||
```
|
||||
Loading…
Reference in New Issue
Block a user