项目初始化-首次提交

This commit is contained in:
handsomeAq 2026-08-04 17:20:08 +08:00
commit 2c1f06c84c
58 changed files with 6252 additions and 0 deletions

27
.env.example Normal file
View File

@ -0,0 +1,27 @@
# Redis 连接
REDIS_URL=redis://localhost:6379/0
# 网关服务
HOST=0.0.0.0
PORT=8000
# 后端运行目录(用于前端代理,预留)
BACKEND_ROOT=backend
# 心跳保活参数(秒)
HEARTBEAT_SCAN_INTERVAL=60
AGENT_HEARTBEAT_TIMEOUT=120
AGENT_HEARTBEAT_EXPECTED=10
# 调度参数(秒)
DISPATCH_INTERVAL=2
TASK_TIMEOUT_CHECK_INTERVAL=5
# 任务默认超时(秒)
DEFAULT_TASK_TIMEOUT=3600
# 任务数据保留 TTL默认 24 小时
TASK_TTL=86400
# 认证CLI 提交任务 / Agent 注册时必须携带的密码
GATEWAY_AUTH=change-me

71
.gitignore vendored Normal file
View File

@ -0,0 +1,71 @@
# ===== Environment / Secrets =====
# 环境变量(含密钥,切勿提交)
.env
.env.*
!.env.example
# ===== Python (backend) =====
__pycache__/
*.py[cod]
*$py.class
*.so
*.egg-info/
.eggs/
build/
dist/
htmlcov/
.venv/
venv/
env/
ENV/
.python-version
.mypy_cache/
.pytest_cache/
.ruff_cache/
.coverage
.tox/
*.egg
# ===== Node (frontend) =====
node_modules/
frontend/node_modules/
frontend/dist/
*.tsbuildinfo
frontend/tsconfig.tsbuildinfo
# ===== Logs =====
*.log
logs/
*.log.*
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
# ===== 运行时 / 生成文件 =====
gateway_restart.log
gateway_restart_err.log
frontend/frontend.err.log
frontend/frontend.log
frontend/vite.log
backend/backend.err.log
backend/backend.log
backend/gateway_restart.err.log
backend/gateway_restart.log
# ===== IDE / 编辑器 =====
.idea/
.vscode/
*.swp
*.swo
*~
.DS_Store
Thumbs.db
# ===== 项目工具数据 =====
.codebuddy/
# ===== 构建产物与压缩包 =====
*.tar.gz
*.zip
*.tgz

View File

@ -0,0 +1,231 @@
# A2A需求分析\-网关\-K8s全链路架构详细设计方案
## 一、架构整体概述
### 1\.1 核心架构分层
整体采用**三层解耦架构:需求分析层 → 智能网关层 → K8s Agent算力层**完全贴合你的设计思路兼顾本地IDE/CLI异步任务、云端Agent弹性调度、全链路任务管控。
- **需求分析层(前端调度端)**CLI终端、VSCode IDE、Claude Code/CodeBuddy等AI终端负责自然语言需求解析、复杂任务拆解、本地轻量任务异步执行、结构化任务上报网关。
- **智能网关层(核心中枢)**基于FastAPI\+ElementUI\+Redis搭建核心能力为任务池管理、Agent池管理、心跳保活、任务\-Agent规则匹配、全链路通信中转、任务生命周期管控、可视化运维。
- **K8s算力层执行端**全权负责Agent容器的生命周期管理创建、扩容、销毁、重启Agent启动/销毁主动通知网关,实现集群弹性伸缩。
### 1\.2 核心设计理念
- 分层解耦:需求拆解、任务调度、算力执行完全隔离,支持独立迭代扩容;
- 异步协同本地CLI可自主处理轻量编程任务复杂任务上云调度K8s Agent轻重任务分流
- 动态感知网关实时维护在线Agent池通过定时心跳剔除异常节点适配K8s动态扩缩容
- 精准匹配基于自定义规则实现任务与Agent能力精准绑定支持负载均衡、优先级调度
- 统一通信网关作为唯一通信中枢承接CLI与云端Agent的双向交互通过AgentID精准关联任务链路。
## 二、各层级详细能力设计
### 2\.1 需求分析层CLI/IDE/AI终端
#### 2\.1\.1 核心角色
包含自研CLI、VSCode插件、Claude Code、CodeBuddy等所有前端需求入口作为**顶层任务调度者**具备LLM自主拆解能力。
#### 2\.1\.2 核心能力
1. **需求智能拆解**:接收用户自然语言需求,自主拆分出可独立执行的原子任务,区分轻量本地任务、重型云端任务;
2. **本地异步执行**简单代码补全、语法校验、本地文件修改等轻量任务由CLI/IDE本地异步处理无需上云
3. **云端任务上报**:编译构建、批量测试、数据分析、流媒体处理等复杂任务,结构化封装后提交至网关;
4. **结果接收与聚合**接收网关转发的Agent执行结果LLM聚合输出最终答案反馈给用户
5. **全链路追踪**每个任务生成全局唯一RequestID贯穿网关、Agent全链路用于结果精准匹配、日志溯源。
#### 2\.1\.3 任务上报协议
采用标准HTTP JSON协议适配所有AI终端工具调用规范请求体核心字段RequestID、任务类型、任务标签、任务载荷、超时时间、本地工作目录信息。
### 2\.2 智能网关层(核心中枢)
技术栈:**FastAPI高性能接口服务\+ ElementUI可视化后台\+ Redis内存数据存储**无状态服务设计支持水平扩容核心维护两大核心资源池Task任务池、Agent算力池。
#### 2\.2\.1 核心模块拆分
- **API接入模块**接收CLI/IDE任务提交、Agent注册/注销、心跳上报、任务结果回传;
- **任务池管理模块**:缓存所有待执行/执行中/已完成任务,维护任务全生命周期状态;
- **Agent池管理模块**维护所有K8s在线Agent信息基于定时心跳保活、剔除异常节点
- **规则调度模块**实现任务与Agent的标签匹配、负载均衡、优先级调度
- **通信中转模块**基于AgentID关联CLI与Agent双向转发任务指令、执行日志、最终结果
- **可视化运维模块**ElementUI实现任务列表、Agent在线状态、执行日志、异常告警可视化展示。
#### 2\.2\.2 Task任务池设计Redis存储
网关接收CLI任务后立即写入Redis任务池同步返回「任务受理成功」响应不阻塞CLI实现异步调度。
**任务数据结构**RequestID、TaskType、TaskTags、Payload、Statuspending/running/success/failed、AgentID、CliSessionID、CreateTime、Timeout、Progress、Result、ErrorInfo。
**任务状态流转**:待受理 → 待调度 → 执行中 → 执行成功/执行失败 → 任务归档TTL自动清理
#### 2\.2\.3 Agent算力池设计Redis存储
全权管理K8s集群所有Agent实例适配K8s动态生命周期实现动态注册、失效剔除。
**Agent数据结构**AgentID、EndpointK8s内部访问地址、AgentTags能力标签、MaxConcurrent最大并发、CurrentLoad当前负载、LastHeartbeat最后心跳时间、Statusonline/offline、CreateTime。
#### 2\.2\.4 心跳保活机制(核心设计)
严格遵循你的设计:**网关定时每分钟轮询维持心跳**,补充工程化容错逻辑:
1. 网关后台启动定时任务每60s扫描一次Redis Agent池
2. 判定规则连续2个周期120s未上报心跳 → 标记Agent为offline从可用算力池剔除
3. 被动更新Agent启动主动注册、销毁主动注销实时更新Agent池状态优先覆盖定时轮询结果
4. 负载同步:心跳同步上报当前任务并发数,为调度算法提供依据。
#### 2\.2\.5 任务\-Agent匹配调度规则
网关核心调度逻辑,实现精准任务分发,优先级从高到低:
1. **标签精准匹配**根据任务TaskTags筛选具备对应能力标签的Agent如代码编译、日志分析、流媒体处理
2. **负载过滤**剔除当前负载已满CurrentLoad ≥ MaxConcurrent的Agent
3. **最优算力选择**优先选择负载最低、在线时间最稳定的Agent
4. **任务绑定**匹配成功后将AgentID写入任务池对应任务标记任务为running下发任务至目标Agent。
#### 2\.2\.6 双向通信机制(核心闭环)
完全贴合你的设计:**网关作为唯一中转携带AgentID完成CLI与Agent双向通信**
1. 正向链路CLI → AgentCLI提交任务 → 网关入库匹配Agent → 携带AgentID下发任务指令
2. 反向链路Agent → CLIAgent执行完成/产生日志 → 结果回传网关 → 网关通过RequestID\+AgentID关联对应CLI会话 → 精准返回结果;
3. 链路隔离多任务、多Agent、多CLI会话通过RequestID\+AgentID双维度隔离杜绝消息错乱。
### 2\.3 K8s算力层
#### 2\.3\.1 核心职责
专注Agent容器生命周期管理不参与业务调度职责单一Agent镜像部署、弹性扩缩容、故障重启、资源配额管控、节点销毁。
#### 2\.3\.2 Agent生命周期联动网关机制
1. **Agent创建启动**Pod启动后自动读取K8s环境变量网关地址、Pod信息主动调用网关`/agent/register`接口完成注册上报能力标签、并发上限、访问地址网关更新Agent池纳入算力调度。
2. **Agent运行中**每10s主动上报心跳同步当前负载网关定时轮询兜底校验
3. **Agent销毁/缩容**Pod收到SIGTERM信号后主动调用网关`/agent/unregister`接口优雅注销网关立即剔除该Agent终止其未完成任务避免任务调度失效
4. **Agent异常崩溃**无主动注销时依赖网关120s心跳超时机制自动剔除实现故障自愈。
## 三、全链路完整执行流程
整合所有模块,端到端闭环流程如下:
1. **需求拆解上报**用户在CLI/IDE输入需求AI终端自主拆解任务轻量任务本地异步执行复杂任务结构化后提交网关
2. **网关任务受理**:网关接收任务,生成/复用RequestID写入Redis任务池立即返回受理响应避免CLI阻塞
3. **算力匹配调度**网关遍历在线Agent池通过标签\+负载规则匹配最优Agent绑定AgentID下发任务
4. **Agent执行任务**K8s Agent接收任务开始执行定时上报执行进度、日志至网关
5. **结果回传中转**Agent执行完成后将结果/错误信息回传网关网关通过AgentID\+RequestID匹配原CLI会话
6. **结果聚合反馈**网关推送结果至CLIAI终端聚合结果、整理答案最终反馈给用户
7. **状态归档清理**:网关更新任务为成功/失败状态,留存日志,到期自动清理冗余数据。
## 四、核心数据存储设计Redis
### 4\.1 任务池存储结构
- Hash结构`task:info:{RequestID}`存储任务全量信息;
- Set结构`task:pending` 待调度任务集合、`task:running` 执行中任务集合;
- TTL策略任务数据默认保留24小时自动过期清理释放资源。
### 4\.2 Agent池存储结构
- Hash结构`agent:info:{AgentID}` 存储Agent全量信息
- ZSet结构`agent:heartbeat` score=最后心跳时间,用于网关定时清理失效节点;
- Set索引`agent:tag:{tag}` 按能力标签索引Agent加速任务匹配效率。
## 五、容错与异常处理机制
### 5\.1 任务异常容错
- 任务超时网关自定义任务超时时间超时自动标记失败释放Agent算力反馈CLI
- Agent离线执行中Agent失联网关立即终止任务标记异常支持CLI重提任务
- 幂等性基于RequestID实现任务幂等避免CLI重试导致重复调度。
### 5\.2 Agent容错
- 心跳兜底:主动心跳\+网关定时轮询双机制,杜绝僵尸节点;
- 重启自愈K8s负责Agent故障重启重启后自动重新注册纳入算力池。
### 5\.3 通信容错
- 断连不丢任务任务数据持久化Redis网关重启、网络波动不丢失任务状态
- 精准重连匹配重连后通过RequestID\+AgentID恢复任务链路继续推送结果。
## 六、可视化运维能力ElementUI
基于ElementUI搭建后台提供全维度可视化管控
- 任务管理查看所有任务状态、进度、关联AgentID、执行日志、结果详情
- Agent管理在线Agent列表、能力标签、实时负载、心跳状态、异常节点告警
- 日志审计全链路任务操作日志、Agent注册注销日志、通信记录溯源
- 手动管控支持手动取消任务、下线异常Agent、重置任务状态。
## 七、架构核心优势
1. **极简解耦**三层架构职责清晰K8s只管生命周期网关只管调度通信CLI只管需求拆解
2. **弹性适配**完美适配K8s扩缩容Agent动态上下线自动感知无需人工干预
3. **高低速任务分流**:本地轻量异步处理,云端重型算力调度,资源利用率最大化;
4. **精准链路管控**AgentID\+RequestID双维度绑定通信不错乱、任务可溯源
5. **轻量易落地**FastAPI\+Redis技术栈轻量化部署简单、开发成本低配套可视化运维
6. **可扩展性强**:后续可扩展多租户、权限管控、负载均衡优化、大文件中转、本地文件远程读写等能力。
## 八、落地实施步骤
1. 搭建网关基础服务FastAPI接口、Redis数据结构、ElementUI可视化基础页面
2. 实现Agent注册、注销、心跳上报、网关定时轮询保活能力
3. 开发任务池管理、任务\-Agent匹配调度核心逻辑
4. 打通CLI任务上报、网关调度、Agent执行、结果回传全链路
5. 完善异常容错、日志审计、可视化运维功能;
6. 压测优化适配K8s大规模Agent集群调度。
> (注:部分内容可能由 AI 生成)

139
README.md Normal file
View File

@ -0,0 +1,139 @@
# A2A 智能网关gateway
基于设计文档《A2A需求分析-网关-K8s全链路架构详细设计方案》实现的智能网关核心服务。
本期范围:**网关核心 + 可视化运维后台**。Agent/CLI 为预留接口K8s 算力层不实际部署。
## 技术栈
- 后端Python + FastAPI + Uvicorn + redis-pyasyncio
- 存储Redis任务池 / Agent 池 / 调度索引 / 心跳 ZSet / 审计日志)
- 前端Vue 3 + Vite + Element Plus完整运维后台
- 后台任务asyncio 循环(心跳扫描 / 调度 / 超时检查),配 Redis 分布式锁防多实例重复执行
## 目录结构
```
gateway/
├── docker-compose.yml # 本地一键启动 Redis
├── requirements.txt # Python 依赖
├── .env.example # 环境变量示例
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI 入口
│ │ ├── config.py # 配置
│ │ ├── constants.py # Redis 键名/TTL/状态常量
│ │ ├── models/schemas.py # Pydantic 模型
│ │ ├── repository/ # Redis 存储层task/agent/log
│ │ ├── services/ # 业务服务(任务/Agent/调度/心跳/通信/超时/锁)
│ │ ├── api/ # 路由cli/agent/admin
│ │ ├── middleware.py # 日志中间件
│ │ └── scheduler_loop.py # 后台任务循环
│ └── tests/ # 单元与集成测试
└── frontend/ # Vue3 + Element Plus 运维后台
```
## 快速启动
### 1. 启动 Redis
```bash
docker compose up -d
```
无 Docker 时需本机提供 Redis 实例,并设置 `REDIS_URL`
### 2. 启动后端
```bash
cd backend
pip install -r ../requirements.txt
# 复制 .env.example 为 .env 并按需修改
uvicorn app.main:app --host 0.0.0.0 --port 8000
```
接口文档Swaggerhttp://localhost:8000/docs
### 3. 启动前端运维后台
```bash
cd frontend
# 使用 Node >= 18推荐 18/20/22
npm install
npm run dev
```
访问 http://localhost:5173 Vite 已将 `/api` 代理到后端 `:8000`
### 4. 运行后端测试
```bash
cd backend
python -m pytest -q
```
## 核心机制
### 任务池Redis
- `task:info:{request_id}`任务全量信息Hash
- `task:pending` / `task:running`调度状态索引Set
- 状态流转pending → running → success/failed
- TTL 默认 24h 自动归档RequestID 幂等去重
### Agent 池Redis
- `agent:info:{agent_id}`Agent 信息Hash
- `agent:heartbeat`心跳时间戳ZSetscore=最后心跳)
- `agent:tag:{tag}`能力标签索引Set
- 心跳保活:网关每 60s 扫描,连续 120s 未心跳标记 offline
### 规则调度
标签精准匹配 → 负载过滤(当前负载 < 并发上限 最低负载停留时间久者优先 绑定 Agent 并下发
### 认证
CLI 提交任务与 Agent 注册时,请求体必须携带 `auth` 字段,值须与网关配置的 `GATEWAY_AUTH``.env` 中设置)一致,否则返回 `401`。示例:
```bash
curl -X POST http://localhost:8000/api/cli/tasks \
-H "Content-Type: application/json" \
-d '{"auth": "<GATEWAY_AUTH>", "task_type": "compile", "task_tags": ["build"], "payload": {"cmd": "python -m build"}}'
```
### 预留 Agent 协议接口
> 所有接口中 `register` 必须携带 `auth` 字段,其余接口以已注册的 `agent_id` 关联身份。
| 接口 | 说明 |
| --- | --- |
| `POST /api/agent/register` | Agent 启动注册(`auth` + 能力标签、并发上限、地址) |
| `POST /api/agent/unregister` | 优雅注销 |
| `POST /api/agent/heartbeat` | 心跳(约每 10s同步负载 |
| `POST /api/agent/result` | 任务结果回传 |
### 通信中转
`RequestID + AgentID` 双维度关联 CLI 会话与 Agent正向下发任务指令、反向回传进度与结果。本期为状态机闭环 + 日志记录,实际网络下发由 Agent 接入时扩展。
## 运维后台
- 任务管理:列表 / 筛选 / 详情 / 进度 / 取消
- Agent 管理:卡片网格 / 标签 / 负载 / 心跳 / 离线高亮
- 日志审计:全链路时间线,按 RequestID / AgentID 筛选
- 手动管控:取消任务 / 重置任务 / 下线 Agent
## 环境变量
`.env.example`,关键参数:
| 变量 | 默认 | 说明 |
| --- | --- | --- |
| `REDIS_URL` | `redis://localhost:6379/0` | Redis 连接 |
| `HEARTBEAT_SCAN_INTERVAL` | `60` | 心跳扫描间隔(秒) |
| `AGENT_HEARTBEAT_TIMEOUT` | `120` | 心跳超时剔除阈值(秒) |
| `DISPATCH_INTERVAL` | `2` | 调度循环间隔(秒) |
| `TASK_TIMEOUT_CHECK_INTERVAL` | `5` | 任务超时检查间隔(秒) |
| `DEFAULT_TASK_TIMEOUT` | `3600` | 任务默认超时(秒) |
| `TASK_TTL` | `86400` | 任务数据保留 TTL |

81
_test_sse.py Normal file
View File

@ -0,0 +1,81 @@
"""端到端验证SSE 订阅 + 任务提交,确认网关主动推送 task_done。"""
import asyncio
import json
import uuid
import httpx
BASE = "http://127.0.0.1:8000"
AUTH = "gw_Hz8Qp3Km9f"
SESSION = f"e2e-{uuid.uuid4().hex[:8]}"
async def main() -> None:
async with httpx.AsyncClient(timeout=30) as client:
# 1) 提交任务(携带 cli_session_id任务类型用 agent 已注册的能力
resp = await client.post(
f"{BASE}/api/cli/tasks",
json={
"auth": AUTH,
"task_type": "dev",
"task_tags": ["code", "dev"],
"description": "SSE 通知验证:创建临时文件",
"cli_session_id": SESSION,
"payload": {
"type": "create_files",
"files": [{"path": "_sse_probe.txt", "content": "hello-sse"}],
},
"timeout": 120,
},
)
print("submit status:", resp.status_code)
task = resp.json()
request_id = task["request_id"]
print("request_id:", request_id, "initial status:", task["status"])
# 2) 订阅 SSE等待 task_done
got = []
async def subscribe():
try:
async with client.stream(
"GET",
f"{BASE}/api/cli/events",
params={"cli_session_id": SESSION, "auth": AUTH},
) as stream:
async for line in stream.aiter_lines():
if line.startswith("data: "):
evt = json.loads(line[6:])
got.append(evt)
print("SSE EVENT:", json.dumps(evt, ensure_ascii=False))
if evt.get("request_id") == request_id:
return
except Exception as exc: # noqa: BLE001
print("SSE error:", exc)
# 3) 并行:订阅 + 轮询兜底打印(仅观察,不作为结论)
await asyncio.gather(
subscribe(),
_poll_until_done(client, request_id),
)
if any(e.get("request_id") == request_id for e in got):
print("\nRESULT: PASS - SSE task_done received without polling")
else:
print("\nRESULT: FAIL - no SSE event for this request_id")
async def _poll_until_done(client, request_id: str) -> None:
for _ in range(60):
r = await client.get(f"{BASE}/api/cli/tasks/{request_id}")
if r.status_code == 404:
await asyncio.sleep(2)
continue
t = r.json()
print("poll status:", t["status"])
if t["status"] in ("success", "failed"):
return
await asyncio.sleep(2)
if __name__ == "__main__":
asyncio.run(main())

0
backend/app/__init__.py Normal file
View File

View File

103
backend/app/api/admin.py Normal file
View File

@ -0,0 +1,103 @@
"""后台管理接口:任务/Agent/日志/手动管控。"""
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Request
from app.constants import TaskStatus
from app.models.schemas import AgentInfo, LogEntry, TaskInfo
from app.repository.log_repo import LogRepo
from app.services.agent_service import AgentService
from app.services.task_service import TaskService
router = APIRouter()
def get_task_service(request: Request) -> TaskService:
return TaskService(request.app.state.redis)
def get_agent_service(request: Request) -> AgentService:
return AgentService(request.app.state.redis)
def get_log_repo(request: Request) -> LogRepo:
return LogRepo(request.app.state.redis)
# ---------- 任务管理 ----------
@router.get("/tasks", response_model=list[TaskInfo], summary="任务列表")
async def list_tasks(
status: Optional[TaskStatus] = None,
limit: int = 100,
svc: TaskService = Depends(get_task_service),
):
return await svc.list(status=status, limit=limit)
@router.get("/tasks/{request_id}", response_model=TaskInfo, summary="任务详情")
async def task_detail(request_id: str, svc: TaskService = Depends(get_task_service)):
task = await svc.get(request_id)
if not task:
raise HTTPException(status_code=404, detail="task not found")
return task
@router.post("/tasks/{request_id}/cancel", summary="取消任务")
async def admin_cancel(request_id: str, svc: TaskService = Depends(get_task_service)):
if not await svc.cancel(request_id):
raise HTTPException(status_code=400, detail="cannot cancel task")
return {"ok": True}
@router.post("/tasks/{request_id}/reset", summary="重置任务状态")
async def admin_reset(request_id: str, svc: TaskService = Depends(get_task_service)):
if not await svc.reset(request_id):
raise HTTPException(status_code=400, detail="cannot reset task")
return {"ok": True}
# ---------- Agent 管理 ----------
@router.get("/agents", response_model=list[AgentInfo], summary="Agent 列表")
async def list_agents(svc: AgentService = Depends(get_agent_service)):
return await svc.all()
@router.post("/agents/{agent_id}/offline", summary="下线 Agent")
async def agent_offline(agent_id: str, svc: AgentService = Depends(get_agent_service)):
if not await svc.take_offline(agent_id):
raise HTTPException(status_code=404, detail="agent not found")
return {"ok": True}
# ---------- 日志审计 ----------
@router.get("/logs", response_model=list[LogEntry], summary="日志审计")
async def logs(
limit: int = 200,
request_id: Optional[str] = None,
agent_id: Optional[str] = None,
repo: LogRepo = Depends(get_log_repo),
):
return await repo.list(limit=limit, request_id=request_id, agent_id=agent_id)
# ---------- 概览 ----------
@router.get("/overview", summary="概览统计")
async def overview(
task_svc: TaskService = Depends(get_task_service),
agent_svc: AgentService = Depends(get_agent_service),
):
tasks = await task_svc.list(limit=1000)
agents = await agent_svc.all()
from collections import Counter
status_count = Counter(t.status.value for t in tasks)
return {
"task_total": len(tasks),
"task_pending": status_count.get(TaskStatus.PENDING.value, 0),
"task_running": status_count.get(TaskStatus.RUNNING.value, 0),
"task_success": status_count.get(TaskStatus.SUCCESS.value, 0),
"task_failed": status_count.get(TaskStatus.FAILED.value, 0),
"agent_total": len(agents),
"agent_online": sum(1 for a in agents if a.status.value == "online"),
"agent_offline": sum(1 for a in agents if a.status.value == "offline"),
}

53
backend/app/api/agent.py Normal file
View File

@ -0,0 +1,53 @@
"""预留 Agent 协议接口register / unregister / heartbeat / result。"""
from fastapi import APIRouter, Depends, Request
from app.models.schemas import AgentHeartbeat, AgentInfo, AgentRegister, TaskResult
from app.services.agent_service import AgentService
from app.services.relay import RelayService
from app.services.security import require_auth
router = APIRouter()
def get_agent_service(request: Request) -> AgentService:
return AgentService(request.app.state.redis)
def get_relay_service(request: Request) -> RelayService:
return RelayService(request.app.state.redis)
@router.post("/register", response_model=AgentInfo, summary="Agent 注册")
async def register(body: AgentRegister, svc: AgentService = Depends(get_agent_service)):
require_auth(body.auth)
return await svc.register(body)
@router.post("/unregister", summary="Agent 注销")
async def unregister(body: dict, svc: AgentService = Depends(get_agent_service)):
from fastapi import HTTPException
agent_id = body.get("agent_id")
if not agent_id:
raise HTTPException(status_code=400, detail="agent_id required")
if not await svc.unregister(agent_id):
raise HTTPException(status_code=404, detail="agent not found")
return {"ok": True}
@router.post("/heartbeat", summary="Agent 心跳")
async def heartbeat(body: AgentHeartbeat, svc: AgentService = Depends(get_agent_service)):
from fastapi import HTTPException
if not await svc.heartbeat(body):
raise HTTPException(status_code=404, detail="agent not registered")
return {"ok": True}
@router.post("/result", summary="Agent 结果回传")
async def result(body: TaskResult, relay: RelayService = Depends(get_relay_service)):
from fastapi import HTTPException
if not await relay.on_result(body):
raise HTTPException(status_code=400, detail="result rejected")
return {"ok": True}

88
backend/app/api/cli.py Normal file
View File

@ -0,0 +1,88 @@
"""CLI 接口:任务提交/查询/取消/完成事件订阅。"""
from fastapi import APIRouter, Depends, Request
from fastapi.responses import StreamingResponse
from app.models.schemas import TaskInfo, TaskSubmit
from app.services.notifier import TaskNotifier
from app.services.security import require_auth
from app.services.task_service import TaskService
router = APIRouter()
def get_task_service(request: Request) -> TaskService:
return TaskService(request.app.state.redis)
def get_notifier(request: Request) -> TaskNotifier:
return TaskNotifier(request.app.state.redis)
@router.get("/events", summary="订阅任务完成事件SSE 长连接)")
async def subscribe_events(
request: Request,
cli_session_id: str,
auth: str = "",
notifier: TaskNotifier = Depends(get_notifier),
):
"""任务到达终态时网关主动推送完成事件,无需轮询。
返回 SSE text/event-stream每帧形如
data: {"event": "task_done", "request_id": "...", "status": "success", "result": {...}}
"""
require_auth(auth)
channel = notifier.channel(cli_session_id)
pubsub = request.app.state.redis.pubsub()
await pubsub.subscribe(channel)
replay = await notifier.replay(cli_session_id)
async def event_gen():
try:
# 先回放已完成的(连接晚于任务完成的场景)
for evt in replay:
yield f"data: {evt}\n\n"
# 再实时监听
async for message in pubsub.listen():
if await request.is_disconnected():
break
if message["type"] != "message":
continue
data = message["data"]
if isinstance(data, bytes):
data = data.decode()
yield f"data: {data}\n\n"
finally:
await pubsub.unsubscribe(channel)
await pubsub.aclose()
return StreamingResponse(
event_gen(),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
@router.post("/tasks", response_model=TaskInfo, summary="提交任务")
async def submit_task(body: TaskSubmit, svc: TaskService = Depends(get_task_service)):
require_auth(body.auth)
return await svc.submit(body)
@router.get("/tasks/{request_id}", response_model=TaskInfo, summary="查询任务")
async def get_task(request_id: str, svc: TaskService = Depends(get_task_service)):
from fastapi import HTTPException
task = await svc.get(request_id)
if not task:
raise HTTPException(status_code=404, detail="task not found")
return task
@router.post("/tasks/{request_id}/cancel", summary="取消任务")
async def cancel_task(request_id: str, svc: TaskService = Depends(get_task_service)):
from fastapi import HTTPException
if not await svc.cancel(request_id):
raise HTTPException(status_code=400, detail="cannot cancel task")
return {"ok": True, "request_id": request_id}

52
backend/app/config.py Normal file
View File

@ -0,0 +1,52 @@
"""应用配置:通过环境变量 / .env 读取。"""
from pathlib import Path
from pydantic_settings import BaseSettings, SettingsConfigDict
# 项目根目录backend/ 的上一级)
PROJECT_ROOT = Path(__file__).resolve().parents[2]
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_file=str(PROJECT_ROOT / ".env"),
env_file_encoding="utf-8",
extra="ignore",
)
# Redis
redis_url: str = "redis://localhost:6379/0"
# 网关服务
host: str = "0.0.0.0"
port: int = 8000
# 心跳保活参数(秒)
heartbeat_scan_interval: int = 60
agent_heartbeat_timeout: int = 120
agent_heartbeat_expected: int = 10
# 调度参数(秒)
dispatch_interval: int = 2
task_timeout_check_interval: int = 5
# 任务默认超时(秒)
default_task_timeout: int = 3600
# 任务数据保留 TTL
task_ttl: int = 86400
# 日志级别
log_level: str = "INFO"
# 认证CLI 提交任务 / Agent 注册时必须携带的密码
gateway_auth: str = "dev-gateway-auth"
# 分布式锁相关
lock_timeout: int = 30
# 最大并发调度轮次
dispatch_batch_size: int = 20
settings = Settings()

38
backend/app/constants.py Normal file
View File

@ -0,0 +1,38 @@
"""Redis 键名与 TTL 常量、状态枚举。"""
from enum import Enum
class TaskStatus(str, Enum):
PENDING = "pending"
RUNNING = "running"
SUCCESS = "success"
FAILED = "failed"
class AgentStatus(str, Enum):
ONLINE = "online"
OFFLINE = "offline"
# ---- 任务池键 ----
# 任务全量信息hash task:info:{request_id}
TASK_INFO_KEY = "task:info:{request_id}"
# 待调度任务集合
TASK_PENDING_SET = "task:pending"
# 执行中任务集合
TASK_RUNNING_SET = "task:running"
# ---- Agent 池键 ----
# Agent 全量信息hash agent:info:{agent_id}
AGENT_INFO_KEY = "agent:info:{agent_id}"
# 心跳时间戳 ZSetscore = 最后心跳时间
AGENT_HEARTBEAT_ZSET = "agent:heartbeat"
# 按能力标签索引 Agentset agent:tag:{tag}
AGENT_TAG_KEY = "agent:tag:{tag}"
# 全部在线 Agent 索引
AGENT_ALL_SET = "agent:all"
# ---- 分布式锁键 ----
LOCK_SCHEDULER = "lock:scheduler"
LOCK_HEARTBEAT = "lock:heartbeat"
LOCK_TIMEOUT = "lock:timeout"

62
backend/app/main.py Normal file
View File

@ -0,0 +1,62 @@
"""FastAPI 应用入口注册路由、CORS、启动后台任务循环。"""
import logging
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app.config import settings
from app.api import admin, agent, cli
from app.middleware import RequestLogMiddleware
from app.scheduler_loop import SchedulerLoop
from app.repository.redis_client import get_redis_pool
logging.basicConfig(
level=getattr(logging, settings.log_level.upper(), logging.INFO),
format="%(asctime)s %(levelname)s [%(name)s] %(message)s",
)
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
# 启动时会话
redis = await get_redis_pool()
loop = SchedulerLoop(redis)
app.state.redis = redis
app.state.scheduler_loop = loop
await loop.start()
logger.info("Gateway started, host=%s port=%s", settings.host, settings.port)
try:
yield
finally:
await loop.stop()
await redis.aclose()
logger.info("Gateway stopped")
app = FastAPI(
title="A2A Gateway",
description="A2A 智能网关:任务池 / Agent 池 / 规则调度 / 通信中转 / 可视化运维",
version="1.0.0",
lifespan=lifespan,
)
# CORS允许前端开发服务器访问
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.add_middleware(RequestLogMiddleware)
app.include_router(cli.router, prefix="/api/cli", tags=["CLI"])
app.include_router(agent.router, prefix="/api/agent", tags=["Agent"])
app.include_router(admin.router, prefix="/api/admin", tags=["Admin"])
@app.get("/health", tags=["health"])
async def health():
return {"status": "ok"}

33
backend/app/middleware.py Normal file
View File

@ -0,0 +1,33 @@
"""日志中间件:记录 RequestID/AgentID 链路与访问日志,避免敏感载荷与日志刷屏。"""
import logging
import time
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
logger = logging.getLogger("access")
class RequestLogMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
start = time.perf_counter()
# 心跳类接口降级为 debug避免刷屏
is_heartbeat = request.url.path.endswith("/heartbeat")
if logger.isEnabledFor(logging.DEBUG) or not is_heartbeat:
logger.info(
"%s %s path=%s",
request.method,
request.client.host if request.client else "-",
request.url.path,
)
response = await call_next(request)
cost_ms = (time.perf_counter() - start) * 1000
if not is_heartbeat or logger.isEnabledFor(logging.DEBUG):
logger.info(
"%s %s -> %s (%dms)",
request.method,
request.url.path,
response.status_code,
round(cost_ms, 2),
)
return response

View File

View File

@ -0,0 +1,92 @@
"""Pydantic 数据模型。"""
import time
import uuid
from typing import Any
from pydantic import BaseModel, Field
from app.constants import AgentStatus, TaskStatus
def new_request_id() -> str:
return uuid.uuid4().hex
# ---------- 任务 ----------
class TaskSubmit(BaseModel):
"""CLI 任务上报请求体。"""
auth: str = Field(..., description="网关认证密码,需与 GATEWAY_AUTH 一致")
request_id: str | None = Field(default=None, description="幂等键,不传则自动生成")
task_type: str = Field(..., description="任务类型,如 compile/build/test")
task_tags: list[str] = Field(default_factory=list, description="任务能力标签")
description: str | None = Field(default=None, description="任务描述")
payload: dict[str, Any] = Field(default_factory=dict, description="任务载荷")
cli_session_id: str | None = Field(default=None, description="CLI 会话标识")
timeout: int = Field(default=0, description="超时秒数0 使用默认值")
local_workdir: str | None = Field(default=None, description="本地工作目录信息")
class TaskInfo(BaseModel):
request_id: str
task_type: str
task_tags: list[str] = Field(default_factory=list)
description: str | None = None
payload: dict[str, Any] = Field(default_factory=dict)
status: TaskStatus = TaskStatus.PENDING
agent_id: str | None = None
cli_session_id: str | None = None
create_time: float = Field(default_factory=time.time)
timeout: int = 0
progress: int = 0
result: dict[str, Any] | None = None
error_info: str | None = None
# ---------- Agent ----------
class AgentRegister(BaseModel):
"""Agent 启动注册请求体。"""
auth: str = Field(..., description="网关认证密码,需与 GATEWAY_AUTH 一致")
agent_id: str
endpoint: str
agent_tags: list[str] = Field(default_factory=list)
max_concurrent: int = Field(default=1, ge=1)
current_load: int = Field(default=0, ge=0)
class AgentHeartbeat(BaseModel):
agent_id: str
current_load: int = Field(default=0, ge=0)
class AgentInfo(BaseModel):
agent_id: str
endpoint: str
agent_tags: list[str] = Field(default_factory=list)
max_concurrent: int = 1
current_load: int = 0
last_heartbeat: float = Field(default_factory=time.time)
status: AgentStatus = AgentStatus.ONLINE
create_time: float = Field(default_factory=time.time)
# ---------- 结果回传 ----------
class TaskResult(BaseModel):
request_id: str
agent_id: str
status: TaskStatus = TaskStatus.SUCCESS
progress: int = Field(default=100, ge=0, le=100)
result: dict[str, Any] | None = None
error_info: str | None = None
# ---------- 日志 ----------
class LogEntry(BaseModel):
ts: float = Field(default_factory=time.time)
level: str = "info"
source: str = "gateway" # gateway / agent / cli / admin
scope: str = "task" # task / agent / system
request_id: str | None = None
agent_id: str | None = None
message: str

View File

View File

@ -0,0 +1,146 @@
"""Agent 池 Redis 存储层。"""
import time
from typing import Any
import redis.asyncio as aioredis
from app import constants as C
from app.constants import AgentStatus
from app.models.schemas import AgentInfo
class AgentRepo:
def __init__(self, redis: aioredis.Redis):
self.redis = redis
@staticmethod
def _info_key(agent_id: str) -> str:
return C.AGENT_INFO_KEY.format(agent_id=agent_id)
@staticmethod
def _tag_key(tag: str) -> str:
return C.AGENT_TAG_KEY.format(tag=tag)
# ---------- 写入 ----------
async def upsert(self, agent: AgentInfo, *, heartbeat: bool = False) -> bool:
"""写入 Agent 信息,返回是否新建。非心跳时为全量注册。"""
key = self._info_key(agent.agent_id)
existed = await self.redis.exists(key)
if not existed:
await self.redis.sadd(C.AGENT_ALL_SET, agent.agent_id)
# 建立标签索引
for tag in agent.agent_tags:
await self.redis.sadd(self._tag_key(tag), agent.agent_id)
mapping: dict[str, Any] = {
"agent_id": agent.agent_id,
"endpoint": agent.endpoint,
"agent_tags": ",".join(agent.agent_tags),
"max_concurrent": str(agent.max_concurrent),
"current_load": str(agent.current_load),
"last_heartbeat": str(agent.last_heartbeat),
"status": agent.status.value,
"create_time": str(agent.create_time),
}
await self.redis.hset(key, mapping=mapping)
# 心跳 ZSet 记录时间戳
await self.redis.zadd(C.AGENT_HEARTBEAT_ZSET, {agent.agent_id: agent.last_heartbeat})
return not existed
async def update_status(self, agent_id: str, status: AgentStatus) -> bool:
key = self._info_key(agent_id)
if not await self.redis.exists(key):
return False
await self.redis.hset(key, "status", status.value)
if status == AgentStatus.OFFLINE:
await self.redis.zrem(C.AGENT_HEARTBEAT_ZSET, agent_id)
return True
async def beat(self, agent_id: str, current_load: int) -> bool:
"""更新心跳时间与负载。"""
key = self._info_key(agent_id)
if not await self.redis.exists(key):
return False
now = time.time()
await self.redis.hset(
key,
mapping={"last_heartbeat": str(now), "current_load": str(current_load), "status": AgentStatus.ONLINE.value},
)
await self.redis.zadd(C.AGENT_HEARTBEAT_ZSET, {agent_id: now})
return True
async def adjust_load(self, agent_id: str, delta: int) -> bool:
"""原子调整 Agent 负载delta 可为正/负),负载下限为 0。"""
key = self._info_key(agent_id)
if not await self.redis.exists(key):
return False
# hincrby 为原子增量操作,兼容 fakeredis不支持 Lua eval
new = await self.redis.hincrby(key, "current_load", delta)
if new < 0:
await self.redis.hset(key, "current_load", "0")
return True
# ---------- 读取 ----------
async def get(self, agent_id: str) -> AgentInfo | None:
raw = await self.redis.hgetall(self._info_key(agent_id))
if not raw:
return None
return AgentInfo(
agent_id=raw.get("agent_id", ""),
endpoint=raw.get("endpoint", ""),
agent_tags=[t for t in raw.get("agent_tags", "").split(",") if t],
max_concurrent=int(raw.get("max_concurrent", 1) or 1),
current_load=int(raw.get("current_load", 0) or 0),
last_heartbeat=float(raw.get("last_heartbeat", 0) or 0),
status=AgentStatus(raw.get("status", AgentStatus.ONLINE.value)),
create_time=float(raw.get("create_time", 0) or 0),
)
async def online(self) -> list[AgentInfo]:
return [a for a in await self.all() if a.status == AgentStatus.ONLINE]
async def all(self) -> list[AgentInfo]:
ids = list(await self.redis.smembers(C.AGENT_ALL_SET))
out = []
for aid in ids:
a = await self.get(aid)
if a:
out.append(a)
return out
async def by_tags(self, tags: list[str]) -> list[AgentInfo]:
"""按标签索引取 Agent 池(取所有标签的交集,无标签则返回全部在线)。"""
if not tags:
return await self.online()
keys = [self._tag_key(t) for t in tags]
if len(keys) == 1:
ids = list(await self.redis.smembers(keys[0]))
else:
await self.redis.sinterstore("agent:tmp:intersect", keys)
ids = list(await self.redis.smembers("agent:tmp:intersect"))
await self.redis.delete("agent:tmp:intersect")
out = []
for aid in ids:
a = await self.get(aid)
if a and a.status == AgentStatus.ONLINE:
out.append(a)
return out
async def remove(self, agent_id: str) -> None:
"""注销:删除信息、心跳 ZSet、标签索引、全量索引。"""
a = await self.get(agent_id)
if a:
for tag in a.agent_tags:
await self.redis.srem(self._tag_key(tag), agent_id)
await self.redis.delete(self._info_key(agent_id))
await self.redis.zrem(C.AGENT_HEARTBEAT_ZSET, agent_id)
await self.redis.srem(C.AGENT_ALL_SET, agent_id)
async def stale_agents(self, timeout: float) -> list[str]:
"""返回超过 timeout 秒未心跳的 Agent ID基于 ZSet score"""
cutoff = time.time() - timeout
scored = await self.redis.zrangebyscore(C.AGENT_HEARTBEAT_ZSET, 0, cutoff)
return list(scored)
async def heartbeat_map(self) -> dict[str, float]:
"""返回 {agent_id: last_heartbeat}。"""
return {k: float(v) for k, v in await self.redis.zrange(C.AGENT_HEARTBEAT_ZSET, 0, -1, withscores=True)}

View File

@ -0,0 +1,50 @@
"""日志审计存储层:全链路操作/注册注销/通信记录。"""
import time
import redis.asyncio as aioredis
from app.models.schemas import LogEntry
LOG_KEY = "log:audit"
LOG_MAX = 2000
class LogRepo:
def __init__(self, redis: aioredis.Redis):
self.redis = redis
async def append(
self,
message: str,
*,
level: str = "info",
source: str = "gateway",
scope: str = "system",
request_id: str | None = None,
agent_id: str | None = None,
) -> None:
entry = LogEntry(
level=level,
source=source,
scope=scope,
request_id=request_id,
agent_id=agent_id,
message=message,
)
await self.redis.lpush(LOG_KEY, entry.model_dump_json())
await self.redis.ltrim(LOG_KEY, 0, LOG_MAX - 1)
async def list(self, limit: int = 200, request_id: str | None = None, agent_id: str | None = None) -> list[LogEntry]:
raw = await self.redis.lrange(LOG_KEY, 0, limit - 1)
out = []
for item in raw:
try:
entry = LogEntry.model_validate_json(item)
except Exception:
continue
if request_id and entry.request_id != request_id:
continue
if agent_id and entry.agent_id != agent_id:
continue
out.append(entry)
return out

View File

@ -0,0 +1,14 @@
"""Redis 连接池与共享访问封装。"""
import redis.asyncio as aioredis
from app.config import settings
_pool: aioredis.Redis | None = None
async def get_redis_pool() -> aioredis.Redis:
"""返回全局共享的 Redis 客户端(懒初始化)。"""
global _pool
if _pool is None:
_pool = aioredis.from_url(settings.redis_url, decode_responses=True)
return _pool

View File

@ -0,0 +1,166 @@
"""任务池 Redis 存储层。"""
import time
from typing import Any
import redis.asyncio as aioredis
from app import constants as C
from app.config import settings
from app.constants import TaskStatus
from app.models.schemas import TaskInfo
class TaskRepo:
def __init__(self, redis: aioredis.Redis):
self.redis = redis
@staticmethod
def _info_key(request_id: str) -> str:
return C.TASK_INFO_KEY.format(request_id=request_id)
# ---------- 写入 ----------
async def create(self, task: TaskInfo) -> bool:
"""录入任务(幂等),返回是否新建。"""
key = self._info_key(task.request_id)
existed = await self.redis.exists(key)
if existed:
return False
await self.redis.hset(
key,
mapping={
"request_id": task.request_id,
"task_type": task.task_type,
"task_tags": ",".join(task.task_tags),
"description": task.description or "",
"payload": _json(task.payload),
"status": task.status.value,
"agent_id": task.agent_id or "",
"cli_session_id": task.cli_session_id or "",
"create_time": str(task.create_time),
"timeout": str(task.timeout),
"progress": str(task.progress),
"result": _json(task.result),
"error_info": task.error_info or "",
},
)
await self.redis.expire(key, settings.task_ttl)
await self.redis.sadd(C.TASK_PENDING_SET, task.request_id)
return True
async def update(
self,
request_id: str,
*,
status: TaskStatus | None = None,
agent_id: str | None = None,
progress: int | None = None,
result: dict | None = None,
error_info: str | None = None,
) -> bool:
key = self._info_key(request_id)
if not await self.redis.exists(key):
return False
mapping: dict[str, Any] = {}
if status is not None:
mapping["status"] = status.value
if agent_id is not None:
mapping["agent_id"] = agent_id
if progress is not None:
mapping["progress"] = str(progress)
if result is not None:
mapping["result"] = _json(result)
if error_info is not None:
mapping["error_info"] = error_info
if mapping:
await self.redis.hset(key, mapping=mapping)
return True
# ---------- 状态集合维护 ----------
async def mark_pending(self, request_id: str) -> None:
await self.redis.sadd(C.TASK_PENDING_SET, request_id)
await self.redis.srem(C.TASK_RUNNING_SET, request_id)
async def mark_running(self, request_id: str) -> None:
await self.redis.sadd(C.TASK_RUNNING_SET, request_id)
await self.redis.srem(C.TASK_PENDING_SET, request_id)
async def mark_finished(self, request_id: str) -> None:
await self.redis.srem(C.TASK_PENDING_SET, request_id)
await self.redis.srem(C.TASK_RUNNING_SET, request_id)
async def pending_tasks(self) -> list[str]:
return list(await self.redis.smembers(C.TASK_PENDING_SET))
async def running_tasks(self) -> list[str]:
return list(await self.redis.smembers(C.TASK_RUNNING_SET))
async def remove_pending(self, request_id: str) -> None:
await self.redis.srem(C.TASK_PENDING_SET, request_id)
# ---------- 读取 ----------
async def get(self, request_id: str) -> TaskInfo | None:
raw = await self.redis.hgetall(self._info_key(request_id))
if not raw:
return None
return await self._to_task(raw)
async def _to_task(self, raw: dict) -> TaskInfo:
return TaskInfo(
request_id=raw.get("request_id", ""),
task_type=raw.get("task_type", ""),
task_tags=[t for t in raw.get("task_tags", "").split(",") if t],
description=raw.get("description") or None,
payload=_unjson(raw.get("payload")),
status=TaskStatus(raw.get("status", TaskStatus.PENDING.value)),
agent_id=raw.get("agent_id") or None,
cli_session_id=raw.get("cli_session_id") or None,
create_time=float(raw.get("create_time", 0) or 0),
timeout=int(raw.get("timeout", 0) or 0),
progress=int(raw.get("progress", 0) or 0),
result=_unjson(raw.get("result")),
error_info=raw.get("error_info") or None,
)
async def list(self, status: TaskStatus | None = None, limit: int = 100) -> list[TaskInfo]:
"""按状态过滤返回任务列表(含终态任务,通过 scan 全量扫描 task:info:*)。"""
ids = await self.scan_ids()
out = []
for rid in ids:
t = await self.get(rid)
if not t:
continue
if status is not None and t.status != status:
continue
out.append(t)
# 按创建时间倒序
out.sort(key=lambda x: x.create_time, reverse=True)
return out[:limit]
async def scan_ids(self) -> list[str]:
"""扫描所有 task:info:* 键,返回 RequestID 列表。"""
prefix = C.TASK_INFO_KEY.replace("{request_id}", "")
ids = []
async for key in self.redis.scan_iter(match=f"{prefix}*", count=1000):
ids.append(key[len(prefix):])
return ids
async def delete(self, request_id: str) -> None:
await self.redis.delete(self._info_key(request_id))
await self.mark_finished(request_id)
def _json(obj: Any) -> str:
import json
return json.dumps(obj, ensure_ascii=False) if obj is not None else ""
def _unjson(s: str | None) -> Any:
import json
if not s:
return None
try:
return json.loads(s)
except json.JSONDecodeError:
return None

View File

@ -0,0 +1,70 @@
"""后台任务循环启动器:心跳扫描 / 任务调度 / 任务超时,带分布式锁防多实例重复执行。"""
import asyncio
import logging
import redis.asyncio as aioredis
from app.config import settings
from app.constants import LOCK_HEARTBEAT, LOCK_SCHEDULER, LOCK_TIMEOUT
from app.services.heartbeat import HeartbeatService
from app.services.lock import Lock
from app.services.relay import RelayService
from app.services.scheduler import Scheduler
from app.services.timeout import TimeoutService
logger = logging.getLogger(__name__)
class SchedulerLoop:
def __init__(self, redis: aioredis.Redis):
self.redis = redis
self._tasks: list[asyncio.Task] = []
async def start(self) -> None:
self._tasks.append(asyncio.create_task(self._heartbeat_loop()))
self._tasks.append(asyncio.create_task(self._dispatch_loop()))
self._tasks.append(asyncio.create_task(self._timeout_loop()))
logger.info("scheduler loops started")
async def stop(self) -> None:
for t in self._tasks:
t.cancel()
for t in self._tasks:
try:
await t
except asyncio.CancelledError:
pass
self._tasks.clear()
async def _heartbeat_loop(self) -> None:
hb = HeartbeatService(self.redis)
while True:
await asyncio.sleep(settings.heartbeat_scan_interval)
lock = Lock(self.redis, LOCK_HEARTBEAT)
if await lock.acquire():
try:
await hb.scan()
finally:
await lock.release()
async def _dispatch_loop(self) -> None:
sched = Scheduler(self.redis, relay=RelayService(self.redis))
while True:
await asyncio.sleep(settings.dispatch_interval)
lock = Lock(self.redis, LOCK_SCHEDULER)
if await lock.acquire():
try:
await sched.dispatch_pending(settings.dispatch_batch_size)
finally:
await lock.release()
async def _timeout_loop(self) -> None:
to = TimeoutService(self.redis)
while True:
await asyncio.sleep(settings.task_timeout_check_interval)
lock = Lock(self.redis, LOCK_TIMEOUT)
if await lock.acquire():
try:
await to.check()
finally:
await lock.release()

View File

View File

@ -0,0 +1,72 @@
"""Agent 池服务:注册/注销/心跳/负载/离线剔除。"""
import logging
import time
import redis.asyncio as aioredis
from app.constants import AgentStatus
from app.models.schemas import AgentHeartbeat, AgentInfo, AgentRegister
from app.repository.agent_repo import AgentRepo
logger = logging.getLogger(__name__)
class AgentService:
def __init__(self, redis: aioredis.Redis, repo: AgentRepo | None = None):
self.redis = redis
self.repo = repo or AgentRepo(redis)
async def register(self, reg: AgentRegister) -> AgentInfo:
agent = AgentInfo(
agent_id=reg.agent_id,
endpoint=reg.endpoint,
agent_tags=reg.agent_tags,
max_concurrent=reg.max_concurrent,
current_load=reg.current_load,
status=AgentStatus.ONLINE,
create_time=time.time(),
last_heartbeat=time.time(),
)
created = await self.repo.upsert(agent)
logger.info("agent registered agent_id=%s created=%s", agent.agent_id, created)
return agent
async def unregister(self, agent_id: str) -> bool:
existed = await self.repo.get(agent_id)
if not existed:
return False
await self.repo.remove(agent_id)
logger.info("agent unregistered agent_id=%s", agent_id)
return True
async def heartbeat(self, hb: AgentHeartbeat) -> bool:
ok = await self.repo.beat(hb.agent_id, hb.current_load)
if not ok:
logger.warning("heartbeat from unknown agent agent_id=%s", hb.agent_id)
return ok
async def online(self) -> list[AgentInfo]:
return await self.repo.online()
async def all(self) -> list[AgentInfo]:
return await self.repo.all()
async def get(self, agent_id: str) -> AgentInfo | None:
return await self.repo.get(agent_id)
async def take_offline(self, agent_id: str) -> bool:
"""手动下线 Agent。"""
a = await self.repo.get(agent_id)
if not a:
return False
await self.repo.update_status(agent_id, AgentStatus.OFFLINE)
logger.info("agent taken offline agent_id=%s", agent_id)
return True
async def purge_stale(self, timeout: float) -> list[str]:
"""剔除超时未心跳的 Agent。"""
stale = await self.repo.stale_agents(timeout)
for aid in stale:
await self.repo.update_status(aid, AgentStatus.OFFLINE)
logger.info("agent purged (stale) agent_id=%s", aid)
return stale

View File

@ -0,0 +1,24 @@
"""心跳保活服务:扫描 Agent 池,剔除超时未心跳节点。"""
import logging
import redis.asyncio as aioredis
from app.config import settings
from app.constants import AgentStatus
from app.repository.agent_repo import AgentRepo
logger = logging.getLogger(__name__)
class HeartbeatService:
def __init__(self, redis: aioredis.Redis, agent_repo: AgentRepo | None = None):
self.redis = redis
self.agent_repo = agent_repo or AgentRepo(redis)
async def scan(self) -> int:
"""扫描并剔除超时 Agent返回剔除数量。"""
stale = await self.agent_repo.stale_agents(settings.agent_heartbeat_timeout)
for aid in stale:
await self.agent_repo.update_status(aid, AgentStatus.OFFLINE)
logger.info("heartbeat: agent offline (stale) agent_id=%s", aid)
return len(stale)

View File

@ -0,0 +1,35 @@
"""Redis 分布式锁SET NX EX"""
import secrets
import time
import redis.asyncio as aioredis
from app.config import settings
class Lock:
def __init__(self, redis: aioredis.Redis, name: str, timeout: int = settings.lock_timeout):
self.redis = redis
self.name = name
self.timeout = timeout
self._token = secrets.token_hex(8)
async def acquire(self) -> bool:
ok = await self.redis.set(self.name, self._token, nx=True, ex=self.timeout)
return bool(ok)
async def release(self) -> None:
# 仅在持有相同 token 时释放,避免误删他人锁
val = await self.redis.get(self.name)
if val == self._token:
await self.redis.delete(self.name)
async def guarded(self, fn):
"""上下文封装:获取锁执行,释放锁。"""
if not await self.acquire():
return False
try:
await fn()
return True
finally:
await self.release()

View File

@ -0,0 +1,78 @@
"""任务完成事件通知服务。
设计CLI 提交任务时可携带 cli_session_id并通过 SSE 长连接订阅
GET /api/cli/events?cli_session_id=xxx 事件流任务到达终态
success / failed含取消超时网关通过 Redis Pub/Sub
对应会话推送事件避免 CLI 轮询
为保证订阅晚于任务完成也不丢事件publish 时同时写入一份
回放缓存Redis List保留最近 N SSE 连接建立后先回放再实时推送
"""
import json
import logging
from typing import Any
import redis.asyncio as aioredis
from app.models.schemas import TaskInfo
logger = logging.getLogger(__name__)
CHANNEL_PREFIX = "task:event:cli:" # Redis Pub/Sub channel
REPLAY_KEY_PREFIX = "task:event:replay:" # 回放缓存 List key
REPLAY_TTL = 600
REPLAY_MAX = 200
class TaskNotifier:
def __init__(self, redis: aioredis.Redis):
self.redis = redis
@staticmethod
def channel(cli_session_id: str) -> str:
return f"{CHANNEL_PREFIX}{cli_session_id}"
@staticmethod
def _replay_key(cli_session_id: str) -> str:
return f"{REPLAY_KEY_PREFIX}{cli_session_id}"
# ---------- 发布 ----------
async def publish_task_done(self, task: TaskInfo) -> None:
"""任务到达终态时调用:写回放缓存并广播到该 CLI 会话。"""
if not task.cli_session_id:
return
payload = json.dumps(
{
"event": "task_done",
"request_id": task.request_id,
"status": task.status.value,
"result": task.result,
"error_info": task.error_info,
},
ensure_ascii=False,
)
key = self._replay_key(task.cli_session_id)
channel = self.channel(task.cli_session_id)
await self.redis.lpush(key, payload)
await self.redis.ltrim(key, 0, REPLAY_MAX - 1)
await self.redis.expire(key, REPLAY_TTL)
await self.redis.publish(channel, payload)
logger.info("notify task done session=%s request=%s status=%s",
task.cli_session_id, task.request_id, task.status.value)
# ---------- 消费 ----------
async def replay(self, cli_session_id: str) -> list[str]:
"""返回该会话的历史完成事件(新 → 旧),供 SSE 连接回放。"""
items = await self.redis.lrange(self._replay_key(cli_session_id), 0, -1)
return [s.decode() if isinstance(s, bytes) else s for s in items]
async def drain_replay(self, cli_session_id: str) -> None:
"""SSE 连接回放结束后清空回放缓存(已消费)。"""
await self.redis.delete(self._replay_key(cli_session_id))
def format_sse(data: str | dict[str, Any]) -> str:
"""格式化为 SSE 帧。"""
if isinstance(data, dict):
data = json.dumps(data, ensure_ascii=False)
return f"data: {data}\n\n"

View File

@ -0,0 +1,101 @@
"""通信中转服务:正向下发任务指令、反向回传结果/日志RequestID+AgentID 关联。"""
import logging
import httpx
import redis.asyncio as aioredis
from app.config import settings
from app.constants import TaskStatus
from app.models.schemas import TaskResult
from app.repository.agent_repo import AgentRepo
from app.repository.task_repo import TaskRepo
from app.services.notifier import TaskNotifier
logger = logging.getLogger(__name__)
class RelayService:
"""网关作为唯一通信中枢:正向把任务指令推送到 Agent 端点,反向接收结果回传。"""
def __init__(self, redis: aioredis.Redis, task_repo: TaskRepo | None = None):
self.redis = redis
self.task_repo = task_repo or TaskRepo(redis)
self.agent_repo = AgentRepo(redis)
async def dispatch_command(self, agent_id: str, request_id: str, payload: dict) -> bool:
"""正向:向 Agent 真实 HTTP 推送任务指令POST {agent.endpoint}/tasks/{request_id})。
成功返回 TrueAgent 不存在 / 端点缺失 / 网络失败 / 202 均返回 False
由调度器回退任务状态并释放负载
"""
agent = await self.agent_repo.get(agent_id)
if not agent:
logger.warning("relay dispatch failed: agent not found agent=%s request=%s", agent_id, request_id)
return False
if not agent.endpoint:
logger.warning("relay dispatch failed: endpoint empty agent=%s request=%s", agent_id, request_id)
return False
url = f"{agent.endpoint.rstrip('/')}/tasks/{request_id}"
body = {
"auth": settings.gateway_auth,
"request_id": request_id,
"payload": payload,
}
try:
async with httpx.AsyncClient(timeout=5.0) as client:
resp = await client.post(url, json=body)
except httpx.HTTPError as e:
logger.error("relay dispatch network error agent=%s request=%s err=%s", agent_id, request_id, e)
await self._log(agent_id, request_id, "dispatch", f"push failed: {e}")
return False
if resp.status_code != 202:
logger.warning("relay dispatch http %s agent=%s request=%s body=%s",
resp.status_code, agent_id, request_id, resp.text[:200])
await self._log(agent_id, request_id, "dispatch", f"push failed http {resp.status_code}")
return False
await self._log(agent_id, request_id, "dispatch", f"command pushed to {agent.endpoint}")
logger.info("relay dispatch ok agent=%s request=%s url=%s", agent_id, request_id, url)
return True
async def on_result(self, result: TaskResult) -> bool:
"""反向Agent 回传结果,更新任务状态并记录日志。"""
task = await self.task_repo.get(result.request_id)
if not task:
logger.warning("result for unknown task request=%s", result.request_id)
return False
if task.agent_id and task.agent_id != result.agent_id:
logger.warning("result agent mismatch request=%s expected=%s got=%s",
result.request_id, task.agent_id, result.agent_id)
return False
await self.task_repo.update(
result.request_id,
status=result.status,
progress=result.progress,
result=result.result,
error_info=result.error_info,
)
await self.task_repo.mark_finished(result.request_id)
# 释放 Agent 算力(任务确实绑定在该 Agent 时才释放)
if task.agent_id == result.agent_id:
await self.agent_repo.adjust_load(result.agent_id, -1)
await self._log(result.agent_id, result.request_id, "result", f"result received status={result.status.value}")
# 任务到达终态,通知对应 CLI 会话(若携带了 cli_session_id
if task.cli_session_id:
finished = await self.task_repo.get(result.request_id)
if finished:
await TaskNotifier(self.redis).publish_task_done(finished)
logger.info("task result request=%s agent=%s status=%s", result.request_id, result.agent_id, result.status.value)
return True
async def on_progress(self, request_id: str, agent_id: str, progress: int) -> bool:
await self.task_repo.update(request_id, progress=progress)
await self._log(agent_id, request_id, "progress", f"progress {progress}%")
return True
async def _log(self, agent_id: str, request_id: str, action: str, message: str) -> None:
from app.repository.log_repo import LogRepo
await LogRepo(self.redis).append(
source="gateway", scope="task", message=message,
request_id=request_id, agent_id=agent_id,
)

View File

@ -0,0 +1,76 @@
"""规则调度服务:标签匹配→负载过滤→最优选择→任务绑定→推送下发。"""
import logging
import redis.asyncio as aioredis
from app.constants import TaskStatus
from app.models.schemas import AgentInfo
from app.repository.agent_repo import AgentRepo
from app.repository.task_repo import TaskRepo
from app.services.relay import RelayService
logger = logging.getLogger(__name__)
class Scheduler:
def __init__(
self,
redis: aioredis.Redis,
task_repo: TaskRepo | None = None,
agent_repo: AgentRepo | None = None,
relay: RelayService | None = None,
):
self.redis = redis
self.task_repo = task_repo or TaskRepo(redis)
self.agent_repo = agent_repo or AgentRepo(redis)
self.relay = relay or RelayService(redis)
async def pick_candidate(self, task_tags: list[str]) -> AgentInfo | None:
"""按规则挑选最优 Agent标签匹配→负载过滤→最低负载。"""
candidates = await self.agent_repo.by_tags(task_tags)
# 负载过滤:剔除满载
candidates = [a for a in candidates if a.current_load < a.max_concurrent]
if not candidates:
return None
# 最优:负载最低、在线时间最长(稳定)
candidates.sort(key=lambda a: (a.current_load, -a.create_time))
return candidates[0]
async def dispatch(self, request_id: str) -> bool:
"""将单个 pending 任务下发给匹配 Agent绑定 → 推送 payload → 失败回退)。"""
task = await self.task_repo.get(request_id)
if not task or task.status != TaskStatus.PENDING:
return False
agent = await self.pick_candidate(task.task_tags)
if not agent:
return False # 暂无可用 Agent保持 pending等待下次调度
# 绑定 Agent标记 running
await self.task_repo.update(
request_id,
status=TaskStatus.RUNNING,
agent_id=agent.agent_id,
progress=0,
)
await self.task_repo.mark_running(request_id)
# 更新 Agent 负载
await self.agent_repo.adjust_load(agent.agent_id, 1)
# 真实推送 payload 到 Agent 端点
pushed = await self.relay.dispatch_command(agent.agent_id, request_id, task.payload or {})
if not pushed:
# 推送失败:回退 pending 并释放负载,等待下次调度(避免重复扣负载)
await self.task_repo.update(request_id, status=TaskStatus.PENDING, agent_id="", progress=0)
await self.task_repo.mark_pending(request_id)
await self.agent_repo.adjust_load(agent.agent_id, -1)
logger.warning("task dispatch push failed, reverted request_id=%s agent=%s", request_id, agent.agent_id)
return False
logger.info("task dispatched request_id=%s -> agent=%s", request_id, agent.agent_id)
return True
async def dispatch_pending(self, batch_size: int = 20) -> int:
"""调度所有可调度的 pending 任务,返回成功下发数。"""
pending = await self.task_repo.pending_tasks()
dispatched = 0
for rid in pending[:batch_size]:
if await self.dispatch(rid):
dispatched += 1
return dispatched

View File

@ -0,0 +1,10 @@
"""认证校验CLI / Agent 请求必须携带与 GATEWAY_AUTH 一致的密码。"""
from fastapi import HTTPException
from app.config import settings
def require_auth(auth: str) -> None:
"""auth 不匹配时抛出 401。"""
if auth != settings.gateway_auth:
raise HTTPException(status_code=401, detail="invalid auth")

View File

@ -0,0 +1,106 @@
"""任务池服务:受理、状态流转、幂等、超时失败。"""
import logging
import time
import redis.asyncio as aioredis
from app.config import settings
from app.constants import TaskStatus
from app.models.schemas import TaskInfo, TaskSubmit
from app.repository.agent_repo import AgentRepo
from app.repository.task_repo import TaskRepo
from app.services.notifier import TaskNotifier
logger = logging.getLogger(__name__)
class TaskService:
def __init__(self, redis: aioredis.Redis, repo: TaskRepo | None = None):
self.redis = redis
self.repo = repo or TaskRepo(redis)
self.agent_repo = AgentRepo(redis)
async def _release_if_running(self, task: TaskInfo) -> None:
"""若任务处于 running 且绑定 Agent释放其算力。"""
if task.status == TaskStatus.RUNNING and task.agent_id:
await self.agent_repo.adjust_load(task.agent_id, -1)
async def submit(self, submit: TaskSubmit) -> TaskInfo:
"""受理任务:生成/复用 RequestID幂等写入任务池。"""
task = TaskInfo(
request_id=submit.request_id or submit.request_id or _new_id(),
task_type=submit.task_type,
task_tags=submit.task_tags,
description=submit.description,
payload=submit.payload,
cli_session_id=submit.cli_session_id,
timeout=submit.timeout or settings.default_task_timeout,
status=TaskStatus.PENDING,
)
created = await self.repo.create(task)
if not created:
# 幂等:返回已有任务
existing = await self.repo.get(task.request_id)
if existing:
return existing
logger.info("task submitted request_id=%s type=%s", task.request_id, task.task_type)
return task
async def get(self, request_id: str) -> TaskInfo | None:
return await self.repo.get(request_id)
async def list(self, status: TaskStatus | None = None, limit: int = 100) -> list[TaskInfo]:
return await self.repo.list(status=status, limit=limit)
async def cancel(self, request_id: str) -> bool:
"""取消任务(仅 pending/running 可取消)。"""
task = await self.repo.get(request_id)
if not task:
return False
if task.status in (TaskStatus.SUCCESS, TaskStatus.FAILED):
return False
await self._release_if_running(task)
await self.repo.update(request_id, status=TaskStatus.FAILED, error_info="cancelled by user")
await self.repo.mark_finished(request_id)
if task.cli_session_id:
finished = await self.repo.get(request_id)
if finished:
await TaskNotifier(self.redis).publish_task_done(finished)
logger.info("task cancelled request_id=%s", request_id)
return True
async def reset(self, request_id: str) -> bool:
"""重置任务状态为 pending重新调度"""
task = await self.repo.get(request_id)
if not task:
return False
await self.repo.update(request_id, status=TaskStatus.PENDING, agent_id=None, error_info=None, progress=0)
await self.repo.mark_pending(request_id)
logger.info("task reset request_id=%s", request_id)
return True
async def check_timeouts(self) -> int:
"""扫描 running 任务,超时自动标记失败。"""
now = time.time()
failed = 0
for rid in await self.repo.running_tasks():
task = await self.repo.get(rid)
if not task:
continue
if task.timeout and (now - task.create_time) > task.timeout:
await self._release_if_running(task)
await self.repo.update(rid, status=TaskStatus.FAILED, error_info="timeout")
await self.repo.mark_finished(rid)
if task.cli_session_id:
finished = await self.repo.get(rid)
if finished:
await TaskNotifier(self.redis).publish_task_done(finished)
failed += 1
logger.info("task timeout request_id=%s", rid)
return failed
def _new_id() -> str:
import uuid
return uuid.uuid4().hex

View File

@ -0,0 +1,19 @@
"""任务超时检查服务。"""
import logging
import redis.asyncio as aioredis
from app.repository.task_repo import TaskRepo
from app.services.task_service import TaskService
logger = logging.getLogger(__name__)
class TimeoutService:
def __init__(self, redis: aioredis.Redis, task_repo: TaskRepo | None = None):
self.redis = redis
self.task_repo = task_repo or TaskRepo(redis)
async def check(self) -> int:
"""扫描 running 任务,超时自动标记失败,返回失败数。"""
return await TaskService(self.redis, self.task_repo).check_timeouts()

View File

25
backend/tests/conftest.py Normal file
View File

@ -0,0 +1,25 @@
"""pytest 共享 fixture。"""
import fakeredis.aioredis
import pytest
import pytest_asyncio
from app.repository.agent_repo import AgentRepo
from app.repository.task_repo import TaskRepo
@pytest_asyncio.fixture
async def redis():
server = fakeredis.FakeServer()
client = fakeredis.aioredis.FakeRedis(server=server, decode_responses=True)
yield client
await client.aclose()
@pytest_asyncio.fixture
def task_repo(redis):
return TaskRepo(redis)
@pytest_asyncio.fixture
def agent_repo(redis):
return AgentRepo(redis)

View File

@ -0,0 +1,55 @@
"""Agent 池存储层单元测试。"""
import time
import pytest
from app.constants import AgentStatus
from app.models.schemas import AgentInfo
@pytest.mark.asyncio
async def test_upsert_and_get(agent_repo):
a = AgentInfo(agent_id="a1", endpoint="agent-1:5000", agent_tags=["compile", "test"], max_concurrent=2)
created = await agent_repo.upsert(a)
assert created is True
got = await agent_repo.get("a1")
assert got is not None
assert got.agent_tags == ["compile", "test"]
assert got.status == AgentStatus.ONLINE
@pytest.mark.asyncio
async def test_beat_and_stale(agent_repo):
a = AgentInfo(agent_id="a1", endpoint="agent-1:5000", agent_tags=["compile"])
await agent_repo.upsert(a)
assert await agent_repo.beat("a1", 1) is True
# 心跳为最新,不应 stale
stale = await agent_repo.stale_agents(1)
assert "a1" not in stale
@pytest.mark.asyncio
async def test_by_tags(agent_repo):
a1 = AgentInfo(agent_id="a1", endpoint="e1", agent_tags=["compile"], max_concurrent=2)
a2 = AgentInfo(agent_id="a2", endpoint="e2", agent_tags=["compile", "test"], max_concurrent=2)
a3 = AgentInfo(agent_id="a3", endpoint="e3", agent_tags=["test"], max_concurrent=2)
await agent_repo.upsert(a1)
await agent_repo.upsert(a2)
await agent_repo.upsert(a3)
ids = {a.agent_id for a in await agent_repo.by_tags(["compile"])}
assert ids == {"a1", "a2"}
ids = {a.agent_id for a in await agent_repo.by_tags(["compile", "test"])}
assert ids == {"a2"}
@pytest.mark.asyncio
async def test_remove(agent_repo):
a1 = AgentInfo(agent_id="a1", endpoint="e1", agent_tags=["compile"])
await agent_repo.upsert(a1)
await agent_repo.remove("a1")
assert await agent_repo.get("a1") is None
assert await agent_repo.by_tags(["compile"]) == []

90
backend/tests/test_api.py Normal file
View File

@ -0,0 +1,90 @@
"""接口集成测试。"""
import fakeredis.aioredis
import pytest
import pytest_asyncio
from httpx import ASGITransport, AsyncClient
from app.config import settings
from app.main import app
AUTH = settings.gateway_auth
@pytest_asyncio.fixture
async def client():
server = fakeredis.FakeServer()
redis = fakeredis.aioredis.FakeRedis(server=server, decode_responses=True)
app.state.redis = redis
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as c:
yield c
await redis.aclose()
@pytest.mark.asyncio
async def test_health(client):
r = await client.get("/health")
assert r.status_code == 200
assert r.json()["status"] == "ok"
@pytest.mark.asyncio
async def test_submit_and_query_task(client):
r = await client.post("/api/cli/tasks", json={"auth": AUTH, "task_type": "compile", "task_tags": ["build"]})
assert r.status_code == 200
data = r.json()
assert data["status"] == "pending"
rid = data["request_id"]
r2 = await client.get(f"/api/cli/tasks/{rid}")
assert r2.status_code == 200
assert r2.json()["request_id"] == rid
@pytest.mark.asyncio
async def test_agent_register_heartbeat_result(client):
# 注册
r = await client.post("/api/agent/register", json={"auth": AUTH, "agent_id": "a1", "endpoint": "e1", "agent_tags": ["compile"], "max_concurrent": 2})
assert r.status_code == 200
# 心跳
r = await client.post("/api/agent/heartbeat", json={"agent_id": "a1", "current_load": 0})
assert r.status_code == 200
# 提交任务并等待调度(手动触发一次调度)
r = await client.post("/api/cli/tasks", json={"auth": AUTH, "task_type": "compile", "task_tags": ["compile"]})
rid = r.json()["request_id"]
from app.services.scheduler import Scheduler
await Scheduler(app.state.redis).dispatch_pending(10)
# 回传结果
r = await client.post("/api/agent/result", json={"request_id": rid, "agent_id": "a1", "status": "success", "progress": 100, "result": {"ok": True}})
assert r.status_code == 200
detail = await client.get(f"/api/cli/tasks/{rid}")
assert detail.json()["status"] == "success"
@pytest.mark.asyncio
async def test_admin_overview(client):
r = await client.get("/api/admin/overview")
assert r.status_code == 200
assert "task_total" in r.json()
@pytest.mark.asyncio
async def test_submit_task_rejects_bad_auth(client):
r = await client.post("/api/cli/tasks", json={"auth": "wrong-password", "task_type": "compile"})
assert r.status_code == 401
@pytest.mark.asyncio
async def test_submit_task_requires_auth(client):
r = await client.post("/api/cli/tasks", json={"task_type": "compile"})
assert r.status_code == 422 # 缺 auth 字段
@pytest.mark.asyncio
async def test_agent_register_rejects_bad_auth(client):
r = await client.post("/api/agent/register", json={"auth": "wrong-password", "agent_id": "a9", "endpoint": "e9"})
assert r.status_code == 401

View File

@ -0,0 +1,67 @@
"""调度服务单元测试。"""
import pytest
from app.constants import TaskStatus
from app.models.schemas import AgentInfo, TaskInfo
from app.repository.agent_repo import AgentRepo
from app.repository.task_repo import TaskRepo
from app.services.scheduler import Scheduler
@pytest.mark.asyncio
async def test_dispatch_binds_min_load_agent(redis):
task_repo = TaskRepo(redis)
agent_repo = AgentRepo(redis)
# 两个 compile Agentb2 负载更低
await agent_repo.upsert(AgentInfo(agent_id="b1", endpoint="e1", agent_tags=["compile"], max_concurrent=2, current_load=1))
await agent_repo.upsert(AgentInfo(agent_id="b2", endpoint="e2", agent_tags=["compile"], max_concurrent=2, current_load=0))
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
sched = Scheduler(redis, task_repo, agent_repo)
ok = await sched.dispatch("t1")
assert ok is True
task = await task_repo.get("t1")
assert task.agent_id == "b2"
assert task.status == TaskStatus.RUNNING
assert "t1" in await task_repo.running_tasks()
@pytest.mark.asyncio
async def test_dispatch_no_agent_stays_pending(redis):
task_repo = TaskRepo(redis)
agent_repo = AgentRepo(redis)
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
sched = Scheduler(redis, task_repo, agent_repo)
ok = await sched.dispatch("t1")
assert ok is False
task = await task_repo.get("t1")
assert task.status == TaskStatus.PENDING
@pytest.mark.asyncio
async def test_dispatch_skips_full_agent(redis):
task_repo = TaskRepo(redis)
agent_repo = AgentRepo(redis)
# 唯一 agent 已满载
await agent_repo.upsert(AgentInfo(agent_id="b1", endpoint="e1", agent_tags=["compile"], max_concurrent=1, current_load=1))
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
sched = Scheduler(redis, task_repo, agent_repo)
assert await sched.dispatch("t1") is False
@pytest.mark.asyncio
async def test_dispatch_pending(redis):
task_repo = TaskRepo(redis)
agent_repo = AgentRepo(redis)
await agent_repo.upsert(AgentInfo(agent_id="b1", endpoint="e1", agent_tags=["compile"], max_concurrent=2))
await task_repo.create(TaskInfo(request_id="t1", task_type="compile", task_tags=["compile"]))
await task_repo.create(TaskInfo(request_id="t2", task_type="compile", task_tags=["compile"]))
sched = Scheduler(redis, task_repo, agent_repo)
n = await sched.dispatch_pending(10)
assert n == 2

View File

@ -0,0 +1,51 @@
"""任务池存储层单元测试。"""
import pytest
from app.constants import TaskStatus
from app.models.schemas import TaskInfo
@pytest.mark.asyncio
async def test_create_and_get(task_repo):
t = TaskInfo(request_id="r1", task_type="compile", task_tags=["build"], payload={"cmd": "go build"})
created = await task_repo.create(t)
assert created is True
got = await task_repo.get("r1")
assert got is not None
assert got.task_type == "compile"
assert got.task_tags == ["build"]
assert got.payload == {"cmd": "go build"}
assert got.status == TaskStatus.PENDING
@pytest.mark.asyncio
async def test_idempotent_create(task_repo):
t = TaskInfo(request_id="r1", task_type="compile")
assert await task_repo.create(t) is True
assert await task_repo.create(t) is False # 幂等
@pytest.mark.asyncio
async def test_state_sets(task_repo):
t = TaskInfo(request_id="r1", task_type="compile")
await task_repo.create(t)
assert "r1" in await task_repo.pending_tasks()
await task_repo.mark_running("r1")
assert "r1" not in await task_repo.pending_tasks()
assert "r1" in await task_repo.running_tasks()
await task_repo.mark_finished("r1")
assert "r1" not in await task_repo.running_tasks()
@pytest.mark.asyncio
async def test_update(task_repo):
t = TaskInfo(request_id="r1", task_type="compile")
await task_repo.create(t)
await task_repo.update("r1", status=TaskStatus.RUNNING, agent_id="a1", progress=50)
got = await task_repo.get("r1")
assert got.status == TaskStatus.RUNNING
assert got.agent_id == "a1"
assert got.progress == 50

15
docker-compose.yml Normal file
View File

@ -0,0 +1,15 @@
version: "3.8"
services:
redis:
image: redis:7-alpine
container_name: gateway-redis
restart: unless-stopped
ports:
- "6379:6379"
volumes:
- redis-data:/data
command: ["redis-server", "--appendonly", "yes"]
volumes:
redis-data:

12
frontend/index.html Normal file
View File

@ -0,0 +1,12 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>A2A 智能网关运维台</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="/src/main.ts"></script>
</body>
</html>

2751
frontend/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

27
frontend/package.json Normal file
View File

@ -0,0 +1,27 @@
{
"name": "gateway-admin",
"version": "1.0.0",
"private": true,
"type": "module",
"scripts": {
"dev": "vite",
"build": "vue-tsc -b && vite build",
"preview": "vite preview"
},
"dependencies": {
"@element-plus/icons-vue": "^2.3.1",
"element-plus": "^2.9.1",
"lucide-vue-next": "^0.469.0",
"vue": "^3.5.13",
"vue-router": "^4.5.0"
},
"devDependencies": {
"@vitejs/plugin-vue": "^5.2.1",
"autoprefixer": "^10.5.4",
"postcss": "^8.5.25",
"tailwindcss": "^3.4.0",
"typescript": "~5.6.2",
"vite": "^5.4.11",
"vue-tsc": "^2.1.10"
}
}

View File

@ -0,0 +1,6 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}

160
frontend/src/App.vue Normal file
View File

@ -0,0 +1,160 @@
<script setup lang="ts">
import { computed, onMounted, onUnmounted, ref } from 'vue'
import { useRoute } from 'vue-router'
import { api } from './api'
import type { Overview } from './api'
const route = useRoute()
const overview = ref<Overview | null>(null)
const isCollapse = ref(false)
let timer: number | undefined
const title = computed(() => (route.meta.title as string) || '')
const menus = [
{ path: '/tasks', label: '任务管理', icon: 'List' },
{ path: '/agents', label: 'Agent 管理', icon: 'Cpu' },
{ path: '/logs', label: '日志审计', icon: 'Document' },
{ path: '/control', label: '手动管控', icon: 'Operation' },
]
async function refresh() {
try {
overview.value = await api.overview()
} catch {
overview.value = null
}
}
onMounted(() => {
refresh()
timer = window.setInterval(refresh, 5000)
})
onUnmounted(() => {
if (timer) window.clearInterval(timer)
})
</script>
<template>
<div class="flex h-full">
<!-- 可折叠侧边菜单 -->
<aside
class="flex flex-col shrink-0 h-full bg-[#0f172a] transition-all duration-200 overflow-hidden"
:class="isCollapse ? 'w-16' : 'w-56'"
>
<!-- Logo -->
<div class="h-16 flex items-center gap-2.5 px-4 border-b border-white/10 overflow-hidden shrink-0">
<div
class="w-9 h-9 shrink-0 rounded-lg bg-gradient-to-br from-[#1e63e8] to-[#3b82f6] flex items-center justify-center font-bold text-white"
>
A
</div>
<div v-show="!isCollapse" class="min-w-0">
<div class="font-semibold text-sm text-white leading-tight">A2A 智能网关</div>
<div class="text-[11px] text-slate-400">运维控制台</div>
</div>
</div>
<!-- 菜单 -->
<el-menu
:default-active="route.path"
:collapse="isCollapse"
:collapse-transition="false"
router
background-color="#0f172a"
text-color="#94a3b8"
active-text-color="#ffffff"
class="gw-menu flex-1 overflow-y-auto overflow-x-hidden"
>
<el-menu-item v-for="m in menus" :key="m.path" :index="m.path">
<el-icon><component :is="m.icon" /></el-icon>
<template #title>{{ m.label }}</template>
</el-menu-item>
</el-menu>
<div
v-show="!isCollapse"
class="px-5 py-4 text-[11px] text-slate-500 border-t border-white/10 shrink-0"
>
Gateway v1.0.0
</div>
<div
v-show="isCollapse"
class="py-4 text-center text-[10px] text-slate-600 border-t border-white/10 shrink-0"
>
v1.0
</div>
</aside>
<!-- 主区域 -->
<div class="flex-1 flex flex-col min-w-0 h-full">
<header class="h-16 bg-white border-b flex items-center justify-between px-5 gap-4 shrink-0">
<div class="flex items-center gap-3 min-w-0">
<button class="gw-toggle" title="折叠 / 展开菜单" @click="isCollapse = !isCollapse">
<el-icon :size="18"><Expand v-if="isCollapse" /><Fold v-else /></el-icon>
</button>
<h1 class="text-base font-semibold text-slate-800 truncate">{{ title }}</h1>
</div>
<div class="flex items-center gap-5 text-sm shrink-0">
<div class="flex items-center gap-1.5">
<span class="badge-dot bg-[#10b981] pulse"></span>
<span class="text-slate-600">在线 Agent</span>
<span class="font-bold text-[#10b981]">{{ overview?.agent_online ?? '-' }}</span>
</div>
<div class="flex items-center gap-1.5">
<span class="badge-dot bg-[#1e63e8]"></span>
<span class="text-slate-600">任务总数</span>
<span class="font-bold text-[#1e63e8]">{{ overview?.task_total ?? '-' }}</span>
</div>
<div class="flex items-center gap-1.5">
<span class="badge-dot bg-[#ef4444]"></span>
<span class="text-slate-600">告警</span>
<span class="font-bold text-[#ef4444]">
{{ (overview?.task_failed ?? 0) + (overview?.agent_offline ?? 0) }}
</span>
</div>
</div>
</header>
<main class="flex-1 overflow-auto p-6">
<router-view />
</main>
</div>
</div>
</template>
<style scoped>
.gw-toggle {
width: 36px;
height: 36px;
border-radius: 8px;
display: flex;
align-items: center;
justify-content: center;
color: #64748b;
background: transparent;
border: none;
cursor: pointer;
transition: background 0.2s;
}
.gw-toggle:hover {
background: #f1f5f9;
}
/* 深色侧栏菜单细节修正 */
.gw-menu {
--el-menu-hover-bg-color: rgba(255, 255, 255, 0.06);
--el-menu-active-color: #ffffff;
border-right: none !important;
}
.gw-menu .el-menu-item {
margin: 2px 8px;
border-radius: 8px;
}
.gw-menu .el-menu-item.is-active {
background: linear-gradient(90deg, rgba(30, 99, 232, 0.9), rgba(59, 130, 246, 0.7));
}
.gw-menu:not(.el-menu--collapse) .el-menu-item.is-active {
box-shadow: 0 4px 12px rgba(30, 99, 232, 0.35);
}
</style>

78
frontend/src/api/index.ts Normal file
View File

@ -0,0 +1,78 @@
const BASE = '/api/admin'
async function request<T>(path: string, options: RequestInit = {}): Promise<T> {
const res = await fetch(`${BASE}${path}`, {
headers: { 'Content-Type': 'application/json' },
...options,
})
if (!res.ok) {
const text = await res.text()
throw new Error(text || `HTTP ${res.status}`)
}
return res.json() as Promise<T>
}
export interface TaskInfo {
request_id: string
task_type: string
task_tags: string[]
description: string | null
status: 'pending' | 'running' | 'success' | 'failed'
agent_id: string | null
create_time: number
progress: number
result: unknown
error_info: string | null
}
export interface AgentInfo {
agent_id: string
endpoint: string
agent_tags: string[]
max_concurrent: number
current_load: number
last_heartbeat: number
status: 'online' | 'offline'
create_time: number
}
export interface LogEntry {
ts: number
level: string
source: string
scope: string
request_id: string | null
agent_id: string | null
message: string
}
export interface Overview {
task_total: number
task_pending: number
task_running: number
task_success: number
task_failed: number
agent_total: number
agent_online: number
agent_offline: number
}
export const api = {
listTasks: (status?: string) => {
const q = status ? `?status=${status}` : ''
return request<TaskInfo[]>(`/tasks${q}`)
},
taskDetail: (id: string) => request<TaskInfo>(`/tasks/${id}`),
cancelTask: (id: string) => request<{ ok: boolean }>(`/tasks/${id}/cancel`, { method: 'POST' }),
resetTask: (id: string) => request<{ ok: boolean }>(`/tasks/${id}/reset`, { method: 'POST' }),
listAgents: () => request<AgentInfo[]>('/agents'),
offlineAgent: (id: string) => request<{ ok: boolean }>(`/agents/${id}/offline`, { method: 'POST' }),
listLogs: (params?: { request_id?: string; agent_id?: string }) => {
const q = new URLSearchParams()
if (params?.request_id) q.set('request_id', params.request_id)
if (params?.agent_id) q.set('agent_id', params.agent_id)
const s = q.toString()
return request<LogEntry[]>(`/logs${s ? `?${s}` : ''}`)
},
overview: () => request<Overview>('/overview'),
}

7
frontend/src/env.d.ts vendored Normal file
View File

@ -0,0 +1,7 @@
/// <reference types="vite/client" />
declare module '*.vue' {
import type { DefineComponent } from 'vue'
const component: DefineComponent<{}, {}, any>
export default component
}

15
frontend/src/main.ts Normal file
View File

@ -0,0 +1,15 @@
import { createApp } from 'vue'
import ElementPlus from 'element-plus'
import 'element-plus/dist/index.css'
import * as ElementPlusIconsVue from '@element-plus/icons-vue'
import App from './App.vue'
import router from './router'
import './style.css'
const app = createApp(App)
for (const [key, component] of Object.entries(ElementPlusIconsVue)) {
app.component(key, component)
}
app.use(ElementPlus)
app.use(router)
app.mount('#app')

View File

@ -0,0 +1,18 @@
import { createRouter, createWebHistory } from 'vue-router'
import TaskView from '../views/TaskView.vue'
import AgentView from '../views/AgentView.vue'
import LogView from '../views/LogView.vue'
import ControlView from '../views/ControlView.vue'
const router = createRouter({
history: createWebHistory(),
routes: [
{ path: '/', redirect: '/tasks' },
{ path: '/tasks', name: 'tasks', component: TaskView, meta: { title: '任务管理' } },
{ path: '/agents', name: 'agents', component: AgentView, meta: { title: 'Agent 管理' } },
{ path: '/logs', name: 'logs', component: LogView, meta: { title: '日志审计' } },
{ path: '/control', name: 'control', component: ControlView, meta: { title: '手动管控' } },
],
})
export default router

93
frontend/src/style.css Normal file
View File

@ -0,0 +1,93 @@
@tailwind components;
@tailwind utilities;
:root {
--gw-primary: #1e63e8;
--gw-primary-light: #3b82f6;
--gw-dark: #0f172a;
--gw-bg: #f5f7fa;
--gw-success: #10b981;
--gw-warning: #f59e0b;
--gw-danger: #ef4444;
--gw-info: #3b82f6;
--gw-text: #1e293b;
--gw-text-sub: #64748b;
}
* {
box-sizing: border-box;
}
html,
body,
#app {
margin: 0;
padding: 0;
height: 100%;
font-family: 'PingFang SC', 'Microsoft YaHei', -apple-system, sans-serif;
background: var(--gw-bg);
color: var(--gw-text);
}
.stat-card {
border-radius: 12px;
background: linear-gradient(135deg, #ffffff 0%, #f8fafc 100%);
border: 1px solid #e2e8f0;
padding: 18px 20px;
transition: transform 0.2s ease, box-shadow 0.2s ease;
}
.stat-card:hover {
transform: translateY(-3px);
box-shadow: 0 8px 24px rgba(30, 99, 232, 0.10);
}
.stat-value {
font-size: 28px;
font-weight: 700;
line-height: 1.2;
}
.badge-dot {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 50%;
margin-right: 6px;
}
.pulse {
animation: pulse 1.6s ease-in-out infinite;
}
@keyframes pulse {
0%,
100% {
opacity: 1;
}
50% {
opacity: 0.35;
}
}
.page-card {
background: #fff;
border-radius: 12px;
border: 1px solid #e2e8f0;
padding: 20px;
}
.fade-in {
animation: fadeIn 0.3s ease;
}
@keyframes fadeIn {
from {
opacity: 0;
transform: translateY(6px);
}
to {
opacity: 1;
transform: translateY(0);
}
}

View File

@ -0,0 +1,92 @@
<script setup lang="ts">
import { onMounted, onUnmounted, ref } from 'vue'
import { api, type AgentInfo } from '../api'
const agents = ref<AgentInfo[]>([])
const loading = ref(false)
let timer: number | undefined
async function load() {
loading.value = true
try {
agents.value = await api.listAgents()
} catch (e) {
console.error(e)
} finally {
loading.value = false
}
}
function fmtTime(ts: number) {
if (!ts) return '-'
return new Date(ts * 1000).toLocaleString()
}
function loadPct(a: AgentInfo) {
return a.max_concurrent ? Math.round((a.current_load / a.max_concurrent) * 100) : 0
}
onMounted(() => {
load()
timer = window.setInterval(load, 5000)
})
onUnmounted(() => {
if (timer) window.clearInterval(timer)
})
</script>
<template>
<div class="space-y-5 fade-in">
<div class="flex items-center justify-between">
<h2 class="font-semibold">Agent 池状态</h2>
<span class="text-sm text-slate-500">在线 {{ agents.filter((a) => a.status === 'online').length }} / {{ agents.length }}</span>
</div>
<div v-if="loading && agents.length === 0" class="page-card text-center text-slate-400 py-16">加载中...</div>
<div v-else-if="agents.length === 0" class="page-card text-center text-slate-400 py-16">
暂无 Agent 接入请等待 K8s Agent 注册
</div>
<div class="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-4">
<div
v-for="a in agents"
:key="a.agent_id"
class="stat-card"
:class="{ 'ring-2 ring-[#ef4444]': a.status === 'offline' }"
>
<div class="flex items-start justify-between">
<div class="flex items-center gap-2">
<el-icon :size="18" :color="a.status === 'online' ? '#10b981' : '#ef4444'"><Cpu /></el-icon>
<span class="font-mono font-semibold">{{ a.agent_id }}</span>
</div>
<el-tag :type="a.status === 'online' ? 'success' : 'danger'" effect="light" size="small">
{{ a.status === 'online' ? '在线' : '离线' }}
</el-tag>
</div>
<div class="mt-3 text-sm text-slate-500 truncate">{{ a.endpoint }}</div>
<div class="mt-2 flex flex-wrap gap-1">
<el-tag v-for="t in a.agent_tags" :key="t" size="small" type="info" effect="plain">{{ t }}</el-tag>
</div>
<div class="mt-4">
<div class="flex justify-between text-xs text-slate-500 mb-1">
<span>负载 {{ a.current_load }}/{{ a.max_concurrent }}</span>
<span>{{ loadPct(a) }}%</span>
</div>
<el-progress
:percentage="loadPct(a)"
:stroke-width="10"
:color="loadPct(a) >= 100 ? '#ef4444' : loadPct(a) >= 70 ? '#f59e0b' : '#10b981'"
/>
</div>
<div class="mt-3 text-xs text-slate-400 flex items-center gap-1">
<span class="badge-dot" :class="a.status === 'online' ? 'bg-[#10b981] pulse' : 'bg-[#ef4444]'"></span>
心跳 {{ fmtTime(a.last_heartbeat) }} · 注册 {{ fmtTime(a.create_time) }}
</div>
</div>
</div>
</div>
</template>

View File

@ -0,0 +1,115 @@
<script setup lang="ts">
import { onMounted, ref } from 'vue'
import { ElMessage, ElMessageBox } from 'element-plus'
import { api, type AgentInfo, type TaskInfo } from '../api'
const tasks = ref<TaskInfo[]>([])
const agents = ref<AgentInfo[]>([])
const loading = ref(false)
async function load() {
loading.value = true
try {
const [t, a] = await Promise.all([api.listTasks(), api.listAgents()])
tasks.value = t
agents.value = a
} catch (e) {
console.error(e)
} finally {
loading.value = false
}
}
async function cancelTask(id: string) {
try {
await ElMessageBox.confirm(`确认取消任务 ${id} `, '取消任务', { type: 'warning' })
await api.cancelTask(id)
ElMessage.success('任务已取消')
load()
} catch (e) {
if (e !== 'cancel') console.error(e)
}
}
async function resetTask(id: string) {
try {
await ElMessageBox.confirm(`确认重置任务 ${id} 为待调度?`, '重置任务', { type: 'warning' })
await api.resetTask(id)
ElMessage.success('任务已重置')
load()
} catch (e) {
if (e !== 'cancel') console.error(e)
}
}
async function offlineAgent(id: string) {
try {
await ElMessageBox.confirm(`确认下线 Agent ${id} `, '下线 Agent', { type: 'warning' })
await api.offlineAgent(id)
ElMessage.success('Agent 已下线')
load()
} catch (e) {
if (e !== 'cancel') console.error(e)
}
}
onMounted(load)
</script>
<template>
<div class="space-y-5 fade-in">
<div class="page-card">
<h2 class="font-semibold mb-4">取消任务</h2>
<el-table :data="tasks.filter((t) => t.status === 'pending' || t.status === 'running')" v-loading="loading" stripe>
<el-table-column label="RequestID" prop="request_id" min-width="220" show-overflow-tooltip />
<el-table-column label="任务描述" min-width="220" show-overflow-tooltip>
<template #default="{ row }">{{ row.description || '-' }}</template>
</el-table-column>
<el-table-column label="状态" width="100">
<template #default="{ row }">
<el-tag :type="row.status === 'running' ? 'primary' : 'warning'" effect="light">
{{ row.status === 'running' ? '执行中' : '待调度' }}
</el-tag>
</template>
</el-table-column>
<el-table-column label="操作" width="160">
<template #default="{ row }">
<el-button link type="danger" @click="cancelTask(row.request_id)">取消</el-button>
<el-button link type="warning" @click="resetTask(row.request_id)">重置</el-button>
</template>
</el-table-column>
</el-table>
<div v-if="loading === false && tasks.filter((t) => t.status === 'pending' || t.status === 'running').length === 0" class="text-center text-slate-400 py-8">
当前没有可管控的任务
</div>
</div>
<div class="page-card">
<h2 class="font-semibold mb-4">下线 Agent</h2>
<el-table :data="agents" v-loading="loading" stripe>
<el-table-column label="AgentID" prop="agent_id" min-width="180" />
<el-table-column label="地址" prop="endpoint" min-width="160" show-overflow-tooltip />
<el-table-column label="状态" width="100">
<template #default="{ row }">
<el-tag :type="row.status === 'online' ? 'success' : 'danger'" effect="light">
{{ row.status === 'online' ? '在线' : '离线' }}
</el-tag>
</template>
</el-table-column>
<el-table-column label="操作" width="160">
<template #default="{ row }">
<el-button
link
type="danger"
:disabled="row.status !== 'online'"
@click="offlineAgent(row.agent_id)"
>
下线
</el-button>
</template>
</el-table-column>
</el-table>
<div v-if="loading === false && agents.length === 0" class="text-center text-slate-400 py-8">暂无 Agent</div>
</div>
</div>
</template>

View File

@ -0,0 +1,71 @@
<script setup lang="ts">
import { onMounted, ref } from 'vue'
import { api, type LogEntry } from '../api'
const logs = ref<LogEntry[]>([])
const loading = ref(false)
const requestId = ref('')
const agentId = ref('')
const scopeLabel: Record<string, string> = { task: '任务', agent: 'Agent', system: '系统' }
const levelColor: Record<string, string> = {
info: '#10b981',
warning: '#f59e0b',
error: '#ef4444',
}
async function load() {
loading.value = true
try {
logs.value = await api.listLogs({
request_id: requestId.value || undefined,
agent_id: agentId.value || undefined,
})
} catch (e) {
console.error(e)
} finally {
loading.value = false
}
}
function fmtTime(ts: number) {
return new Date(ts * 1000).toLocaleString()
}
onMounted(load)
</script>
<template>
<div class="page-card fade-in">
<div class="flex flex-wrap items-center gap-3 mb-5">
<h2 class="font-semibold mr-2">全链路审计日志</h2>
<el-input v-model="requestId" placeholder="RequestID 筛选" clearable style="width: 220px" @keyup.enter="load" />
<el-input v-model="agentId" placeholder="AgentID 筛选" clearable style="width: 200px" @keyup.enter="load" />
<el-button type="primary" @click="load">查询</el-button>
</div>
<el-timeline v-loading="loading">
<el-timeline-item
v-for="(log, i) in logs"
:key="i"
:color="levelColor[log.level] || '#cbd5e1'"
:timestamp="fmtTime(log.ts)"
placement="top"
>
<div class="border border-slate-100 rounded-lg p-3 bg-slate-50/50">
<div class="flex items-center gap-2 mb-1">
<el-tag size="small" :type="log.scope === 'agent' ? 'success' : log.scope === 'system' ? 'info' : 'primary'" effect="light">
{{ scopeLabel[log.scope] || log.scope }}
</el-tag>
<span class="text-xs text-slate-400">{{ log.source }}</span>
<span v-if="log.request_id" class="text-xs font-mono text-slate-500">#{{ log.request_id }}</span>
<span v-if="log.agent_id" class="text-xs font-mono text-slate-500">agent:{{ log.agent_id }}</span>
</div>
<div class="text-sm">{{ log.message }}</div>
</div>
</el-timeline-item>
</el-timeline>
<div v-if="!loading && logs.length === 0" class="text-center text-slate-400 py-16">暂无日志</div>
</div>
</template>

View File

@ -0,0 +1,175 @@
<script setup lang="ts">
import { onMounted, onUnmounted, ref } from 'vue'
import { ElMessage } from 'element-plus'
import { api, type TaskInfo, type Overview } from '../api'
const tasks = ref<TaskInfo[]>([])
const overview = ref<Overview | null>(null)
const statusFilter = ref('')
const loading = ref(false)
const detail = ref<TaskInfo | null>(null)
const detailVisible = ref(false)
let timer: number | undefined
const statusMeta: Record<string, { label: string; type: string; color: string }> = {
pending: { label: '待调度', type: 'warning', color: '#f59e0b' },
running: { label: '执行中', type: 'primary', color: '#1e63e8' },
success: { label: '成功', type: 'success', color: '#10b981' },
failed: { label: '失败', type: 'danger', color: '#ef4444' },
}
async function load(silent = false) {
if (!silent) loading.value = true
try {
tasks.value = await api.listTasks(statusFilter.value || undefined)
overview.value = await api.overview()
} catch (e) {
console.error(e)
} finally {
loading.value = false
}
}
function fmtTime(ts: number) {
if (!ts) return '-'
return new Date(ts * 1000).toLocaleString()
}
// result {"reply": "", "output": ""}
function resultReply(t: TaskInfo | null): string {
const r = t?.result as Record<string, unknown> | null | undefined
return r && typeof r.reply === 'string' && r.reply.trim() ? r.reply : '-'
}
function resultOutput(t: TaskInfo | null): string {
const r = t?.result as Record<string, unknown> | null | undefined
if (r && typeof r.output === 'string' && r.output.trim()) return r.output
return t?.result ? JSON.stringify(t.result, null, 2) : '-'
}
async function openDetail(id: string) {
try {
detail.value = await api.taskDetail(id)
detailVisible.value = true
} catch (e) {
console.error(e)
}
}
async function cancelTask(id: string) {
try {
await api.cancelTask(id)
ElMessage.success('任务已取消')
load()
} catch (e) {
ElMessage.error('取消失败')
console.error(e)
}
}
onMounted(() => {
load()
timer = window.setInterval(() => load(true), 5000)
})
onUnmounted(() => {
if (timer) window.clearInterval(timer)
})
</script>
<template>
<div class="space-y-5 fade-in">
<!-- 状态统计卡片 -->
<div class="grid grid-cols-2 md:grid-cols-4 gap-4">
<div class="stat-card">
<div class="text-sm text-slate-500">待调度</div>
<div class="stat-value text-[#f59e0b]">{{ overview?.task_pending ?? 0 }}</div>
</div>
<div class="stat-card">
<div class="text-sm text-slate-500">执行中</div>
<div class="stat-value text-[#1e63e8]">{{ overview?.task_running ?? 0 }}</div>
</div>
<div class="stat-card">
<div class="text-sm text-slate-500">成功</div>
<div class="stat-value text-[#10b981]">{{ overview?.task_success ?? 0 }}</div>
</div>
<div class="stat-card">
<div class="text-sm text-slate-500">失败</div>
<div class="stat-value text-[#ef4444]">{{ overview?.task_failed ?? 0 }}</div>
</div>
</div>
<!-- 任务表格 -->
<div class="page-card">
<div class="flex justify-between items-center mb-4">
<h2 class="font-semibold">任务列表</h2>
<el-select v-model="statusFilter" placeholder="全部状态" clearable style="width: 160px" @change="load">
<el-option label="待调度" value="pending" />
<el-option label="执行中" value="running" />
<el-option label="成功" value="success" />
<el-option label="失败" value="failed" />
</el-select>
</div>
<el-table :data="tasks" v-loading="loading" stripe>
<el-table-column label="RequestID" prop="request_id" min-width="220" show-overflow-tooltip />
<el-table-column label="任务描述" prop="description" min-width="220" show-overflow-tooltip>
<template #default="{ row }">{{ row.description || '-' }}</template>
</el-table-column>
<el-table-column label="状态" width="100">
<template #default="{ row }">
<el-tag :type="statusMeta[row.status]?.type" effect="light">{{ statusMeta[row.status]?.label }}</el-tag>
</template>
</el-table-column>
<el-table-column label="Agent" width="150">
<template #default="{ row }">{{ row.agent_id || '-' }}</template>
</el-table-column>
<el-table-column label="进度" width="160">
<template #default="{ row }">
<el-progress :percentage="row.progress" :stroke-width="8" />
</template>
</el-table-column>
<el-table-column label="创建时间" width="170">
<template #default="{ row }">{{ fmtTime(row.create_time) }}</template>
</el-table-column>
<el-table-column label="操作" width="150" fixed="right">
<template #default="{ row }">
<el-button link type="primary" @click="openDetail(row.request_id)">详情</el-button>
<el-button
v-if="row.status === 'pending' || row.status === 'running'"
link
type="danger"
@click="cancelTask(row.request_id)"
>
取消
</el-button>
</template>
</el-table-column>
</el-table>
</div>
<!-- 详情抽屉 -->
<el-drawer v-model="detailVisible" title="任务详情" size="440px">
<div v-if="detail" class="space-y-4">
<div><div class="text-slate-500 text-sm">RequestID</div><div class="font-mono text-sm">{{ detail.request_id }}</div></div>
<div><div class="text-slate-500 text-sm">任务描述</div><div>{{ detail.description || '-' }}</div></div>
<div><div class="text-slate-500 text-sm">状态</div>
<el-tag :type="statusMeta[detail.status]?.type">{{ statusMeta[detail.status]?.label }}</el-tag>
</div>
<div><div class="text-slate-500 text-sm">Agent</div><div>{{ detail.agent_id || '-' }}</div></div>
<div><div class="text-slate-500 text-sm">进度</div>
<el-progress :percentage="detail.progress" />
</div>
<div><div class="text-slate-500 text-sm">创建时间</div><div>{{ fmtTime(detail.create_time) }}</div></div>
<div><div class="text-slate-500 text-sm">Agent 接受任务回复</div>
<pre class="bg-slate-50 rounded p-2 text-xs overflow-auto whitespace-pre-wrap">{{ resultReply(detail) }}</pre>
</div>
<div><div class="text-slate-500 text-sm">Agent 总结</div>
<pre class="bg-slate-50 rounded p-2 text-xs overflow-auto whitespace-pre-wrap">{{ resultOutput(detail) }}</pre>
</div>
<div v-if="detail.error_info"><div class="text-slate-500 text-sm">错误信息</div>
<div class="text-[#ef4444] text-sm">{{ detail.error_info }}</div>
</div>
</div>
</el-drawer>
</div>
</template>

View File

@ -0,0 +1,8 @@
/** @type {import('tailwindcss').Config} */
export default {
content: ['./index.html', './src/**/*.{vue,js,ts,jsx,tsx}'],
theme: {
extend: {},
},
plugins: [],
}

20
frontend/tsconfig.json Normal file
View File

@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"module": "ESNext",
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "preserve",
"strict": true,
"noUnusedLocals": false,
"noUnusedParameters": false,
"verbatimModuleSyntax": false
},
"include": ["src/**/*.ts", "src/**/*.d.ts", "src/**/*.tsx", "src/**/*.vue"]
}

16
frontend/vite.config.ts Normal file
View File

@ -0,0 +1,16 @@
import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue'
export default defineConfig({
plugins: [vue()],
server: {
host: '0.0.0.0',
port: 5173,
proxy: {
'/api': {
target: 'http://localhost:8000',
changeOrigin: true,
},
},
},
})

12
requirements.txt Normal file
View File

@ -0,0 +1,12 @@
fastapi==0.115.12
uvicorn[standard]==0.34.0
redis==5.2.1
pydantic==2.12.4
pydantic-settings==2.10.1
python-multipart==0.0.20
# 测试
pytest==8.3.4
pytest-asyncio==0.25.3
httpx==0.28.1
fakeredis==2.26.2

236
网关架构流程图.md Normal file
View File

@ -0,0 +1,236 @@
# 网关Gateway项目架构流程图
> 基于代码现状绘制的架构图backend + frontend + agent 联动)。
> 端口:后端 `:8000`、前端 `:5173`、Agent `:8001`、Redis 远程实例 `45.207.192.237:56987/0`
---
## 一、系统总体架构图(分层)
```mermaid
flowchart TB
subgraph OUT["外部调用方"]
CLI["CLI 客户端 / 模型对话<br/>提交任务(带 auth"]
end
subgraph FE["前端运维后台Vue3 + Vite + Element Plus:5173"]
TV["TaskView 任务管理<br/>列表/筛选/详情/取消/重置"]
AV["AgentView Agent 管理<br/>卡片/标签/负载/下线"]
LV["LogView 日志审计<br/>按 request_id/agent_id 筛选"]
CV["ControlView 总览<br/>任务/Agent 统计"]
API["api/index.ts<br/>fetch 封装(/api/admin/*"]
end
subgraph GW["网关后端FastAPI + Uvicorn:8000"]
direction TB
subgraph API_L["API 层app/api/"]
A1["/api/cli/*<br/>任务提交"]
A2["/api/agent/*<br/>注册/心跳/结果回传"]
A3["/api/admin/*<br/>运维后台查询与管控"]
end
subgraph SVC["服务层app/services/"]
S1["TaskService 任务池<br/>受理/取消/重置/超时"]
S2["AgentService Agent 池<br/>注册/心跳/下线/剔除"]
S3["Scheduler 规则调度<br/>选人→绑定→推送→回退"]
S4["RelayService 通信中转<br/>正向推送/反向收结果"]
S5["HeartbeatService<br/>心跳扫描60s"]
S6["TimeoutService<br/>任务超时检查5s"]
end
subgraph REPO["存储层app/repository/"]
R1["TaskRepo"]
R2["AgentRepo"]
R3["LogRepo"]
end
subgraph LOOP["后台循环scheduler_loop.py"]
L1["heartbeat_loop<br/>Lock:lock:heartbeat"]
L2["dispatch_loop2s<br/>Lock:lock:scheduler"]
L3["timeout_loop<br/>Lock:lock:timeout"]
end
end
subgraph RD["Redis 存储DB0"]
K1["task:info:{id} Hash<br/>task:pending / task:running Set"]
K2["agent:info:{id} Hash<br/>agent:all / agent:tag:{tag} Set"]
K3["agent:heartbeat ZSet"]
K4["lock:* 分布式锁"]
K5["log:audit List"]
end
subgraph AG["Agent 服务ADK ApiServer:8001"]
direction TB
AG0["api_server.py<br/>REST /run /run_sse<br/>SQLite 会话 / InMemory 记忆"]
AG1["task_receiver.py<br/>POST /tasks/{id}<br/>校验 auth → 202 → 后台线程"]
AG2["gateway_client.py<br/>注册 / 心跳 / 回传结果"]
AG3["ADK InMemoryRunner<br/>执行 dev_appmy_agent"]
end
CLI -->|"POST /api/cli/tasks"| A1
FE --> API
API -->|"/api/admin/*"| A3
A1 --> S1
A2 --> S2
A3 --> S1 & S2
S1 & S2 & S3 & S4 & S5 & S6 --> R1 & R2 & R3
L1 -->|"调用"| S5
L2 -->|"调用"| S3
L3 -->|"调用"| S6
S5 -->|"Lock"| K4
S3 -->|"Lock"| K4
R1 & R2 & R3 --> K1 & K2 & K3 & K5
S3 -->|"Scheduler 选中 Agent"| S4
S4 -->|"POST {endpoint}/tasks/{request_id}<br/>期望 202"| AG1
AG1 -->|"校验通过后后台线程"| AG3
AG3 -->|"执行完成"| AG2
AG2 -->|"POST /api/agent/result"| A2
AG2 -->|"心跳 /api/agent/heartbeat10s"| A2
```
---
## 二、任务全生命周期时序图(完整闭环)
```mermaid
sequenceDiagram
autonumber
participant CLI as CLI / 对话模型
participant GW as 网关后端FastAPI :8000
participant RD as Redis
participant LOOP as SchedulerLoop 后台循环
participant AG as Agent 服务(:8001
participant ADK as ADK InMemoryRunner
Note over AG,ADK: Agent 启动register → 每 10s heartbeat
CLI->>GW: POST /api/cli/tasks {auth, task_type, tags, payload}
GW->>GW: TaskService.submit幂等RequestID 复用)
GW->>RD: hset task:info:{id} + sadd task:pending
GW-->>CLI: 200 {request_id}
loop dispatch_loop每 2s持 lock:scheduler
LOOP->>RD: 取 task:pending 成员
LOOP->>GW: Scheduler.dispatch_pending(batch)
GW->>RD: AgentRepo.by_tags(tags) 标签匹配
GW->>GW: 负载过滤load < max_concurrent
GW->>GW: 最低负载 + 在线最久者优先
GW->>RD: 任务置 running + 绑定 agent_id<br/>task:pending→task:runningagent 负载 +1
GW->>AG: RelayService.dispatch_command<br/>POST {endpoint}/tasks/{id} {auth, payload}
alt 推送成功202
AG->>AG: 校验 auth → 后台线程接收任务
AG-->>GW: 202 {ok, request_id, status:"accepted"}
AG->>ADK: 运行 agentInMemoryRunner.run_async
ADK-->>AG: 最终输出文本
AG->>GW: POST /api/agent/result {success, progress:100, result}
GW->>RD: 更新任务 success + 释放 agent 负载<br/>task:running 移除
GW-->>CLI: (前端可查)
else 推送失败(网络错误 / 非 202
AG-->>GW: 4xx/5xx 或无响应
GW->>RD: 回退:任务置 pending + 解绑 + 释放负载<br/>task:running→task:pending
end
end
Note over LOOP,RD: heartbeat_loop60s 扫 agent:heartbeat ZSet<br/>超 120s 未心跳 → 标记 offline<br/>timeout_loop5s 扫 running超时 → failed
```
---
## 三、规则调度决策流程Scheduler
```mermaid
flowchart LR
START(["dispatch_pending<br/>取 task:pending 批量"]) --> T1{"任务存在且为 pending?"}
T1 -- 否 --> END(["跳过"])
T1 -- 是 --> T2{"按 task_tags 标签匹配<br/>AgentRepo.by_tags(tags)"}
T2 -- 无候选 --> END2(["保持 pending<br/>等待下次调度"])
T2 -- 有候选 --> T3["负载过滤<br/>current_load < max_concurrent"]
T3 -- 空 --> END2
T3 -- 非空 --> T4["排序:最低负载优先<br/>负载相同时在线最久优先"]
T4 --> T5["绑定 Agent<br/>task:info 置 running + agent_id<br/>task:pending→task:running"]
T5 --> T6["agent 负载 +1<br/>adjust_load(agent, +1)"]
T6 --> T7["RelayService.dispatch_command<br/>POST {endpoint}/tasks/{id}"]
T7 --> T8{"HTTP 202?"}
T8 -- 是 --> OK(["下发成功<br/>记录 dispatch 日志"])
T8 -- 否 --> T9["回退task 置 pending + 解绑<br/>task:running→task:pending<br/>agent 负载 -1"]
T9 --> END2
```
---
## 四、Agent 端任务处理流程
```mermaid
flowchart TB
RECV["POST {endpoint}/tasks/{request_id}"] --> AUTH{"auth == GATEWAY_AUTH?"}
AUTH -- 否 --> 401["401 invalid auth"]
AUTH -- 是 --> THREAD["启动 daemon 后台线程<br/>(立即返回 202 accepted"]
THREAD --> PROMPT["payload → prompt<br/>prompt / cmd / 兜底序列化)"]
PROMPT --> RUN["InMemoryRunner.run_async<br/>user_id=gateway, session=task-{id}"]
RUN --> OUT["收集最终文本输出"]
OUT --> REP["gateway_client.report_result<br/>POST /api/agent/result"]
RUN --> ERR["异常捕获"]
ERR --> REPF["report_result(status=failed, error_info)"]
```
---
## 五、Redis 数据结构一览
| 前缀 / 键 | 类型 | 用途 |
| --- | --- | --- |
| `task:info:{request_id}` | Hash | 任务全量信息(状态/绑定 Agent/进度/结果TTL 24h |
| `task:pending` | Set | 待调度任务索引 |
| `task:running` | Set | 执行中任务索引 |
| `agent:info:{agent_id}` | Hash | Agent 信息endpoint/标签/并发上限/负载/心跳时间) |
| `agent:all` | Set | 全部 Agent 索引 |
| `agent:tag:{tag}` | Set | 能力标签 → Agent 索引(调度匹配用) |
| `agent:heartbeat` | ZSet | score=最后心跳时间戳(剔除离线用) |
| `lock:heartbeat` / `lock:scheduler` / `lock:timeout` | String | 分布式锁,防多实例重复执行后台循环 |
| `log:audit` | List | 全链路审计日志dispatch/progress/result |
---
## 六、技术栈与端口汇总
```mermaid
flowchart LR
subgraph 后端
B["Python + FastAPI + Uvicorn<br/>redis-py(asyncio) + httpx<br/>端口 8000"]
end
subgraph 前端
F["Vue 3 + Vite + Element Plus<br/>端口 5173/api 代理到 8000"]
end
subgraph Agent
A["Python + Google ADK v2.5<br/>FastAPI ApiServer + InMemoryRunner<br/>端口 8001"]
end
subgraph 存储
R["Redis远程 45.207.192.237:56987/0<br/>任务池 + Agent 池 + 索引 + 日志 + 锁"]
end
B <--> R
B <-->|"HTTP 推送 / 结果回传"| A
F -->|"/api/admin/*"| B
```
---
## 七、任务状态机
```mermaid
stateDiagram-v2
[*] --> pending: submit 受理(幂等)
pending --> running: 调度器选 Agent 并推送成功
pending --> pending: 无可用 Agent / 推送失败回退
running --> success: Agent 回传 result(success)
running --> failed: Agent 回传 failed
running --> failed: 超时 / 用户取消
pending --> failed: 用户取消
success --> pending: 手动重置(重新调度)
failed --> pending: 手动重置(重新调度)
success --> [*]: TTL 归档
failed --> [*]: TTL 归档
```