179 lines
6.9 KiB
Markdown
179 lines
6.9 KiB
Markdown
|
|
# 01 - 系统总览
|
|||
|
|
|
|||
|
|
> **一句话概括**:Gen2D 采用经典分层架构,Gin HTTP Server -> Handler -> Service -> 基础设施 -> 外部依赖,各层职责清晰、可独立替换。
|
|||
|
|
|
|||
|
|
## 架构全景
|
|||
|
|
|
|||
|
|
```mermaid
|
|||
|
|
graph TB
|
|||
|
|
subgraph Browser["浏览器 (React)"]
|
|||
|
|
UI["三栏工作台"]
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
subgraph Gin["Gin HTTP Server"]
|
|||
|
|
MW["中间件链<br/>Logger / Recovery / Metrics / Auth / RateLimit"]
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
subgraph Handler["Handler 层"]
|
|||
|
|
GH["Generate"]
|
|||
|
|
SH["SSE"]
|
|||
|
|
PH["Prompt"]
|
|||
|
|
OH["其他 Handler"]
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
subgraph Infra["基础设施层"]
|
|||
|
|
WP["WorkerPool<br/>协程池"]
|
|||
|
|
TQ["TaskQueue<br/>Memory / RabbitMQ"]
|
|||
|
|
EB["EventBus<br/>Pub/Sub"]
|
|||
|
|
RL["RateLimiter<br/>Redis 令牌桶"]
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
subgraph Pipeline["Service 层 (Eino Pipeline)"]
|
|||
|
|
PO["PromptOptimizer"]
|
|||
|
|
AG["AssetGenerator"]
|
|||
|
|
QS["QualitySupervisor"]
|
|||
|
|
FA["FormatAdapter"]
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
subgraph External["外部依赖"]
|
|||
|
|
DB["MySQL"]
|
|||
|
|
CDN["七牛云 Kodo"]
|
|||
|
|
REDIS["Redis"]
|
|||
|
|
LLM["LLM / Image API"]
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
UI -->|"HTTP / SSE"| MW
|
|||
|
|
MW --> Handler
|
|||
|
|
GH --> TQ
|
|||
|
|
GH --> WP
|
|||
|
|
SH --> EB
|
|||
|
|
TQ -->|"Consumer"| WP
|
|||
|
|
WP --> Pipeline
|
|||
|
|
PO --> AG --> QS --> FA
|
|||
|
|
AG --> LLM
|
|||
|
|
FA --> CDN
|
|||
|
|
Handler --> DB
|
|||
|
|
RL --> REDIS
|
|||
|
|
Pipeline -->|"Progress"| EB
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 分层详解
|
|||
|
|
|
|||
|
|
| 层级 | 目录 | 核心职责 | 代表组件 |
|
|||
|
|
|------|------|----------|----------|
|
|||
|
|
| **Entry / DI** | `cmd/main.go` | 启动入口、依赖注入、信号处理 | `main()` |
|
|||
|
|
| **HTTP 层** | `internal/handler/` | 请求绑定、参数校验、响应封装 | `Generate`, `SSEHandler`, `PromptOptimize` |
|
|||
|
|
| **中间件** | `internal/mildware/` | 日志、恢复、指标采集、认证、限流 | `Logger`, `Recovery`, `Metrics`, `AuthMiddleware` |
|
|||
|
|
| **业务层** | `internal/service/` | 生成管线、提示词优化、质检、推理调用 | `RunPipeline`, `CheckQuality`, `GenerateImages` |
|
|||
|
|
| **领域模型** | `internal/model/` | 数据库实体、响应体定义 | `User`, `Project`, `Task`, `Asset` |
|
|||
|
|
| **基础设施** | `internal/pkg/` | 协程池、任务队列、事件总线、限流器 | `workerpool.Pool`, `taskqueue.TaskQueue`, `eventbus.Broker` |
|
|||
|
|
| **精灵处理** | `pkg/splitsprite/`, `pkg/gifmaker/` | 精灵表切割、GIF 预览生成 | `splitsprite.Process`, `gifmaker.Encode` |
|
|||
|
|
| **配置** | `internal/config/` | YAML 加载、环境变量绑定、默认值 | `config.Load()` |
|
|||
|
|
|
|||
|
|
## 依赖注入模式
|
|||
|
|
|
|||
|
|
Gen2D 采用轻量级的 **Set\*/Init\* 函数注入** 模式,避免引入 DI 框架。
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
main.go 中的注入链路:
|
|||
|
|
|
|||
|
|
config.Load() → 加载全局配置
|
|||
|
|
handler.InitAuthService() → 注入 JWT 配置
|
|||
|
|
service.InitLLMConfig() → 注入 LLM 配置
|
|||
|
|
service.InitImageGenConfig() → 注入文生图配置
|
|||
|
|
handler.InitStorageService() → 注入存储服务
|
|||
|
|
handler.SetWorkerPool() → 注入协程池
|
|||
|
|
handler.SetTaskQueue() → 注入任务队列
|
|||
|
|
eventbus.Init() → 初始化事件总线
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**设计要点**:
|
|||
|
|
|
|||
|
|
- Handler 层不直接 import `workerpool`、`taskqueue` 等基础设施包,仅通过注入的接口交互
|
|||
|
|
- `service.InitImageGenConfig()` 将配置缓存为包级变量,避免在函数签名中传递大量参数
|
|||
|
|
- 每个 `Set*` 函数对应一个包级全局变量,简单但足够清晰
|
|||
|
|
|
|||
|
|
> :bulb: **为什么不用 Wire / Fx?** 项目规模可控,`cmd/main.go` 约 220 行即可完成全部注入,框架级 DI 的复杂度收益比不高。
|
|||
|
|
|
|||
|
|
## 配置级联
|
|||
|
|
|
|||
|
|
Gen2D 使用 Viper 实现三层配置覆盖,优先级从高到低:
|
|||
|
|
|
|||
|
|
```mermaid
|
|||
|
|
graph LR
|
|||
|
|
ENV["环境变量<br/>GEN2D_*"] -->|"最高优先级"| VIPER["Viper"]
|
|||
|
|
YAML["YAML 配置文件<br/>config.yaml"] -->|"中等优先级"| VIPER
|
|||
|
|
DEFAULT["代码默认值<br/>SetDefault()"] -->|"最低优先级"| VIPER
|
|||
|
|
VIPER --> CONFIG["Config 结构体"]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**覆盖规则**:`环境变量 > YAML 文件 > 默认值`
|
|||
|
|
|
|||
|
|
| 配置源 | 示例 | 说明 |
|
|||
|
|
|--------|------|------|
|
|||
|
|
| 环境变量 | `GEN2D_PORT=9090` | 部署时覆盖,适合容器化场景 |
|
|||
|
|
| YAML 文件 | `server.port: 8080` | 开发时配置,集中管理 |
|
|||
|
|
| 默认值 | `v.SetDefault("server.port", 8080)` | 代码内置,零配置即可启动 |
|
|||
|
|
|
|||
|
|
配置结构体涵盖 10 个子模块:
|
|||
|
|
|
|||
|
|
| 配置块 | 对应环境变量前缀 | 关键字段 |
|
|||
|
|
|--------|------------------|----------|
|
|||
|
|
| `server` | `GEN2D_*` | port, mode, max_file_size |
|
|||
|
|
| `database` | `GEN2D_DSN` | dsn |
|
|||
|
|
| `jwt` | `GEN2D_JWT_*` | secret, expire |
|
|||
|
|
| `log` | `GEN2D_LOG_*` | level, format |
|
|||
|
|
| `redis` | `GEN2D_REDIS_*` | addr, password, db |
|
|||
|
|
| `llm` | `GEN2D_LLM_*` | base_url, api_key, model |
|
|||
|
|
| `image_gen` | `GEN2D_IMAGE_*` | base_url, api_key, model, timeout, max_retries |
|
|||
|
|
| `qiniu` | `GEN2D_QINIU_*` | access_key, secret_key, bucket, cdn_host |
|
|||
|
|
| `workerpool` | `GEN2D_WORKERPOOL_*` | workers, queue_size, max_per_user |
|
|||
|
|
| `taskqueue` | `GEN2D_TASKQUEUE_*` | driver, memory.buffer_size, rabbitmq.* |
|
|||
|
|
|
|||
|
|
## 请求全链路
|
|||
|
|
|
|||
|
|
一个素材生成请求的完整生命周期:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
1. 浏览器 → POST /api/v1/generate
|
|||
|
|
2. Gin 中间件链 → Logger → Recovery → Metrics → Auth → RateLimit
|
|||
|
|
3. Handler.Generate()
|
|||
|
|
├── 参数绑定 & 校验
|
|||
|
|
├── 保存任务到 MySQL(status=pending)
|
|||
|
|
├── 构建 TaskMessage
|
|||
|
|
└── 提交到 TaskQueue(优先)或 WorkerPool(fallback)
|
|||
|
|
4. 返回 { taskId } 给浏览器
|
|||
|
|
5. Consumer 从 TaskQueue 消费消息
|
|||
|
|
6. Consumer → WorkerPool.Submit()
|
|||
|
|
7. WorkerPool 的 Worker 执行任务:
|
|||
|
|
├── 注入 ProgressReporter 到 context
|
|||
|
|
├── RunPipeline() — Eino Graph 执行
|
|||
|
|
│ ├── PromptOptimizer(合并风格 + 技术参数)
|
|||
|
|
│ ├── AssetGenerator(调用 AI 出图 API)
|
|||
|
|
│ ├── QualitySupervisor(质检,最多重试 3 次)
|
|||
|
|
│ └── FormatAdapter(精灵表切割 + GIF 预览)
|
|||
|
|
├── 上传素材到七牛云
|
|||
|
|
└── 更新 MySQL 任务状态
|
|||
|
|
8. EventBus.Publish() → SSE 推送给浏览器
|
|||
|
|
9. 浏览器通过 EventSource 实时接收进度
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 关键设计决策
|
|||
|
|
|
|||
|
|
| 决策 | 选择 | 理由 |
|
|||
|
|
|------|------|------|
|
|||
|
|
| 分层架构 | Handler-Service-Infra 三层 | 解耦各层职责,便于独立测试和替换 |
|
|||
|
|
| 依赖注入 | Set\*/Init\* 函数 | 轻量、零依赖,项目规模可控 |
|
|||
|
|
| 配置管理 | Viper 三层级联 | 容器化友好,零配置可启动 |
|
|||
|
|
| 异步任务 | 提交-队列-消费-执行 | API 快速返回,长任务不阻塞请求 |
|
|||
|
|
| 事件推送 | EventBus + SSE | 比 WebSocket 轻量,HTTP 原生支持 |
|
|||
|
|
| 限流 | Redis 令牌桶 | 分布式一致,Fail-Open 保可用性 |
|
|||
|
|
|
|||
|
|
## 关联文档
|
|||
|
|
|
|||
|
|
- [索引](00-index.md) — 文档导航与架构总览图
|
|||
|
|
- [协程池](02-worker-pool.md) — 有界并发与 per-user 限流
|
|||
|
|
- [任务队列](03-task-queue.md) — 可插拔队列接口与双实现
|
|||
|
|
- [RabbitMQ 集成](04-rabbitmq.md) — 持久化消息与重试机制
|
|||
|
|
- [生成管线](05-generation-pipeline.md) — Eino 4 阶段管线与质量回退
|