221 lines
7.2 KiB
Markdown
221 lines
7.2 KiB
Markdown
|
---
|
|||
|
|
tags: [grpc, production-readiness, retries, tls, performance-tuning]
|
|||
|
|
create time: 2026-05-07 16:00
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# 生产环境最佳实践
|
|||
|
|
|
|||
|
|
## 概述
|
|||
|
|
|
|||
|
|
本章汇总 gRPC 在生产部署时需要关注的各项配置与策略：重试机制、TLS/mTLS、代码生成工具链、性能调优。这些知识点往往是"知道能解决问题，不知道就是故障"的存在。
|
|||
|
|
|
|||
|
|
> [!question] 最后的思考题
|
|||
|
|
>
|
|||
|
|
> 如果你的 gRPC 服务 QPS 达到 10 万级别，但仍然发现延迟偏高，你觉得最可能的瓶颈在哪里？是序列化、网络、还是连接管理？带着这个问题去实际压测一遍，答案会比看十篇文章深刻。
|
|||
|
|
|
|||
|
|
## 重试策略
|
|||
|
|
|
|||
|
|
生产环境不建议无条件重试，但要针对可恢复错误配置自动重试：
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// 客户端配置自动重试
|
|||
|
|
retryPolicy := `{
|
|||
|
|
"retryPolicy": {
|
|||
|
|
"maxAttempts": 3,
|
|||
|
|
"initialBackoff": "0.1s",
|
|||
|
|
"maxBackoff": "1s",
|
|||
|
|
"backoffMultiplier": 2,
|
|||
|
|
"retryableStatusCodes": ["UNAVAILABLE", "DEADLINE_EXCEEDED"]
|
|||
|
|
}
|
|||
|
|
}`
|
|||
|
|
|
|||
|
|
conn, err := grpc.Dial(target,
|
|||
|
|
grpc.WithDefaultServiceConfig(retryPolicy),
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```mermaid
|
|||
|
|
flowchart LR
|
|||
|
|
attempt1["第1次调用<br/>UNAVAILABLE"] -->|指数退避 100ms| attempt2["第2次调用<br/>UNAVAILABLE"] -->|指数退避 200ms| attempt3["第3次调用<br/>SUCCESS"]
|
|||
|
|
|
|||
|
|
attempt1 -.->|INTERNAL → 不重试| final["终止"]
|
|||
|
|
|
|||
|
|
style attempt1 fill:#fff3e0
|
|||
|
|
style attempt2 fill:#ffe0b2
|
|||
|
|
style attempt3 fill:#c8e6c9
|
|||
|
|
style final fill:#ffcdd2
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 重试的安全边界
|
|||
|
|
|
|||
|
|
> [!warning] 重试的三条红线
|
|||
|
|
>
|
|||
|
|
> 1. **仅幂等操作**（GET、DELETE）可以安全重试
|
|||
|
|
> 2. **POST/create 操作**重试可能产生重复数据，必须在业务层加幂等键（如 `idempotency-key` header）
|
|||
|
|
> 3. 重试会增加**读放大**，特别是涉及 DB 的场景
|
|||
|
|
|
|||
|
|
| 操作类型 | 可重试？ | 注意事项 |
|
|||
|
|
|---------|---------|---------|
|
|||
|
|
| Query / Get | ✅ 安全 | 本身幂等，可放心重试 |
|
|||
|
|
| Update / Patch | ⚠️ 有条件 | 需要在 DB 层加乐观锁或唯一索引 |
|
|||
|
|
| Create / Insert | ❌ 谨慎 | 必须有幂等键机制，否则可能重复插入 |
|
|||
|
|
| Delete | ✅ 安全 | 幂等操作 |
|
|||
|
|
|
|||
|
|
## TLS / mTLS 配置
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
tlsConfig := &tls.Config{
|
|||
|
|
Certificates: []tls.Certificate{cert},
|
|||
|
|
ClientAuth: tls.RequireAndVerifyClientCert,
|
|||
|
|
ClientCertPool: certPool,
|
|||
|
|
MinVersion: tls.VersionTLS12,
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
对于大规模微服务，推荐使用 **mTLS**（双向证书认证）作为服务间信任的基础：
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// 服务端：需要客户端证书
|
|||
|
|
conn, _ := grpc.Dial(target,
|
|||
|
|
grpc.WithTransportCredentials(credentials.NewTLS(tlsConfig)),
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
// 客户端也需要携带自己的证书
|
|||
|
|
creds := credentials.NewTLS(tlsConfig)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!tip] Istio 集成
|
|||
|
|
>
|
|||
|
|
> 在服务网格中，mTLS 由 Sidecar Proxy（Envoy）自动处理，业务代码无需关心 TLS 细节。gRPC 连接经过 Sidecar 时透明加密，业务层仍然使用明文连接（localhost）。
|
|||
|
|
|
|||
|
|
## 代码生成工具链
|
|||
|
|
|
|||
|
|
```mermaid
|
|||
|
|
graph LR
|
|||
|
|
Proto["*.proto files"] --> Buf["buf generate"]
|
|||
|
|
Buf --> Go["go_proto_plugin<br/>生成 Go 代码"]
|
|||
|
|
Buf --> JS["js_proto_plugin<br/>生成 TS/JS 代码"]
|
|||
|
|
Buf --> Validate["validate.proto<br/>生成校验代码"]
|
|||
|
|
Buf --> GRPC["go_grpc_plugin<br/>生成 gRPC 桩"]
|
|||
|
|
|
|||
|
|
style Proto fill:#e3f2fd
|
|||
|
|
style Buf fill:#fff3e0
|
|||
|
|
style Go fill:#c8e6c9
|
|||
|
|
style JS fill:#fce4ec
|
|||
|
|
style GRPC fill:#e8f5e9
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### buf.gen.yaml 示例
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
version: v2
|
|||
|
|
plugins:
|
|||
|
|
- remote: buf.build/protocolbuffers/go
|
|||
|
|
out: gen/go
|
|||
|
|
opt: paths=source_relative
|
|||
|
|
- remote: buf.build/grpc/go
|
|||
|
|
out: gen/go
|
|||
|
|
opt: paths=source_relative
|
|||
|
|
- remote: buf.build/bufbuild/validate-go
|
|||
|
|
out: gen/go
|
|||
|
|
opt: paths=source_relative
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!tip] buf.lock —— 锁定依赖版本
|
|||
|
|
>
|
|||
|
|
> 像 go.mod 锁定 Go 模块版本一样，`buf.lock` 锁定 proto 依赖的确切 commit，避免上游变更导致构建不一致。CI 中应加入 `buf dep update --lock` 的检查步骤。
|
|||
|
|
|
|||
|
|
### CI/CD 集成
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
# GitHub Actions 示例
|
|||
|
|
- name: Generate gRPC code
|
|||
|
|
uses: bufbuild/buf-action@v1
|
|||
|
|
with:
|
|||
|
|
command: generate
|
|||
|
|
input: proto/
|
|||
|
|
|
|||
|
|
- name: Check generated code is up to date
|
|||
|
|
run: buf mod update && git diff --exit-code
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 配置管理
|
|||
|
|
|
|||
|
|
完整的 Dial 配置参考：
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
conn, err := grpc.DialContext(ctx, target,
|
|||
|
|
// 基础选项
|
|||
|
|
grpc.WithTransportCredentials(credentials.NewTLS(tlsConfig)),
|
|||
|
|
grpc.WithInitialWindowSize(1<<20), // 窗口大小 1MB
|
|||
|
|
grpc.WithInitialConnWindowSize(1<<20), // 连接窗口 1MB
|
|||
|
|
grpc.MaxCallRecvMsgSize(10 << 20), // 最大收包 10MB
|
|||
|
|
grpc.MaxCallSendMsgSize(10 << 20), // 最大发包 10MB
|
|||
|
|
grpc.WithConnectParams(grpc.ConnectParams{ // 连接参数
|
|||
|
|
MinConnectTimeout: 5 * time.Second,
|
|||
|
|
BackoffConfig: backoff.Config{
|
|||
|
|
BaseDelay: 100 * time.Millisecond,
|
|||
|
|
Multiplier: 1.6,
|
|||
|
|
MaxDelay: 3 * time.Second,
|
|||
|
|
},
|
|||
|
|
}),
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Dial 选项速查
|
|||
|
|
|
|||
|
|
| 配置项 | 默认值 | 建议值 | 作用域 |
|
|||
|
|
|--------|--------|--------|--------|
|
|||
|
|
| MaxCallRecvMsgSize | 4MB | 10MB | 仅客户端 |
|
|||
|
|
| MaxCallSendMsgSize | 4MB | 10MB | 仅服务端 |
|
|||
|
|
| InitialWindowSize | 64KB | 1MB（高吞吐） | 连接级 |
|
|||
|
|
| BackoffBaseDelay | 100ms | 100ms | 连接断开重连 |
|
|||
|
|
| BackoffMaxDelay | 10s | 3s | 连接断开重连 |
|
|||
|
|
|
|||
|
|
## 性能调优要点
|
|||
|
|
|
|||
|
|
| 优化项 | 建议值 | 影响 |
|
|||
|
|
|--------|--------|------|
|
|||
|
|
| 单个消息大小上限 | 4MB~10MB | 防止 OOM，超出则分块传 |
|
|||
|
|
| Initial Window Size | 1MB | 吞吐瓶颈时常需调大 |
|
|||
|
|
| Keepalive Time | 10~30s | 太短增加开销，太长被代理杀 |
|
|||
|
|
| Compressor | gzip（按需启用） | CPU vs 带宽权衡 |
|
|||
|
|
| Connection Pooling | 让 gRPC 自动管理 | 不要手动开连接 |
|
|||
|
|
|
|||
|
|
### gzip 压缩的取舍
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// 客户端对特定大 Payload 调用启用压缩
|
|||
|
|
resp, err := client.GetBigData(
|
|||
|
|
ctx,
|
|||
|
|
req,
|
|||
|
|
grpc.UseCompressor("gzip"),
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!summary] 何时启用 gzip
|
|||
|
|
>
|
|||
|
|
> | 场景 | 是否建议 gzip | 理由 |
|
|||
|
|
> |------|-------------|------|
|
|||
|
|
> | 小消息（< 1KB） | 否 | 压缩开销大于节省的带宽 |
|
|||
|
|
> | 大消息（> 10KB）且 CPU 充裕 | 是 | 带宽通常是更大瓶颈 |
|
|||
|
|
> | 高 QPS 短命请求 | 否 | CPU 反而成为瓶颈 |
|
|||
|
|
> | 跨数据中心调用 | 是 | 网络 RTT 高，减少数据传输量有意义 |
|
|||
|
|
|
|||
|
|
## 连接管理与 Keepalive 回顾
|
|||
|
|
|
|||
|
|
连接配置的最佳实践详见 [[06-连接管理]]。总结来说，生产环境至少要做到三件事：
|
|||
|
|
|
|||
|
|
1. **开启 Keepalive**，ping 间隔 10~30s，防止被代理切断
|
|||
|
|
2. **配置 MaxConnectionAge**，让旧连接平滑退役，新版本自动接盘
|
|||
|
|
3. **设置合理 backoff**，连接断开时指数退避重连，不要打满服务器
|
|||
|
|
|
|||
|
|
## 关联笔记
|
|||
|
|
|
|||
|
|
- [[01-协议与架构]] — 理解协议栈有助于调优每一个参数的意义
|
|||
|
|
- [[04-拦截器]] — 重试策略可以通过 interceptor 实现更复杂的逻辑
|
|||
|
|
- [[05-错误处理]] — 重试策略根据错误状态码来决定是否重试
|
|||
|
|
- [[06-连接管理]] — Keepalive、负载均衡、Name Resolver 的详细配置
|
|||
|
|
- [[02-服务治理/02-安全机制]] — mTLS 与服务间身份认证的更深内容
|
|||
|
|
- [[02-服务治理/06-容错模式]] — 熔断器、限流器与重试策略的组合配置
|