Init
This commit is contained in:
@@ -0,0 +1,155 @@
|
||||
---
|
||||
tags: [microservice, docker, container, image-optimization]
|
||||
create time: 2026-05-05
|
||||
---
|
||||
|
||||
# 容器化
|
||||
|
||||
## 概述
|
||||
|
||||
Docker 容器是微服务交付的标准单元。它解决了"在我机器上是好的"这个经典问题——**开发、测试、生产使用完全一致的运行时环境**。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
CODE["源代码"] --> BUILD["CI 构建镜像"]
|
||||
BUILD --> REGISTRY["镜像仓库<br/>Harbor / ECR / ACR"]
|
||||
REGISTRY --> K8s["Kubernetes 部署"]
|
||||
|
||||
style BUILD fill:#e3f2fd
|
||||
style REGISTRY fill:#fff3e0
|
||||
style K8s fill:#e8f5e9
|
||||
```
|
||||
|
||||
## Dockerfile 最佳实践
|
||||
|
||||
### Go 多阶段构建(推荐)
|
||||
|
||||
```dockerfile
|
||||
# ========== 阶段 1: 构建 ==========
|
||||
FROM golang:1.22-alpine AS builder
|
||||
|
||||
RUN apk --no-cache add git ca-certificates
|
||||
|
||||
WORKDIR /app
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
COPY . .
|
||||
ARG LDFLAGS="-s -w -extldflags '-static'"
|
||||
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags "$LDFLAGS" -o server .
|
||||
|
||||
# ========== 阶段 2: 运行时 ==========
|
||||
FROM alpine:latest
|
||||
|
||||
RUN apk --no-cache add ca-certificates tzdata && \
|
||||
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
|
||||
echo "Asia/Shanghai" > /etc/timezone
|
||||
|
||||
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
|
||||
USER appuser
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=builder /app/server .
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
|
||||
CMD wget -qO- http://localhost:8080/healthz || exit 1
|
||||
|
||||
CMD ["./server"]
|
||||
```
|
||||
|
||||
### 关键优化点
|
||||
|
||||
| 优化项 | 方法 | 效果 |
|
||||
|--------|------|------|
|
||||
| **多阶段构建** | 编译和运行分离 | 镜像从 800MB → 15MB |
|
||||
| **alpine 基础镜像** | 替代 debian/ubuntu | 减小体积 |
|
||||
| **非 root 运行** | `USER appuser` | 安全合规 |
|
||||
| **静态链接** | `CGO_ENABLED=0` | 不依赖系统库 |
|
||||
| **layer cache** | `go.mod` 先 COPY | CI 加速构建 |
|
||||
| **健康检查** | HEALTHCHECK 指令 | K8s 原生支持 |
|
||||
|
||||
### `.dockerignore`
|
||||
|
||||
```
|
||||
.git
|
||||
.gitignore
|
||||
*.md
|
||||
vendor/
|
||||
tests/
|
||||
*.log
|
||||
.DS_Store
|
||||
.idea/
|
||||
.vscode/
|
||||
```
|
||||
|
||||
> [!tip] 为什么 .dockerignore 很重要?
|
||||
>
|
||||
> 如果不排除 `.git` 目录,整个版本历史都会被打包进镜像(增加数百 MB)。如果排除不当,可能遗漏必要的配置文件。
|
||||
|
||||
## 镜像安全
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Scan["镜像扫描 (Trivy/Snyk)"] --> Clean{"漏洞等级?"}
|
||||
|
||||
Clean -- "CRITICAL/HIGH" --> Block["❌ 阻止推送"]
|
||||
Clean -- "MEDIUM" --> Review["⚠️ 人工审核"]
|
||||
Clean -- "LOW/INFO" --> Allow["✅ 允许推送"]
|
||||
|
||||
style Block fill:#ffebee
|
||||
style Review fill:#fff3e0
|
||||
style Allow fill:#e8f5e9
|
||||
```
|
||||
|
||||
### 安全检查清单
|
||||
|
||||
- [ ] 不使用 `latest` tag(永远用具体版本号)
|
||||
- [ ] 不安装不必要的软件包(`apk del --purge .build-deps`)
|
||||
- [ ] 定期更新基础镜像(修复 CVE)
|
||||
- [ ] 使用非 root 用户运行
|
||||
- [ ] 镜像不包含密钥、密码、私钥
|
||||
- [ ] 使用最小基础镜像(scratch / distroless)
|
||||
|
||||
### Distroless 镜像
|
||||
|
||||
Google 推出的**无 shell、无包管理器**的极简运行时镜像:
|
||||
|
||||
```dockerfile
|
||||
# 最极致的精简
|
||||
FROM gcr.io/distroless/static-debian12
|
||||
COPY --from=builder /app/server .
|
||||
USER nonroot
|
||||
CMD ["./server"]
|
||||
```
|
||||
|
||||
> ⚠️ 注意:distroless 镜像没有 shell (`/bin/sh`),调试时需要额外工具(如 debug 镜像或 `kubectl exec` 到 sidecar)。
|
||||
|
||||
## 镜像仓库管理
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Dev["开发者本地"] -->|"docker push"| DEV_REPO["dev 仓库<br/>v1.2.3-dev"]
|
||||
|
||||
Staging["Staging 验证"] -->|"通过"| PROD_REPO["prod 仓库<br/>v1.2.3"]
|
||||
|
||||
K8s["K8s Cluster"] -->|"pull"| PROD_REPO
|
||||
|
||||
PR["PR Merge"] --> Tag["打标签 v1.2.3"]
|
||||
Tag --> ProdRepoMove["移动到 prod 仓库"]
|
||||
|
||||
style DEV_REPO fill:#fff3e0
|
||||
style PROD_REPO fill:#e8f5e9
|
||||
```
|
||||
|
||||
| 仓库方案 | 特点 | 适用场景 |
|
||||
|---------|------|---------|
|
||||
| **Harbor** | 自托管,RBAC + 扫描 | 国内企业首选 |
|
||||
| **ECR / ACR / GCR** | 云厂商托管 | 全云环境 |
|
||||
| **Docker Hub** | 公共免费 | 开源项目 |
|
||||
|
||||
## 关联笔记
|
||||
|
||||
- [[05-部署运维/02-Kubernetes]] — K8s 以 Pod 为部署单元,镜像来自 Docker
|
||||
- [[05-部署运维/03-CICD与GitOps]] — CI/CD 流水线中的镜像构建环节
|
||||
@@ -0,0 +1,266 @@
|
||||
---
|
||||
tags: [microservice, kubernetes, k8s, container-orchestration]
|
||||
create time: 2026-05-05
|
||||
---
|
||||
|
||||
# Kubernetes
|
||||
|
||||
## 概述
|
||||
|
||||
Kubernetes (K8s) 是微服务架构的事实标准编排引擎。它将容器化的服务组织成声明式的资源对象,自动处理部署、扩展、故障恢复。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph CLUSTER["K8s Cluster"]
|
||||
Master["控制面<br/>API Server / Scheduler / Controller Manager / etcd"]
|
||||
|
||||
subgraph NODES["工作节点"]
|
||||
N1["Node A<br/>kubelet + kube-proxy"]
|
||||
N2["Node B<br/>kubelet + kube-proxy"]
|
||||
end
|
||||
|
||||
Master --> N1
|
||||
Master --> N2
|
||||
|
||||
subgraph APPS["应用层"]
|
||||
Order["Order Service Deployment"]
|
||||
Pay["Payment Service Deployment"]
|
||||
end
|
||||
|
||||
N1 --> Order
|
||||
N2 --> Order
|
||||
N1 --> Pay
|
||||
N2 --> Pay
|
||||
end
|
||||
|
||||
SVC["Service (ClusterIP)"] -->|Load Balance| Order
|
||||
Ingress["Ingress (HTTP Routing)"] --> SVC
|
||||
|
||||
style Master fill:#e3f2fd
|
||||
style N1 fill:#fff3e0
|
||||
style N2 fill:#fff3e0
|
||||
```
|
||||
|
||||
## 核心概念速查
|
||||
|
||||
| K8s 对象 | 用途 | 类比 |
|
||||
|---------|------|------|
|
||||
| **Pod** | 最小部署单元,包含一个或多个容器 | 应用实例 |
|
||||
| **Deployment** | 管理 Pod 的副本数和滚动更新 | 应用的"模板" |
|
||||
| **Service** | 稳定的网络入口,负载均衡 | 内部 VIP |
|
||||
| **Ingress** | HTTP/HTTPS 路由规则 | 外部网关 |
|
||||
| **ConfigMap** | 配置注入(明文) | 环境变量/配置文件 |
|
||||
| **Secret** | 敏感配置注入(base64) | 密码/API Key |
|
||||
| **HPA** | 根据指标自动扩缩容 | 弹性伸缩 |
|
||||
| **StatefulSet** | 有状态应用的有序管理 | DB、ZK |
|
||||
| **Job/CronJob** | 一次性任务 / 定时任务 | 批处理 |
|
||||
|
||||
## Deployment 详解
|
||||
|
||||
### 完整示例
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: order-service
|
||||
labels:
|
||||
app: order
|
||||
version: v1.2.3
|
||||
spec:
|
||||
replicas: 3 # 期望副本数
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: 1 # 最多超额 1 个 Pod
|
||||
maxUnavailable: 0 # 滚动更新期间不允许不可用
|
||||
|
||||
selector:
|
||||
matchLabels:
|
||||
app: order
|
||||
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: order
|
||||
version: v1.2.3
|
||||
spec:
|
||||
containers:
|
||||
- name: order-service
|
||||
image: registry.example.com/order:v1.2.3
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
|
||||
# ========== 资源配置 ==========
|
||||
resources:
|
||||
requests: # 调度依据:保证至少有这些
|
||||
cpu: "250m"
|
||||
memory: "256Mi"
|
||||
limits: # 硬上限:超过则 OOMKill/CPU Throttle
|
||||
cpu: "500m"
|
||||
memory: "512Mi"
|
||||
|
||||
# ========== 探针 ==========
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 8080
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 10
|
||||
failureThreshold: 3 # 连续失败 3 次才重启
|
||||
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ready
|
||||
port: 8080
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
failureThreshold: 3
|
||||
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 8080
|
||||
failureThreshold: 30 # 最长等待 300s (慢启动友好)
|
||||
|
||||
# ========== 环境变量 & 挂载 ==========
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: order-service-config
|
||||
- secretRef:
|
||||
name: order-service-secrets
|
||||
|
||||
lifecycle:
|
||||
preStop:
|
||||
exec:
|
||||
command: ["sh", "-c", "sleep 15"] # 优雅退出,给 LB 摘流时间
|
||||
```
|
||||
|
||||
### Probe 选择指南
|
||||
|
||||
| 探针类型 | 触发条件 | 动作 | 适用场景 |
|
||||
|---------|---------|------|---------|
|
||||
| **Liveness** | `/healthz` 返回非 2xx | 重启容器 | 死锁、无法恢复的崩溃 |
|
||||
| **Readiness** | `/ready` 返回非 2xx | 摘除 Service 流量 | 依赖未就绪、热加载中 |
|
||||
| **Startup** | 首次成功前持续失败 | 不重启,只等待 | 大模型/JVM 冷启动 |
|
||||
|
||||
> [!warning] 经典陷阱:CrashLoopBackOff
|
||||
>
|
||||
> 如果 Liveness Probe 因为 DB 连接超时而返回 503,K8s 会认为容器挂了并反复重启它——这就是 CrashLoopBackOff。正确做法是:让 `/healthz` 做降级判断(DB 不可用时返回 200),用 `/ready` 来摘除流量。
|
||||
|
||||
## Service 与 Ingress
|
||||
|
||||
### Service 类型
|
||||
|
||||
| 类型 | 特点 | 使用场景 |
|
||||
|------|------|---------|
|
||||
| **ClusterIP** | 集群内 IP,外部不可访问 | 默认,内部服务间调用 |
|
||||
| **NodePort** | 在每个 Node 上开端口 | 调试、临时访问 |
|
||||
| **LoadBalancer** | 云厂商分配公网 IP | 对外暴露的服务 |
|
||||
| **ExternalName** | CNAME 到外部域名 | 对接外部系统 |
|
||||
|
||||
```yaml
|
||||
# ClusterIP Service — 服务发现的载体
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: order-service
|
||||
spec:
|
||||
selector:
|
||||
app: order
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 8080
|
||||
protocol: TCP
|
||||
type: ClusterIP
|
||||
```
|
||||
|
||||
调用方只需 `http://order-service:80`,K8s 通过 iptables/IPVS 自动实现负载均衡。
|
||||
|
||||
### Ingress — HTTP 路由
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: main-ingress
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/rewrite-target: /
|
||||
spec:
|
||||
rules:
|
||||
- host: api.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /orders
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: order-service
|
||||
port:
|
||||
number: 80
|
||||
- path: /users
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: user-service
|
||||
port:
|
||||
number: 80
|
||||
```
|
||||
|
||||
## HPA 弹性伸缩
|
||||
|
||||
```yaml
|
||||
apiVersion: autoscaling/v2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: order-service-hpa
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: order-service
|
||||
minReplicas: 3
|
||||
maxReplicas: 20
|
||||
metrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 70 # CPU > 70% 时扩容
|
||||
- type: Resource
|
||||
resource:
|
||||
name: memory
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 80
|
||||
behavior:
|
||||
scaleUp:
|
||||
stabilizationWindowSeconds: 60 # 扩容稳定期
|
||||
policies:
|
||||
- type: Pods
|
||||
value: 2
|
||||
periodSeconds: 60 # 每分钟最多扩 2 个
|
||||
scaleDown:
|
||||
stabilizationWindowSeconds: 300 # 缩容稳定期 5min(防抖动)
|
||||
```
|
||||
|
||||
## K8s 运维 Checklist
|
||||
|
||||
每次上线前过一遍这个清单:
|
||||
|
||||
| # | 检查项 | 说明 |
|
||||
|---|--------|------|
|
||||
| 1 | **Probe 已配置** | liveness/readiness/startup 都设定了阈值 |
|
||||
| 2 | **Resources Limits** | 防止单个 Pod OOMKill 拖垮整台机器 |
|
||||
| 3 | **日志输出到 stdout/stderr** | 可被采集器解析为 JSON |
|
||||
| 4 | **trace_id 透传** | 跨服务调用链 trace_id 不丢失 |
|
||||
| 5 | **回滚预案** | `kubectl rollout undo deployment/order-service` 能用 |
|
||||
| 6 | **告警已配置** | 关键指标异常时有人收到通知 |
|
||||
| 7 | **镜像 Tag** | 不用 latest,用语义化版本或 commit SHA |
|
||||
|
||||
## 关联笔记
|
||||
|
||||
- [[02-服务治理/04-服务发现]] — K8s Service 是服务端发现模式的代表
|
||||
- [[02-服务治理/08-流量治理]] — Istio VirtualService 在 K8s 上的高级路由
|
||||
- [[05-部署运维/04-SRE实践]] — SLO/Error Budget 在 K8s 中的落地
|
||||
@@ -0,0 +1,176 @@
|
||||
---
|
||||
tags: [microservice, cicd, gitops, github-actions, argocd]
|
||||
create time: 2026-05-05
|
||||
---
|
||||
|
||||
# CI/CD 与 GitOps
|
||||
|
||||
## 概述
|
||||
|
||||
微服务需要**独立部署**。上百个服务的手工发布是不可想象的——必须用自动化流水线保证每次变更都能安全、快速地推送到生产环境。
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
CODE["代码提交"] --> TEST["测试 & 静态分析"]
|
||||
TEST --> SCAN["安全扫描"]
|
||||
SCAN --> BUILD["构建镜像"]
|
||||
BUILD --> PUSH["推送仓库"]
|
||||
PUSH --> STAGING["Staging 验证"]
|
||||
STAGING -->|"人工审批"| PROD["Production 部署"]
|
||||
|
||||
style CODE fill:#e3f2fd
|
||||
style TEST fill:#fff3e0
|
||||
style SCAN fill:#fce4ec
|
||||
style BUILD fill:#e8f5e9
|
||||
style PUSH fill:#f3e5f5
|
||||
style STAGING fill:#e0f7fa
|
||||
style PROD fill:#c8e6c9
|
||||
```
|
||||
|
||||
## CI/CD 设计原则
|
||||
|
||||
| 原则 | 说明 |
|
||||
|------|------|
|
||||
| **一次构建,多处部署** | 镜像不随环境重新编译,只改 K8s ConfigMap/环境变量 |
|
||||
| **语义化版本** | 镜像 tag 用 `v1.2.3`,tag 即版本溯源 |
|
||||
| **路径过滤** | 只对相关服务的代码变更触发构建 |
|
||||
| **Commit SHA 作为镜像 tag** | 保证精确回滚 |
|
||||
| **分阶段部署** | Staging → Production 的审批关卡不可跳过 |
|
||||
|
||||
## GitHub Actions Pipeline 实战
|
||||
|
||||
```yaml
|
||||
# .github/workflows/deploy.yml
|
||||
name: Deploy order-service
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "services/order/**"
|
||||
|
||||
env:
|
||||
REGISTRY: registry.example.com
|
||||
IMAGE: order-service
|
||||
|
||||
jobs:
|
||||
# ========== Stage 1: Build & Test ==========
|
||||
build-and-test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Run tests
|
||||
run: make test
|
||||
|
||||
- name: Security scan
|
||||
uses: aquasecurity/trivy-action@master
|
||||
with:
|
||||
scan-type: 'fs'
|
||||
severity: 'CRITICAL,HIGH'
|
||||
|
||||
- name: Build Docker image
|
||||
run: |
|
||||
docker build -t ${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ github.sha }} \
|
||||
-t ${{ env.REGISTRY }}/${{ env.IMAGE }}:v${{ github.run_number }} \
|
||||
-f services/order/Dockerfile \
|
||||
services/order
|
||||
|
||||
- name: Push to registry
|
||||
run: |
|
||||
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login -u ${{ secrets.REGISTRY_USER }} --password-stdin
|
||||
docker push ${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ github.sha }}
|
||||
docker push ${{ env.REGISTRY }}/${{ env.IMAGE }}:v${{ github.run_number }}
|
||||
|
||||
# ========== Stage 2: Deploy to Staging ==========
|
||||
deploy-staging:
|
||||
needs: build-and-test
|
||||
runs-on: ubuntu-latest
|
||||
environment: staging
|
||||
steps:
|
||||
- name: Deploy to staging
|
||||
run: |
|
||||
kubectl set image deployment/order-service \
|
||||
order=${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ github.sha }} \
|
||||
--namespace=staging
|
||||
kubectl rollout status deployment/order-service \
|
||||
--namespace=staging --timeout=120s
|
||||
|
||||
# ========== Stage 3: Deploy to Production ==========
|
||||
deploy-production:
|
||||
needs: deploy-staging
|
||||
runs-on: ubuntu-latest
|
||||
environment: production
|
||||
steps:
|
||||
- name: Canary release (10% → 50% → 100%)
|
||||
run: |
|
||||
kubectl patch canary order-service --type merge \
|
||||
-p '{"spec":{"weight":10}}'
|
||||
|
||||
# ... 等待监控确认,逐步放大流量
|
||||
echo "Monitor metrics before proceeding..."
|
||||
```
|
||||
|
||||
## GitOps 工作流
|
||||
|
||||
GitOps 的核心思想:**K8s 集群的状态 = Git 仓库中声明式配置的当前状态。**
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Dev["开发者 PR"] -->|"修改 K8s manifest"| Git[(Git Repo)]
|
||||
|
||||
subgraph Cluster["K8s Cluster"]
|
||||
Argo["ArgoCD / Flux"] -->|同步| K8sState["Pod/Service/ConfigMap"]
|
||||
end
|
||||
|
||||
Git -.->|Webhook| Argo
|
||||
|
||||
Argo -->|"检测到差异"| Diff{"状态一致?"}
|
||||
Diff -- 否 --> Sync["自动同步到 K8s ✅"]
|
||||
Diff -- 是 --> OK["已一致 ⏸️"]
|
||||
|
||||
style Git fill:#e3f2fd
|
||||
style Argo fill:#fff3e0
|
||||
style K8sState fill:#e8f5e9
|
||||
```
|
||||
|
||||
### 与传统 CI/CD 的区别
|
||||
|
||||
| 维度 | 传统 CI/CD | GitOps |
|
||||
|------|-----------|--------|
|
||||
| **部署驱动** | CI 服务器主动推送 | Git 仓库变动触发拉取 |
|
||||
| **状态源** | CI pipeline 的历史记录 | Git commit history |
|
||||
| **回滚方式** | 回到上一次的 pipeline | `git revert` + 自动同步 |
|
||||
| **漂移检测** | 通常无 | 持续比对,自动修复不一致 |
|
||||
| **代表工具** | Jenkins / GitLab CI / GitHub Actions | ArgoCD / Flux |
|
||||
|
||||
### GitOps 的优势
|
||||
|
||||
1. **审计完整** — 所有变更都在 Git 中可追溯
|
||||
2. **回滚简单** — `git revert` 就是回滚操作
|
||||
3. **自修复** — ArgoCD 持续监测并修复集群状态偏离
|
||||
4. **多人协作** — 通过 PR Review 流程管控配置变更
|
||||
|
||||
## Helm — K8s 的包管理
|
||||
|
||||
当每个服务都有几十行 YAML 时,Helm 能大幅简化部署:
|
||||
|
||||
```bash
|
||||
# 创建 Chart 模板
|
||||
helm create order-service
|
||||
|
||||
# 使用 values.yaml 参数化部署
|
||||
helm upgrade --install order-service ./charts/order-service \
|
||||
--set image.tag=v1.2.3 \
|
||||
--set replicas=3 \
|
||||
--namespace=production
|
||||
```
|
||||
|
||||
> [!tip] 为什么需要 Helm?
|
||||
>
|
||||
> 没有 Helm 时,每个服务都要手动维护 Deployment、Service、Ingress、ConfigMap 等几十个 YAML 文件。Helm 允许你把通用模板抽出来,只在 values.yaml 里改差异化配置。
|
||||
|
||||
## 关联笔记
|
||||
|
||||
- [[05-部署运维/01-容器化]] — Docker 镜像构建是 CI/CD 的第一步
|
||||
- [[05-部署运维/02-Kubernetes]] — K8s 是部署的目标平台
|
||||
- [[05-部署运维/04-SRE实践]] — 错误预算影响发布策略
|
||||
@@ -0,0 +1,160 @@
|
||||
---
|
||||
tags: [microservice, sre, slo, error-budget, postmortem]
|
||||
create time: 2026-05-05
|
||||
---
|
||||
|
||||
# SRE 实践
|
||||
|
||||
## 概述
|
||||
|
||||
站点可靠性工程 (SRE) 把运维问题看作**软件工程问题**。它的核心理念是用 SLI/SLO/SLA 来量化服务质量,避免"我觉得系统很慢"这类模糊描述。
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
User["用户体验"] --> SLI{"实际测量"}
|
||||
SLI -->|"达标"| SLO_OK["✅ SLO 达成"]
|
||||
SLI -->|"不达标"| Budget["消耗错误预算"]
|
||||
|
||||
Budget --> Left{"预算剩余?"}
|
||||
Left -- "> 50%" --> Ship["快速迭代 🚀"]
|
||||
Left -- "< 10%" --> Stabilize["稳定优先 ❄️"]
|
||||
|
||||
style User fill:#e3f2fd
|
||||
style SLO_OK fill:#e8f5e9
|
||||
style Ship fill:#fff3e0
|
||||
style Stabilize fill:#ffebee
|
||||
```
|
||||
|
||||
## SLI / SLO / SLA 详解
|
||||
|
||||
### 三层模型
|
||||
|
||||
| 术语 | 全称 | 定义 | 谁制定 | 变更频率 |
|
||||
|------|------|------|--------|---------|
|
||||
| **SLI** | Service Level Indicator | 实际度量:用户请求的成功率是多少? | 观测系统自动产出 | 持续更新 |
|
||||
| **SLO** | Service Level Objective | 内部目标:我们承诺达到 99.9% 可用性 | SRE + 研发 | 季度回顾 |
|
||||
| **SLA** | Service Level Agreement | 对外契约:达不到就赔钱 | 法务 + 商务 | 按合同约定 |
|
||||
|
||||
### 实用性比例对照表
|
||||
|
||||
| SLO | 每年停机时间 | 每月停机时间 | 适用级别 |
|
||||
|-----|-------------|-------------|---------|
|
||||
| **99%** | ~87.6 小时 | ~7.2 小时 | 内部工具(几乎无意义) |
|
||||
| **99.9% ("三个九")** | ~8.76 小时 | ~43 分钟 | 大多数后端服务 ✅ |
|
||||
| **99.95%** | ~4.38 小时 | ~21 分钟 | 核心交易链路 |
|
||||
| **99.99% ("四个九")** | ~52.6 分钟 | ~4.3 分钟 | 金融级 / 支付系统 |
|
||||
| **99.999%** | ~5.26 分钟 | ~26 秒 | 电信级,极难实现 |
|
||||
|
||||
> [!warning] "三个九"是底线
|
||||
>
|
||||
> 如果团队宣称 SLO = 99%,这意味着每月可以容忍近 **7 小时的不可用**——这在生产环境中基本等于没有可用性目标。
|
||||
|
||||
## Error Budget (错误预算) 深度解析
|
||||
|
||||
### 计算方式
|
||||
|
||||
```
|
||||
可用率 = (总时间 - 故障时间) / 总时间 × 100%
|
||||
错误预算 = 1 - SLO
|
||||
|
||||
SLO = 99.9% → 错误预算 = 0.001
|
||||
每月可容忍故障 = 30 × 24 × 60 × 0.001 = 43.2 分钟
|
||||
每周可容忍故障 = 7 × 24 × 60 × 0.001 = 10.1 分钟
|
||||
|
||||
SLO = 99.99% → 错误预算 = 0.0001
|
||||
每月可容忍故障 ≈ 4.3 分钟
|
||||
```
|
||||
|
||||
### 基于错误预算的决策矩阵
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Ratio["错误预算使用率 = 已消耗 / 总预算"]
|
||||
|
||||
Ratio -- "< 50%" --> Green["🟢 绿灯阶段<br/>预算充足: 可以大胆发布<br/>新功能、尝试激进方案<br/>常规发布节奏"]
|
||||
Ratio -- "50~90%" --> Yellow["🟡 黄灯阶段<br/>预算紧张: 收紧发布频率<br/>增加人工审查<br/>暂停非关键功能开发"]
|
||||
Ratio -- "\> 90%" --> Orange["🟠 橙灯阶段<br/>严重不足: 冻结发布<br/>专注稳定性修复<br/>全员 On-Call"]
|
||||
Ratio -- "\> 100%" --> Red["🔴 红灯阶段<br/>预算耗尽: 禁止所有<br/>功能性变更<br/>SLO 事故复盘"]
|
||||
|
||||
style Green fill:#c8e6c9
|
||||
style Yellow fill:#fff9c4
|
||||
style Orange fill:#ffe0b2
|
||||
style Red fill:#ffcdd2
|
||||
```
|
||||
|
||||
### 错误预算告警联动
|
||||
|
||||
```yaml
|
||||
# Prometheus 告警规则示例
|
||||
groups:
|
||||
- name: error-budget
|
||||
rules:
|
||||
- alert: ErrorBudgetBurnRateHigh
|
||||
expr: |
|
||||
sum(rate(http_requests_total{status=~"5.."}[1h])) /
|
||||
sum(rate(http_requests_total[1h])) > 0.001
|
||||
for: 1h
|
||||
labels:
|
||||
severity: warning
|
||||
budget_phase: yellow
|
||||
annotations:
|
||||
summary: "错误预算消耗加速"
|
||||
message: "当前错误率 {{ $value }}% > SLO 阈值 0.1%"
|
||||
|
||||
- alert: ErrorBudgetExhausted
|
||||
expr: |
|
||||
(sum(rate(http_requests_total{status=~"5.."}[24h])) /
|
||||
sum(rate(http_requests_total[24h]))) >= 0.001
|
||||
labels:
|
||||
severity: critical
|
||||
budget_phase: red
|
||||
annotations:
|
||||
summary: "错误预算已耗尽!"
|
||||
message: "本月 SLO 已破线,冻结非紧急变更"
|
||||
```
|
||||
|
||||
## SRE 心法
|
||||
|
||||
1. **用户视角定义 SLO** — 不是"API P99 < 200ms",而是"用户在正常网络下加载页面 < 2s"
|
||||
2. **错误预算用完 = 停止功能开发** — 全力修 bug、加稳定性
|
||||
3. **Blameless Postmortem** — 不问"谁干的",问"流程哪里可以改进"
|
||||
4. **自动化一切重复劳动** — 手动操作一定会出错
|
||||
5. **接受一定程度的失败** — 在可控范围内快速迭代,比追求完美更重要
|
||||
|
||||
## Blameless Postmortem 模板
|
||||
|
||||
| 字段 | 填写内容 |
|
||||
|------|---------|
|
||||
| **事件名称** | `2026-05-05 支付服务超时事故` |
|
||||
| **影响范围** | 约 15% 的支付请求失败,持续 23 分钟 |
|
||||
| **发现时间** | 14:32 (On-Call 接到告警) |
|
||||
| **恢复时间** | 14:55 (回滚后确认恢复) |
|
||||
| **根本原因** | 某次变更后支付网关的连接池大小从 50 降到 10 |
|
||||
| **时间线** | 14:00 发布 v1.2.3 → 14:30 错误率开始升高 → 14:32 收到告警 → 14:35 开始排查 → 14:45 定位根因 → 14:50 回滚 → 14:55 完全恢复 |
|
||||
| **改进行动** | ① 连接池参数变动必须经过压测验证;② 监控中补充连接池活跃数指标 |
|
||||
| **跟进人** | @zhangsan (行动 ①)、@lisi (行动 ②) |
|
||||
|
||||
## SRE Metrics 看板
|
||||
|
||||
除了 SLO,SRE 还需要关注以下运营指标:
|
||||
|
||||
| 指标 | 说明 | 目标 |
|
||||
|------|------|------|
|
||||
| **MTTR** | Mean Time To Recovery | 核心服务 < 30min |
|
||||
| **Change Failure Rate** | 变更导致故障的比例 | < 5% |
|
||||
| **Lead Time for Changes** | 代码提交到上线的时间 | < 2h |
|
||||
| **Deployment Frequency** | 日均部署次数 | > 5 (大规模团队) |
|
||||
|
||||
> [!tip] DORA Metrics
|
||||
>
|
||||
> Google 提出的四大 DevOps 指标,广泛用于评估工程效能:
|
||||
> - **部署频率** — 交付速度
|
||||
> - **变更前置时间** — 从代码提交到生产部署需要多久
|
||||
> - **变更失败率** — 多少部署导致了故障或回滚
|
||||
> - **MTTR** — 恢复服务的平均时间
|
||||
|
||||
## 关联笔记
|
||||
|
||||
- [[04-可观测性/04-告警管理]] — SLO/Error Budget 与告警体系的联动
|
||||
- [[05-部署运维/02-Kubernetes]] — K8s HPA 弹性伸缩支撑 SLO 保障
|
||||
- [[05-部署运维/03-CICD与GitOps]] — GitOps 支持安全的持续交付
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
tags: [microservice, deployment, ops]
|
||||
create time: 2026-04-29 12:05
|
||||
---
|
||||
|
||||
# 部署运维
|
||||
|
||||
## 概述
|
||||
|
||||
微服务架构下,服务数量从几个增长到几百个。**人工运维完全不可行**。本文涵盖容器化编排、K8s 资源管理、发布策略、弹性伸缩、CI/CD 流水线和 SRE 核心概念。
|
||||
|
||||
## 知识体系
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A["容器化"] --> B["Kubernetes"]
|
||||
B --> C["CI/CD 与 GitOps"]
|
||||
C --> D["SRE 实践"]
|
||||
|
||||
style A fill:#e3f2fd
|
||||
style B fill:#fff3e0
|
||||
style C fill:#fce4ec
|
||||
style D fill:#e8f5e9
|
||||
```
|
||||
|
||||
| # | 主题 | 核心问题 |
|
||||
|---|------|----------|
|
||||
| 1 | [[05-部署运维/01-容器化]] | Docker 镜像怎么优化?最佳实践有哪些? |
|
||||
| 2 | [[05-部署运维/02-Kubernetes]] | K8s 核心概念和资源管理怎么做? |
|
||||
| 3 | [[05-部署运维/03-CICD与GitOps]] | 自动化流水线怎么设计?GitOps 流程怎么走? |
|
||||
| 4 | [[05-部署运维/04-SRE实践]] | SLI/SLO/Error Budget 如何落地? |
|
||||
|
||||
### 学习建议
|
||||
|
||||
> [!tip] 学习路径
|
||||
> 先掌握容器化(Docker),再学 K8s 核心概念,CI/CD 和 SRE 可以在实践中逐步深入。
|
||||
|
||||
### 关联笔记
|
||||
|
||||
- [[02-服务治理]] — K8s Service 提供原生的服务发现和负载均衡
|
||||
- [[04-可观测性]] — K8s Liveness/Readiness Probe 是可观测性的基础
|
||||
- [[hzh/MS/README.md]] — 完整微服务知识索引
|
||||
Reference in New Issue
Block a user