412 lines
15 KiB
Markdown
412 lines
15 KiB
Markdown
|
|
---
|
|||
|
|
tags: [gRPC, RPC, Streaming, Go, Microservice, API Design]
|
|||
|
|
create time: 2026-05-11 16:40
|
|||
|
|
update time: 2026-05-13 00:00
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# RPC 调用模式总览
|
|||
|
|
|
|||
|
|
## 概述
|
|||
|
|
|
|||
|
|
gRPC 提供四种 RPC 调用模式,从最简单的请求-响应到完全的双向流。选择合适的模式是设计高性能 API 的第一步。搞懂它们的区别,你就知道什么时候该用简单调用、什么时候需要双向通信。
|
|||
|
|
|
|||
|
|
> [!question] 如果只选一种模式能走天下吗?
|
|||
|
|
> 技术上可以——Unary RPC 确实能解决几乎所有问题。但强行用 Unary 实现实时推送,意味着你要轮询(polling),这会产生大量无效请求和延迟。模式选择本质上是在"延迟 vs 资源消耗"之间做 tradeoff。
|
|||
|
|
|
|||
|
|
## RPC 模式全景图
|
|||
|
|
|
|||
|
|
```mermaid
|
|||
|
|
flowchart LR
|
|||
|
|
U["Unary\n一问一答"] --> S1["简单 · 阻塞 · 一次往返"]
|
|||
|
|
SS["Server Stream\n一问多答"] --> S2["广播式 · 服务端推送"]
|
|||
|
|
CS["Client Stream\n多问一答"] --> S3["收集式 · 分批上传"]
|
|||
|
|
BD["BiDi Stream\n多问多答"] --> S4["全双工 · 独立收发"]
|
|||
|
|
|
|||
|
|
style U fill:#00B6BC,color:#fff
|
|||
|
|
style SS fill:#00D866,color:#fff
|
|||
|
|
style CS fill:#4FC3F7,color:#fff
|
|||
|
|
style BD fill:#EE5A24,color:#fff
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## RPC 数据流示意图
|
|||
|
|
|
|||
|
|
```mermaid
|
|||
|
|
sequenceDiagram
|
|||
|
|
participant C as Client
|
|||
|
|
participant S as Server
|
|||
|
|
|
|||
|
|
rect rgba(0, 182, 188, 0.1)
|
|||
|
|
Note over C,S: Unary — 一次请求,一次响应
|
|||
|
|
C->>S: request
|
|||
|
|
S-->>C: response
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
rect rgba(0, 216, 102, 0.1)
|
|||
|
|
Note over C,S: Server Stream — 一次请求,多次响应
|
|||
|
|
C->>S: request
|
|||
|
|
S-->>C: response 1
|
|||
|
|
S-->>C: response 2
|
|||
|
|
S-->>C: ... n (EOF)
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
rect rgba(79, 195, 247, 0.1)
|
|||
|
|
Note over C,S: Client Stream — 多次请求,一次响应
|
|||
|
|
C->>S: chunk 1
|
|||
|
|
C->>S: chunk 2
|
|||
|
|
C->>S: ... n (done)
|
|||
|
|
S-->>C: result
|
|||
|
|
end
|
|||
|
|
|
|||
|
|
rect rgba(238, 90, 36, 0.1)
|
|||
|
|
Note over C,S: BiDi Stream — 双向独立通信
|
|||
|
|
C->>S: msg 1
|
|||
|
|
S-->>C: reply 1
|
|||
|
|
C->>S: msg 2
|
|||
|
|
S-->>C: reply 2
|
|||
|
|
end
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Unary RPC(普通调用)
|
|||
|
|
|
|||
|
|
最经典、最常见的模式,等同于 REST 的 request-response。
|
|||
|
|
|
|||
|
|
**特点:**
|
|||
|
|
- 一次客户端请求,一次服务端响应
|
|||
|
|
- 简单、易调试、可直接映射 HTTP GET/POST
|
|||
|
|
- 适合:CRUD 操作、短查询、标准 API 端点
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// proto 定义
|
|||
|
|
// rpc GetUser(GetUserRequest) returns (GetUserResponse);
|
|||
|
|
|
|||
|
|
// Client side
|
|||
|
|
func main() {
|
|||
|
|
conn, _ := grpc.Dial("localhost:50051", grpc.WithTransportCredentials(insecure.NewCredentials()))
|
|||
|
|
defer conn.Close()
|
|||
|
|
client := pb.NewUserServiceClient(conn)
|
|||
|
|
|
|||
|
|
resp, err := client.GetUser(context.Background(), &pb.GetUserRequest{Id: 42})
|
|||
|
|
if err != nil {
|
|||
|
|
log.Fatal(err) // error comes from transport or server handler
|
|||
|
|
}
|
|||
|
|
fmt.Println(resp.Name)
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
// Server side
|
|||
|
|
type Server struct {
|
|||
|
|
pb.UnimplementedUserServiceServer
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
func (s *Server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.GetUserResponse, error) {
|
|||
|
|
user := fetchFromDB(req.GetId())
|
|||
|
|
return &pb.GetUserResponse{Name: user.Name}, nil
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**思考题:**如果一个接口需要 30 秒才能返回结果,你应该用 Unary 还是其他模式?(提示:考虑超时和连接的持有时间)
|
|||
|
|
|
|||
|
|
> [!answer]+ 参考答案
|
|||
|
|
> **可以用 Unary,但要注意三件事:**
|
|||
|
|
>
|
|||
|
|
> 1. **设置合理的 context timeout**:`context.WithTimeout`,避免无限期等待
|
|||
|
|
> 2. **HTTP/2 ping keepalive**:gRPC 默认会发送 keepalive ping 防止代理(Nginx/LB)因"连接空闲"而断开
|
|||
|
|
> 3. **是否真的需要阻塞等待**:如果是异步任务(如报表生成),更好的做法是——Unary 提交任务 + 轮询/回调通知结果,而非让一个 RPC 连接挂 30 秒。
|
|||
|
|
>
|
|||
|
|
> Streaming 并不会延长超时时间——超时由 context 控制,与使用哪种 RPC 模式无关。
|
|||
|
|
|
|||
|
|
## Server Streaming RPC(服务端流)
|
|||
|
|
|
|||
|
|
Client 发一个请求,Server 持续返回多个响应。
|
|||
|
|
|
|||
|
|
**经典场景:**
|
|||
|
|
- 实时通知推送
|
|||
|
|
- 大列表分批返回
|
|||
|
|
- 日志/事件流订阅
|
|||
|
|
|
|||
|
|
```protobuf
|
|||
|
|
rpc Subscribe(SubscribeRequest) returns (stream Event);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// Server side - 持续发送事件(注意不要用 log.Fatal,会杀死整个进程)
|
|||
|
|
func (s *Server) Subscribe(req *pb.SubscribeRequest, stream pb.UserService_SubscribeServer) error {
|
|||
|
|
for _, event := range s.watchEvents(req.Topic) {
|
|||
|
|
if err := stream.Send(&pb.Event{Data: event}); err != nil {
|
|||
|
|
return err // channel closed or context canceled
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
return nil
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!note] import 提示
|
|||
|
|
> 流式示例中用到以下包:`context`, `fmt`, `io`, `log`.
|
|||
|
|
> 每个文件只需 import 实际用到的即可,不必照抄。
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// Client side - 遍历接收事件(生产环境建议加 context timeout 和 recover)
|
|||
|
|
func subscribeEvents(client pb.UserServiceClient, topic string) error {
|
|||
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|||
|
|
defer cancel()
|
|||
|
|
|
|||
|
|
stream, err := client.Subscribe(ctx, &pb.SubscribeRequest{Topic: topic})
|
|||
|
|
if err != nil {
|
|||
|
|
return err // dial failed — 上层 main 可以做 Fatal
|
|||
|
|
}
|
|||
|
|
defer func() {
|
|||
|
|
if r := recover(); r != nil {
|
|||
|
|
log.Printf("panic recovered: %v", r)
|
|||
|
|
}
|
|||
|
|
}()
|
|||
|
|
for {
|
|||
|
|
event, err := stream.Recv()
|
|||
|
|
if err == io.EOF {
|
|||
|
|
break // server finished sending
|
|||
|
|
}
|
|||
|
|
if err != nil {
|
|||
|
|
log.Printf("stream error: %v", err) // ⚠️ 这里用 return/log,不能用 Fatal
|
|||
|
|
return err
|
|||
|
|
}
|
|||
|
|
fmt.Printf("received: %s\n", event.Data)
|
|||
|
|
}
|
|||
|
|
return nil
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!tip] Context Cancel 与 Recv 的关系
|
|||
|
|
> Client 仍可通过 cancel 随时中断流——服务端 `Recv()` 将返回一个 context canceled 错误。这也是调试流式问题时最容易忽略的一点:**不是 Server 主动关了连接,而是 Client 放弃了**。
|
|||
|
|
|
|||
|
|
## Client Streaming RPC(客户端流)
|
|||
|
|
|
|||
|
|
Client 持续发送多个请求,Server 在所有数据发送完毕后返回一个响应。
|
|||
|
|
|
|||
|
|
**经典场景:**
|
|||
|
|
- 大批量数据上传
|
|||
|
|
- 文件分片聚合处理
|
|||
|
|
- 批量日志采集
|
|||
|
|
|
|||
|
|
```protobuf
|
|||
|
|
rpc Upload(stream FileChunk) returns (UploadResult);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// Client side - 流式发送数据分片(注意 defer CleanupSend 做资源清理)
|
|||
|
|
func uploadFile(client pb.FileServiceClient, chunks [][]byte) error {
|
|||
|
|
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
|||
|
|
defer cancel()
|
|||
|
|
|
|||
|
|
stream, err := client.Upload(ctx)
|
|||
|
|
if err != nil {
|
|||
|
|
return err
|
|||
|
|
}
|
|||
|
|
defer func() {
|
|||
|
|
if r := recover(); r != nil {
|
|||
|
|
stream.CloseSend() // panic 时确保 send 方向关闭
|
|||
|
|
}
|
|||
|
|
}()
|
|||
|
|
for _, chunk := range chunks {
|
|||
|
|
if err := stream.Send(&pb.FileChunk{Data: chunk}); err != nil {
|
|||
|
|
return err
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
result, err := stream.CloseAndRecv() // 结束发送,获取最终结果
|
|||
|
|
if err != nil {
|
|||
|
|
return err
|
|||
|
|
}
|
|||
|
|
fmt.Printf("upload complete, size: %d\n", result.Size)
|
|||
|
|
return nil
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
// Server side - 逐块接收后聚合(用 return 而非 Fatal,让 gRPC 框架处理错误上报)
|
|||
|
|
func (s *Server) Upload(stream pb.FileService_UploadServer) error {
|
|||
|
|
var buffer bytes.Buffer
|
|||
|
|
for {
|
|||
|
|
chunk, err := stream.Recv()
|
|||
|
|
if err == io.EOF {
|
|||
|
|
break // client closed send direction
|
|||
|
|
}
|
|||
|
|
if err != nil {
|
|||
|
|
return err
|
|||
|
|
}
|
|||
|
|
buffer.Write(chunk.Data)
|
|||
|
|
}
|
|||
|
|
result, err := stream.SendAndReceive(&pb.UploadResult{Size: int32(buffer.Len())})
|
|||
|
|
return err
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!tip] CloseAndRecv vs CloseSend + Recv
|
|||
|
|
> `CloseAndRecv()` 是 Client Stream 中的便捷方法——它同时完成"关闭发送方向"和"读取响应"两步。等价于先 `stream.CloseSend()` 再 `stream.Recv()`。在 Client Stream 中两者效果相同,但 `CloseAndRecv` 更简洁、出错概率更低。
|
|||
|
|
|
|||
|
|
**优势:**内存友好——不需要一次性 load 全部数据到内存中,每个 chunk 独立收发。
|
|||
|
|
|
|||
|
|
## 错误处理模式总结
|
|||
|
|
|
|||
|
|
四种模式共用同一套错误处理原则:
|
|||
|
|
|
|||
|
|
| 场景 | Unary | Server Stream | Client Stream | BiDi Stream |
|
|||
|
|
|------|:-----:|:-------------:|:-------------:|:-----------:|
|
|||
|
|
| **Stream 创建失败(Client)** | — | `log.Fatal` ✅ | `log.Fatal` ✅ | `log.Fatal` ✅ |
|
|||
|
|
| **Recv 返回 io.EOF** | — | `break` ✅ | `break` ✅ | `close(done)` ✅ |
|
|||
|
|
| **Recv 返回其他 error** | `return/log` ✅ | `return/log` ✅ | `return/log` ✅ | 分别处理两端 ✅ |
|
|||
|
|
| **Send 返回 error** | — | `return` ✅ | `return` ✅ | `return` ✅ |
|
|||
|
|
| **Context canceled** | RPC 自动取消 | 同 error ✅ | 同 error ✅ | `select <-ctx.Done()` ✅ |
|
|||
|
|
|
|||
|
|
> [!important] 黄金法则
|
|||
|
|
> 1. **只有连接建立阶段的 dial/send 错误才能用 `log.Fatal`**——handler 内部永远用 return
|
|||
|
|
> 2. **io.EOF 不是错误**——它表示对方完成了发送方向,是正常退出信号
|
|||
|
|
> 3. **每次 stream recv/send 都要检查 error**,哪怕代码看起来"不可能失败"
|
|||
|
|
> 4. **context timeout 对所有模式生效**——Streaming 不会因为"流式"就自动获得更长超时
|
|||
|
|
|
|||
|
|
## Bidirectional Streaming RPC(双向流)
|
|||
|
|
|
|||
|
|
Client 和 Server 可以同时独立地发送消息,是全双工通信。
|
|||
|
|
|
|||
|
|
**经典场景:**
|
|||
|
|
- 聊天室
|
|||
|
|
- 实时协作编辑
|
|||
|
|
- 游戏状态同步
|
|||
|
|
|
|||
|
|
```protobuf
|
|||
|
|
rpc Chat(stream ChatMessage) returns (stream ChatMessage);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// Server side - 转发逻辑,两端各自独立循环(增加 context cancel 处理)
|
|||
|
|
func (s *Server) Chat(stream pb.ChatService_ChatServer) error {
|
|||
|
|
ctx := stream.Context()
|
|||
|
|
done := make(chan struct{})
|
|||
|
|
|
|||
|
|
// goroutine 1: 读取客户端消息
|
|||
|
|
go func() {
|
|||
|
|
for {
|
|||
|
|
msg, err := stream.Recv()
|
|||
|
|
if err == io.EOF {
|
|||
|
|
close(done)
|
|||
|
|
return
|
|||
|
|
}
|
|||
|
|
if err != nil {
|
|||
|
|
log.Printf("recv error: %v", err)
|
|||
|
|
return
|
|||
|
|
}
|
|||
|
|
// 广播给其他 connected clients...
|
|||
|
|
s.broadcast(msg)
|
|||
|
|
}
|
|||
|
|
}()
|
|||
|
|
|
|||
|
|
// goroutine 2: 定时推送服务器消息
|
|||
|
|
ticker := time.NewTicker(5 * time.Second)
|
|||
|
|
defer ticker.Stop()
|
|||
|
|
for {
|
|||
|
|
select {
|
|||
|
|
case <-ctx.Done():
|
|||
|
|
return ctx.Err() // client disconnected → clean exit
|
|||
|
|
case <-done:
|
|||
|
|
return nil
|
|||
|
|
case t := <-ticker.C:
|
|||
|
|
if sendErr := stream.Send(&pb.ChatMessage{Text: fmt.Sprintf("heartbeat: %s", t)}); sendErr != nil {
|
|||
|
|
log.Printf("send heartbeat error: %v", sendErr)
|
|||
|
|
return sendErr
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**关键点:**
|
|||
|
|
- 两端的 Send 和 Recv 是独立的——一端 Recv 完(EOF)不影响另一端继续 Send
|
|||
|
|
- 必须用两个 goroutine 分别处理 recv 和 send 循环——单线程无法同时读写
|
|||
|
|
- `io.EOF` 只表示对方的关闭,不代表己方也要停止发送
|
|||
|
|
- **Context Cancel 优先级高于 EOF**:Client 断开连接时,`stream.Context().Done()` 先触发。生产代码中永远要检查它
|
|||
|
|
- 建议在 handler 中使用 `defer stream.SendAndClose(...)` 或 defer 清理逻辑确保资源释放
|
|||
|
|
|
|||
|
|
### 背压与心跳(生产级要点)
|
|||
|
|
|
|||
|
|
BiDi Stream 在长连接场景下,有三个必须考虑的问题:context cancel、io.EOF、背压和心跳保活。
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
// 背压控制:如果 Send 堆积过多,buffer 满时自动阻塞发送端
|
|||
|
|
func (s *Server) handleStream(stream pb.ChatService_ChatServer) error {
|
|||
|
|
sendCh := make(chan *pb.ChatMessage, 100) // buffer size = 100
|
|||
|
|
go func() {
|
|||
|
|
for msg := range sendCh {
|
|||
|
|
// Send 是阻塞的——buffer 满时自动触发背压
|
|||
|
|
if err := stream.Send(msg); err != nil {
|
|||
|
|
return // connection lost, goroutine exits cleanly
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
}()
|
|||
|
|
// ...recv loop 往 sendCh 里塞消息即可
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!tip] 背压原理
|
|||
|
|
> gRPC 的 `Send()` 是**有缓冲阻塞**的。当 internal buffer 写满时,发送端会自动 pause——这就是 HTTP/2 Flow Control 提供的天然背压机制,不需要手动实现。但你应该设置合理的 buffer size,过大浪费内存,过小影响吞吐。建议从 100 起步,根据实际监控调整。
|
|||
|
|
|
|||
|
|
> [!note] Keepalive 配置示例
|
|||
|
|
> ```go
|
|||
|
|
> conn, _ := grpc.Dial(addr,
|
|||
|
|
> grpc.WithKeepaliveParams(keepalive.ClientParameters{
|
|||
|
|
> Time: 10 * time.Second, // ping interval
|
|||
|
|
> Timeout: 20 * time.Second, // wait for ping ack
|
|||
|
|
> PermitWithoutStream: true, // 即使无活跃 RPC 也发 ping
|
|||
|
|
> }),
|
|||
|
|
> )
|
|||
|
|
> ```
|
|||
|
|
> 这对穿越 Nginx / AWS ALB 等负载均衡器至关重要——它们通常会对空闲连接执行 TCP idle timeout 断开。服务端也需要配置类似的 keepalive,否则 Server→Client 方向的心跳缺失会导致 Client 误判连接死亡。
|
|||
|
|
|
|||
|
|
> [!warning] 复杂度警告
|
|||
|
|
> BiDi Streaming 是最强大但也最容易出错的模式。你必须同时处理:context cancel、io.EOF、网络异常、心跳保活、背压(backpressure)。生产环境中除非必要,否则优先考虑其他三种模式。
|
|||
|
|
|
|||
|
|
> [!question] 为什么需要 PermitWithoutStream?
|
|||
|
|
> 因为某些场景中连接处于"空闲状态"——没有正在进行的 RPC 调用——此时 LB 会因为检测到 TCP 层无任何流量而主动断开连接。设置 `PermitWithoutStream: true` 确保即使没有活跃流,gRPC 仍会持续发送 ping 包维持连接。
|
|||
|
|
|
|||
|
|
## 模式选型决策指南
|
|||
|
|
|
|||
|
|
```mermaid
|
|||
|
|
flowchart TD
|
|||
|
|
Start["是否需要\n实时交互?"] -->|否| Simple["单次请求?"]
|
|||
|
|
Start -->|是| BiDi["高频交互?"]
|
|||
|
|
|
|||
|
|
Simple -->|是| U["Unary RPC\n最简单 ⭐"]
|
|||
|
|
Simple -->|否| SS["Server Stream\n一次请求多次返回"]
|
|||
|
|
|
|||
|
|
BiDi -->|是| BD["Bidirectional Stream\n全双工通信 🔥"]
|
|||
|
|
BiDi -->|否| CS["数据量超大?"]
|
|||
|
|
|
|||
|
|
CS -->|是| CB["Client Stream\n分批上传"]
|
|||
|
|
CS -->|否| SS
|
|||
|
|
|
|||
|
|
style U fill:#00D866,color:#fff
|
|||
|
|
style BD fill:#FF6B35,color:#fff
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> [!tip] 选型原则
|
|||
|
|
> **默认选 Unary**。只有在三种情况下考虑其他模式:(1) 需要实时推送——用 Server Stream;(2) 数据量大且需流式发送——用 Client Stream;(3) 双向高频交互——用 BiDi Stream。永远不要为了炫技而选择更复杂的模式。
|
|||
|
|
|
|||
|
|
## 性能对比
|
|||
|
|
|
|||
|
|
| 维度 | Unary | Server Stream | Client Stream | BiDi Stream |
|
|||
|
|
|------|-------|---------------|---------------|-------------|
|
|||
|
|
| **RTT** | 1 | 1+N | M+1 | M+N |
|
|||
|
|
| **实现复杂度** | ⭐ | ⭐⭐ | ⭐⭐ | ⭐⭐⭐⭐ |
|
|||
|
|
| **内存占用** | 高(一次加载完整响应) | 低(逐条处理) | 低(分片发送) | 低(双向流控) |
|
|||
|
|
| **超时风险** | 高(连接全程持有一直到完整响应) | 低(请求已发出,可取消) | 中(大文件需合理 timeout) | 中(长连接需 keepalive) |
|
|||
|
|
| **典型场景** | CRUD、短查询 | 订阅推送、列表分页 | 大文件上传、批量采集 | 聊天室、实时协作、游戏同步 |
|
|||
|
|
|
|||
|
|
## 与 HTTP 方法的映射类比
|
|||
|
|
|
|||
|
|
| gRPC 模式 | 近似的 HTTP 模式 |
|
|||
|
|
|-----------|-----------------|
|
|||
|
|
| Unary | GET / POST |
|
|||
|
|
| Server Stream | SSE (Server-Sent Events) |
|
|||
|
|
| Client Stream | Multipart Upload |
|
|||
|
|
| BiDi Stream | WebSocket |
|
|||
|
|
|
|||
|
|
> [!note] 类比 ≠ 等价
|
|||
|
|
> 这些只是功能层面的类比。gRPC 是二进制协议且基于 HTTP/2,行为特征和 HTTP 层语义有本质差异——比如 HTTP/2 的多路复用让 gRPC Stream 比 WebSocket 更高效。
|
|||
|
|
|
|||
|
|
## 关联笔记
|
|||
|
|
- [[hhs/gRPC/2. gRPC 核心篇/06-Service 定义与代码生成]]
|
|||
|
|
- [[hhs/gRPC/3. 服务端实现/09-Streaming Handler]]
|