test: add unit tests and sample notes
This commit is contained in:
@@ -0,0 +1,35 @@
|
||||
---
|
||||
tags: [eino, agent, go, framework]
|
||||
---
|
||||
|
||||
## EINO 框架简介
|
||||
|
||||
EINO 是字节跳动 CloudWeGo 团队开发的 Go 语言 AI 应用框架,提供了从组件到编排到 Agent 的完整开发栈。
|
||||
|
||||
## 三大核心模块
|
||||
|
||||
### 1. Components(组件)
|
||||
Eino 定义了一系列标准化的组件接口:
|
||||
- **ChatModel**:与大语言模型交互
|
||||
- **Embedding**:文本向量化
|
||||
- **Retriever**:文档检索
|
||||
- **Tool**:工具调用
|
||||
- **Indexer**:文档索引
|
||||
|
||||
### 2. Chain/Graph(编排)
|
||||
通过有向无环图将组件编排成复杂的数据处理流程:
|
||||
- 类型安全的边连接
|
||||
- 自动流式转换
|
||||
- Callback 机制
|
||||
|
||||
### 3. ADK(Agent Development Kit)
|
||||
提供多种 Agent 编排模式:
|
||||
- **ChatModelAgent**:ReAct 模式的单 Agent
|
||||
- **Supervisor**:中心化的多 Agent 协作
|
||||
- **Plan-Execute**:先规划后执行
|
||||
|
||||
## 为什么选择 EINO
|
||||
|
||||
1. Go 原生协程,性能优异
|
||||
2. 编译时类型检查,减少运行时错误
|
||||
3. 开箱即用的 Agent 模式
|
||||
@@ -0,0 +1,46 @@
|
||||
---
|
||||
tags: [go, concurrency, goroutine]
|
||||
---
|
||||
|
||||
## Go 并发模式概述
|
||||
|
||||
Go 语言的并发模型基于 CSP(Communicating Sequential Processes)理论,核心思想是"通过通信共享内存,而不是通过共享内存通信"。
|
||||
|
||||
## Goroutine 基础
|
||||
|
||||
Goroutine 是 Go 运行时管理的轻量级线程。创建一个 goroutine 只需要在函数调用前加上 `go` 关键字:
|
||||
|
||||
```go
|
||||
go func() {
|
||||
fmt.Println("Hello from goroutine")
|
||||
}()
|
||||
```
|
||||
|
||||
Goroutine 的初始栈大小只有 2KB,远小于操作系统线程的 1-2MB,因此可以轻松创建数十万个 goroutine。
|
||||
|
||||
## Channel 通信
|
||||
|
||||
Channel 是 goroutine 之间通信的管道:
|
||||
|
||||
```go
|
||||
ch := make(chan int, 10) // 带缓冲的 channel
|
||||
ch <- 42 // 发送
|
||||
value := <-ch // 接收
|
||||
```
|
||||
|
||||
## 常见并发模式
|
||||
|
||||
### Fan-out/Fan-in
|
||||
将任务分发给多个 goroutine 并行处理,然后汇总结果。
|
||||
|
||||
### Pipeline
|
||||
将处理流程分成多个阶段,每个阶段是一个 goroutine,通过 channel 串联。
|
||||
|
||||
### Worker Pool
|
||||
固定数量的 worker goroutine 从任务队列中取任务执行。
|
||||
|
||||
## 常见陷阱
|
||||
|
||||
1. **Goroutine 泄漏**:goroutine 阻塞在 channel 上无法退出
|
||||
2. **Race Condition**:多个 goroutine 同时读写共享变量
|
||||
3. **Deadlock**:所有 goroutine 都在等待对方
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
tags: [prompt, llm, engineering]
|
||||
---
|
||||
|
||||
## Prompt Engineering 最佳实践
|
||||
|
||||
Prompt Engineering 是设计和优化输入给大语言模型的提示词的技术。
|
||||
|
||||
## 核心原则
|
||||
|
||||
### 1. 明确指令
|
||||
告诉模型具体要做什么,而不是模糊描述:
|
||||
- 好:将以下文本翻译成英文,保持专业术语不变
|
||||
- 差:处理一下这个文本
|
||||
|
||||
### 2. 提供示例
|
||||
Few-shot learning 通过给出示例来引导模型行为。
|
||||
|
||||
### 3. 分步思考
|
||||
Chain-of-Thought 提示让模型逐步推理:
|
||||
"请一步步思考,首先分析问题,然后给出解决方案"
|
||||
|
||||
### 4. 角色设定
|
||||
通过 System Prompt 设定模型的角色和行为边界。
|
||||
|
||||
## 常用技巧
|
||||
|
||||
- **结构化输出**:要求 JSON/Markdown 格式输出
|
||||
- **约束条件**:明确输出的限制和要求
|
||||
- **上下文管理**:合理控制上下文窗口的使用
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
tags: [rag, llm, search, embedding]
|
||||
---
|
||||
|
||||
## RAG 检索增强生成
|
||||
|
||||
RAG(Retrieval-Augmented Generation)是一种将外部知识库与大语言模型结合的技术范式。
|
||||
|
||||
## RAG 的核心流程
|
||||
|
||||
### 1. 索引阶段(Indexing)
|
||||
- 加载文档(Document Loader)
|
||||
- 文本分块(Chunking)
|
||||
- 向量化(Embedding)
|
||||
- 存入向量数据库(Indexer)
|
||||
|
||||
### 2. 检索阶段(Retrieval)
|
||||
- 用户查询向量化
|
||||
- 向量相似度搜索
|
||||
- 返回 Top-K 相关文档
|
||||
|
||||
### 3. 生成阶段(Generation)
|
||||
- 将检索结果注入 Prompt
|
||||
- LLM 基于上下文生成回答
|
||||
|
||||
## 混合检索
|
||||
|
||||
结合语义搜索(向量相似度)和关键词搜索(BM25),取长补短:
|
||||
- 语义搜索:理解用户意图,找到表述不同但意思相近的内容
|
||||
- 关键词搜索:精确匹配专业术语和专有名词
|
||||
|
||||
## 向量数据库选型
|
||||
|
||||
- **Milvus**:分布式,适合大规模生产环境
|
||||
- **Chroma**:轻量级,适合原型开发
|
||||
- **内存实现**:最简单,适合小规模和个人项目
|
||||
@@ -0,0 +1,74 @@
|
||||
package index
|
||||
|
||||
import (
|
||||
"math"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestInMemoryVectorIndex_AddSearch(t *testing.T) {
|
||||
idx := NewInMemoryVectorIndex()
|
||||
|
||||
idx.Add("a", []float64{1, 0, 0})
|
||||
idx.Add("b", []float64{0, 1, 0})
|
||||
idx.Add("c", []float64{0, 0, 1})
|
||||
|
||||
if idx.Len() != 3 {
|
||||
t.Fatalf("Len = %d, want 3", idx.Len())
|
||||
}
|
||||
|
||||
hits := idx.Search([]float64{1, 0, 0}, 3)
|
||||
if len(hits) != 3 {
|
||||
t.Fatalf("Search: got %d hits, want 3", len(hits))
|
||||
}
|
||||
if hits[0].ID != "a" {
|
||||
t.Errorf("top hit = %q, want %q", hits[0].ID, "a")
|
||||
}
|
||||
if math.Abs(hits[0].Score-1.0) > 1e-9 {
|
||||
t.Errorf("top score = %f, want 1.0", hits[0].Score)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInMemoryVectorIndex_Remove(t *testing.T) {
|
||||
idx := NewInMemoryVectorIndex()
|
||||
idx.Add("a", []float64{1, 0})
|
||||
idx.Add("b", []float64{0, 1})
|
||||
|
||||
idx.Remove("a")
|
||||
if idx.Len() != 1 {
|
||||
t.Fatalf("Len after remove = %d, want 1", idx.Len())
|
||||
}
|
||||
|
||||
hits := idx.Search([]float64{0, 1}, 5)
|
||||
if len(hits) != 1 || hits[0].ID != "b" {
|
||||
t.Errorf("expected only 'b' in results")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInMemoryVectorIndex_RemoveByPrefix(t *testing.T) {
|
||||
idx := NewInMemoryVectorIndex()
|
||||
idx.Add("note1_0", []float64{1, 0})
|
||||
idx.Add("note1_1", []float64{0.9, 0.1})
|
||||
idx.Add("note2_0", []float64{0, 1})
|
||||
|
||||
idx.RemoveByPrefix("note1_")
|
||||
if idx.Len() != 1 {
|
||||
t.Fatalf("Len after RemoveByPrefix = %d, want 1", idx.Len())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCosine(t *testing.T) {
|
||||
tests := []struct {
|
||||
a, b []float64
|
||||
want float64
|
||||
}{
|
||||
{[]float64{1, 0}, []float64{1, 0}, 1.0},
|
||||
{[]float64{1, 0}, []float64{0, 1}, 0.0},
|
||||
{[]float64{1, 0}, []float64{-1, 0}, -1.0},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
got := cosine(tt.a, tt.b)
|
||||
if math.Abs(got-tt.want) > 1e-9 {
|
||||
t.Errorf("cosine(%v, %v) = %f, want %f", tt.a, tt.b, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package rag
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestChunkMarkdown_ByHeaders(t *testing.T) {
|
||||
md := `# Title
|
||||
|
||||
## Section 1
|
||||
|
||||
Content of section 1.
|
||||
|
||||
## Section 2
|
||||
|
||||
Content of section 2.`
|
||||
|
||||
chunks := ChunkMarkdown(md, 1000)
|
||||
if len(chunks) < 2 {
|
||||
t.Errorf("expected at least 2 chunks, got %d", len(chunks))
|
||||
}
|
||||
}
|
||||
|
||||
func TestChunkMarkdown_SmallContent(t *testing.T) {
|
||||
md := "Just a short note with no headers."
|
||||
chunks := ChunkMarkdown(md, 500)
|
||||
if len(chunks) != 1 {
|
||||
t.Errorf("expected 1 chunk, got %d", len(chunks))
|
||||
}
|
||||
}
|
||||
|
||||
func TestChunkMarkdown_SplitLargeSection(t *testing.T) {
|
||||
md := `## Big Section
|
||||
|
||||
Paragraph one with enough content to make this section reasonably long for testing purposes.
|
||||
|
||||
Paragraph two with more content that should be in a separate chunk when the max size is small.`
|
||||
|
||||
chunks := ChunkMarkdown(md, 80)
|
||||
if len(chunks) < 2 {
|
||||
t.Errorf("expected at least 2 chunks for large section, got %d", len(chunks))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMemoryNoteStore_CRUD(t *testing.T) {
|
||||
s := NewMemoryNoteStore()
|
||||
ctx := context.Background()
|
||||
|
||||
note := &Note{ID: "test-1", Title: "Test Note", Content: "Hello", Tags: []string{"test"}}
|
||||
if err := s.Create(ctx, note); err != nil {
|
||||
t.Fatalf("Create: %v", err)
|
||||
}
|
||||
|
||||
got, err := s.GetByID(ctx, "test-1")
|
||||
if err != nil {
|
||||
t.Fatalf("GetByID: %v", err)
|
||||
}
|
||||
if got.Title != "Test Note" {
|
||||
t.Errorf("Title = %q, want %q", got.Title, "Test Note")
|
||||
}
|
||||
|
||||
list, err := s.List(ctx)
|
||||
if err != nil || len(list) != 1 {
|
||||
t.Fatalf("List: len=%d, err=%v", len(list), err)
|
||||
}
|
||||
|
||||
if err := s.Delete(ctx, "test-1"); err != nil {
|
||||
t.Fatalf("Delete: %v", err)
|
||||
}
|
||||
_, err = s.GetByID(ctx, "test-1")
|
||||
if err == nil {
|
||||
t.Fatal("expected error after delete")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryNoteStore_SearchByKeyword(t *testing.T) {
|
||||
s := NewMemoryNoteStore()
|
||||
ctx := context.Background()
|
||||
|
||||
s.Create(ctx, &Note{ID: "1", Title: "Go Concurrency", Content: "goroutine patterns"})
|
||||
s.Create(ctx, &Note{ID: "2", Title: "Python Basics", Content: "variables and types"})
|
||||
|
||||
results, err := s.SearchByKeyword(ctx, "goroutine")
|
||||
if err != nil {
|
||||
t.Fatalf("SearchByKeyword: %v", err)
|
||||
}
|
||||
if len(results) != 1 || results[0].ID != "1" {
|
||||
t.Errorf("expected 1 result with ID=1, got %d", len(results))
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user