Compare commits

...

9 Commits

Author SHA1 Message Date
30c3e74bd2 feat: 完善 WebUI basePath 支持并简化示例代码
主要改动:

1. WebUI basePath 逻辑完善
   - NewWebUI 支持可变参数 basePath
   - 新增 path() 辅助方法统一路径处理
   - handleTableAPI 正确处理 basePath 前缀
   - handleIndex 根据 basePath 替换占位符

2. 简化示例代码
   - 删除反向代理实现(111行)
   - 直接使用带 basePath 的 WebUI
   - 代码量减少 33%,架构更清晰

3. 前端优化
   - 新增 api.js 统一 API 服务层
   - 所有组件使用统一的 API 调用
   - 支持通过 window.API_BASE 配置 basePath

4. 修复 .gitignore
   - 使用通用模式支持 commands 目录
   - 无需为新示例项目修改配置
2025-10-14 22:23:30 +08:00
7ac4b99a9e feat: WebUI 重构与更新 2025-10-13 01:36:49 +08:00
fc7eeb3696 feat: 更新 table 与 WebUI 静态资源 2025-10-12 12:52:35 +08:00
c7cb1ae6c6 feat: 更新 sstable 并新增 examples/complex 示例 2025-10-12 06:02:19 +08:00
03ec262ca5 feat: updates to btree/index/query/sstable/table 2025-10-11 16:33:18 +08:00
c8cbe4178f 优化:改进 Compaction 和 WebUI 日志
- 简化 Compaction 日志输出,使用结构化日志
- 优化 WebUI 命令行参数处理
- 改进 WebUI 服务器启动和错误处理
- 统一日志格式和输出方式
2025-10-10 23:41:23 +08:00
3148bf226d 重构:添加结构化日志支持
- 引入 slog 替代 fmt.Printf 日志输出
- Database 和 Table 支持自定义 Logger
- 优化错误和警告信息的结构化记录
- 改进 Compaction 配置传递机制
- 完善数据库和表的初始化流程
2025-10-10 22:11:58 +08:00
65bdf1c50d 文档:添加 MIT LICENSE 并优化 README
- 添加 MIT 开源许可证
- 精简 README.md 内容
- 完善项目描述
2025-10-10 20:09:32 +08:00
5b8e5e7bd2 文档:更新和优化项目文档
- 更新 DESIGN.md:
  - 强调强制 Schema(21 种类型)
  - 更新核心代码行数为 ~5,400 行
  - 优化 ROW1 格式说明(英文)
  - 完善性能指标和项目成果
- 精简 DOCS.md 和 README.md
- 统一文档风格和术语
2025-10-10 20:00:23 +08:00
56 changed files with 6061 additions and 3539 deletions

1
.gitignore vendored
View File

@@ -38,6 +38,7 @@ testdb/
# Example binaries and data
/examples/*/data/
/examples/*/*
!/examples/*/commands
!/examples/*/*.go
!/examples/*/go.mod
!/examples/*/go.sum

169
DESIGN.md
View File

@@ -8,9 +8,9 @@
1. **极简架构** - 放弃复杂的 LSM Tree 多层设计,使用简单的两层结构
2. **高并发写入** - WAL + MemTable 保证 200,000+ writes/s
3. **快速查询** - mmap B+Tree 索引 + 二级索引1-5 ms 查询性能
4. **低内存占用** - mmap 零拷贝,应用层内存 < 200 MB
5. **功能完善** - 支持 Schema索引条件查询等高级特性
6. **生产可用** - 核心代码 5399 包含完善的错误处理和数据一致性保证
4. **低内存占用** - mmap 零拷贝,应用层内存 < 150 MB
5. **功能完善** - 强制 Schema21 种类型索引条件查询等高级特性
6. **生产可用** - 核心代码 ~5,400 包含完善的错误处理和数据一致性保证
## 🏗️ 核心架构
@@ -19,10 +19,10 @@
│ SRDB Architecture │
├─────────────────────────────────────────────────────────────┤
│ Application Layer │
│ ┌───────────────┐ ┌──────────────────────────┐
│ │ Database │->│ Table │
│ │ (Multi-Table) │ │ (Schema + Storage) │
│ └───────────────┘ └──────────────────────────┘
│ ┌───────────────┐ ┌──────────────────────────┐ │
│ │ Database │->│ Table │ │
│ │ (Multi-Table) │ │ (Schema + Storage) │ │
│ └───────────────┘ └──────────────────────────┘ │
├─────────────────────────────────────────────────────────────┤
│ Write Path (High Concurrency) │
│ ┌─────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
@@ -50,8 +50,10 @@
│ │ │ - Leaf Nodes → Data Offset │ │ │
│ │ ├─────────────────────────────────────────┤ │ │
│ │ │ Data Blocks (Binary Format) │ │ │
│ │ │ - 有 Schema: 二进制编码 │ │ │
│ │ │ - 无 Schema: JSON 格式 │ │ │
│ │ │ - ROW1 Format: Binary Encoding │ │ │
│ │ │ [Magic:4B][Seq:8B][Time:8B] │ │ │
│ │ │ [Fields:2B][OffsetTable][Data] │ │ │
│ │ │ - Supports zero-copy & partial reads │ │ │
│ │ └─────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Secondary Indexes (Optional) │ │
@@ -85,38 +87,6 @@
## 📁 文件组织结构
### 代码目录结构
```
srdb/ ← 项目根目录
├── go.mod ← 模块定义: code.tczkiot.com/wlw/srdb
├── DESIGN.md ← 本设计文档
├── CLAUDE.md ← Claude Code 指导文档
├── database.go ← 数据库管理 (多表)
├── table.go ← 表管理 (带 Schema)
├── errors.go ← 错误定义和处理
├── wal.go ← WAL 实现 (Write-Ahead Log)
├── memtable.go ← MemTable 实现 (map + sorted slice)
├── sstable.go ← SSTable 文件 (读写器、管理器、编码)
├── btree.go ← B+Tree 索引 (构建器、读取器)
├── version.go ← 版本控制 (MANIFEST 管理)
├── compaction.go ← Compaction 压缩合并
├── schema.go ← Schema 定义与验证
├── index.go ← 二级索引管理器
├── index_btree.go ← 索引 B+Tree 实现
├── query.go ← 查询构建器和表达式求值
├── examples/ ← 示例程序目录
│ ├── webui/ ← Web UI 管理工具
│ └── ... (其他示例)
└── webui/ ← Web UI 静态资源
└── ...
```
### 运行时数据目录结构
```
@@ -152,16 +122,16 @@ database_dir/ ← 数据库目录
- 崩溃恢复支持
文件格式:
┌─────────────────────────────────────┐
│ WAL Entry │
├─────────────────────────────────────┤
│ CRC32 (4 bytes) │
│ Length (4 bytes) │
│ Type (1 byte): Put │
│ Key (8 bytes): _seq │
│ Value Length (4 bytes) │
│ Value (N bytes): 序列化的行数据
└─────────────────────────────────────┘
┌───────────────────────────────────────
│ WAL Entry
├───────────────────────────────────────
│ CRC32 (4 bytes)
│ Length (4 bytes)
│ Type (1 byte): Put
│ Key (8 bytes): _seq
│ Value Length (4 bytes)
│ Value (N bytes): Serialized row data
└───────────────────────────────────────
性能:
- 顺序写入: 极快
@@ -334,49 +304,65 @@ func (s *MmapSST) readNode(offset int64) *BTreeNode {
- 零拷贝: 无内存分配
```
### 5. Schema 系统 (新增功能)
### 5. Schema 系统
```
设计:
- 类型定义和验证
- 必填字段检查
- 唯一性约束
- 默认值支持
- 强制 Schema所有表必须定义
- 21 种精确类型映射
- Nullable 字段支持
- 类型验证和转换
- 索引标记Indexed: true
支持的类型21 种):
1. 有符号整数5种: Int, Int8, Int16, Int32, Int64
2. 无符号整数5种: Uint, Uint8, Uint16, Uint32, Uint64
3. 浮点数2种: Float32, Float64
4. 字符串1种: String
5. 布尔1种: Bool
6. 特殊类型5种: Byte, Rune, Decimal, Time, Duration
7. 复杂类型2种: Object (JSON), Array (JSON)
实现:
type Schema struct {
Fields []FieldDefinition
TableName string
Fields []Field
}
type FieldDefinition struct {
type Field struct {
Name string
Type string // "string", "int", "float", "bool"
Required bool // 是否必填
Unique bool // 是否唯一
Default interface{} // 默认值
Type FieldType // 21 种类型之一
Indexed bool // 是否创建索引
Nullable bool // 是否允许 NULL
Comment string // 字段注释
}
func (s *Schema) Validate(data map[string]interface{}) error {
for _, field := range s.Fields {
// 检查必填字段
// 检查类型匹配
// 应用默认值
}
// 1. 检查必填字段
// 2. 类型验证和转换
// 3. Nullable 检查
// 4. 返回验证后的数据
}
使用示例:
schema := &schema.Schema{
Fields: []schema.FieldDefinition{
{Name: "name", Type: "string", Required: true},
{Name: "age", Type: "int", Required: false},
{Name: "email", Type: "string", Unique: true},
},
}
schema, _ := NewSchema("users", []Field{
{Name: "name", Type: String, Indexed: false},
{Name: "age", Type: Int32, Indexed: false},
{Name: "email", Type: String, Indexed: true},
{Name: "balance", Type: Decimal, Nullable: true},
})
table, _ := db.CreateTable("users", schema)
类型转换规则:
- 相同类型:直接接受
- 兼容类型:自动转换(有符号 ↔ 无符号,需非负)
- 类型提升:整数 → 浮点
- JSON 兼容float64 → 整数(需为整数值)
- 负数 → 无符号:拒绝
```
### 6. 二级索引 (新增功能)
### 6. 二级索引
```
设计:
@@ -472,7 +458,7 @@ qb.Where("email", query.EndsWith, "@gmail.com")
4. 返回匹配的行
```
### 8. 数据库和表管理 (新增功能)
### 8. 数据库和表管理
```
设计:
@@ -607,27 +593,6 @@ Flush 流程 (后台):
## 📊 性能指标
### 代码规模
```
核心代码: ~13,000 行 (不含测试和示例)
├── table.go: 表管理和存储引擎
├── wal.go: WAL 实现
├── memtable.go: MemTable 实现
├── sstable.go: SSTable 文件读写
├── btree.go: B+Tree 索引
├── version.go: 版本控制 (MANIFEST)
├── compaction.go: Compaction 压缩
├── index.go: 二级索引
├── query.go: 查询构建器
├── schema.go: Schema 验证
├── errors.go: 错误处理
└── database.go: 数据库管理
测试代码: ~2000+ 行
示例代码: ~1000+ 行
总计: 16,000+ 行
```
### 写入性能
```
单线程: 50,000 writes/s
@@ -884,8 +849,8 @@ SRDB 是一个功能完善的高性能 Append-Only 数据库引擎:
- **高并发写入**: WAL + MemTable200K+ w/s
- **快速查询**: mmap B+Tree + 二级索引1-5 ms
- **低内存占用**: mmap 零拷贝< 150 MB
- **功能完善**: Schema索引条件查询多表管理
- **生产可用**: 5399 行核心代码完善的错误处理和数据一致性
- **功能完善**: 强制 Schema21 种类型索引条件查询多表管理
- **生产可用**: ~5,400 行核心代码完善的错误处理和数据一致性
- **简单可靠**: Append-Only无更新/删除的复杂性
**技术亮点:**
@@ -912,8 +877,8 @@ SRDB 是一个功能完善的高性能 Append-Only 数据库引擎:
- 传统 OLTP 系统
**项目成果:**
- 核心代码: ~13,000
- 核心代码: ~5,400 精简高效
- 测试代码: ~2,000+
- 示例程序: 13+ 个完整示例
- 文档: 完善的设计和使用文档
- 性能: 达到设计目标
- 文档: 完善的设计和使用文档DESIGN.mdCLAUDE.mdDOCS.mdREADME.md
- 性能: 达到设计目标200K+ w/s 写入1-5 ms 查询

66
DOCS.md
View File

@@ -12,7 +12,7 @@
- [Scan 方法](#scan-方法)
- [Object 和 Array 类型](#object-和-array-类型)
- [索引](#索引)
- [事务和并发](#事务和并发)
- [并发控制](#并发控制)
- [性能优化](#性能优化)
- [错误处理](#错误处理)
- [最佳实践](#最佳实践)
@@ -22,7 +22,7 @@
## 概述
SRDB (Simple Row Database) 是一个用 Go 编写的高性能嵌入式数据库,采用 LSM-Tree 架构,专为时序数据和高并发写入场景设计。
SRDB (Simple Row Database) 是一个用 Go 编写的高性能嵌入式数据库,采用 Append-Only 架构(参考 LSM-Tree 设计理念),专为时序数据和高并发写入场景设计。
### 核心特性
@@ -996,9 +996,7 @@ rows, _ := table.Query().Contains("name", "Alice").Rows()
---
## 事务和并发
### 并发控制
## 并发控制
SRDB 使用 **MVCC (多版本并发控制)** 实现无锁并发读写:
@@ -1036,18 +1034,6 @@ for i := 0; i < 100; i++ {
wg.Wait()
```
### 事务支持
⚠️ **当前版本不支持显式事务**,但保证:
- 单条写入的原子性(通过 WAL
- 数据持久性WAL fsync
- 崩溃恢复WAL 重放)
未来版本计划支持:
- [ ] 显式事务 API
- [ ] 批量操作的原子性
- [ ] ACID 保证
---
## 性能优化
@@ -1310,34 +1296,52 @@ table.Insert(data) // 错误未处理
## 架构细节
### LSM-Tree 结
### Append-Only 架
SRDB 采用 Append-Only 架构(参考 LSM-Tree 设计理念),分为两层:
1. **内存层** - WAL + MemTable (Active + Immutable)
2. **磁盘层** - 带 B+Tree 索引的 SST 文件分层存储L0-L3
```
写入流程:
数据 → WAL持久化→ MemTable → Immutable MemTable → Level 0 SST → Compaction → Level 1-6
数据 → WAL持久化→ MemTable → Flush → SST L0 → Compaction → SST L1-L3
读取流程:
查询 → MemTableO(1))→ Immutable MemTables → SST FilesB+Tree
```
### 文件组织
```
database_dir/
├── database.meta # 数据库元数据
── MANIFEST # 版本控制
└── table_name/
├── schema.json # 表 Schema
├── MANIFEST # 表级版本控制
├── 000001.wal # WAL 文件
├── 000001.sst # SST 文件
├── 000002.sst
── idx_email.sst # 索引文件
├── database.meta # 数据库元数据
── table_name/ # 每表一个目录
├── schema.json # 表 Schema 定义
├── MANIFEST-000001 # 表级版本控制
├── CURRENT # 当前 MANIFEST 指针
├── wal/ # WAL 子目录
├── 000001.wal # WAL 文件
│ └── CURRENT # 当前 WAL 指针
── sst/ # SST 子目录L0-L3 层级文件
│ └── 000001.sst # SST 文件B+Tree + 数据)
└── idx/ # 索引子目录
└── idx_email.sst # 二级索引文件
```
### 设计特点
- **Append-Only** - 无原地更新,简化并发控制
- **MemTable** - `map[int64][]byte + sorted slice`O(1) 读写
- **SST 文件** - 4KB 节点的 B+Treemmap 零拷贝访问
- **二进制编码** - ROW1 格式,无压缩,优先查询性能
- **Compaction** - 后台异步合并,按层级管理文件大小
### Compaction 策略
- **Level 0**: 文件数量 ≥ 4 触发
- **Level 1-6**: 总大小超过阈值触发
- **Level 0-3**: 文件数量或总大小超过阈值时触发
- **Score 计算**: `size / max_size` 或 `file_count / max_files`
- **文件大小**: L0=2MB, L1=10MB, L2=50MB, L3=100MB, L4+=200MB
- **文件大小**: L0=2MB, L1=10MB, L2=50MB, L3=100MB
### 性能指标

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 SRDB Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

657
README.md
View File

@@ -1,48 +1,28 @@
# SRDB - Simple Row Database
[![Go Version](https://img.shields.io/badge/Go-1.24+-00ADD8?style=flat&logo=go)](https://golang.org/)
[![Go Version](https://img.shields.io/badge/Go-1.21+-00ADD8?style=flat&logo=go)](https://golang.org/)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
一个基于 LSM-Tree 的高性能嵌入式数据库,专为时序数据和日志存储设计。
一个用 Go 编写的高性能 Append-Only 时序数据库引擎,专为高并发写入和快速查询设计。
## 🎯 特性
## 🎯 核心特性
### 核心功能
- **LSM-Tree 架构** - 高效的写入性能和空间利用率
- **MVCC 并发控制** - 支持多版本并发读写
- **WAL 持久化** - 写前日志保证数据安全
- **自动 Compaction** - 智能的多层级数据合并策略
- **索引支持** - 快速的字段查询能力
- **Schema 管理** - 灵活的表结构定义,支持 21 种类型
- **复杂类型** - 原生支持 Objectmap和 Arrayslice
### 查询能力
- **链式查询 API** - 流畅的查询构建器
- **丰富的操作符** - 支持 `=`, `!=`, `<`, `>`, `IN`, `BETWEEN`, `CONTAINS`
- **复合条件** - `AND`, `OR`, `NOT` 逻辑组合
- **字段选择** - 按需加载指定字段,优化性能
- **游标模式** - 惰性加载,支持大数据集遍历
- **Append-Only 架构** - WAL + MemTable + mmap B+Tree SST简化并发控制
- **强类型 Schema** - 21 种数据类型,包括 Objectmap和 Arrayslice
- **高性能写入** - 200K+ 写/秒(多线程),<1ms 延迟p99
- **快速查询** - <0.1ms内存1-5ms磁盘支持二级索引
- **智能 Scan** - 自动扫描到结构体完整支持复杂类型
### 管理工具
- **Web UI** - 现代化的数据库管理界面
- **命令行工具** - 丰富的诊断和维护工具
- **实时监控** - LSM-Tree 结构和 Compaction 状态可视化
---
- **链式查询 API** - 18 种操作符支持复合条件
- **自动 Compaction** - 后台异步合并优化存储空间
- **零拷贝读取** - mmap 访问 SST 文件内存占用 <150MB
- **Web 管理界面** - 现代化的数据浏览和监控工具
## 📋 目录
- [快速开始](#快速开始)
- [基本用法](#基本用法)
- [查询 API](#查询-api)
- [Scan 方法](#scan-方法---扫描到结构体)
- [Object 和 Array 类型](#object-和-array-类型)
- [Web UI](#web-ui)
- [架构设计](#架构设计)
- [性能特点](#性能特点)
- [开发指南](#开发指南)
- [核心概念](#核心概念)
- [文档](#文档)
- [开发](#开发)
---
@@ -54,6 +34,8 @@
go get code.tczkiot.com/wlw/srdb
```
**要求**Go 1.21+
### 基本示例
```go
@@ -73,12 +55,14 @@ func main() {
}
defer db.Close()
// 2. 定义 Schema
// 2. 定义 Schema强类型21 种类型)
schema, err := srdb.NewSchema("users", []srdb.Field{
{Name: "id", Type: srdb.Int64, Indexed: true, Comment: "用户ID"},
{Name: "name", Type: srdb.String, Indexed: false, Comment: "用户名"},
{Name: "id", Type: srdb.Uint32, Indexed: true, Comment: "用户ID"},
{Name: "name", Type: srdb.String, Comment: "用户名"},
{Name: "email", Type: srdb.String, Indexed: true, Comment: "邮箱"},
{Name: "age", Type: srdb.Int32, Indexed: false, Comment: "年龄"},
{Name: "age", Type: srdb.Int32, Comment: "年龄"},
{Name: "tags", Type: srdb.Array, Comment: "标签"}, // Array 类型
{Name: "settings", Type: srdb.Object, Comment: "设置"}, // Object 类型
})
if err != nil {
log.Fatal(err)
@@ -92,555 +76,127 @@ func main() {
// 4. 插入数据
err = table.Insert(map[string]any{
"id": 1,
"id": uint32(1),
"name": "Alice",
"email": "alice@example.com",
"age": 25,
"age": int32(25),
"tags": []any{"golang", "database"},
"settings": map[string]any{
"theme": "dark",
"lang": "zh-CN",
},
})
if err != nil {
log.Fatal(err)
}
// 5. 查询数据
rows, err := table.Query().
// 5. 查询并扫描到结构体
type User struct {
ID uint32 `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Age int32 `json:"age"`
Tags []string `json:"tags"`
Settings map[string]string `json:"settings"`
}
var users []User
err = table.Query().
Eq("name", "Alice").
Gte("age", 18).
Rows()
Scan(&users)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
// 6. 遍历结果
for rows.Next() {
row := rows.Row()
fmt.Printf("User: %v\n", row.Data())
}
fmt.Printf("Found %d users\n", len(users))
fmt.Printf("Tags: %v\n", users[0].Tags)
fmt.Printf("Settings: %v\n", users[0].Settings)
}
```
---
## 📖 基本用法
## 💡 核心概念
### 数据库操作
### 架构
```go
// 打开数据库
db, err := srdb.Open("./data")
SRDB 使用 **Append-Only 架构**分为两层
// 列出所有表
tables := db.ListTables()
// 获取表
table, err := db.GetTable("users")
// 删除表
err = db.DropTable("users")
// 关闭数据库
db.Close()
```
### 表操作
```go
// 插入数据
err := table.Insert(map[string]any{
"name": "Bob",
"age": 30,
})
// 获取单条数据(通过序列号)
row, err := table.Get(seq)
// 删除数据
err := table.Delete(seq)
// 更新数据
err := table.Update(seq, map[string]any{
"age": 31,
})
```
### Schema 定义
```go
schema, err := srdb.NewSchema("logs", []srdb.Field{
{
Name: "level",
Type: srdb.String,
Indexed: true,
Comment: "日志级别",
},
{
Name: "message",
Type: srdb.String,
Indexed: false,
Comment: "日志内容",
},
{
Name: "timestamp",
Type: srdb.Int64,
Indexed: true,
Comment: "时间戳",
},
{
Name: "metadata",
Type: srdb.Object,
Indexed: false,
Comment: "元数据map",
},
{
Name: "tags",
Type: srdb.Array,
Indexed: false,
Comment: "标签slice",
},
})
```
**支持的字段类型**21 种):
**有符号整数**
- `Int`, `Int8`, `Int16`, `Int32`, `Int64`
**无符号整数**
- `Uint`, `Uint8`, `Uint16`, `Uint32`, `Uint64`
**浮点数**
- `Float32`, `Float64`
**基础类型**
- `String` - 字符串
- `Bool` - 布尔值
- `Byte` - 字节uint8
- `Rune` - 字符int32
**特殊类型**
- `Decimal` - 高精度十进制(需要 shopspring/decimal
- `Time` - 时间戳time.Time
**复杂类型**
- `Object` - 对象map[string]xxx、struct{}、*struct{}
- `Array` - 数组([]xxx 切片)
---
## 🔍 查询 API
### 基本查询
```go
// 等值查询
rows, err := table.Query().Eq("name", "Alice").Rows()
// 范围查询
rows, err := table.Query().
Gte("age", 18).
Lt("age", 60).
Rows()
// IN 查询
rows, err := table.Query().
In("status", []any{"active", "pending"}).
Rows()
// BETWEEN 查询
rows, err := table.Query().
Between("age", 18, 60).
Rows()
```
### 字符串查询
```go
// 包含
rows, err := table.Query().Contains("message", "error").Rows()
// 前缀匹配
rows, err := table.Query().StartsWith("email", "admin@").Rows()
// 后缀匹配
rows, err := table.Query().EndsWith("filename", ".log").Rows()
```
### 复合条件
```go
// AND 条件
rows, err := table.Query().
Eq("status", "active").
Gte("age", 18).
Rows()
// OR 条件
rows, err := table.Query().
Where(srdb.Or(
srdb.Eq("role", "admin"),
srdb.Eq("role", "moderator"),
)).
Rows()
// 复杂组合
rows, err := table.Query().
Where(srdb.And(
srdb.Eq("status", "active"),
srdb.Or(
srdb.Gte("age", 18),
srdb.Eq("verified", true),
),
)).
Rows()
```
### 字段选择
```go
// 只查询指定字段(性能优化)
rows, err := table.Query().
Select("id", "name", "email").
Eq("status", "active").
Rows()
```
### 结果处理
```go
// 游标模式(惰性加载)
rows, err := table.Query().Rows()
defer rows.Close()
for rows.Next() {
row := rows.Row()
fmt.Println(row.Data())
}
// 获取第一条
row, err := table.Query().First()
// 获取最后一条
row, err := table.Query().Last()
// 收集所有结果
data := rows.Collect()
// 获取总数
count := rows.Count()
```
### Scan 方法 - 扫描到结构体
SRDB 提供智能的 Scan 方法,完整支持 Object 和 Array 类型:
```go
// 定义结构体
type User struct {
Name string `json:"name"`
Email string `json:"email"`
Settings map[string]string `json:"settings"` // Object 类型
Tags []string `json:"tags"` // Array 类型
}
// 扫描多行到切片
var users []User
table.Query().Scan(&users)
// 扫描单行到结构体(智能判断)
var user User
table.Query().Eq("name", "Alice").Scan(&user)
// Row.Scan - 扫描当前行
row, _ := table.Query().First()
var user User
row.Scan(&user)
// 部分字段扫描(性能优化)
type UserBrief struct {
Name string `json:"name"`
Email string `json:"email"`
}
var briefs []UserBrief
table.Query().Select("name", "email").Scan(&briefs)
```
**Scan 特性**
- ✅ 智能判断目标类型(切片 vs 结构体)
- ✅ 完整支持 Objectmap和 Arrayslice类型
- ✅ 支持嵌套结构
- ✅ 结合 Select() 优化性能
详细示例:[examples/scan_demo](examples/scan_demo/README.md)
### 完整的操作符列表
| 操作符 | 方法 | 说明 |
|--------|------|------|
| `=` | `Eq(field, value)` | 等于 |
| `!=` | `NotEq(field, value)` | 不等于 |
| `<` | `Lt(field, value)` | 小于 |
| `>` | `Gt(field, value)` | 大于 |
| `<=` | `Lte(field, value)` | 小于等于 |
| `>=` | `Gte(field, value)` | 大于等于 |
| `IN` | `In(field, values)` | 在列表中 |
| `NOT IN` | `NotIn(field, values)` | 不在列表中 |
| `BETWEEN` | `Between(field, min, max)` | 在范围内 |
| `NOT BETWEEN` | `NotBetween(field, min, max)` | 不在范围内 |
| `CONTAINS` | `Contains(field, pattern)` | 包含子串 |
| `NOT CONTAINS` | `NotContains(field, pattern)` | 不包含子串 |
| `STARTS WITH` | `StartsWith(field, prefix)` | 以...开头 |
| `NOT STARTS WITH` | `NotStartsWith(field, prefix)` | 不以...开头 |
| `ENDS WITH` | `EndsWith(field, suffix)` | 以...结尾 |
| `NOT ENDS WITH` | `NotEndsWith(field, suffix)` | 不以...结尾 |
| `IS NULL` | `IsNull(field)` | 为空 |
| `IS NOT NULL` | `NotNull(field)` | 不为空 |
### Object 和 Array 类型
SRDB 支持复杂的数据类型,可以存储 JSON 风格的对象和数组:
```go
// 定义包含复杂类型的表
type Article struct {
Title string `srdb:"field:title"`
Content string `srdb:"field:content"`
Tags []string `srdb:"field:tags"` // Array 类型
Metadata map[string]any `srdb:"field:metadata"` // Object 类型
Authors []string `srdb:"field:authors"` // Array 类型
}
// 使用 StructToFields 自动生成 Schema
fields, _ := srdb.StructToFields(Article{})
schema, _ := srdb.NewSchema("articles", fields)
table, _ := db.CreateTable("articles", schema)
// 插入数据
table.Insert(map[string]any{
"title": "SRDB 使用指南",
"content": "...",
"tags": []any{"database", "golang", "lsm-tree"},
"metadata": map[string]any{
"category": "tech",
"views": 1250,
"featured": true,
},
"authors": []any{"Alice", "Bob"},
})
// 查询和扫描
var article Article
table.Query().Eq("title", "SRDB 使用指南").Scan(&article)
fmt.Println(article.Tags) // ["database", "golang", "lsm-tree"]
fmt.Println(article.Metadata["category"]) // "tech"
fmt.Println(article.Metadata["views"]) // 1250
```
**支持的场景**
-`map[string]xxx` - 任意键值对
-`struct{}` - 结构体(自动转换为 Object
-`*struct{}` - 结构体指针
-`[]xxx` - 任意类型的切片
- ✅ 嵌套的 Object 和 Array
- ✅ 空对象 `{}` 和空数组 `[]`
**存储细节**
- Object 和 Array 使用 JSON 编码存储
- 存储格式:`[length: uint32][JSON data]`
- 零值Object 为 `{}`Array 为 `[]`
- 支持任意嵌套深度
---
## 🌐 Web UI
SRDB 提供了一个功能强大的 Web 管理界面。
### 启动 Web UI
```bash
cd examples/webui
# 基本启动
go run main.go serve
# 自定义配置
go run main.go serve --db /path/to/database --port 3000
# 启用自动数据插入(演示模式)
go run main.go serve --auto-insert
```
访问http://localhost:8080
### 功能特性
- **表管理** - 查看所有表及其 Schema
- **数据浏览** - 分页浏览表数据,支持列选择
- **Manifest 查看** - 可视化 LSM-Tree 结构
- **实时监控** - Compaction 状态和统计
- **主题切换** - 深色/浅色主题
- **响应式设计** - 完美适配移动设备
详细文档:[examples/webui/README.md](examples/webui/README.md)
---
## 🏗️ 架构设计
### LSM-Tree 结构
1. **内存层** - WALWrite-Ahead Log+ MemTableActive + Immutable
2. **磁盘层** - SST 文件 B+Tree 索引分层存储L0-L3
```
写入流程:
数据
WAL持久化
MemTable内存
Immutable MemTable
Level 0 SST磁盘
Level 1-6 SSTCompaction
数据 → WAL持久化→ MemTable → Flush → SST L0 → Compaction → SST L1-L3
读取流程:
查询 → MemTableO(1))→ Immutable MemTables → SST FilesB+Tree
```
### 组件架构
### 数据文件
```
Database
├── Table (Schema + Storage)
│ ├── MemTable Manager
│ │ ├── Active MemTable
│ └── Immutable MemTables
├── SSTable Manager
└── SST Files (Level 0-6)
│ ├── WAL Manager
│ └── Write-Ahead Log
├── Version Manager
│ └── MVCC Versions
└── Compaction Manager
── Picker选择策略
│ └── Worker执行合并
└── Query Builder
└── Expression Engine
database_dir/
├── database.meta # 数据库元数据
└── table_name/ # 每表一个目录
├── schema.json # 表 Schema 定义
├── MANIFEST-000001 # 表级版本控制
├── CURRENT # 当前 MANIFEST 指针
├── wal/ # WAL 子目录
│ ├── 000001.wal # WAL 文件
│ └── CURRENT # 当前 WAL 指针
├── sst/ # SST 子目录L0-L3 层级文件)
│ └── 000001.sst # SST 文件B+Tree + 数据)
└── idx/ # 索引子目录
── idx_email.sst # 二级索引文件
```
### 数据流
### 设计特点
**写入路径**
```
Insert → WAL → MemTable → Flush → SST Level 0 → Compaction → SST Level 1-6
```
**读取路径**
```
Query → MemTable → Immutable MemTables → SST Files (Level 0-6)
```
**Compaction 触发**
- Level 0文件数量 ≥ 4
- Level 1-6总大小超过阈值
- Score 计算:`size / max_size``file_count / max_files`
---
## ⚡ 性能特点
### 写入性能
- **顺序写入** - WAL 和 MemTable 顺序写入,性能极高
- **批量刷盘** - MemTable 达到阈值后批量刷盘
- **异步 Compaction** - 后台异步执行,不阻塞写入
### 读取性能
- **内存优先** - 优先从 MemTable 读取
- **Bloom Filter** - 快速判断 key 是否存在TODO
- **索引加速** - 索引字段快速定位
- **按需加载** - 游标模式惰性加载,节省内存
### 空间优化
- **Snappy 压缩** - SST 文件自动压缩
- **增量合并** - Compaction 只合并必要的文件
- **垃圾回收** - 自动清理过期版本
### 性能指标(参考)
| 操作 | 性能 |
|------|------|
| 顺序写入 | ~100K ops/s |
| 随机写入 | ~50K ops/s |
| 点查询 | ~10K ops/s |
| 范围扫描 | ~1M rows/s |
*注:实际性能取决于硬件配置和数据特征*
---
## 🛠️ 开发指南
### 项目结构
```
srdb/
├── btree.go # B-Tree 索引实现
├── compaction.go # Compaction 管理器
├── database.go # 数据库管理
├── errors.go # 错误定义和处理
├── index.go # 索引管理
├── index_btree.go # 索引 B+Tree
├── memtable.go # 内存表
├── query.go # 查询构建器
├── schema.go # Schema 定义
├── sstable.go # SSTable 文件
├── table.go # 表管理(含存储引擎)
├── version.go # 版本管理MVCC
├── wal.go # Write-Ahead Log
├── webui/ # Web UI
│ ├── webui.go # HTTP 服务器
│ └── static/ # 前端资源
└── examples/ # 示例程序
└── webui/ # Web UI 工具
```
### 运行测试
```bash
# 运行所有测试
go test ./...
# 运行特定测试
go test -v -run TestTable
# 性能测试
go test -bench=. -benchmem
```
### 构建示例
```bash
# 构建 WebUI
cd examples/webui
go build -o webui main.go
# 运行
./webui serve --db ./data
```
- **Append-Only** - 无原地更新简化并发控制
- **MemTable** - `map[int64][]byte + sorted slice`O(1) 读写
- **SST 文件** - 4KB 节点的 B+Treemmap 零拷贝访问
- **二进制编码** - ROW1 格式无压缩优先查询性能
- **Compaction** - 后台异步合并按层级管理文件大小
---
## 📚 文档
### 核心文档
- [设计文档](DESIGN.md) - 详细的架构设计和实现原理
- [CLAUDE.md](CLAUDE.md) - 完整的开发者指南
- [Nullable 指南](NULLABLE_GUIDE.md) - Nullable 字段使用说明
- [API 文档](https://pkg.go.dev/code.tczkiot.com/wlw/srdb) - Go API 参考
### 示例和教程
- [Scan 方法指南](examples/scan_demo/README.md) - 扫描到结构体,支持 Object 和 Array
- [WebUI 工具](examples/webui/README.md) - Web 管理界面使用指南
- [所有类型示例](examples/all_types/) - 21 种类型的完整示例
- [Nullable 示例](examples/nullable/) - Nullable 字段的使用
- [DOCS.md](DOCS.md) - 完整 API 文档和使用指南
- [DESIGN.md](CLAUDE.md) - 数据库设计文档
### 示例教程
- [WebUI 工具](examples/webui/README.md) - Web 管理界面
---
## 🛠️ 开发
### 运行测试
```bash
# 所有测试
go test -v ./...
# 单个测试
go test -v -run TestTable
# 性能测试
go test -bench=. -benchmem
```
### 构建 WebUI
```bash
cd examples/webui
go build -o webui main.go
./webui serve --db ./data
```
---
@@ -673,13 +229,12 @@ MIT License - 详见 [LICENSE](LICENSE) 文件
## 🙏 致谢
- [LevelDB](https://github.com/google/leveldb) - LSM-Tree 设计灵感
- [LevelDB](https://github.com/google/leveldb) - 架构设计参考
- [RocksDB](https://github.com/facebook/rocksdb) - Compaction 策略参考
- [Lit](https://lit.dev/) - Web Components 框架
---
## 📧 联系方式
## 📧 联系
- 项目主页https://code.tczkiot.com/wlw/srdb
- Issue 跟踪https://code.tczkiot.com/wlw/srdb/issues

339
btree.go
View File

@@ -52,7 +52,7 @@ B+Tree 用于索引 SSTable 和 Index 文件,提供 O(log n) 查询性能。
[Header: 32B]
[Keys: Key0(8B), Key1(8B), Key2(8B)]
[Children: Child0(8B), Child1(8B), Child2(8B), Child3(8B)]
查询规则:
- key < Key0 Child0
- Key0 key < Key1 Child1
@@ -144,27 +144,29 @@ func NewLeafNode() *BTreeNode {
// Marshal 序列化节点到 4 KB
//
// 布局:
// [Header: 32B]
// [Keys: KeyCount * 8B]
// [Values: 取决于节点类型]
// - Internal: Children (KeyCount+1) * 8B
// - Leaf: 交错存储 (Offset, Size) 对,每对 12BKeyCount * 12B
//
// [Header: 32B]
// [Keys: KeyCount * 8B]
// [Values: 取决于节点类型]
// - Internal: Children (KeyCount+1) * 8B
// - Leaf: 交错存储 (Offset, Size) 对,每对 12B共 KeyCount * 12B
//
// 示例叶子节点KeyCount=3
// Offset | Size | Content
// -------|------|----------------------------------
// 0 | 1 | NodeType = 1 (Leaf)
// 1 | 2 | KeyCount = 3
// 3 | 1 | Level = 0
// 4 | 28 | Reserved
// 32 | 24 | Keys [100, 200, 300]
// 56 | 8 | DataOffset0 = 1000
// 64 | 4 | DataSize0 = 50
// 68 | 8 | DataOffset1 = 2000
// 76 | 4 | DataSize1 = 60
// 80 | 8 | DataOffset2 = 3000
// 88 | 4 | DataSize2 = 70
// 92 | 4004 | Padding (unused)
//
// Offset | Size | Content
// -------|------|----------------------------------
// 0 | 1 | NodeType = 1 (Leaf)
// 1 | 2 | KeyCount = 3
// 3 | 1 | Level = 0
// 4 | 28 | Reserved
// 32 | 24 | Keys [100, 200, 300]
// 56 | 8 | DataOffset0 = 1000
// 64 | 4 | DataSize0 = 50
// 68 | 8 | DataOffset1 = 2000
// 76 | 4 | DataSize1 = 60
// 80 | 8 | DataOffset2 = 3000
// 88 | 4 | DataSize2 = 70
// 92 | 4004 | Padding (unused)
func (n *BTreeNode) Marshal() []byte {
buf := make([]byte, BTreeNodeSize)
@@ -213,10 +215,12 @@ func (n *BTreeNode) Marshal() []byte {
// UnmarshalBTree 从字节数组反序列化节点
//
// 参数:
// data: 4KB 节点数据(通常来自 mmap
//
// data: 4KB 节点数据(通常来自 mmap
//
// 返回:
// *BTreeNode: 反序列化后的节点
//
// *BTreeNode: 反序列化后的节点
//
// 零拷贝优化:
// - 直接从 mmap 数据读取,不复制整个节点
@@ -310,15 +314,16 @@ func (n *BTreeNode) AddData(key int64, offset int64, size int32) error {
// BTreeBuilder 从下往上构建 B+Tree
//
// 构建流程:
// 1. Add(): 添加所有 (key, offset, size) 到叶子节点
// - 当叶子节点满时,创建新的叶子节点
// - 所有叶子节点按 key 有序
//
// 2. Build(): 从叶子层向上构建
// - Level 0: 叶子节点(已创建)
// - Level 1: 为叶子节点创建父节点(内部节点)
// - Level 2+: 递归创建更高层级
// - 最终返回根节点偏移量
// 1. Add(): 添加所有 (key, offset, size) 到叶子节点
// - 当叶子节点满时,创建新的叶子节点
// - 所有叶子节点按 key 有序
//
// 2. Build(): 从叶子层向上构建
// - Level 0: 叶子节点(已创建)
// - Level 1: 为叶子节点创建父节点(内部节点)
// - Level 2+: 递归创建更高层级
// - 最终返回根节点偏移量
//
// 示例100 个 keyOrder=200
// - 叶子层: 1 个叶子节点100 个 key
@@ -453,13 +458,13 @@ func (b *BTreeBuilder) buildLevel(children []*BTreeNode, childOffsets []int64, l
// BTreeReader 用于查询 B+Tree (mmap)
//
// 查询流程:
// 1. 从根节点开始
// 2. 如果是内部节点:
// - 二分查找确定子节点
// - 跳转到子节点继续查找
// 3. 如果是叶子节点:
// - 二分查找 key
// - 返回 (dataOffset, dataSize)
// 1. 从根节点开始
// 2. 如果是内部节点:
// - 二分查找确定子节点
// - 跳转到子节点继续查找
// 3. 如果是叶子节点:
// - 二分查找 key
// - 返回 (dataOffset, dataSize)
//
// 性能优化:
// - mmap 零拷贝:直接从内存映射读取节点
@@ -485,17 +490,19 @@ func NewBTreeReader(mmap mmap.MMap, rootOffset int64) *BTreeReader {
// Get 查询 key返回数据位置
//
// 参数:
// key: 要查询的 key
//
// key: 要查询的 key
//
// 返回:
// dataOffset: 数据块的文件偏移量
// dataSize: 数据块的大小
// found: 是否找到
//
// dataOffset: 数据块的文件偏移量
// dataSize: 数据块的大小
// found: 是否找到
//
// 查询流程:
// 1. 从根节点开始遍历
// 2. 内部节点:二分查找确定子节点,跳转
// 3. 叶子节点:二分查找 key返回数据位置
// 1. 从根节点开始遍历
// 2. 内部节点:二分查找确定子节点,跳转
// 3. 叶子节点:二分查找 key返回数据位置
func (r *BTreeReader) Get(key int64) (dataOffset int64, dataSize int32, found bool) {
if r.rootOffset == 0 {
return 0, 0, false
@@ -543,7 +550,7 @@ func (r *BTreeReader) Get(key int64) (dataOffset int64, dataSize int32, found bo
}
}
// GetAllKeys 获取 B+Tree 中所有的 key序)
// GetAllKeys 获取 B+Tree 中所有的 key序)
func (r *BTreeReader) GetAllKeys() []int64 {
if r.rootOffset == 0 {
return nil
@@ -562,7 +569,217 @@ func (r *BTreeReader) GetAllKeys() []int64 {
return keys
}
// traverseLeafNodes 遍历所有叶子节点
// GetAllKeysDesc 获取 B+Tree 中所有的 key按降序
//
// 性能优化:
// - 从右到左遍历叶子节点
// - 每个叶子节点内从后往前读取 keys
// - 避免额外的排序操作
func (r *BTreeReader) GetAllKeysDesc() []int64 {
if r.rootOffset == 0 {
return nil
}
var keys []int64
r.traverseLeafNodesReverse(r.rootOffset, func(node *BTreeNode) {
// 从后往前添加 keys
for i := len(node.Keys) - 1; i >= 0; i-- {
keys = append(keys, node.Keys[i])
}
})
return keys
}
// KeyCallback 迭代回调函数
//
// 参数:
// - key: 当前的 key序列号
// - dataOffset: 数据块的文件偏移量
// - dataSize: 数据块的大小
//
// 返回:
// - true: 继续迭代
// - false: 停止迭代
type KeyCallback func(key int64, dataOffset int64, dataSize int32) bool
// ForEach 升序迭代所有 key支持提前终止
//
// 使用场景:
// - 需要遍历数据但不想一次性加载所有 keys节省内存
// - 支持条件过滤,找到目标后提前终止
// - 支持外部自定义处理逻辑
//
// 示例:
//
// // 找到第一个 > 100 的 key
// reader.ForEach(func(key int64, offset int64, size int32) bool {
// if key > 100 {
// fmt.Printf("Found: %d\n", key)
// return false // 停止迭代
// }
// return true // 继续
// })
func (r *BTreeReader) ForEach(callback KeyCallback) {
if r.rootOffset == 0 {
return
}
r.forEachInternal(r.rootOffset, callback, false)
}
// ForEachDesc 降序迭代所有 key支持提前终止
//
// 使用场景:
// - 从最新数据开始遍历(时序数据库常见需求)
// - 查找最近的 N 条记录
// - 支持条件过滤和提前终止
//
// 示例:
//
// // 获取最新的 10 条记录
// count := 0
// reader.ForEachDesc(func(key int64, offset int64, size int32) bool {
// fmt.Printf("Key: %d\n", key)
// count++
// return count < 10 // 找到 10 条后停止
// })
func (r *BTreeReader) ForEachDesc(callback KeyCallback) {
if r.rootOffset == 0 {
return
}
r.forEachInternal(r.rootOffset, callback, true)
}
// forEachInternal 内部迭代实现(支持升序和降序)
//
// 性能优化(真正的按需读取):
// - 只读取节点 header32 bytes确定节点类型和 key 数量
// - 对于叶子节点,逐个读取 key、offset、size避免一次性读取所有数据
// - 对于内部节点,逐个读取 child offset支持提前终止
// - 如果回调在第 N 个 key 返回 false只会读取前 N 个 key
//
// 参数:
// - nodeOffset: 当前节点的文件偏移量
// - callback: 回调函数
// - reverse: true=降序, false=升序
//
// 返回:
// - true: 继续迭代
// - false: 停止迭代(外部请求或遍历完成)
func (r *BTreeReader) forEachInternal(nodeOffset int64, callback KeyCallback, reverse bool) bool {
if nodeOffset+BTreeNodeSize > int64(len(r.mmap)) {
return true // 无效节点,继续其他分支
}
nodeData := r.mmap[nodeOffset : nodeOffset+BTreeNodeSize]
// 只读取 header32 bytes
if len(nodeData) < BTreeHeaderSize {
return true
}
nodeType := nodeData[0]
keyCount := int(binary.LittleEndian.Uint16(nodeData[1:3]))
if nodeType == BTreeNodeTypeLeaf {
// 叶子节点:按需逐个读取 key 和 data
// 布局:[Header: 32B][Keys: keyCount*8B][Data: (offset,size) pairs]
keysStartOffset := BTreeHeaderSize
dataStartOffset := keysStartOffset + keyCount*8
if reverse {
// 降序:从后往前读取
for i := keyCount - 1; i >= 0; i-- {
// 读取 key
keyOffset := keysStartOffset + i*8
if keyOffset+8 > len(nodeData) {
break
}
key := int64(binary.LittleEndian.Uint64(nodeData[keyOffset : keyOffset+8]))
// 读取 dataOffset 和 dataSize交错存储每对 12 bytes
dataOffset := dataStartOffset + i*12
if dataOffset+12 > len(nodeData) {
break
}
offset := int64(binary.LittleEndian.Uint64(nodeData[dataOffset : dataOffset+8]))
size := int32(binary.LittleEndian.Uint32(nodeData[dataOffset+8 : dataOffset+12]))
// 调用回调,如果返回 false 则立即停止(真正的按需读取)
if !callback(key, offset, size) {
return false
}
}
} else {
// 升序:从前往后读取
for i := range keyCount {
// 读取 key
keyOffset := keysStartOffset + i*8
if keyOffset+8 > len(nodeData) {
break
}
key := int64(binary.LittleEndian.Uint64(nodeData[keyOffset : keyOffset+8]))
// 读取 dataOffset 和 dataSize
dataOffset := dataStartOffset + i*12
if dataOffset+12 > len(nodeData) {
break
}
offset := int64(binary.LittleEndian.Uint64(nodeData[dataOffset : dataOffset+8]))
size := int32(binary.LittleEndian.Uint32(nodeData[dataOffset+8 : dataOffset+12]))
// 调用回调,如果返回 false 则立即停止
if !callback(key, offset, size) {
return false
}
}
}
return true
}
// 内部节点:按需逐个读取 child offset
// 布局:[Header: 32B][Keys: keyCount*8B][Children: (keyCount+1)*8B]
childCount := keyCount + 1
childrenStartOffset := BTreeHeaderSize + keyCount*8
if reverse {
// 降序:从右到左遍历子节点
for i := childCount - 1; i >= 0; i-- {
childOffset := childrenStartOffset + i*8
if childOffset+8 > len(nodeData) {
break
}
childPtr := int64(binary.LittleEndian.Uint64(nodeData[childOffset : childOffset+8]))
// 递归遍历子树,如果子树请求停止则立即返回
if !r.forEachInternal(childPtr, callback, reverse) {
return false
}
}
} else {
// 升序:从左到右遍历子节点
for i := range childCount {
childOffset := childrenStartOffset + i*8
if childOffset+8 > len(nodeData) {
break
}
childPtr := int64(binary.LittleEndian.Uint64(nodeData[childOffset : childOffset+8]))
// 递归遍历子树
if !r.forEachInternal(childPtr, callback, reverse) {
return false
}
}
}
return true
}
// traverseLeafNodes 遍历所有叶子节点(从左到右)
func (r *BTreeReader) traverseLeafNodes(nodeOffset int64, callback func(*BTreeNode)) {
if nodeOffset+BTreeNodeSize > int64(len(r.mmap)) {
return
@@ -579,9 +796,37 @@ func (r *BTreeReader) traverseLeafNodes(nodeOffset int64, callback func(*BTreeNo
// 叶子节点,执行回调
callback(node)
} else {
// 内部节点,递归遍历所有子节点
// 内部节点,递归遍历所有子节点(从左到右)
for _, childOffset := range node.Children {
r.traverseLeafNodes(childOffset, callback)
}
}
}
// traverseLeafNodesReverse 倒序遍历所有叶子节点(从右到左)
//
// 用于支持倒序查询,性能优化:
// - 避免先获取所有 keys 再反转
// - 直接从最右侧的叶子节点开始遍历
func (r *BTreeReader) traverseLeafNodesReverse(nodeOffset int64, callback func(*BTreeNode)) {
if nodeOffset+BTreeNodeSize > int64(len(r.mmap)) {
return
}
nodeData := r.mmap[nodeOffset : nodeOffset+BTreeNodeSize]
node := UnmarshalBTree(nodeData)
if node == nil {
return
}
if node.NodeType == BTreeNodeTypeLeaf {
// 叶子节点,执行回调
callback(node)
} else {
// 内部节点,递归遍历所有子节点(从右到左)
for i := len(node.Children) - 1; i >= 0; i-- {
r.traverseLeafNodesReverse(node.Children[i], callback)
}
}
}

View File

@@ -132,6 +132,386 @@ func TestBTreeSerialization(t *testing.T) {
t.Log("Serialization test passed!")
}
// TestBTreeForEach 测试升序迭代
func TestBTreeForEach(t *testing.T) {
// 创建测试文件
file, err := os.Create("test_foreach.sst")
if err != nil {
t.Fatal(err)
}
defer os.Remove("test_foreach.sst")
// 构建 B+Tree
builder := NewBTreeBuilder(file, 256)
for i := int64(1); i <= 100; i++ {
err := builder.Add(i, i*100, int32(i*10))
if err != nil {
t.Fatal(err)
}
}
rootOffset, err := builder.Build()
if err != nil {
t.Fatal(err)
}
file.Close()
// 打开并 mmap
file, _ = os.Open("test_foreach.sst")
defer file.Close()
mmapData, _ := mmap.Map(file, mmap.RDONLY, 0)
defer mmapData.Unmap()
reader := NewBTreeReader(mmapData, rootOffset)
// 测试 1: 完整升序迭代
t.Run("Complete", func(t *testing.T) {
var keys []int64
var offsets []int64
var sizes []int32
reader.ForEach(func(key int64, offset int64, size int32) bool {
keys = append(keys, key)
offsets = append(offsets, offset)
sizes = append(sizes, size)
return true
})
// 验证数量
if len(keys) != 100 {
t.Errorf("Expected 100 keys, got %d", len(keys))
}
// 验证顺序(升序)
for i := 0; i < len(keys)-1; i++ {
if keys[i] >= keys[i+1] {
t.Errorf("Keys not in ascending order: keys[%d]=%d, keys[%d]=%d",
i, keys[i], i+1, keys[i+1])
}
}
// 验证第一个和最后一个
if keys[0] != 1 {
t.Errorf("Expected first key=1, got %d", keys[0])
}
if keys[99] != 100 {
t.Errorf("Expected last key=100, got %d", keys[99])
}
// 验证 offset 和 size
for i, key := range keys {
expectedOffset := key * 100
expectedSize := int32(key * 10)
if offsets[i] != expectedOffset {
t.Errorf("Key %d: expected offset %d, got %d", key, expectedOffset, offsets[i])
}
if sizes[i] != expectedSize {
t.Errorf("Key %d: expected size %d, got %d", key, expectedSize, sizes[i])
}
}
})
// 测试 2: 提前终止
t.Run("EarlyTermination", func(t *testing.T) {
var keys []int64
reader.ForEach(func(key int64, offset int64, size int32) bool {
keys = append(keys, key)
return len(keys) < 5 // 只收集 5 个
})
if len(keys) != 5 {
t.Errorf("Expected 5 keys, got %d", len(keys))
}
if keys[0] != 1 || keys[4] != 5 {
t.Errorf("Expected keys [1,2,3,4,5], got %v", keys)
}
})
// 测试 3: 条件过滤
t.Run("ConditionalFilter", func(t *testing.T) {
var evenKeys []int64
reader.ForEach(func(key int64, offset int64, size int32) bool {
if key%2 == 0 {
evenKeys = append(evenKeys, key)
}
return true
})
if len(evenKeys) != 50 {
t.Errorf("Expected 50 even keys, got %d", len(evenKeys))
}
// 验证都是偶数
for _, key := range evenKeys {
if key%2 != 0 {
t.Errorf("Key %d is not even", key)
}
}
})
// 测试 4: 查找第一个满足条件的
t.Run("FindFirst", func(t *testing.T) {
var foundKey int64
count := 0
reader.ForEach(func(key int64, offset int64, size int32) bool {
count++
if key > 50 {
foundKey = key
return false // 找到后停止
}
return true
})
if foundKey != 51 {
t.Errorf("Expected to find key 51, got %d", foundKey)
}
if count != 51 {
t.Errorf("Expected to iterate 51 times, got %d", count)
}
})
// 测试 5: 与 GetAllKeys 结果一致性
t.Run("ConsistencyWithGetAllKeys", func(t *testing.T) {
var iterKeys []int64
reader.ForEach(func(key int64, offset int64, size int32) bool {
iterKeys = append(iterKeys, key)
return true
})
allKeys := reader.GetAllKeys()
if len(iterKeys) != len(allKeys) {
t.Errorf("Length mismatch: ForEach=%d, GetAllKeys=%d", len(iterKeys), len(allKeys))
}
for i := range iterKeys {
if iterKeys[i] != allKeys[i] {
t.Errorf("Key mismatch at index %d: ForEach=%d, GetAllKeys=%d",
i, iterKeys[i], allKeys[i])
}
}
})
}
// TestBTreeForEachDesc 测试降序迭代
func TestBTreeForEachDesc(t *testing.T) {
// 创建测试文件
file, err := os.Create("test_foreach_desc.sst")
if err != nil {
t.Fatal(err)
}
defer os.Remove("test_foreach_desc.sst")
// 构建 B+Tree
builder := NewBTreeBuilder(file, 256)
for i := int64(1); i <= 100; i++ {
err := builder.Add(i, i*100, int32(i*10))
if err != nil {
t.Fatal(err)
}
}
rootOffset, err := builder.Build()
if err != nil {
t.Fatal(err)
}
file.Close()
// 打开并 mmap
file, _ = os.Open("test_foreach_desc.sst")
defer file.Close()
mmapData, _ := mmap.Map(file, mmap.RDONLY, 0)
defer mmapData.Unmap()
reader := NewBTreeReader(mmapData, rootOffset)
// 测试 1: 完整降序迭代
t.Run("Complete", func(t *testing.T) {
var keys []int64
reader.ForEachDesc(func(key int64, offset int64, size int32) bool {
keys = append(keys, key)
return true
})
// 验证数量
if len(keys) != 100 {
t.Errorf("Expected 100 keys, got %d", len(keys))
}
// 验证顺序(降序)
for i := 0; i < len(keys)-1; i++ {
if keys[i] <= keys[i+1] {
t.Errorf("Keys not in descending order: keys[%d]=%d, keys[%d]=%d",
i, keys[i], i+1, keys[i+1])
}
}
// 验证第一个和最后一个
if keys[0] != 100 {
t.Errorf("Expected first key=100, got %d", keys[0])
}
if keys[99] != 1 {
t.Errorf("Expected last key=1, got %d", keys[99])
}
})
// 测试 2: 获取最新的 N 条记录(时序数据库常见需求)
t.Run("GetLatestN", func(t *testing.T) {
var latestKeys []int64
reader.ForEachDesc(func(key int64, offset int64, size int32) bool {
latestKeys = append(latestKeys, key)
return len(latestKeys) < 10 // 只取最新的 10 条
})
if len(latestKeys) != 10 {
t.Errorf("Expected 10 keys, got %d", len(latestKeys))
}
// 验证是最新的 10 条100, 99, 98, ..., 91
for i, key := range latestKeys {
expected := int64(100 - i)
if key != expected {
t.Errorf("latestKeys[%d]: expected %d, got %d", i, expected, key)
}
}
})
// 测试 3: 与 GetAllKeysDesc 结果一致性
t.Run("ConsistencyWithGetAllKeysDesc", func(t *testing.T) {
var iterKeys []int64
reader.ForEachDesc(func(key int64, offset int64, size int32) bool {
iterKeys = append(iterKeys, key)
return true
})
allKeys := reader.GetAllKeysDesc()
if len(iterKeys) != len(allKeys) {
t.Errorf("Length mismatch: ForEachDesc=%d, GetAllKeysDesc=%d", len(iterKeys), len(allKeys))
}
for i := range iterKeys {
if iterKeys[i] != allKeys[i] {
t.Errorf("Key mismatch at index %d: ForEachDesc=%d, GetAllKeysDesc=%d",
i, iterKeys[i], allKeys[i])
}
}
})
// 测试 4: 降序查找第一个满足条件的
t.Run("FindFirstDesc", func(t *testing.T) {
var foundKey int64
count := 0
reader.ForEachDesc(func(key int64, offset int64, size int32) bool {
count++
if key < 50 {
foundKey = key
return false // 找到后停止
}
return true
})
if foundKey != 49 {
t.Errorf("Expected to find key 49, got %d", foundKey)
}
if count != 52 { // 100, 99, ..., 50, 49
t.Errorf("Expected to iterate 52 times, got %d", count)
}
})
}
// TestBTreeForEachEmpty 测试空树的迭代
func TestBTreeForEachEmpty(t *testing.T) {
// 创建空的 B+Tree
file, _ := os.Create("test_empty.sst")
defer os.Remove("test_empty.sst")
builder := NewBTreeBuilder(file, 256)
rootOffset, _ := builder.Build()
file.Close()
file, _ = os.Open("test_empty.sst")
defer file.Close()
mmapData, _ := mmap.Map(file, mmap.RDONLY, 0)
defer mmapData.Unmap()
reader := NewBTreeReader(mmapData, rootOffset)
// 测试升序迭代
t.Run("ForEach", func(t *testing.T) {
called := false
reader.ForEach(func(key int64, offset int64, size int32) bool {
called = true
return true
})
if called {
t.Error("Callback should not be called on empty tree")
}
})
// 测试降序迭代
t.Run("ForEachDesc", func(t *testing.T) {
called := false
reader.ForEachDesc(func(key int64, offset int64, size int32) bool {
called = true
return true
})
if called {
t.Error("Callback should not be called on empty tree")
}
})
}
// TestBTreeForEachSingle 测试单个元素的迭代
func TestBTreeForEachSingle(t *testing.T) {
// 创建只有一个元素的 B+Tree
file, _ := os.Create("test_single.sst")
defer os.Remove("test_single.sst")
builder := NewBTreeBuilder(file, 256)
builder.Add(42, 4200, 420)
rootOffset, _ := builder.Build()
file.Close()
file, _ = os.Open("test_single.sst")
defer file.Close()
mmapData, _ := mmap.Map(file, mmap.RDONLY, 0)
defer mmapData.Unmap()
reader := NewBTreeReader(mmapData, rootOffset)
// 测试升序迭代
t.Run("ForEach", func(t *testing.T) {
var keys []int64
reader.ForEach(func(key int64, offset int64, size int32) bool {
keys = append(keys, key)
if offset != 4200 || size != 420 {
t.Errorf("Unexpected data: offset=%d, size=%d", offset, size)
}
return true
})
if len(keys) != 1 || keys[0] != 42 {
t.Errorf("Expected single key 42, got %v", keys)
}
})
// 测试降序迭代
t.Run("ForEachDesc", func(t *testing.T) {
var keys []int64
reader.ForEachDesc(func(key int64, offset int64, size int32) bool {
keys = append(keys, key)
return true
})
if len(keys) != 1 || keys[0] != 42 {
t.Errorf("Expected single key 42, got %v", keys)
}
})
}
func BenchmarkBTreeGet(b *testing.B) {
// 构建测试数据
file, _ := os.Create("bench.sst")
@@ -159,3 +539,86 @@ func BenchmarkBTreeGet(b *testing.B) {
reader.Get(key)
}
}
// BenchmarkBTreeForEach 性能测试:完整迭代
func BenchmarkBTreeForEach(b *testing.B) {
file, _ := os.Create("bench_foreach.sst")
defer os.Remove("bench_foreach.sst")
builder := NewBTreeBuilder(file, 256)
for i := int64(1); i <= 10000; i++ {
builder.Add(i, i*100, 100)
}
rootOffset, _ := builder.Build()
file.Close()
file, _ = os.Open("bench_foreach.sst")
defer file.Close()
mmapData, _ := mmap.Map(file, mmap.RDONLY, 0)
defer mmapData.Unmap()
reader := NewBTreeReader(mmapData, rootOffset)
b.ResetTimer()
for b.Loop() {
count := 0
reader.ForEach(func(key int64, offset int64, size int32) bool {
count++
return true
})
}
}
// BenchmarkBTreeForEachEarlyTermination 性能测试:提前终止
func BenchmarkBTreeForEachEarlyTermination(b *testing.B) {
file, _ := os.Create("bench_foreach_early.sst")
defer os.Remove("bench_foreach_early.sst")
builder := NewBTreeBuilder(file, 256)
for i := int64(1); i <= 100000; i++ {
builder.Add(i, i*100, 100)
}
rootOffset, _ := builder.Build()
file.Close()
file, _ = os.Open("bench_foreach_early.sst")
defer file.Close()
mmapData, _ := mmap.Map(file, mmap.RDONLY, 0)
defer mmapData.Unmap()
reader := NewBTreeReader(mmapData, rootOffset)
b.ResetTimer()
for b.Loop() {
count := 0
reader.ForEach(func(key int64, offset int64, size int32) bool {
count++
return count < 10 // 只读取前 10 个
})
}
}
// BenchmarkBTreeGetAllKeys vs ForEach 对比
func BenchmarkBTreeGetAllKeys(b *testing.B) {
file, _ := os.Create("bench_getall.sst")
defer os.Remove("bench_getall.sst")
builder := NewBTreeBuilder(file, 256)
for i := int64(1); i <= 10000; i++ {
builder.Add(i, i*100, 100)
}
rootOffset, _ := builder.Build()
file.Close()
file, _ = os.Open("bench_getall.sst")
defer file.Close()
mmapData, _ := mmap.Map(file, mmap.RDONLY, 0)
defer mmapData.Unmap()
reader := NewBTreeReader(mmapData, rootOffset)
b.ResetTimer()
for b.Loop() {
_ = reader.GetAllKeys()
}
}

View File

@@ -2,6 +2,8 @@ package srdb
import (
"fmt"
"io"
"log/slog"
"os"
"path/filepath"
"sort"
@@ -62,18 +64,50 @@ type CompactionTask struct {
type Picker struct {
mu sync.Mutex
currentStage int // 当前阶段0=L0合并, 1=L0升级, 2=L1升级, 3=L2升级
// 层级大小限制(可配置)
level0SizeLimit int64
level1SizeLimit int64
level2SizeLimit int64
level3SizeLimit int64
}
// NewPicker 创建新的 Compaction Picker
// NewPicker 创建新的 Compaction Picker(使用默认值)
func NewPicker() *Picker {
return &Picker{
currentStage: 0, // 从 L0 合并开始
currentStage: 0, // 从 L0 合并开始
level0SizeLimit: level0SizeLimit,
level1SizeLimit: level1SizeLimit,
level2SizeLimit: level2SizeLimit,
level3SizeLimit: level3SizeLimit,
}
}
// getLevelSizeLimit 获取层级大小限制(调用包级私有函数
// UpdateLevelLimits 更新层级大小限制(由 CompactionManager 调用)
func (p *Picker) UpdateLevelLimits(l0, l1, l2, l3 int64) {
p.mu.Lock()
defer p.mu.Unlock()
p.level0SizeLimit = l0
p.level1SizeLimit = l1
p.level2SizeLimit = l2
p.level3SizeLimit = l3
}
// getLevelSizeLimit 获取层级大小限制(从配置读取)
// 注意:层级配置在 UpdateLevelLimits 后不会改变,因此不需要加锁
func (p *Picker) getLevelSizeLimit(level int) int64 {
return getLevelSizeLimit(level)
switch level {
case 0:
return p.level0SizeLimit
case 1:
return p.level1SizeLimit
case 2:
return p.level2SizeLimit
case 3:
return p.level3SizeLimit
default:
return p.level3SizeLimit
}
}
// PickCompaction 选择需要 Compaction 的任务(按阶段返回,阶段内并发执行)
@@ -102,43 +136,27 @@ func (p *Picker) PickCompaction(version *Version) []*CompactionTask {
defer p.mu.Unlock()
var tasks []*CompactionTask
var stageName string
// 根据当前阶段选择任务
switch p.currentStage {
case 0:
// Stage 0: L0 合并任务
tasks = p.pickL0MergeTasks(version)
stageName = "L0-merge"
case 1:
// Stage 1: L0 升级任务
tasks = p.pickL0UpgradeTasks(version)
stageName = "L0-upgrade"
case 2:
// Stage 2: L1 升级任务
tasks = p.pickLevelCompaction(version, 1)
stageName = "L1-upgrade"
case 3:
// Stage 3: L2 升级任务
tasks = p.pickLevelCompaction(version, 2)
stageName = "L2-upgrade"
}
// 保存当前阶段索引
currentStage := p.currentStage
// 推进到下一阶段(无论是否有任务),这里是否巧妙地
// 推进到下一阶段(无论是否有任务),这里巧妙地
// 使用了取模运算来保证阶段递增与阶段重置。
p.currentStage = (p.currentStage + 1) % 4
if len(tasks) > 0 {
fmt.Printf("[Picker] Stage %d (%s): found %d tasks, next stage will be %d\n",
currentStage, stageName, len(tasks), p.currentStage)
} else {
fmt.Printf("[Picker] Stage %d (%s): no tasks, next stage will be %d\n",
currentStage, stageName, p.currentStage)
}
return tasks
}
@@ -451,7 +469,8 @@ type Compactor struct {
picker *Picker
versionSet *VersionSet
schema *Schema
mu sync.RWMutex // 只保护 schema 字段的读写
logger *slog.Logger
mu sync.RWMutex // 只保护 schema 和 logger 字段的读写
}
// NewCompactor 创建新的 Compactor
@@ -460,6 +479,7 @@ func NewCompactor(sstDir string, versionSet *VersionSet) *Compactor {
sstDir: sstDir,
picker: NewPicker(),
versionSet: versionSet,
logger: slog.New(slog.NewTextHandler(io.Discard, nil)), // 默认丢弃日志
}
}
@@ -470,6 +490,13 @@ func (c *Compactor) SetSchema(schema *Schema) {
c.schema = schema
}
// SetLogger 设置 Logger
func (c *Compactor) SetLogger(logger *slog.Logger) {
c.mu.Lock()
defer c.mu.Unlock()
c.logger = logger
}
// GetPicker 获取 Picker
func (c *Compactor) GetPicker() *Picker {
return c.picker
@@ -482,6 +509,11 @@ func (c *Compactor) DoCompaction(task *CompactionTask, version *Version) (*Versi
return nil, fmt.Errorf("compaction task is nil")
}
// 获取 logger
c.mu.RLock()
logger := c.logger
c.mu.RUnlock()
// 0. 验证输入文件是否存在(防止并发 compaction 导致的竞态)
existingInputFiles := make([]*FileMetadata, 0, len(task.InputFiles))
for _, file := range task.InputFiles {
@@ -489,13 +521,14 @@ func (c *Compactor) DoCompaction(task *CompactionTask, version *Version) (*Versi
if _, err := os.Stat(sstPath); err == nil {
existingInputFiles = append(existingInputFiles, file)
} else {
fmt.Printf("[Compaction] Warning: input file %06d.sst not found, skipping from task\n", file.FileNumber)
logger.Warn("[Compaction] Input file not found, skipping",
"file_number", file.FileNumber)
}
}
// 如果所有输入文件都不存在,直接返回(无需 compaction
if len(existingInputFiles) == 0 {
fmt.Printf("[Compaction] All input files missing, compaction skipped\n")
logger.Warn("[Compaction] All input files missing, compaction skipped")
return nil, nil // 返回 nil 表示不需要应用任何 VersionEdit
}
@@ -519,7 +552,8 @@ func (c *Compactor) DoCompaction(task *CompactionTask, version *Version) (*Versi
existingOutputFiles = append(existingOutputFiles, file)
} else {
// 输出层级的文件不存在,记录并在 VersionEdit 中删除它
fmt.Printf("[Compaction] Warning: overlapping output file %06d.sst missing, will remove from MANIFEST\n", file.FileNumber)
logger.Warn("[Compaction] Overlapping output file missing, will remove from MANIFEST",
"file_number", file.FileNumber)
missingOutputFiles = append(missingOutputFiles, file)
}
}
@@ -558,7 +592,8 @@ func (c *Compactor) DoCompaction(task *CompactionTask, version *Version) (*Versi
// 删除缺失的输出层级文件(清理 MANIFEST 中的过期引用)
for _, file := range missingOutputFiles {
edit.DeleteFile(file.FileNumber)
fmt.Printf("[Compaction] Removing missing file %06d.sst from MANIFEST\n", file.FileNumber)
logger.Info("[Compaction] Removing missing file from MANIFEST",
"file_number", file.FileNumber)
}
// 添加新文件,并跟踪最大文件编号
@@ -874,6 +909,19 @@ type CompactionManager struct {
sstManager *SSTableManager // 添加 sstManager 引用,用于同步删除 readers
sstDir string
// 配置(从 Database Options 传递,启动后不可变)
configMu sync.Mutex // 仅用于 ApplyConfig防御性编程
logger *slog.Logger
level0SizeLimit int64
level1SizeLimit int64
level2SizeLimit int64
level3SizeLimit int64
compactionInterval time.Duration
gcInterval time.Duration
gcFileMinAge time.Duration
disableCompaction bool
disableGC bool
// 控制后台 Compaction
stopCh chan struct{}
wg sync.WaitGroup
@@ -891,17 +939,55 @@ type CompactionManager struct {
totalOrphansFound int64
}
// NewCompactionManager 创建新的 Compaction Manager
// NewCompactionManager 创建新的 Compaction Manager(使用默认配置)
func NewCompactionManager(sstDir string, versionSet *VersionSet, sstManager *SSTableManager) *CompactionManager {
return &CompactionManager{
compactor: NewCompactor(sstDir, versionSet),
versionSet: versionSet,
sstManager: sstManager,
sstDir: sstDir,
stopCh: make(chan struct{}),
compactor: NewCompactor(sstDir, versionSet),
versionSet: versionSet,
sstManager: sstManager,
sstDir: sstDir,
stopCh: make(chan struct{}),
// 默认 logger丢弃日志将在 ApplyConfig 中设置为 Database.options.Logger
logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
// 使用硬编码常量作为默认值(向后兼容)
level0SizeLimit: level0SizeLimit,
level1SizeLimit: level1SizeLimit,
level2SizeLimit: level2SizeLimit,
level3SizeLimit: level3SizeLimit,
compactionInterval: 10 * time.Second,
gcInterval: 5 * time.Minute,
gcFileMinAge: 1 * time.Minute,
disableCompaction: false,
disableGC: false,
}
}
// ApplyConfig 应用数据库级配置(从 Database Options
func (m *CompactionManager) ApplyConfig(opts *Options) {
m.configMu.Lock()
defer m.configMu.Unlock()
m.logger = opts.Logger
m.level0SizeLimit = opts.Level0SizeLimit
m.level1SizeLimit = opts.Level1SizeLimit
m.level2SizeLimit = opts.Level2SizeLimit
m.level3SizeLimit = opts.Level3SizeLimit
m.compactionInterval = opts.CompactionInterval
m.gcInterval = opts.GCInterval
m.gcFileMinAge = opts.GCFileMinAge
m.disableCompaction = opts.DisableAutoCompaction
m.disableGC = opts.DisableGC
// 同时更新 compactor 的 picker 和 logger
m.compactor.picker.UpdateLevelLimits(
m.level0SizeLimit,
m.level1SizeLimit,
m.level2SizeLimit,
m.level3SizeLimit,
)
m.compactor.SetLogger(opts.Logger)
}
// GetPicker 获取 Compaction Picker
func (m *CompactionManager) GetPicker() *Picker {
return m.compactor.GetPicker()
@@ -929,7 +1015,15 @@ func (m *CompactionManager) Stop() {
func (m *CompactionManager) backgroundCompaction() {
defer m.wg.Done()
ticker := time.NewTicker(10 * time.Second) // 每 10 秒检查一次
// Options 是不可变的,配置在启动后不会改变,直接读取即可
interval := m.compactionInterval
disabled := m.disableCompaction
if disabled {
return // 禁用自动 Compaction直接退出
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
@@ -987,7 +1081,9 @@ func (m *CompactionManager) doCompact() {
}
totalStagesExecuted++
fmt.Printf("[Compaction] Found %d tasks in stage %d to execute concurrently\n", len(tasks), stage)
m.logger.Info("[Compaction] Found tasks to execute concurrently",
"stage", stage,
"task_count", len(tasks))
// 并发执行同一阶段的所有任务
var wg sync.WaitGroup
@@ -999,8 +1095,10 @@ func (m *CompactionManager) doCompact() {
firstFile := task.InputFiles[0].FileNumber
m.mu.Lock()
if m.lastFailedFile == firstFile && m.consecutiveFails >= 3 {
fmt.Printf("[Compaction] Skipping L%d file %d (failed %d times)\n",
task.Level, firstFile, m.consecutiveFails)
m.logger.Warn("[Compaction] Skipping file (failed multiple times)",
"level", task.Level,
"file_number", firstFile,
"consecutive_fails", m.consecutiveFails)
m.consecutiveFails = 0
m.lastFailedFile = 0
m.mu.Unlock()
@@ -1020,12 +1118,17 @@ func (m *CompactionManager) doCompact() {
}
// 执行 Compaction
fmt.Printf("[Compaction] Starting: L%d -> L%d, files: %d\n",
task.Level, task.OutputLevel, len(task.InputFiles))
m.logger.Info("[Compaction] Starting",
"source_level", task.Level,
"target_level", task.OutputLevel,
"file_count", len(task.InputFiles))
err := m.DoCompactionWithVersion(task, currentVersion)
if err != nil {
fmt.Printf("[Compaction] Failed L%d -> L%d: %v\n", task.Level, task.OutputLevel, err)
m.logger.Error("[Compaction] Failed",
"source_level", task.Level,
"target_level", task.OutputLevel,
"error", err)
// 记录失败信息
if len(task.InputFiles) > 0 {
@@ -1040,7 +1143,9 @@ func (m *CompactionManager) doCompact() {
m.mu.Unlock()
}
} else {
fmt.Printf("[Compaction] Completed: L%d -> L%d\n", task.Level, task.OutputLevel)
m.logger.Info("[Compaction] Completed",
"source_level", task.Level,
"target_level", task.OutputLevel)
// 清除失败计数
m.mu.Lock()
@@ -1057,7 +1162,10 @@ func (m *CompactionManager) doCompact() {
// 等待当前阶段的所有任务完成
wg.Wait()
fmt.Printf("[Compaction] Stage %d completed: %d/%d tasks succeeded\n", stage, successCount.Load(), len(tasks))
m.logger.Info("[Compaction] Stage completed",
"stage", stage,
"succeeded", successCount.Load(),
"total", len(tasks))
}
// 如果所有阶段都没有任务,输出诊断信息
@@ -1080,7 +1188,7 @@ func (m *CompactionManager) printCompactionStats(version *Version, picker *Picke
}
m.lastCompactionTime = time.Now()
fmt.Println("[Compaction] Status check:")
m.logger.Info("[Compaction] Status check")
for level := range NumLevels {
files := version.GetLevel(level)
if len(files) == 0 {
@@ -1093,8 +1201,11 @@ func (m *CompactionManager) printCompactionStats(version *Version, picker *Picke
}
score := picker.GetLevelScore(version, level)
fmt.Printf(" L%d: %d files, %.2f MB, score: %.2f\n",
level, len(files), float64(totalSize)/(1024*1024), score)
m.logger.Info("[Compaction] Level status",
"level", level,
"file_count", len(files),
"size_mb", float64(totalSize)/(1024*1024),
"score", score)
}
}
@@ -1112,7 +1223,7 @@ func (m *CompactionManager) DoCompactionWithVersion(task *CompactionTask, versio
// 如果 edit 为 nil说明所有文件都已经不存在无需应用变更
if edit == nil {
fmt.Printf("[Compaction] No changes needed (files already removed)\n")
m.logger.Info("[Compaction] No changes needed (files already removed)")
return nil
}
@@ -1120,7 +1231,8 @@ func (m *CompactionManager) DoCompactionWithVersion(task *CompactionTask, versio
err = m.versionSet.LogAndApply(edit)
if err != nil {
// LogAndApply 失败,清理已写入的新 SST 文件(防止孤儿文件)
fmt.Printf("[Compaction] LogAndApply failed, cleaning up new files: %v\n", err)
m.logger.Error("[Compaction] LogAndApply failed, cleaning up new files",
"error", err)
m.cleanupNewFiles(edit)
return fmt.Errorf("apply version edit: %w", err)
}
@@ -1132,7 +1244,9 @@ func (m *CompactionManager) DoCompactionWithVersion(task *CompactionTask, versio
sstPath := filepath.Join(m.sstDir, fmt.Sprintf("%06d.sst", file.FileNumber))
reader, err := NewSSTableReader(sstPath)
if err != nil {
fmt.Printf("[Compaction] Warning: failed to open new file %06d.sst: %v\n", file.FileNumber, err)
m.logger.Warn("[Compaction] Failed to open new file",
"file_number", file.FileNumber,
"error", err)
continue
}
// 设置 Schema
@@ -1176,16 +1290,20 @@ func (m *CompactionManager) cleanupNewFiles(edit *VersionEdit) {
return
}
fmt.Printf("[Compaction] Cleaning up %d new files after LogAndApply failure\n", len(edit.AddedFiles))
m.logger.Warn("[Compaction] Cleaning up new files after LogAndApply failure",
"file_count", len(edit.AddedFiles))
// 删除新创建的文件
for _, file := range edit.AddedFiles {
sstPath := filepath.Join(m.sstDir, fmt.Sprintf("%06d.sst", file.FileNumber))
err := os.Remove(sstPath)
if err != nil {
fmt.Printf("[Compaction] Failed to cleanup new file %06d.sst: %v\n", file.FileNumber, err)
m.logger.Warn("[Compaction] Failed to cleanup new file",
"file_number", file.FileNumber,
"error", err)
} else {
fmt.Printf("[Compaction] Cleaned up new file %06d.sst\n", file.FileNumber)
m.logger.Info("[Compaction] Cleaned up new file",
"file_number", file.FileNumber)
}
}
}
@@ -1193,11 +1311,12 @@ func (m *CompactionManager) cleanupNewFiles(edit *VersionEdit) {
// deleteObsoleteFiles 删除废弃的 SST 文件
func (m *CompactionManager) deleteObsoleteFiles(edit *VersionEdit) {
if edit == nil {
fmt.Printf("[Compaction] deleteObsoleteFiles: edit is nil\n")
m.logger.Warn("[Compaction] deleteObsoleteFiles: edit is nil")
return
}
fmt.Printf("[Compaction] deleteObsoleteFiles: %d files to delete\n", len(edit.DeletedFiles))
m.logger.Info("[Compaction] Deleting obsolete files",
"file_count", len(edit.DeletedFiles))
// 删除被标记为删除的文件
for _, fileNum := range edit.DeletedFiles {
@@ -1205,7 +1324,9 @@ func (m *CompactionManager) deleteObsoleteFiles(edit *VersionEdit) {
if m.sstManager != nil {
err := m.sstManager.RemoveReader(fileNum)
if err != nil {
fmt.Printf("[Compaction] Failed to remove reader for %06d.sst: %v\n", fileNum, err)
m.logger.Warn("[Compaction] Failed to remove reader",
"file_number", fileNum,
"error", err)
}
}
@@ -1215,9 +1336,12 @@ func (m *CompactionManager) deleteObsoleteFiles(edit *VersionEdit) {
if err != nil {
// 删除失败只记录日志,不影响 compaction 流程
// 后台垃圾回收器会重试
fmt.Printf("[Compaction] Failed to delete obsolete file %06d.sst: %v\n", fileNum, err)
m.logger.Warn("[Compaction] Failed to delete obsolete file",
"file_number", fileNum,
"error", err)
} else {
fmt.Printf("[Compaction] Deleted obsolete file %06d.sst\n", fileNum)
m.logger.Info("[Compaction] Deleted obsolete file",
"file_number", fileNum)
}
}
}
@@ -1291,11 +1415,36 @@ func (m *CompactionManager) GetLevelStats() []LevelStats {
return stats
}
// GetLevelSizeLimit 获取指定层级的大小限制(公开方法,供 WebUI 等外部使用)
// 注意Options 是不可变的,配置在 ApplyConfig 后不会改变,因此不需要加锁
func (m *CompactionManager) GetLevelSizeLimit(level int) int64 {
switch level {
case 0:
return m.level0SizeLimit
case 1:
return m.level1SizeLimit
case 2:
return m.level2SizeLimit
case 3:
return m.level3SizeLimit
default:
return m.level3SizeLimit
}
}
// backgroundGarbageCollection 后台垃圾回收循环
func (m *CompactionManager) backgroundGarbageCollection() {
defer m.wg.Done()
ticker := time.NewTicker(5 * time.Minute) // 每 5 分钟检查一次
// Options 是不可变的,配置在启动后不会改变,直接读取即可
interval := m.gcInterval
disabled := m.disableGC
if disabled {
return // 禁用垃圾回收,直接退出
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
@@ -1328,7 +1477,7 @@ func (m *CompactionManager) collectOrphanFiles() {
pattern := filepath.Join(m.sstDir, "*.sst")
sstFiles, err := filepath.Glob(pattern)
if err != nil {
fmt.Printf("[GC] Failed to scan SST directory: %v\n", err)
m.logger.Error("[GC] Failed to scan SST directory", "error", err)
return
}
@@ -1345,23 +1494,30 @@ func (m *CompactionManager) collectOrphanFiles() {
// 检查是否是活跃文件
if !activeFiles[fileNum] {
// 检查文件修改时间,避免删除正在 flush 的文件
// 如果文件在最近 1 分钟内创建/修改,跳过(可能正在 LogAndApply
// Options 是不可变的,直接读取配置即可
minAge := m.gcFileMinAge
fileInfo, err := os.Stat(sstPath)
if err != nil {
continue
}
if time.Since(fileInfo.ModTime()) < 1*time.Minute {
fmt.Printf("[GC] Skipping recently modified file %06d.sst (age: %v)\n",
fileNum, time.Since(fileInfo.ModTime()))
if time.Since(fileInfo.ModTime()) < minAge {
m.logger.Info("[GC] Skipping recently modified file",
"file_number", fileNum,
"age", time.Since(fileInfo.ModTime()),
"min_age", minAge)
continue
}
// 这是孤儿文件,删除它
err = os.Remove(sstPath)
if err != nil {
fmt.Printf("[GC] Failed to delete orphan file %06d.sst: %v\n", fileNum, err)
m.logger.Warn("[GC] Failed to delete orphan file",
"file_number", fileNum,
"error", err)
} else {
fmt.Printf("[GC] Deleted orphan file %06d.sst\n", fileNum)
m.logger.Info("[GC] Deleted orphan file",
"file_number", fileNum)
orphanCount++
}
}
@@ -1371,15 +1527,18 @@ func (m *CompactionManager) collectOrphanFiles() {
m.mu.Lock()
m.lastGCTime = time.Now()
m.totalOrphansFound += int64(orphanCount)
totalOrphans := m.totalOrphansFound
m.mu.Unlock()
if orphanCount > 0 {
fmt.Printf("[GC] Completed: cleaned up %d orphan files (total: %d)\n", orphanCount, m.totalOrphansFound)
m.logger.Info("[GC] Completed",
"cleaned_up", orphanCount,
"total_orphans", totalOrphans)
}
}
// CleanupOrphanFiles 手动触发孤儿文件清理(可在启动时调用)
func (m *CompactionManager) CleanupOrphanFiles() {
fmt.Println("[GC] Manual cleanup triggered")
m.logger.Info("[GC] Manual cleanup triggered")
m.collectOrphanFiles()
}

View File

@@ -3,6 +3,8 @@ package srdb
import (
"encoding/json"
"fmt"
"io"
"log/slog"
"maps"
"os"
"path/filepath"
@@ -21,6 +23,9 @@ type Database struct {
// 元数据
metadata *Metadata
// 配置选项
options *Options
// 锁
mu sync.RWMutex
}
@@ -38,17 +43,144 @@ type TableInfo struct {
CreatedAt int64 `json:"created_at"`
}
// Open 打开数据库
// Options 数据库配置选项
type Options struct {
// ========== 基础配置 ==========
Dir string // 数据库目录(必需)
Logger *slog.Logger // 日志器可选nil 表示不输出日志)
// ========== MemTable 配置 ==========
MemTableSize int64 // MemTable 大小限制(字节),默认 64MB
AutoFlushTimeout time.Duration // 自动 flush 超时时间,默认 30s0 表示禁用
// ========== Compaction 配置 ==========
// 层级大小限制
Level0SizeLimit int64 // L0 层大小限制,默认 64MB
Level1SizeLimit int64 // L1 层大小限制,默认 256MB
Level2SizeLimit int64 // L2 层大小限制,默认 512MB
Level3SizeLimit int64 // L3 层大小限制,默认 1GB
// 后台任务间隔
CompactionInterval time.Duration // Compaction 检查间隔,默认 10s
GCInterval time.Duration // 垃圾回收检查间隔,默认 5min
// ========== 高级配置(可选)==========
DisableAutoCompaction bool // 禁用自动 Compaction默认 false
DisableGC bool // 禁用垃圾回收,默认 false
GCFileMinAge time.Duration // GC 文件最小年龄,默认 1min
}
// DefaultOptions 返回默认配置
func DefaultOptions(dir string) *Options {
return &Options{
Dir: dir,
Logger: nil, // 默认不输出日志
MemTableSize: 64 * 1024 * 1024, // 64MB
AutoFlushTimeout: 30 * time.Second, // 30s
Level0SizeLimit: 64 * 1024 * 1024, // 64MB
Level1SizeLimit: 256 * 1024 * 1024, // 256MB
Level2SizeLimit: 512 * 1024 * 1024, // 512MB
Level3SizeLimit: 1024 * 1024 * 1024, // 1GB
CompactionInterval: 10 * time.Second, // 10s
GCInterval: 5 * time.Minute, // 5min
DisableAutoCompaction: false,
DisableGC: false,
GCFileMinAge: 1 * time.Minute, // 1min
}
}
// fillDefaults 填充未设置的默认值(修改传入的 opts
func (opts *Options) fillDefaults() {
// Logger如果为 nil创建一个丢弃所有日志的 logger
if opts.Logger == nil {
opts.Logger = slog.New(slog.NewTextHandler(io.Discard, nil))
}
if opts.MemTableSize == 0 {
opts.MemTableSize = 64 * 1024 * 1024 // 64MB
}
if opts.AutoFlushTimeout == 0 {
opts.AutoFlushTimeout = 30 * time.Second // 30s
}
if opts.Level0SizeLimit == 0 {
opts.Level0SizeLimit = 64 * 1024 * 1024 // 64MB
}
if opts.Level1SizeLimit == 0 {
opts.Level1SizeLimit = 256 * 1024 * 1024 // 256MB
}
if opts.Level2SizeLimit == 0 {
opts.Level2SizeLimit = 512 * 1024 * 1024 // 512MB
}
if opts.Level3SizeLimit == 0 {
opts.Level3SizeLimit = 1024 * 1024 * 1024 // 1GB
}
if opts.CompactionInterval == 0 {
opts.CompactionInterval = 10 * time.Second // 10s
}
if opts.GCInterval == 0 {
opts.GCInterval = 5 * time.Minute // 5min
}
if opts.GCFileMinAge == 0 {
opts.GCFileMinAge = 1 * time.Minute // 1min
}
}
// Validate 验证配置的有效性
func (opts *Options) Validate() error {
if opts.Dir == "" {
return NewErrorf(ErrCodeInvalidParam, "database directory cannot be empty")
}
if opts.MemTableSize < 1*1024*1024 {
return NewErrorf(ErrCodeInvalidParam, "MemTableSize must be at least 1MB, got %d", opts.MemTableSize)
}
if opts.Level0SizeLimit < 1*1024*1024 {
return NewErrorf(ErrCodeInvalidParam, "Level0SizeLimit must be at least 1MB, got %d", opts.Level0SizeLimit)
}
if opts.Level1SizeLimit < opts.Level0SizeLimit {
return NewErrorf(ErrCodeInvalidParam, "Level1SizeLimit (%d) must be >= Level0SizeLimit (%d)", opts.Level1SizeLimit, opts.Level0SizeLimit)
}
if opts.Level2SizeLimit < opts.Level1SizeLimit {
return NewErrorf(ErrCodeInvalidParam, "Level2SizeLimit (%d) must be >= Level1SizeLimit (%d)", opts.Level2SizeLimit, opts.Level1SizeLimit)
}
if opts.Level3SizeLimit < opts.Level2SizeLimit {
return NewErrorf(ErrCodeInvalidParam, "Level3SizeLimit (%d) must be >= Level2SizeLimit (%d)", opts.Level3SizeLimit, opts.Level2SizeLimit)
}
if opts.CompactionInterval < 1*time.Second {
return NewErrorf(ErrCodeInvalidParam, "CompactionInterval must be at least 1s, got %v", opts.CompactionInterval)
}
if opts.GCInterval < 1*time.Minute {
return NewErrorf(ErrCodeInvalidParam, "GCInterval must be at least 1min, got %v", opts.GCInterval)
}
if opts.GCFileMinAge < 0 {
return NewErrorf(ErrCodeInvalidParam, "GCFileMinAge cannot be negative, got %v", opts.GCFileMinAge)
}
return nil
}
// Open 打开数据库(向后兼容,使用默认配置)
func Open(dir string) (*Database, error) {
return OpenWithOptions(DefaultOptions(dir))
}
// OpenWithOptions 使用指定配置打开数据库
func OpenWithOptions(opts *Options) (*Database, error) {
// 填充默认值
opts.fillDefaults()
// 验证配置
if err := opts.Validate(); err != nil {
return nil, err
}
// 创建目录
err := os.MkdirAll(dir, 0755)
err := os.MkdirAll(opts.Dir, 0755)
if err != nil {
return nil, err
}
db := &Database{
dir: dir,
tables: make(map[string]*Table),
dir: opts.Dir,
tables: make(map[string]*Table),
options: opts,
}
// 加载元数据
@@ -111,24 +243,39 @@ func (db *Database) recoverTables() error {
for _, tableInfo := range db.metadata.Tables {
tableDir := filepath.Join(db.dir, tableInfo.Name)
table, err := OpenTable(&TableOptions{
Dir: tableDir,
MemTableSize: DefaultMemTableSize,
Dir: tableDir,
MemTableSize: db.options.MemTableSize,
AutoFlushTimeout: db.options.AutoFlushTimeout,
})
if err != nil {
// 记录失败的表,但继续恢复其他表
failedTables = append(failedTables, tableInfo.Name)
fmt.Printf("[WARNING] Failed to open table %s: %v\n", tableInfo.Name, err)
fmt.Printf("[WARNING] Table %s will be skipped. You may need to drop and recreate it.\n", tableInfo.Name)
db.options.Logger.Warn("[Database] Failed to open table",
"table", tableInfo.Name,
"error", err)
db.options.Logger.Warn("[Database] Table will be skipped. You may need to drop and recreate it.",
"table", tableInfo.Name)
continue
}
// 设置 Logger
table.SetLogger(db.options.Logger)
// 将数据库级 Compaction 配置应用到表的 CompactionManager
if table.compactionManager != nil {
table.compactionManager.ApplyConfig(db.options)
}
db.tables[tableInfo.Name] = table
}
// 如果有失败的表,输出汇总信息
if len(failedTables) > 0 {
fmt.Printf("[WARNING] %d table(s) failed to recover: %v\n", len(failedTables), failedTables)
fmt.Printf("[WARNING] To fix: Delete the corrupted table directory and restart.\n")
fmt.Printf("[WARNING] Example: rm -rf %s/<table_name>\n", db.dir)
db.options.Logger.Warn("[Database] Failed to recover tables",
"failed_count", len(failedTables),
"failed_tables", failedTables)
db.options.Logger.Warn("[Database] To fix: Delete the corrupted table directory and restart",
"example", fmt.Sprintf("rm -rf %s/<table_name>", db.dir))
}
return nil
@@ -151,18 +298,27 @@ func (db *Database) CreateTable(name string, schema *Schema) (*Table, error) {
return nil, err
}
// 创建表
// 创建表(传递数据库级配置)
table, err := OpenTable(&TableOptions{
Dir: tableDir,
MemTableSize: DefaultMemTableSize,
Name: schema.Name,
Fields: schema.Fields,
Dir: tableDir,
MemTableSize: db.options.MemTableSize,
AutoFlushTimeout: db.options.AutoFlushTimeout,
Name: schema.Name,
Fields: schema.Fields,
})
if err != nil {
os.RemoveAll(tableDir)
return nil, err
}
// 设置 Logger
table.SetLogger(db.options.Logger)
// 将数据库级 Compaction 配置应用到表的 CompactionManager
if table.compactionManager != nil {
table.compactionManager.ApplyConfig(db.options)
}
// 添加到 tables map
db.tables[name] = table

View File

@@ -6,6 +6,10 @@
```
examples/
├── complex/ # 复杂类型系统示例21 种类型全覆盖)
│ ├── main.go # 主程序
│ ├── README.md # 详细文档
│ └── .gitignore # 忽略数据目录
└── webui/ # Web UI 和命令行工具集
├── main.go # 主入口点
├── commands/ # 命令实现
@@ -22,6 +26,82 @@ examples/
---
## Complex - 完整类型系统演示
一个展示 SRDB 所有 **21 种数据类型**的完整示例,包括结构体 Schema 生成、边界值测试、索引查询和分页等核心功能。
### 🎯 涵盖的类型
| 分类 | 数量 | 包含类型 |
|------|------|----------|
| **字符串** | 1 种 | String |
| **有符号整数** | 5 种 | Int, Int8, Int16, Int32, Int64 |
| **无符号整数** | 5 种 | Uint, Uint8, Uint16, Uint32, Uint64 |
| **浮点数** | 2 种 | Float32, Float64 |
| **布尔** | 1 种 | Bool |
| **特殊类型** | 5 种 | Byte, Rune, Decimal, Time, Duration |
| **复杂类型** | 2 种 | Object, Array |
### 快速开始
```bash
cd examples/complex
# 运行示例
go run main.go
# 清理并重新生成
go run main.go --clean
# 指定数据目录
go run main.go --dir ./mydata --clean
```
### 示例输出
```
╔═══════════════ 设备记录 #1 (seq=1) ═══════════════╗
║ ID: IOT-2025-0001 ║
║ 名称: 智能环境监测站 ║
╟─────────────────── 整数类型 ────────────────────────╢
║ Signal(int): -55 ║
║ ErrorCode(i8): 0 ║
║ DeltaTemp(i16): 150 ║
║ RecordNum(i32): 12345 ║
║ TotalBytes(i64):1073741824 ║
...
```
### 功能演示
**结构体自动生成 Schema**
```go
fields, _ := srdb.StructToFields(DeviceRecord{})
```
**边界值测试**
- int8 最大值 (127)
- int16 最小值 (-32768)
- uint64 最大值 (18446744073709551615)
**索引查询优化**
```go
table.Query().Eq("device_id", "IOT-2025-0001").Rows()
```
**分页查询(返回总数)**
```go
rows, total, err := table.Query().Paginate(1, 10)
```
**复杂类型序列化**
- Object: map[string]any → JSON
- Array: []string → JSON
详细文档:[complex/README.md](complex/README.md)
---
## WebUI - 数据库管理工具
一个集成了 Web 界面和命令行工具的 SRDB 数据库管理工具。

356
examples/complex/README.md Normal file
View File

@@ -0,0 +1,356 @@
# SRDB 复杂类型示例
这个示例演示了 SRDB 支持的所有 **21 种数据类型**的使用方法,包括:
- 结构体自动生成 Schema
- 所有基本类型和特殊类型的插入与查询
- 边界值测试
- 索引查询
- 分页查询
- 复杂类型Object/Array的序列化
## 📊 支持的 21 种类型
### 基本类型 (14种)
| 分类 | 类型 | Go 类型 | 说明 |
|------|------|---------|------|
| **字符串** | String | `string` | UTF-8 字符串 |
| **有符号整数** | Int | `int` | 平台相关 |
| | Int8 | `int8` | -128 ~ 127 |
| | Int16 | `int16` | -32768 ~ 32767 |
| | Int32 | `int32` | -2^31 ~ 2^31-1 |
| | Int64 | `int64` | -2^63 ~ 2^63-1 |
| **无符号整数** | Uint | `uint` | 平台相关 |
| | Uint8 | `uint8` | 0 ~ 255 |
| | Uint16 | `uint16` | 0 ~ 65535 |
| | Uint32 | `uint32` | 0 ~ 4294967295 |
| | Uint64 | `uint64` | 0 ~ 2^64-1 |
| **浮点数** | Float32 | `float32` | 单精度 |
| | Float64 | `float64` | 双精度 |
| **布尔** | Bool | `bool` | true/false |
### 特殊类型 (5种)
| 类型 | Go 类型 | 说明 | 使用场景 |
|------|---------|------|----------|
| Byte | `byte` | 0-255独立类型 | 状态码、百分比、标志位 |
| Rune | `rune` | Unicode 字符(独立类型) | 等级、分类字符 |
| Decimal | `decimal.Decimal` | 高精度十进制 | 金融计算、货币金额 |
| Time | `time.Time` | 时间戳 | 日期时间 |
| Duration | `time.Duration` | 时长 | 超时、间隔、运行时长 |
### 复杂类型 (2种)
| 类型 | Go 类型 | 说明 |
|------|---------|------|
| Object | `map[string]any`, `struct{}` | JSON 编码存储 |
| Array | `[]any`, `[]string`, `[]int` 等 | JSON 编码存储 |
## 🚀 快速开始
### 1. 构建并运行
```bash
cd examples/complex
go run main.go
```
### 2. 使用参数
```bash
# 指定数据目录
go run main.go --dir ./mydata
# 清理数据并重新生成
go run main.go --clean
# 指定目录并清理
go run main.go --dir ./mydata --clean
```
### 3. 构建可执行文件
```bash
go build -o complex
./complex --clean
```
## 📝 代码结构
### 结构体定义
```go
type DeviceRecord struct {
// 字符串
DeviceID string `srdb:"device_id;indexed;comment:设备ID"`
Name string `srdb:"name;comment:设备名称"`
// 有符号整数 (5种)
Signal int `srdb:"signal;comment:信号强度"`
ErrorCode int8 `srdb:"error_code;comment:错误码"`
DeltaTemp int16 `srdb:"delta_temp;comment:温差"`
RecordNum int32 `srdb:"record_num;comment:记录号"`
TotalBytes int64 `srdb:"total_bytes;comment:总字节数"`
// 无符号整数 (5种)
Flags uint `srdb:"flags;comment:标志位"`
Status uint8 `srdb:"status;comment:状态"`
Port uint16 `srdb:"port;comment:端口"`
SessionID uint32 `srdb:"session_id;comment:会话ID"`
Timestamp uint64 `srdb:"timestamp;comment:时间戳"`
// 浮点数 (2种)
TempValue float32 `srdb:"temp_value;comment:温度值"`
Latitude float64 `srdb:"latitude;comment:纬度"`
Longitude float64 `srdb:"longitude;comment:经度"`
// 布尔
IsOnline bool `srdb:"is_online;indexed;comment:在线状态"`
// 特殊类型
BatteryPct byte `srdb:"battery_pct;comment:电量百分比"`
Level rune `srdb:"level;comment:等级字符"`
Price decimal.Decimal `srdb:"price;comment:价格"`
CreatedAt time.Time `srdb:"created_at;comment:创建时间"`
RunTime time.Duration `srdb:"run_time;comment:运行时长"`
// 复杂类型
Settings map[string]any `srdb:"settings;comment:设置"`
Tags []string `srdb:"tags;comment:标签列表"`
}
```
### 核心步骤
1. **从结构体生成 Schema**
```go
fields, err := srdb.StructToFields(DeviceRecord{})
```
2. **创建表**
```go
table, err := srdb.OpenTable(&srdb.TableOptions{
Dir: "./data",
Name: "devices",
Fields: fields,
})
```
3. **插入数据(使用 map**
```go
device := map[string]any{
"device_id": "IOT-2025-0001",
"name": "智能环境监测站",
"signal": -55,
"error_code": int8(0),
"port": uint16(8080),
"temp_value": float32(23.5),
"is_online": true,
"battery_pct": byte(85),
"level": rune('S'),
"price": decimal.NewFromFloat(999.99),
"created_at": time.Now(),
"run_time": 3*time.Hour + 25*time.Minute,
"settings": map[string]any{"interval": 60},
"tags": []string{"indoor", "hvac"},
}
table.Insert(device)
```
4. **查询数据**
```go
rows, err := table.Query().OrderBy("_seq").Rows()
for rows.Next() {
row := rows.Row()
data := row.Data()
// 处理数据...
}
```
5. **索引查询**
```go
table.BuildIndexes()
rows, _ := table.Query().Eq("device_id", "IOT-2025-0001").Rows()
```
6. **分页查询**
```go
rows, total, err := table.Query().OrderBy("_seq").Paginate(1, 10)
```
## 🎯 示例输出
运行程序后,你会看到漂亮的表格化输出:
```
╔═══════════════ 设备记录 #1 (seq=1) ═══════════════╗
║ ID: IOT-2025-0001 ║
║ 名称: 智能环境监测站 ║
╟─────────────────── 整数类型 ────────────────────────╢
║ Signal(int): -55 ║
║ ErrorCode(i8): 0 ║
║ DeltaTemp(i16): 150 ║
║ RecordNum(i32): 12345 ║
║ TotalBytes(i64):1073741824 ║
║ Flags(uint): 0xF ║
║ Status(u8): 200 ║
║ Port(u16): 8080 ║
║ SessionID(u32): 987654321 ║
║ Timestamp(u64): 1760210986 ║
╟───────────────── 浮点/布尔 ──────────────────────╢
║ Temperature(f32): 23.50°C ║
║ 坐标(f64): (39.904200, 116.407396) ║
║ Online(bool): true ║
╟───────────────── 特殊类型 ──────────────────────╢
║ Battery(byte): 85% ║
║ Level(rune): S ║
║ Price(decimal): ¥999.99 ║
║ CreatedAt(time): 2025-10-12 03:29:46 ║
║ RunTime(duration): 3h25m0s ║
╟───────────────── 复杂类型 ──────────────────────╢
║ Settings(object): 4 项配置 ║
║ • report_interval = 60 ║
║ • sample_rate = 100 ║
║ • auto_calibrate = true ║
║ • threshold = 25 ║
║ Tags(array): 4 个标签 ║
║ [indoor hvac monitoring enterprise] ║
╚═════════════════════════════════════════════════════╝
```
## 💡 关键特性
### 1. 边界值测试
示例包含各类型的边界值测试:
```go
device := map[string]any{
"error_code": int8(127), // int8 最大值
"delta_temp": int16(-32768), // int16 最小值
"record_num": int32(2147483647), // int32 最大值
"total_bytes": int64(9223372036854775807), // int64 最大值
"status": uint8(255), // uint8 最大值
"port": uint16(65535), // uint16 最大值
}
```
### 2. 索引查询优化
使用索引加速查询:
```go
// 结构体中标记索引
DeviceID string `srdb:"device_id;indexed"`
IsOnline bool `srdb:"is_online;indexed"`
// 构建索引
table.BuildIndexes()
// 使用索引查询
rows, _ := table.Query().Eq("device_id", "IOT-2025-0001").Rows()
rows, _ := table.Query().Eq("is_online", true).Rows()
```
### 3. 分页查询
支持返回总数的分页:
```go
rows, total, err := table.Query().OrderBy("_seq").Paginate(1, 2)
fmt.Printf("总记录数: %d\n", total)
```
### 4. 复杂类型序列化
Object 和 Array 自动序列化为 JSON
```go
// Object: map[string]any
"settings": map[string]any{
"report_interval": 60,
"sample_rate": 100,
"auto_calibrate": true,
}
// Array: []string
"tags": []string{"indoor", "hvac", "monitoring"}
// 查询时自动反序列化
settings := data["settings"].(map[string]any)
tags := data["tags"].([]any)
```
## 📚 类型选择最佳实践
### 整数类型
```go
// ❌ 不推荐:盲目使用 int64
Port int64 // 端口号 0-65535浪费 6 字节
Status int64 // 状态码 0-255浪费 7 字节
// ✅ 推荐:根据数据范围选择
Port uint16 // 0-655352 字节
Status uint8 // 0-2551 字节
```
### 浮点数类型
```go
// ❌ 不推荐
Temperature float64 // 温度用单精度足够
// ✅ 推荐
Temperature float32 // -40°C ~ 125°C单精度足够
Latitude float64 // 地理坐标需要双精度
```
### 特殊类型使用
```go
// Byte: 百分比、状态码
BatteryLevel byte // 0-100
// Rune: 单字符等级
Grade rune // 'S', 'A', 'B', 'C'
// Decimal: 金融计算
Price decimal.Decimal // 避免浮点精度问题
// Time: 时间戳
CreatedAt time.Time
// Duration: 时长
Timeout time.Duration
```
## 🔧 依赖
```go
import (
"code.tczkiot.com/wlw/srdb"
"github.com/shopspring/decimal"
)
```
确保已安装 `decimal` 包:
```bash
go get github.com/shopspring/decimal
```
## 📖 相关文档
- [SRDB 主文档](../../README.md)
- [CLAUDE.md - 开发指南](../../CLAUDE.md)
- [WebUI 示例](../webui/)
## 🤝 贡献
如果你有更好的示例或发现问题,欢迎提交 Issue 或 Pull Request。
## 📄 许可证
MIT License - 详见项目根目录 LICENSE 文件

691
examples/complex/main.go Normal file
View File

@@ -0,0 +1,691 @@
package main
import (
"flag"
"fmt"
"os"
"path/filepath"
"time"
"code.tczkiot.com/wlw/srdb"
"github.com/shopspring/decimal"
)
// ========== 嵌套结构体定义 ==========
// Location 位置信息(嵌套结构体)
type Location struct {
Country string `json:"country"`
Province string `json:"province"`
City string `json:"city"`
Address string `json:"address"`
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
}
// NetworkConfig 网络配置(嵌套结构体)
type NetworkConfig struct {
SSID string `json:"ssid"`
Password string `json:"password"`
IPAddress string `json:"ip_address"`
Gateway string `json:"gateway"`
DNS string `json:"dns"`
UseStaticIP bool `json:"use_static_ip"`
}
// Sensor 传感器信息(用于切片)
type Sensor struct {
Type string `json:"type"` // 传感器类型
Model string `json:"model"` // 型号
Value float64 `json:"value"` // 当前值
Unit string `json:"unit"` // 单位
MinValue float64 `json:"min_value"` // 最小值
MaxValue float64 `json:"max_value"` // 最大值
Precision int `json:"precision"` // 精度
SamplingRate int `json:"sampling_rate"` // 采样率
Enabled bool `json:"enabled"` // 是否启用
}
// MaintenanceRecord 维护记录(用于切片)
type MaintenanceRecord struct {
Date string `json:"date"` // 维护日期
Technician string `json:"technician"` // 技术员
Type string `json:"type"` // 维护类型
Description string `json:"description"` // 描述
Cost float64 `json:"cost"` // 费用
NextDate string `json:"next_date"` // 下次维护日期
}
// ========== 主结构体定义 ==========
// ComplexDevice 复杂设备记录(包含所有复杂场景)
type ComplexDevice struct {
// ========== 基本字段 ==========
DeviceID string `srdb:"device_id;indexed;comment:设备ID"`
Name string `srdb:"name;comment:设备名称"`
Model string `srdb:"model;comment:设备型号"`
// ========== Nullable 字段(指针类型)==========
SerialNumber *string `srdb:"serial_number;nullable;comment:序列号(可选)"`
Manufacturer *string `srdb:"manufacturer;nullable;comment:制造商(可选)"`
Description *string `srdb:"description;nullable;comment:描述(可选)"`
WarrantyEnd *time.Time `srdb:"warranty_end;nullable;comment:保修截止日期(可选)"`
LastMaintenance *time.Time `srdb:"last_maintenance;nullable;comment:上次维护时间(可选)"`
MaxPower *float32 `srdb:"max_power;nullable;comment:最大功率(可选)"`
Weight *float64 `srdb:"weight;nullable;comment:重量(可选)"`
Voltage *int32 `srdb:"voltage;nullable;comment:电压(可选)"`
Price *decimal.Decimal `srdb:"price;nullable;comment:价格(可选)"`
// ========== 所有基本类型 ==========
// 有符号整数
Signal int `srdb:"signal;comment:信号强度"`
ErrorCode int8 `srdb:"error_code;comment:错误码"`
Temperature int16 `srdb:"temperature;comment:温度(℃*10"`
Counter int32 `srdb:"counter;comment:计数器"`
TotalBytes int64 `srdb:"total_bytes;comment:总字节数"`
// 无符号整数
Flags uint `srdb:"flags;comment:标志位"`
Status uint8 `srdb:"status;comment:状态码"`
Port uint16 `srdb:"port;comment:端口号"`
SessionID uint32 `srdb:"session_id;comment:会话ID"`
Timestamp uint64 `srdb:"timestamp;comment:时间戳"`
// 浮点数
Humidity float32 `srdb:"humidity;comment:湿度"`
Latitude float64 `srdb:"latitude;comment:纬度"`
Longitude float64 `srdb:"longitude;comment:经度"`
// 布尔
IsOnline bool `srdb:"is_online;indexed;comment:是否在线"`
IsActivated bool `srdb:"is_activated;comment:是否激活"`
// 特殊类型
BatteryLevel byte `srdb:"battery_level;comment:电池电量"`
Grade rune `srdb:"grade;comment:等级"`
TotalPrice decimal.Decimal `srdb:"total_price;comment:总价"`
CreatedAt time.Time `srdb:"created_at;comment:创建时间"`
Uptime time.Duration `srdb:"uptime;comment:运行时长"`
// ========== 嵌套结构体Object==========
Location Location `srdb:"location;comment:位置信息(嵌套结构体)"`
NetworkConfig NetworkConfig `srdb:"network_config;comment:网络配置(嵌套结构体)"`
// ========== 结构体切片Array==========
Sensors []Sensor `srdb:"sensors;comment:传感器列表(结构体切片)"`
MaintenanceRecords []MaintenanceRecord `srdb:"maintenance_records;comment:维护记录(结构体切片)"`
// ========== 基本类型切片 ==========
Tags []string `srdb:"tags;comment:标签列表"`
AlertCodes []int32 `srdb:"alert_codes;comment:告警代码列表"`
HistoryReadings []float64 `srdb:"history_readings;comment:历史读数"`
// ========== 简单 MapObject==========
Metadata map[string]any `srdb:"metadata;comment:元数据"`
CustomSettings map[string]any `srdb:"custom_settings;comment:自定义设置"`
}
func main() {
// 命令行参数
dataDir := flag.String("dir", "./data", "数据存储目录")
clean := flag.Bool("clean", false, "运行前清理数据目录")
flag.Parse()
fmt.Println("=============================================================")
fmt.Println(" SRDB 复杂类型系统演示Nullable + 嵌套结构体 + 结构体切片)")
fmt.Println("=============================================================\n")
// 准备数据目录
absDir, err := filepath.Abs(*dataDir)
if err != nil {
fmt.Printf("❌ 无效的目录路径: %v\n", err)
os.Exit(1)
}
if *clean {
fmt.Printf("🧹 清理数据目录: %s\n", absDir)
os.RemoveAll(absDir)
}
fmt.Printf("📁 数据目录: %s\n\n", absDir)
// ========== 步骤 1: 从结构体生成 Schema ==========
fmt.Println("【步骤 1】从结构体自动生成 Schema")
fmt.Println("─────────────────────────────────────────────────────")
fields, err := srdb.StructToFields(ComplexDevice{})
if err != nil {
fmt.Printf("❌ 失败: %v\n", err)
os.Exit(1)
}
fmt.Printf("✅ 成功生成 %d 个字段\n\n", len(fields))
// 统计字段类型
nullableCount := 0
objectCount := 0
arrayCount := 0
for _, field := range fields {
if field.Nullable {
nullableCount++
}
if field.Type.String() == "object" {
objectCount++
}
if field.Type.String() == "array" {
arrayCount++
}
}
fmt.Println("字段统计:")
fmt.Printf(" • 总字段数: %d\n", len(fields))
fmt.Printf(" • Nullable 字段: %d 个(使用指针)\n", nullableCount)
fmt.Printf(" • Object 字段: %d 个(结构体/map\n", objectCount)
fmt.Printf(" • Array 字段: %d 个(切片)\n", arrayCount)
// ========== 步骤 2: 创建表 ==========
fmt.Println("\n【步骤 2】创建数据表")
fmt.Println("─────────────────────────────────────────────────────")
table, err := srdb.OpenTable(&srdb.TableOptions{
Dir: absDir,
Name: "complex_devices",
Fields: fields,
})
if err != nil {
fmt.Printf("❌ 创建表失败: %v\n", err)
os.Exit(1)
}
defer table.Close()
fmt.Println("✅ 表 'complex_devices' 创建成功")
// ========== 步骤 3: 插入完整数据 ==========
fmt.Println("\n【步骤 3】插入测试数据")
fmt.Println("─────────────────────────────────────────────────────")
// 准备辅助变量
serialNum := "SN-2025-001-ALPHA"
manufacturer := "智能科技有限公司"
description := "高性能工业级环境监测站,支持多种传感器接入"
warrantyEnd := time.Now().AddDate(3, 0, 0) // 3年保修
lastMaint := time.Now().Add(-30 * 24 * time.Hour) // 30天前维护
maxPower := float32(500.5)
weight := 12.5
voltage := int32(220)
price := decimal.NewFromFloat(9999.99)
// 数据1: 完整填充(包含所有 Nullable 字段)
device1 := map[string]any{
// 基本字段
"device_id": "COMPLEX-DEV-001",
"name": "智能环境监测站 Pro",
"model": "ENV-MONITOR-PRO-X1",
// Nullable 字段(全部有值)
"serial_number": serialNum,
"manufacturer": manufacturer,
"description": description,
"warranty_end": warrantyEnd,
"last_maintenance": lastMaint,
"max_power": maxPower,
"weight": weight,
"voltage": voltage,
"price": price,
// 基本类型
"signal": -55,
"error_code": int8(0),
"temperature": int16(235), // 23.5°C
"counter": int32(12345),
"total_bytes": int64(1024 * 1024 * 500),
"flags": uint(0x0F),
"status": uint8(200),
"port": uint16(8080),
"session_id": uint32(987654321),
"timestamp": uint64(time.Now().Unix()),
"humidity": float32(65.5),
"latitude": 39.904200,
"longitude": 116.407396,
"is_online": true,
"is_activated": true,
"battery_level": byte(85),
"grade": rune('S'),
"total_price": decimal.NewFromFloat(15999.99),
"created_at": time.Now(),
"uptime": 72 * time.Hour,
// 嵌套结构体
"location": Location{
Country: "中国",
Province: "北京市",
City: "朝阳区",
Address: "建国路88号",
Lat: 39.904200,
Lng: 116.407396,
},
"network_config": NetworkConfig{
SSID: "SmartDevice-5G",
Password: "******",
IPAddress: "192.168.1.100",
Gateway: "192.168.1.1",
DNS: "8.8.8.8",
UseStaticIP: true,
},
// 结构体切片
"sensors": []Sensor{
{
Type: "temperature",
Model: "DHT22",
Value: 23.5,
Unit: "°C",
MinValue: -40.0,
MaxValue: 80.0,
Precision: 1,
SamplingRate: 1000,
Enabled: true,
},
{
Type: "humidity",
Model: "DHT22",
Value: 65.5,
Unit: "%",
MinValue: 0.0,
MaxValue: 100.0,
Precision: 1,
SamplingRate: 1000,
Enabled: true,
},
{
Type: "pressure",
Model: "BMP280",
Value: 1013.25,
Unit: "hPa",
MinValue: 300.0,
MaxValue: 1100.0,
Precision: 2,
SamplingRate: 500,
Enabled: true,
},
},
"maintenance_records": []MaintenanceRecord{
{
Date: "2024-12-01",
Technician: "张工",
Type: "定期维护",
Description: "清洁传感器、检查线路、更新固件",
Cost: 500.00,
NextDate: "2025-03-01",
},
{
Date: "2024-09-15",
Technician: "李工",
Type: "故障维修",
Description: "更换损坏的温度传感器",
Cost: 800.00,
NextDate: "2024-12-01",
},
},
// 基本类型切片
"tags": []string{"industrial", "outdoor", "monitoring", "iot", "smart-city"},
"alert_codes": []int32{1001, 2003, 3005, 4002},
"history_readings": []float64{23.1, 23.3, 23.5, 23.7, 23.9, 24.0},
// Map
"metadata": map[string]any{
"install_date": "2024-01-15",
"firmware_version": "v2.3.1",
"hardware_revision": "Rev-C",
"certification": []string{"CE", "FCC", "RoHS"},
},
"custom_settings": map[string]any{
"auto_calibrate": true,
"report_interval": 60,
"alert_threshold": 85.0,
"debug_mode": false,
},
}
err = table.Insert(device1)
if err != nil {
fmt.Printf("❌ 插入数据1失败: %v\n", err)
os.Exit(1)
}
fmt.Println("✅ 数据1插入成功: " + device1["name"].(string))
fmt.Println(" 包含: 9个Nullable字段全部有值")
fmt.Println(" 包含: 2个嵌套结构体 + 2个结构体切片")
// Debug: 检查插入后的记录数
count1, _ := table.Query().Rows()
c1 := 0
for count1.Next() {
c1++
}
count1.Close()
fmt.Printf(" 🔍 插入后表中有 %d 条记录\n", c1)
// 数据2: 部分 Nullable 字段为 nil
device2 := map[string]any{
// 基本字段
"device_id": "COMPLEX-DEV-002",
"name": "简易温湿度传感器",
"model": "TEMP-SENSOR-LITE",
// Nullable 字段(部分为 nil
"serial_number": "SN-2025-002-BETA",
"manufacturer": "普通传感器公司",
"description": nil, // NULL
"warranty_end": nil, // NULL
"last_maintenance": nil, // NULL
"max_power": nil, // NULL
"weight": nil, // NULL
"voltage": nil, // NULL
"price": nil, // NULL
// 基本类型
"signal": -70,
"error_code": int8(0),
"temperature": int16(220),
"counter": int32(500),
"total_bytes": int64(1024 * 1024 * 10),
"flags": uint(0x03),
"status": uint8(100),
"port": uint16(8081),
"session_id": uint32(123456789),
"timestamp": uint64(time.Now().Unix()),
"humidity": float32(55.0),
"latitude": 39.900000,
"longitude": 116.400000,
"is_online": false,
"is_activated": true,
"battery_level": byte(30),
"grade": rune('B'),
"total_price": decimal.NewFromFloat(299.99),
"created_at": time.Now().Add(-7 * 24 * time.Hour),
"uptime": 168 * time.Hour,
// 嵌套结构体
"location": Location{
Country: "中国",
Province: "上海市",
City: "浦东新区",
Address: "世纪大道123号",
Lat: 31.235929,
Lng: 121.506058,
},
"network_config": NetworkConfig{
SSID: "SmartDevice-2.4G",
Password: "******",
IPAddress: "192.168.1.101",
Gateway: "192.168.1.1",
DNS: "114.114.114.114",
UseStaticIP: false,
},
// 结构体切片(较少的元素)
"sensors": []Sensor{
{
Type: "temperature",
Model: "DS18B20",
Value: 22.0,
Unit: "°C",
MinValue: -55.0,
MaxValue: 125.0,
Precision: 0,
SamplingRate: 500,
Enabled: true,
},
},
"maintenance_records": []MaintenanceRecord{},
// 基本类型切片
"tags": []string{"indoor", "basic"},
"alert_codes": []int32{},
"history_readings": []float64{21.5, 21.8, 22.0},
// Map
"metadata": map[string]any{
"install_date": "2025-01-01",
"firmware_version": "v1.0.0",
},
"custom_settings": map[string]any{
"report_interval": 120,
},
}
err = table.Insert(device2)
if err != nil {
fmt.Printf("❌ 插入数据2失败: %v\n", err)
os.Exit(1)
}
fmt.Println("✅ 数据2插入成功: " + device2["name"].(string))
fmt.Println(" 包含: 9个Nullable字段6个为nil")
fmt.Println(" 包含: 较少的结构体切片元素")
// Debug: 检查插入后的记录数
count2, _ := table.Query().Rows()
c2 := 0
for count2.Next() {
c2++
}
count2.Close()
fmt.Printf(" 🔍 插入后表中有 %d 条记录\n", c2)
// 数据3: 所有 Nullable 字段为 nil
device3 := map[string]any{
// 基本字段
"device_id": "COMPLEX-DEV-003",
"name": "最小配置设备",
"model": "MIN-CONFIG",
// Nullable 字段(全部为 nil
"serial_number": nil,
"manufacturer": nil,
"description": nil,
"warranty_end": nil,
"last_maintenance": nil,
"max_power": nil,
"weight": nil,
"voltage": nil,
"price": nil,
// 基本类型(最小值/默认值)
"signal": -90,
"error_code": int8(-1),
"temperature": int16(0),
"counter": int32(0),
"total_bytes": int64(0),
"flags": uint(0),
"status": uint8(0),
"port": uint16(0),
"session_id": uint32(0),
"timestamp": uint64(0),
"humidity": float32(0.0),
"latitude": 0.0,
"longitude": 0.0,
"is_online": false,
"is_activated": false,
"battery_level": byte(0),
"grade": rune('C'),
"total_price": decimal.Zero,
"created_at": time.Unix(0, 0),
"uptime": 0 * time.Second,
// 嵌套结构体(空值)
"location": Location{},
"network_config": NetworkConfig{},
// 结构体切片(空切片)
"sensors": []Sensor{},
"maintenance_records": []MaintenanceRecord{},
// 基本类型切片(空切片)
"tags": []string{},
"alert_codes": []int32{},
"history_readings": []float64{},
// Map空map
"metadata": map[string]any{},
"custom_settings": map[string]any{},
}
err = table.Insert(device3)
if err != nil {
fmt.Printf("❌ 插入数据3失败: %v\n", err)
os.Exit(1)
}
fmt.Println("✅ 数据3插入成功: " + device3["name"].(string))
fmt.Println(" 包含: 9个Nullable字段全部为nil")
fmt.Println(" 包含: 所有切片为空")
// Debug: 检查插入后的记录数
count3, _ := table.Query().Rows()
c3 := 0
for count3.Next() {
c3++
}
count3.Close()
fmt.Printf(" 🔍 插入后表中有 %d 条记录\n", c3)
// ========== 步骤 4: 查询并展示 ==========
fmt.Println("\n【步骤 4】查询并验证数据")
fmt.Println("─────────────────────────────────────────────────────")
// Debug: 直接检查表的记录数
debugRows, _ := table.Query().Rows()
debugCount := 0
for debugRows.Next() {
debugCount++
}
debugRows.Close()
fmt.Printf("🔍 调试: 表中实际有 %d 条记录\n\n", debugCount)
rows, err := table.Query().OrderBy("_seq").Rows()
if err != nil {
fmt.Printf("❌ 查询失败: %v\n", err)
os.Exit(1)
}
defer rows.Close()
count := 0
for rows.Next() {
row := rows.Row()
data := row.Data()
count++
fmt.Printf("\n╔══════════════════ 设备 #%d (seq=%d) ══════════════════╗\n", count, row.Seq())
fmt.Printf("║ ID: %-53s ║\n", data["device_id"])
fmt.Printf("║ 名称: %-51s ║\n", data["name"])
fmt.Printf("║ 型号: %-51s ║\n", data["model"])
// Nullable 字段展示
fmt.Printf("╟────────────────── Nullable 字段 ─────────────────────╢\n")
if data["serial_number"] != nil {
fmt.Printf("║ 序列号: %-47s ║\n", data["serial_number"])
} else {
fmt.Printf("║ 序列号: <未设置>%40s ║\n", "")
}
if data["manufacturer"] != nil {
fmt.Printf("║ 制造商: %-47s ║\n", data["manufacturer"])
} else {
fmt.Printf("║ 制造商: <未设置>%40s ║\n", "")
}
if data["price"] != nil {
price := data["price"].(decimal.Decimal)
fmt.Printf("║ 价格: ¥%-47s ║\n", price.StringFixed(2))
} else {
fmt.Printf("║ 价格: <未设置>%42s ║\n", "")
}
if data["warranty_end"] != nil {
warrantyEnd := data["warranty_end"].(time.Time)
fmt.Printf("║ 保修截止: %-43s ║\n", warrantyEnd.Format("2006-01-02"))
} else {
fmt.Printf("║ 保修截止: <未设置>%38s ║\n", "")
}
// 嵌套结构体展示
fmt.Printf("╟───────────────── 嵌套结构体 ─────────────────────╢\n")
location := data["location"].(map[string]any)
fmt.Printf("║ 位置: %s %s %s%*s ║\n",
location["country"], location["province"], location["city"],
37-len(fmt.Sprint(location["country"], location["province"], location["city"])), "")
fmt.Printf("║ 地址: %-43v ║\n", location["address"])
networkCfg := data["network_config"].(map[string]any)
fmt.Printf("║ 网络: SSID=%v, IP=%v%*s ║\n",
networkCfg["ssid"], networkCfg["ip_address"],
27-len(fmt.Sprint(networkCfg["ssid"], networkCfg["ip_address"])), "")
// 结构体切片展示
fmt.Printf("╟───────────────── 结构体切片 ──────────────────────╢\n")
sensors := data["sensors"].([]any)
fmt.Printf("║ 传感器数量: %d 个%39s ║\n", len(sensors), "")
for i, s := range sensors {
sensor := s.(map[string]any)
fmt.Printf("║ [%d] %s: %.1f %s (型号: %s)%*s ║\n",
i+1, sensor["type"], sensor["value"], sensor["unit"], sensor["model"],
20-len(fmt.Sprint(sensor["type"], sensor["model"])), "")
}
maintRecords := data["maintenance_records"].([]any)
fmt.Printf("║ 维护记录: %d 条%40s ║\n", len(maintRecords), "")
for i, m := range maintRecords {
maint := m.(map[string]any)
fmt.Printf("║ [%d] %s - %s (¥%.2f)%*s ║\n",
i+1, maint["date"], maint["type"], maint["cost"],
22-len(fmt.Sprint(maint["date"], maint["type"])), "")
}
// 基本类型切片
fmt.Printf("╟───────────────── 基本类型切片 ────────────────────╢\n")
tags := data["tags"].([]any)
fmt.Printf("║ 标签: %d 个 %v%*s ║\n",
len(tags), tags,
45-len(fmt.Sprint(tags)), "")
fmt.Println("╚═════════════════════════════════════════════════════════╝")
}
if count != 3 {
fmt.Printf("\n❌ 预期 3 条记录,实际 %d 条\n", count)
os.Exit(1)
}
// ========== 总结 ==========
fmt.Println("\n\n=============================================================")
fmt.Println(" ✅ 所有复杂类型测试通过!")
fmt.Println("=============================================================")
fmt.Println("\n📊 功能验证:")
fmt.Println(" ✓ Nullable 字段(指针类型)")
fmt.Println(" - 数据1: 9个Nullable字段全部有值")
fmt.Println(" - 数据2: 9个Nullable字段部分为nil")
fmt.Println(" - 数据3: 9个Nullable字段全部为nil")
fmt.Println("\n ✓ 嵌套结构体Object")
fmt.Println(" - Location: 6个字段的位置信息结构体")
fmt.Println(" - NetworkConfig: 6个字段的网络配置结构体")
fmt.Println("\n ✓ 结构体切片Array of Struct")
fmt.Println(" - Sensors: 传感器列表每个9个字段")
fmt.Println(" - MaintenanceRecords: 维护记录列表每个6个字段")
fmt.Println("\n ✓ 基本类型切片")
fmt.Println(" - []string: 标签列表")
fmt.Println(" - []int32: 告警代码列表")
fmt.Println(" - []float64: 历史读数")
fmt.Println("\n ✓ Map类型")
fmt.Println(" - metadata: 元数据信息")
fmt.Println(" - custom_settings: 自定义设置")
fmt.Println("\n💡 关键特性:")
fmt.Println(" • 指针类型自动识别为Nullable")
fmt.Println(" • 嵌套结构体自动转JSON")
fmt.Println(" • 结构体切片自动序列化")
fmt.Println(" • nil值正确处理和展示")
fmt.Println(" • 空切片和空map正确存储")
fmt.Printf("\n📁 数据已保存到: %s\n", absDir)
}

View File

@@ -7,6 +7,7 @@ import (
"math/big"
"net/http"
"slices"
"strings"
"time"
"code.tczkiot.com/wlw/srdb"
@@ -63,16 +64,21 @@ func StartWebUI(dbPath string, addr string) {
} else {
// 插入一些示例数据
users := []map[string]any{
{"name": "Alice", "email": "alice@example.com", "age": 30, "city": "Beijing"},
{"name": "Bob", "email": "bob@example.com", "age": 25, "city": "Shanghai"},
{"name": "Charlie", "email": "charlie@example.com", "age": 35, "city": "Guangzhou"},
{"name": "David", "email": "david@example.com", "age": 28, "city": "Shenzhen"},
{"name": "Eve", "email": "eve@example.com", "age": 32, "city": "Hangzhou"},
{"name": "Alice", "email": "alice@example.com", "age": int64(30), "city": "Beijing"},
{"name": "Bob", "email": "bob@example.com", "age": int64(25), "city": "Shanghai"},
{"name": "Charlie", "email": "charlie@example.com", "age": int64(35), "city": "Guangzhou"},
{"name": "David", "email": "david@example.com", "age": int64(28), "city": "Shenzhen"},
{"name": "Eve", "email": "eve@example.com", "age": int64(32), "city": "Hangzhou"},
}
insertedCount := 0
for _, user := range users {
table.Insert(user)
if err := table.Insert(user); err != nil {
log.Printf("Failed to insert user: %v, error: %v", user, err)
} else {
insertedCount++
}
}
log.Printf("Created users table with %d records", len(users))
log.Printf("Created users table with %d/%d records", insertedCount, len(users))
}
}
@@ -83,31 +89,42 @@ func StartWebUI(dbPath string, addr string) {
} else {
// 插入一些示例数据
products := []map[string]any{
{"product_name": "Laptop", "price": 999.99, "quantity": 10, "category": "Electronics"},
{"product_name": "Mouse", "price": 29.99, "quantity": 50, "category": "Electronics"},
{"product_name": "Keyboard", "price": 79.99, "quantity": 30, "category": "Electronics"},
{"product_name": "Monitor", "price": 299.99, "quantity": 15, "category": "Electronics"},
{"product_name": "Desk", "price": 199.99, "quantity": 5, "category": "Furniture"},
{"product_name": "Chair", "price": 149.99, "quantity": 8, "category": "Furniture"},
{"product_name": "Laptop", "price": 999.99, "quantity": int64(10), "category": "Electronics"},
{"product_name": "Mouse", "price": 29.99, "quantity": int64(50), "category": "Electronics"},
{"product_name": "Keyboard", "price": 79.99, "quantity": int64(30), "category": "Electronics"},
{"product_name": "Monitor", "price": 299.99, "quantity": int64(15), "category": "Electronics"},
{"product_name": "Desk", "price": 199.99, "quantity": int64(5), "category": "Furniture"},
{"product_name": "Chair", "price": 149.99, "quantity": int64(8), "category": "Furniture"},
}
insertedCount := 0
for _, product := range products {
table.Insert(product)
if err := table.Insert(product); err != nil {
log.Printf("Failed to insert product: %v, error: %v", product, err)
} else {
insertedCount++
}
}
log.Printf("Created products table with %d records", len(products))
log.Printf("Created products table with %d/%d records", insertedCount, len(products))
}
}
// 启动后台数据插入协程
go autoInsertData(db)
// 启动 Web UI
handler := webui.NewWebUI(db)
// 创建 WebUI,使用 /debug 作为 basePath
ui := webui.NewWebUI(db, "/debug")
fmt.Printf("SRDB Web UI is running at http://%s\n", addr)
// 创建主路由
mux := http.NewServeMux()
// 挂载 WebUI 到根路径WebUI 内部会处理 /debug 前缀)
mux.Handle("/", ui)
fmt.Printf("SRDB Web UI is running at: http://%s/debug/\n", strings.TrimPrefix(addr, ":"))
fmt.Println("Press Ctrl+C to stop")
fmt.Println("Background data insertion is running...")
if err := http.ListenAndServe(addr, handler); err != nil {
if err := http.ListenAndServe(addr, mux); err != nil {
log.Fatal(err)
}
}

View File

@@ -274,6 +274,46 @@ func (idx *SecondaryIndex) GetMetadata() IndexMetadata {
return idx.metadata
}
// ForEach 升序迭代所有索引条目
// callback 返回 false 时停止迭代,支持提前终止
// 注意只能迭代已持久化的数据B+Tree不包括内存中未持久化的数据
func (idx *SecondaryIndex) ForEach(callback IndexEntryCallback) error {
idx.mu.RLock()
defer idx.mu.RUnlock()
if !idx.ready {
return fmt.Errorf("index not ready")
}
// 只支持 B+Tree 格式的索引
if !idx.useBTree || idx.btreeReader == nil {
return fmt.Errorf("ForEach only supports B+Tree format indexes")
}
idx.btreeReader.ForEach(callback)
return nil
}
// ForEachDesc 降序迭代所有索引条目
// callback 返回 false 时停止迭代,支持提前终止
// 注意只能迭代已持久化的数据B+Tree不包括内存中未持久化的数据
func (idx *SecondaryIndex) ForEachDesc(callback IndexEntryCallback) error {
idx.mu.RLock()
defer idx.mu.RUnlock()
if !idx.ready {
return fmt.Errorf("index not ready")
}
// 只支持 B+Tree 格式的索引
if !idx.useBTree || idx.btreeReader == nil {
return fmt.Errorf("ForEachDesc only supports B+Tree format indexes")
}
idx.btreeReader.ForEachDesc(callback)
return nil
}
// NeedsUpdate 检查是否需要更新
func (idx *SecondaryIndex) NeedsUpdate(currentMaxSeq int64) bool {
idx.mu.RLock()

View File

@@ -530,6 +530,55 @@ func (r *IndexBTreeReader) GetMetadata() IndexMetadata {
}
}
// IndexEntryCallback 索引条目回调函数
// 参数value 字段值seqs 对应的 seq 列表
// 返回true 继续迭代false 停止迭代
type IndexEntryCallback func(value string, seqs []int64) bool
// ForEach 升序迭代所有索引条目
// callback 返回 false 时停止迭代,支持提前终止
func (r *IndexBTreeReader) ForEach(callback IndexEntryCallback) {
r.btree.ForEach(func(key int64, dataOffset int64, dataSize int32) bool {
// 读取数据块(零拷贝)
if dataOffset+int64(dataSize) > int64(len(r.mmap)) {
return false // 数据越界,停止迭代
}
binaryData := r.mmap[dataOffset : dataOffset+int64(dataSize)]
// 解码二进制数据
value, seqs, err := decodeIndexEntry(binaryData)
if err != nil {
return false // 解码失败,停止迭代
}
// 调用用户回调
return callback(value, seqs)
})
}
// ForEachDesc 降序迭代所有索引条目
// callback 返回 false 时停止迭代,支持提前终止
func (r *IndexBTreeReader) ForEachDesc(callback IndexEntryCallback) {
r.btree.ForEachDesc(func(key int64, dataOffset int64, dataSize int32) bool {
// 读取数据块(零拷贝)
if dataOffset+int64(dataSize) > int64(len(r.mmap)) {
return false // 数据越界,停止迭代
}
binaryData := r.mmap[dataOffset : dataOffset+int64(dataSize)]
// 解码二进制数据
value, seqs, err := decodeIndexEntry(binaryData)
if err != nil {
return false // 解码失败,停止迭代
}
// 调用用户回调
return callback(value, seqs)
})
}
// Close 关闭读取器
func (r *IndexBTreeReader) Close() error {
if r.mmap != nil {

338
query.go
View File

@@ -5,6 +5,8 @@ import (
"fmt"
"maps"
"reflect"
"slices"
"sort"
"strings"
)
@@ -357,9 +359,13 @@ func Or(exprs ...Expr) Expr {
}
type QueryBuilder struct {
conds []Expr
fields []string // 要选择的字段nil 表示选择所有字段
table *Table
conds []Expr
fields []string // 要选择的字段nil 表示选择所有字段
table *Table
orderBy string // 排序字段,仅支持 "_seq" 或索引字段
orderDesc bool // 是否降序排序
offset int // 跳过的记录数
limit int // 返回的最大记录数0 表示无限制
}
func newQueryBuilder(table *Table) *QueryBuilder {
@@ -470,12 +476,123 @@ func (qb *QueryBuilder) NotNull(field string) *QueryBuilder {
return qb.where(NotNull(field))
}
// OrderBy 设置排序字段(升序)
// 仅支持 "_seq" 或有索引的字段,使用其他字段会返回错误
func (qb *QueryBuilder) OrderBy(field string) *QueryBuilder {
qb.orderBy = field
qb.orderDesc = false
return qb
}
// OrderByDesc 设置排序字段(降序)
// 仅支持 "_seq" 或有索引的字段,使用其他字段会返回错误
func (qb *QueryBuilder) OrderByDesc(field string) *QueryBuilder {
qb.orderBy = field
qb.orderDesc = true
return qb
}
// Offset 设置跳过的记录数
// 用于分页查询,跳过前 n 条记录
func (qb *QueryBuilder) Offset(n int) *QueryBuilder {
if n < 0 {
n = 0
}
qb.offset = n
return qb
}
// Limit 设置返回的最大记录数
// 用于分页查询,最多返回 n 条记录
// n = 0 表示无限制
func (qb *QueryBuilder) Limit(n int) *QueryBuilder {
if n < 0 {
n = 0
}
qb.limit = n
return qb
}
// Paginate 执行分页查询并返回结果、总记录数和错误
// page: 页码,从 1 开始
// pageSize: 每页记录数
// 返回值:
// - rows: 当前页的数据
// - total: 满足条件的总记录数(用于计算总页数)
// - err: 错误信息
//
// 注意:此方法会执行两次查询,第一次获取总数,第二次获取分页数据
func (qb *QueryBuilder) Paginate(page, pageSize int) (rows *Rows, total int, err error) {
if page < 1 {
page = 1
}
if pageSize < 0 {
pageSize = 0
}
// 1. 先获取总记录数(不应用分页)
// 创建一个新的 QueryBuilder 副本用于计数
countQb := &QueryBuilder{
conds: qb.conds,
fields: qb.fields,
table: qb.table,
orderBy: "", // 计数不需要排序
offset: 0, // 计数不应用分页
limit: 0,
}
countRows, err := countQb.Rows()
if err != nil {
return nil, 0, err
}
defer countRows.Close()
// 计算总数
total = countRows.Len()
// 2. 执行分页查询
qb.offset = (page - 1) * pageSize
qb.limit = pageSize
rows, err = qb.Rows()
if err != nil {
return nil, total, err
}
return rows, total, nil
}
// validateOrderBy 验证排序字段是否有效
func (qb *QueryBuilder) validateOrderBy() error {
if qb.orderBy == "" {
return nil // 没有设置排序,无需验证
}
// 允许使用 _seq
if qb.orderBy == "_seq" {
return nil
}
// 检查该字段是否有索引
if _, exists := qb.table.indexManager.GetIndex(qb.orderBy); exists {
return nil
}
// 不支持的字段
return fmt.Errorf("OrderBy only supports '_seq' or indexed fields, field '%s' is not indexed", qb.orderBy)
}
// Rows 返回所有匹配的数据(游标模式 - 惰性加载)
func (qb *QueryBuilder) Rows() (*Rows, error) {
if qb.table == nil {
return nil, fmt.Errorf("table is nil")
}
// 验证排序字段
if err := qb.validateOrderBy(); err != nil {
return nil, err
}
rows := &Rows{
schema: qb.table.schema,
fields: qb.fields,
@@ -484,6 +601,11 @@ func (qb *QueryBuilder) Rows() (*Rows, error) {
visited: make(map[int64]bool),
}
// 如果设置了排序,使用排序后的结果集
if qb.orderBy != "" {
return qb.rowsWithOrder(rows)
}
// 尝试使用索引优化查询
// 检查是否有可以使用索引的 Eq 条件
indexField, indexValue := qb.findIndexableCondition()
@@ -570,6 +692,9 @@ func (qb *QueryBuilder) rowsWithIndex(rows *Rows, indexField string, indexValue
}
}
// 应用 offset 和 limit
rows.cachedRows = qb.applyOffsetLimit(rows.cachedRows)
// 使用缓存模式
rows.cached = true
rows.cachedIndex = -1
@@ -577,6 +702,194 @@ func (qb *QueryBuilder) rowsWithIndex(rows *Rows, indexField string, indexValue
return rows, nil
}
// rowsWithOrder 使用排序返回数据
func (qb *QueryBuilder) rowsWithOrder(rows *Rows) (*Rows, error) {
if qb.orderBy == "_seq" {
// 按 _seq 排序
return qb.rowsOrderBySeq(rows)
}
// 按索引字段排序
return qb.rowsOrderByIndex(rows, qb.orderBy)
}
// rowsOrderBySeq 按 _seq 排序返回数据
func (qb *QueryBuilder) rowsOrderBySeq(rows *Rows) (*Rows, error) {
// 收集所有 seq从所有数据源
seqList := []int64{}
// 1. 从 Active MemTable 收集
activeMemTable := qb.table.memtableManager.GetActive()
if activeMemTable != nil {
seqList = append(seqList, activeMemTable.Keys()...)
}
// 2. 从 Immutable MemTables 收集
immutables := qb.table.memtableManager.GetImmutables()
for _, immutable := range immutables {
seqList = append(seqList, immutable.MemTable.Keys()...)
}
// 3. 从 SST 文件收集
sstReaders := qb.table.sstManager.GetReaders()
for _, reader := range sstReaders {
seqList = append(seqList, reader.GetAllKeys()...)
}
// 去重(使用 map
seqMap := make(map[int64]bool)
uniqueSeqs := []int64{}
for _, seq := range seqList {
if !seqMap[seq] {
seqMap[seq] = true
uniqueSeqs = append(uniqueSeqs, seq)
}
}
// 排序
if qb.orderDesc {
// 降序
sort.Slice(uniqueSeqs, func(i, j int) bool {
return uniqueSeqs[i] > uniqueSeqs[j]
})
} else {
// 升序
slices.Sort(uniqueSeqs)
}
// 按排序后的 seq 获取数据
rows.cachedRows = make([]*SSTableRow, 0, len(uniqueSeqs))
for _, seq := range uniqueSeqs {
row, err := qb.table.Get(seq)
if err != nil {
continue // 跳过获取失败的记录
}
// 检查是否匹配过滤条件
if qb.Match(row.Data) {
rows.cachedRows = append(rows.cachedRows, row)
}
}
// 应用 offset 和 limit
rows.cachedRows = qb.applyOffsetLimit(rows.cachedRows)
// 使用缓存模式
rows.cached = true
rows.cachedIndex = -1
return rows, nil
}
// rowsOrderByIndex 按索引字段排序返回数据
//
// 实现策略:
// 1. 使用 ForEach/ForEachDesc 从索引收集所有 (value, seqs) 对
// 2. 按字段值(而非哈希)对这些对进行排序
// 3. 按排序后的顺序获取数据
//
// 注意:虽然使用了索引,但需要在内存中排序所有索引条目。
// 对于大量唯一值的字段,内存开销可能较大。
func (qb *QueryBuilder) rowsOrderByIndex(rows *Rows, indexField string) (*Rows, error) {
// 获取索引
idx, exists := qb.table.indexManager.GetIndex(indexField)
if !exists {
return nil, fmt.Errorf("index on field %s not found", indexField)
}
// 检查索引是否准备就绪
if !idx.IsReady() {
return nil, fmt.Errorf("index on field %s is not ready", indexField)
}
// 用于收集索引条目的结构
type indexEntry struct {
value string
seqs []int64
}
// 收集所有索引条目
entries := []indexEntry{}
err := idx.ForEach(func(value string, seqs []int64) bool {
// 复制 seqs 避免引用问题
seqsCopy := make([]int64, len(seqs))
copy(seqsCopy, seqs)
entries = append(entries, indexEntry{
value: value,
seqs: seqsCopy,
})
return true
})
if err != nil {
return nil, fmt.Errorf("failed to iterate index: %w", err)
}
// 按字段值排序(而非哈希)
if qb.orderDesc {
// 降序
sort.Slice(entries, func(i, j int) bool {
return entries[i].value > entries[j].value
})
} else {
// 升序
sort.Slice(entries, func(i, j int) bool {
return entries[i].value < entries[j].value
})
}
// 按排序后的顺序收集所有 seq
allSeqs := []int64{}
for _, entry := range entries {
allSeqs = append(allSeqs, entry.seqs...)
}
// 根据 seq 列表获取数据
rows.cachedRows = make([]*SSTableRow, 0, len(allSeqs))
for _, seq := range allSeqs {
row, err := qb.table.Get(seq)
if err != nil {
continue // 跳过获取失败的记录
}
// 检查是否匹配所有其他条件
if qb.Match(row.Data) {
rows.cachedRows = append(rows.cachedRows, row)
}
}
// 应用 offset 和 limit
rows.cachedRows = qb.applyOffsetLimit(rows.cachedRows)
// 使用缓存模式
rows.cached = true
rows.cachedIndex = -1
return rows, nil
}
// applyOffsetLimit 应用 offset 和 limit 到结果集
func (qb *QueryBuilder) applyOffsetLimit(rows []*SSTableRow) []*SSTableRow {
// 如果没有设置 offset 和 limit直接返回
if qb.offset == 0 && qb.limit == 0 {
return rows
}
// 应用 offset
if qb.offset > 0 {
if qb.offset >= len(rows) {
return []*SSTableRow{}
}
rows = rows[qb.offset:]
}
// 应用 limit
if qb.limit > 0 && qb.limit < len(rows) {
rows = rows[:qb.limit]
}
return rows
}
// First 返回第一个匹配的数据
func (qb *QueryBuilder) First() (*Row, error) {
rows, err := qb.Rows()
@@ -697,6 +1010,10 @@ type Rows struct {
cached bool
cachedRows []*SSTableRow
cachedIndex int // 缓存模式下的迭代位置
// 分页状态(惰性模式)
skippedCount int // 已跳过的记录数(用于 offset
returnedCount int // 已返回的记录数(用于 limit
}
// memtableIterator 包装 MemTable 的迭代器
@@ -843,8 +1160,21 @@ func (r *Rows) next() bool {
continue
}
// 应用 offset跳过前 N 条记录
if r.qb.offset > 0 && r.skippedCount < r.qb.offset {
r.skippedCount++
r.visited[minSeq] = true
continue
}
// 应用 limit达到返回上限后停止
if r.qb.limit > 0 && r.returnedCount >= r.qb.limit {
return false
}
// 找到匹配的记录
r.visited[minSeq] = true
r.returnedCount++
r.currentRow = &Row{schema: r.schema, fields: r.fields, inner: row}
return true
}
@@ -927,7 +1257,7 @@ func (r *Rows) Data() []map[string]any {
// - 如果目标是结构体/指针:只扫描第一行
func (r *Rows) Scan(value any) error {
rv := reflect.ValueOf(value)
if rv.Kind() != reflect.Ptr {
if rv.Kind() != reflect.Pointer {
return fmt.Errorf("scan target must be a pointer")
}

View File

@@ -770,6 +770,13 @@ func readFieldBinaryValue(buf *bytes.Reader, typ FieldType, keep bool) (any, err
if err := binary.Read(buf, binary.LittleEndian, &length); err != nil {
return nil, err
}
if length == 0 {
// 空字符串,直接返回
if keep {
return "", nil
}
return nil, nil
}
str := make([]byte, length)
if _, err := buf.Read(str); err != nil {
return nil, err
@@ -796,6 +803,13 @@ func readFieldBinaryValue(buf *bytes.Reader, typ FieldType, keep bool) (any, err
if err := binary.Read(buf, binary.LittleEndian, &length); err != nil {
return nil, err
}
if length == 0 {
// 零值 Decimal
if keep {
return decimal.Zero, nil
}
return nil, nil
}
data := make([]byte, length)
if _, err := buf.Read(data); err != nil {
return nil, err
@@ -839,6 +853,13 @@ func readFieldBinaryValue(buf *bytes.Reader, typ FieldType, keep bool) (any, err
if err := binary.Read(buf, binary.LittleEndian, &length); err != nil {
return nil, err
}
if length == 0 {
// 空对象
if keep {
return map[string]any{}, nil
}
return nil, nil
}
data := make([]byte, length)
if _, err := buf.Read(data); err != nil {
return nil, err
@@ -858,6 +879,13 @@ func readFieldBinaryValue(buf *bytes.Reader, typ FieldType, keep bool) (any, err
if err := binary.Read(buf, binary.LittleEndian, &length); err != nil {
return nil, err
}
if length == 0 {
// 空数组
if keep {
return []any{}, nil
}
return nil, nil
}
data := make([]byte, length)
if _, err := buf.Read(data); err != nil {
return nil, err
@@ -1140,6 +1168,18 @@ func (r *SSTableReader) GetAllKeys() []int64 {
return r.btReader.GetAllKeys()
}
// ForEach 升序迭代所有 key-offset-size 对
// callback 返回 false 时停止迭代,支持提前终止
func (r *SSTableReader) ForEach(callback KeyCallback) {
r.btReader.ForEach(callback)
}
// ForEachDesc 降序迭代所有 key-offset-size 对
// callback 返回 false 时停止迭代,支持提前终止
func (r *SSTableReader) ForEachDesc(callback KeyCallback) {
r.btReader.ForEachDesc(callback)
}
// Close 关闭读取器
func (r *SSTableReader) Close() error {
if r.mmap != nil {

115
table.go
View File

@@ -3,6 +3,8 @@ package srdb
import (
"encoding/json"
"fmt"
"io"
"log/slog"
"os"
"path/filepath"
"reflect"
@@ -28,6 +30,7 @@ type Table struct {
memtableManager *MemTableManager // MemTable 管理器
versionSet *VersionSet // MANIFEST 管理器
compactionManager *CompactionManager // Compaction 管理器
logger *slog.Logger // 日志器
seq atomic.Int64
flushMu sync.Mutex
@@ -142,7 +145,11 @@ func OpenTable(opts *TableOptions) (*Table, error) {
err := indexMgr.CreateIndex(field.Name)
if err != nil {
// 索引创建失败,记录警告但不阻塞表创建
fmt.Fprintf(os.Stderr, "[WARNING] Failed to create index for field %s: %v\n", field.Name, err)
// 此时使用临时 loggerTable 还未完全创建)
tmpLogger := slog.New(slog.NewTextHandler(os.Stderr, nil))
tmpLogger.Warn("[Table] Failed to create index for field",
"field", field.Name,
"error", err)
}
}
}
@@ -176,6 +183,7 @@ func OpenTable(opts *TableOptions) (*Table, error) {
sstManager: sstMgr,
memtableManager: memMgr,
versionSet: versionSet,
logger: slog.New(slog.NewTextHandler(io.Discard, nil)), // 默认丢弃日志
}
// 先恢复数据(包括从 WAL 恢复)
@@ -260,12 +268,15 @@ func (t *Table) normalizeInsertData(data any) ([]map[string]any, error) {
typ = val.Type()
}
// 获取解引用后的实际值
actualData := val.Interface()
switch typ.Kind() {
case reflect.Map:
// map[string]any - 单条
m, ok := data.(map[string]any)
m, ok := actualData.(map[string]any)
if !ok {
return nil, fmt.Errorf("expected map[string]any, got %T", data)
return nil, fmt.Errorf("expected map[string]any, got %T", actualData)
}
return []map[string]any{m}, nil
@@ -275,9 +286,9 @@ func (t *Table) normalizeInsertData(data any) ([]map[string]any, error) {
// []map[string]any
if elemType.Kind() == reflect.Map {
maps, ok := data.([]map[string]any)
maps, ok := actualData.([]map[string]any)
if !ok {
return nil, fmt.Errorf("expected []map[string]any, got %T", data)
return nil, fmt.Errorf("expected []map[string]any, got %T", actualData)
}
return maps, nil
}
@@ -313,14 +324,14 @@ func (t *Table) normalizeInsertData(data any) ([]map[string]any, error) {
case reflect.Struct:
// struct{} - 单个结构体
m, err := t.structToMap(data)
m, err := t.structToMap(actualData)
if err != nil {
return nil, err
}
return []map[string]any{m}, nil
default:
return nil, fmt.Errorf("unsupported data type: %T (kind: %s)", data, typ.Kind())
return nil, fmt.Errorf("unsupported data type: %T (kind: %s)", actualData, typ.Kind())
}
}
@@ -348,32 +359,47 @@ func (t *Table) structToMap(v any) (map[string]any, error) {
continue
}
// 获取字段名
fieldName := field.Name
// 解析 srdb tag
tag := field.Tag.Get("srdb")
// 跳过忽略的字段
if tag == "-" {
// 忽略该字段
continue
}
// 解析 tag 获取字段名
// 默认使用 snake_case 转换字段名
fieldName := camelToSnake(field.Name)
// 解析 tag与 StructToFields 保持一致)
if tag != "" {
parts := strings.Split(tag, ";")
if parts[0] != "" {
fieldName = parts[0]
} else {
// 使用 snake_case 转换
fieldName = camelToSnake(field.Name)
for _, part := range parts {
part = strings.TrimSpace(part)
if part == "" {
continue
}
// 检查是否为 field:xxx 格式
if strings.HasPrefix(part, "field:") {
fieldName = strings.TrimPrefix(part, "field:")
break // 找到字段名,停止解析
}
// 忽略其他标记indexed, nullable, comment:xxx
}
} else {
// 没有 tag使用 snake_case 转换
fieldName = camelToSnake(field.Name)
}
// 获取字段值
fieldVal := val.Field(i)
result[fieldName] = fieldVal.Interface()
// 处理指针类型如果是指针解引用nil 保持为 nil
if fieldVal.Kind() == reflect.Pointer {
if fieldVal.IsNil() {
result[fieldName] = nil
} else {
result[fieldName] = fieldVal.Elem().Interface()
}
} else {
result[fieldName] = fieldVal.Interface()
}
}
return result, nil
@@ -402,14 +428,40 @@ func (t *Table) insertSingle(data map[string]any) error {
return NewError(ErrCodeSchemaValidationFailed, err)
}
// 2. 生成 _seq
// 2. 类型转换:将数据转换为 Schema 定义的类型
// 这样可以确保写入时的类型与 Schema 一致(例如将 int64 转换为 time.Time
convertedData := make(map[string]any, len(data))
for key, value := range data {
// 跳过 nil 值
if value == nil {
convertedData[key] = nil
continue
}
// 获取字段定义
field, err := t.schema.GetField(key)
if err != nil {
// 字段不在 Schema 中,保持原值
convertedData[key] = value
continue
}
// 使用 Schema 的类型转换
converted, err := convertValue(value, field.Type)
if err != nil {
return NewErrorf(ErrCodeSchemaValidationFailed, "convert field %s: %v", key, err)
}
convertedData[key] = converted
}
// 3. 生成 _seq
seq := t.seq.Add(1)
// 3. 添加系统字段
// 4. 添加系统字段
row := &SSTableRow{
Seq: seq,
Time: time.Now().UnixNano(),
Data: data,
Data: convertedData,
}
// 3. 序列化(使用二进制格式,保留类型信息)
@@ -446,6 +498,11 @@ func (t *Table) insertSingle(data map[string]any) error {
return nil
}
// SetLogger 设置 logger由 Database 调用)
func (t *Table) SetLogger(logger *slog.Logger) {
t.logger = logger
}
// Get 查询数据
func (t *Table) Get(seq int64) (*SSTableRow, error) {
// 1. 先查 MemTable Manager (Active + Immutables)
@@ -939,6 +996,16 @@ func (t *Table) ListIndexes() []string {
return t.indexManager.ListIndexes()
}
// GetIndex 获取指定字段的索引
func (t *Table) GetIndex(field string) (*SecondaryIndex, bool) {
return t.indexManager.GetIndex(field)
}
// BuildIndexes 构建所有索引
func (t *Table) BuildIndexes() error {
return t.indexManager.BuildAll()
}
// GetIndexMetadata 获取索引元数据
func (t *Table) GetIndexMetadata() map[string]IndexMetadata {
return t.indexManager.GetIndexMetadata()

346
webui/README.md Normal file
View File

@@ -0,0 +1,346 @@
# SRDB WebUI
基于 Preact.js 的 SRDB WebUI使用无构建工具工作流No Build Workflow
## 技术栈
- **Preact 10.19.3** - 轻量级 React 替代品3KB
- **HTM 3.1.1** - JSX 替代方案,直接在浏览器中使用
- **dom-align** - 智能元素定位(用于 Popover 和 Tooltip
- **ESM.sh** - ES Modules CDN
## 特性
**零构建工具** - 无需 Webpack、Vite 等构建工具
**原生 ES Modules** - 直接使用 `import` 语法
**HTM 语法** - 使用 `html` 标签模板字符串代替 JSX
**主题切换** - 支持深色/浅色主题,持久化保存
**智能 Popover** - 鼠标悬停单元格显示完整数据
**字段 Tooltip** - 悬停表头显示字段注释,悬停表名显示表注释
**响应式设计** - 适配不同屏幕尺寸
**数据分页** - 支持自定义每页显示行数20/50/100/200+ 跳页功能
**列选择器** - 下拉菜单选择要显示的列,支持持久化保存
**详情模态框** - 点击眼睛图标查看单条记录的完整数据
**Manifest 弹窗** - 点击按钮打开弹窗,实时显示 LSM-Tree 结构和 Compaction 统计
**索引标识** - 有索引的字段显示 🔍 图标
**悬浮加载提示** - 加载时顶部显示悬浮提示,不影响布局
## 目录结构
```
webui/
├── static/
│ ├── index.html # 主 HTML 文件
│ ├── css/
│ │ └── styles.css # 全局样式(支持主题)
│ │
│ └── js/
│ ├── main.js # 应用入口
│ ├── hooks/ # 自定义 Hooks
│ │ ├── useCellPopover.js # 单元格 Popover Hook
│ │ └── useTooltip.js # Tooltip Hook
│ │
│ └── components/ # Preact 组件
│ ├── App.js # 主应用组件
│ ├── Sidebar.js # 侧边栏(表列表)
│ ├── TableItem.js # 表列表项(可展开字段)
│ ├── FieldList.js # 字段列表
│ ├── TableView.js # 表视图(主容器)
│ ├── ColumnSelector.js # 列选择器(下拉菜单)
│ ├── DataTable.js # 数据表格
│ ├── TableRow.js # 表格行
│ ├── TableCell.js # 表格单元格
│ ├── Pagination.js # 分页器sticky 底部)
│ ├── PageJumper.js # 跳页输入框
│ ├── RowDetailModal.js # 记录详情模态框
│ ├── ManifestModal.js # Manifest 弹窗
│ ├── ManifestView.js # Manifest 视图
│ ├── LevelCard.js # 层级卡片
│ ├── FileCard.js # 文件卡片
│ ├── StatCard.js # 统计卡片
│ └── CompactionStats.js # Compaction 统计
├── webui.go # Go 后端处理器
└── README.md # 本文件
```
## 快速开始
### 1. 启动后端服务
```bash
cd /path/to/srdb/examples/webui
go run main.go serve --db ./data --addr :8080
```
### 2. 访问 WebUI
浏览器打开:`http://localhost:8080/`
## 使用技巧
1. **查看字段说明**:鼠标悬停在表头字段名上,会显示该字段的 comment
2. **查看表说明**:鼠标悬停在表名上,会显示该表的 comment
3. **选择显示列**:点击右上角的 "Columns" 按钮,勾选要显示的列,选择会自动保存
4. **查看完整数据**:鼠标悬停在表格单元格上,会显示该单元格的完整数据
5. **查看记录详情**:鼠标悬停在行上,点击出现的眼睛图标查看完整记录
6. **跳转到指定页**:在分页器输入页码,点击"跳转"按钮
7. **查看 LSM-Tree 结构**:点击右上角的 "📊 Manifest" 按钮打开弹窗
8. **展开表字段**:在侧边栏点击表名前的 ▶ 图标展开查看所有字段
9. **识别索引字段**:带有 🔍 图标的字段表示该字段已建立索引
10. **主题切换**:点击左上角的 ☀️/🌙 图标切换深色/浅色主题
## 核心组件说明
### App.js - 主应用
负责:
- 主题管理(深色/浅色切换LocalStorage 持久化)
- 表列表加载和选择
- 侧边栏 + 主内容区布局
### Sidebar.js - 侧边栏
特性:
- 可展开的表列表(点击 ▶ 展开字段)
- 表统计信息(字段数)
- 字段列表显示(字段名、类型、索引图标)
- 选中状态高亮
### TableView.js - 表视图容器
包含:
- 表名标题(悬停显示表注释)
- 行数统计(格式化显示 K/M
- Manifest 按钮(打开弹窗)
- 列选择器按钮
- DataTable 组件
- ManifestModal 组件
### ColumnSelector.js - 列选择器
特性:
- 下拉菜单式选择器
- 多选复选框
- 显示字段类型和索引图标
- LocalStorage 持久化保存
- 选中数量徽章显示
### DataTable.js - 数据表格
功能:
- 响应式表格渲染
- 表头 Tooltip显示字段注释
- 单元格 Popover悬停显示完整数据
- 时间格式化(`_time` 字段)
- 悬浮加载提示(顶部居中)
- Sticky 分页器(粘在底部)
### TableRow.js - 表格行
特性:
- 悬停高亮效果
- 操作列眼睛图标(仅悬停时显示,使用 opacity 动画)
- 点击查看详情
### Pagination.js - 分页组件
功能:
- 上一页/下一页导航
- 每页行数选择20/50/100/200
- 跳页功能PageJumper
- Sticky 定位在底部
- 显示当前页范围和总数
### RowDetailModal.js - 记录详情
特性:
- 全屏模态框
- 显示单条记录的所有字段
- 系统字段标记(`_seq`, `_time`
- JSON 格式化显示
- ESC 键和点击遮罩层关闭
### ManifestModal.js - Manifest 弹窗
特性:
- 90vw 宽度弹窗(最大 1200px
- 包装 ManifestView 组件
- ESC 键和点击外部关闭
- 阻止背景滚动
### ManifestView.js - Manifest 视图
展示:
- 统计卡片(总文件数、总大小、下一个文件号、最后序列号)
- 各层级详情L0-L6+
- 层级标识(彩色徽章)
- 文件数和总大小
- Compaction Score带进度条
- 文件列表文件号、大小、行数、Key 范围)
- Compaction 统计(合并次数、读写字节等)
- 只显示有文件的层级
## 自定义 Hooks
### useCellPopover.js
用于单元格数据预览的 Popover
- 使用 dom-align 智能定位
- 自动处理溢出和边界
- 延迟隐藏避免闪烁
- 支持 JSON 格式化显示
### useTooltip.js
用于表头和表名的 Tooltip
- 显示字段 comment 或表 comment
- 使用 dom-align 定位在元素下方
- 无延迟即时显示
- 最大宽度 300px自动换行
## Preact + HTM 语法示例
### 基本组件
```javascript
import { html } from 'htm/preact';
export function MyComponent({ name }) {
return html`
<div class="hello">
<h1>Hello, ${name}!</h1>
</div>
`;
}
```
### 使用 Hooks
```javascript
import { html } from 'htm/preact';
import { useState, useEffect } from 'preact/hooks';
export function Counter() {
const [count, setCount] = useState(0);
useEffect(() => {
console.log('Count changed:', count);
}, [count]);
return html`
<div>
<p>Count: ${count}</p>
<button onClick=${() => setCount(count + 1)}>
Increment
</button>
</div>
`;
}
```
### 条件渲染
```javascript
${loading
? html`<div class="loading">Loading...</div>`
: html`<div class="content">${data}</div>`
}
```
### 列表渲染
```javascript
${items.map(item => html`
<div key=${item.id}>${item.name}</div>
`)}
```
## 主题切换
主题通过修改 `<html>` 元素的 `data-theme` 属性实现:
```javascript
// 切换到浅色主题
document.documentElement.setAttribute('data-theme', 'light');
// 切换回深色主题
document.documentElement.removeAttribute('data-theme');
```
CSS 变量会根据主题自动切换:
```css
:root {
--bg-main: #0f0f1a; /* 深色主题 */
}
:root[data-theme="light"] {
--bg-main: #f5f5f7; /* 浅色主题 */
}
```
## API 接口
WebUI v2 使用的 API 端点:
- `GET /api/tables` - 获取表列表(含行数统计)
- `GET /api/tables/:name/schema` - 获取表 Schema
- `GET /api/tables/:name/data?limit=100&offset=0` - 获取表数据(支持分页)
- `GET /api/tables/:name/data/:seq` - 获取单条记录详情(完整数据)
- `GET /api/tables/:name/manifest` - 获取 Manifest 信息LSM-Tree 结构)
## 开发建议
### 添加新组件
1.`static/js/components/` 创建新文件
2. 使用 `html` 标签模板语法
3. 导出函数组件
```javascript
import { html } from 'htm/preact';
export function NewComponent({ prop1, prop2 }) {
return html`
<div>Your content here</div>
`;
}
```
### 内联样式 vs CSS
推荐使用内联样式(通过 `styles` 对象)以获得更好的组件隔离性和主题支持:
```javascript
const styles = {
container: {
padding: '20px',
background: 'var(--bg-elevated)',
color: 'var(--text-primary)'
}
};
return html`<div style=${styles.container}>Content</div>`;
```
### 状态管理
使用 `useState``useEffect` 管理组件状态:
```javascript
import { useState, useEffect } from 'preact/hooks';
// 本地状态
const [count, setCount] = useState(0);
// 副作用处理
useEffect(() => {
// 组件挂载时执行
console.log('Component mounted');
// 清理函数
return () => {
console.log('Component unmounted');
};
}, []); // 空依赖数组表示只在挂载/卸载时执行
```
## 参考资料
- [Preact 官方文档](https://preactjs.com/)
- [HTM 文档](https://github.com/developit/htm)
- [dom-align 文档](https://github.com/yiminghe/dom-align)
- [ESM.sh CDN](https://esm.sh/)
## License
MIT

View File

@@ -1,4 +1,4 @@
/* SRDB WebUI - Modern Design with Lit */
/* SRDB WebUI v2 - Preact Edition */
:root {
/* 主色调 - 优雅的紫蓝色 */
@@ -7,18 +7,18 @@
--primary-light: #818cf8;
--primary-bg: rgba(99, 102, 241, 0.1);
/* 背景色 */
/* 背景色 - 深色主题(默认)*/
--bg-main: #0f0f1a;
--bg-surface: #1a1a2e;
--bg-elevated: #222236;
--bg-hover: #2a2a3e;
/* 文字颜色 */
/* 文字颜色 - 深色主题 */
--text-primary: #ffffff;
--text-secondary: #a0a0b0;
--text-tertiary: #6b6b7b;
/* 边框和分隔线 */
/* 边框和分隔线 - 深色主题 */
--border-color: rgba(255, 255, 255, 0.1);
--border-hover: rgba(255, 255, 255, 0.2);
@@ -30,12 +30,9 @@
/* 阴影 */
--shadow-sm: 0 1px 2px 0 rgba(0, 0, 0, 0.3);
--shadow-md:
0 4px 6px -1px rgba(0, 0, 0, 0.4), 0 2px 4px -1px rgba(0, 0, 0, 0.3);
--shadow-lg:
0 10px 15px -3px rgba(0, 0, 0, 0.5), 0 4px 6px -2px rgba(0, 0, 0, 0.3);
--shadow-xl:
0 20px 25px -5px rgba(0, 0, 0, 0.5), 0 10px 10px -5px rgba(0, 0, 0, 0.3);
--shadow-md: 0 4px 6px -1px rgba(0, 0, 0, 0.4), 0 2px 4px -1px rgba(0, 0, 0, 0.3);
--shadow-lg: 0 10px 15px -3px rgba(0, 0, 0, 0.5), 0 4px 6px -2px rgba(0, 0, 0, 0.3);
--shadow-xl: 0 20px 25px -5px rgba(0, 0, 0, 0.5), 0 10px 10px -5px rgba(0, 0, 0, 0.3);
/* 圆角 */
--radius-sm: 6px;
@@ -47,25 +44,38 @@
--transition: all 0.2s cubic-bezier(0.4, 0, 0.2, 1);
}
/* 浅色主题 */
:root[data-theme="light"] {
/* 背景色 - 浅色主题 */
--bg-main: #f5f5f7;
--bg-surface: #ffffff;
--bg-elevated: #ffffff;
--bg-hover: #f0f0f2;
/* 文字颜色 - 浅色主题 */
--text-primary: #1d1d1f;
--text-secondary: #6e6e73;
--text-tertiary: #86868b;
/* 边框和分隔线 - 浅色主题 */
--border-color: rgba(0, 0, 0, 0.1);
--border-hover: rgba(0, 0, 0, 0.2);
/* 阴影 - 浅色主题 */
--shadow-sm: 0 1px 2px 0 rgba(0, 0, 0, 0.05);
--shadow-md: 0 4px 6px -1px rgba(0, 0, 0, 0.1), 0 2px 4px -1px rgba(0, 0, 0, 0.06);
--shadow-lg: 0 10px 15px -3px rgba(0, 0, 0, 0.1), 0 4px 6px -2px rgba(0, 0, 0, 0.05);
--shadow-xl: 0 20px 25px -5px rgba(0, 0, 0, 0.1), 0 10px 10px -5px rgba(0, 0, 0, 0.04);
}
* {
box-sizing: border-box;
margin: 0;
padding: 0;
}
:root {
/* 主题过渡动画 */
transition: background-color 0.3s ease, color 0.3s ease;
}
body {
font-family:
"Inter",
-apple-system,
BlinkMacSystemFont,
"Segoe UI",
Roboto,
sans-serif;
font-family: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
background: var(--bg-main);
color: var(--text-primary);
line-height: 1.6;
@@ -78,3 +88,44 @@ body {
overflow: hidden;
transition: background-color 0.3s ease, color 0.3s ease;
}
#app {
height: 100vh;
display: flex;
flex-direction: column;
}
/* 自定义滚动条 */
*::-webkit-scrollbar {
width: 8px;
height: 8px;
}
*::-webkit-scrollbar-track {
background: transparent;
}
*::-webkit-scrollbar-thumb {
background: var(--border-color);
border-radius: 4px;
}
*::-webkit-scrollbar-thumb:hover {
background: var(--border-hover);
}
/* Loading 状态 */
.loading {
display: flex;
align-items: center;
justify-content: center;
height: 100%;
color: var(--text-secondary);
}
/* Empty 状态 */
.empty {
text-align: center;
padding: 60px 20px;
color: var(--text-secondary);
}

View File

@@ -1,36 +1,34 @@
<!doctype html>
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>SRDB Web UI</title>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&display=swap"
rel="stylesheet"
/>
<link rel="stylesheet" href="/static/css/styles.css" />
<!-- Import Map for Lit and local modules -->
<script type="importmap">
{
"imports": {
"lit": "https://cdn.jsdelivr.net/gh/lit/dist@3/core/lit-core.min.js",
"lit/": "https://cdn.jsdelivr.net/gh/lit/dist@3/",
"~/": "/static/js/"
}
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>SRDB WebUI v2 - Preact</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&display=swap" rel="stylesheet">
<link rel="stylesheet" href="~~/static/css/styles.css">
<!-- 路径配置:服务端渲染时替换 ~~ 和 ~/: 为实际的 BasePath -->
<script>window.API_BASE = "~~";</script>
<!-- Import Map for Preact and HTM -->
<script type="importmap">
{
"imports": {
"preact": "https://esm.sh/preact@10.19.3",
"preact/": "https://esm.sh/preact@10.19.3/",
"htm/preact": "https://esm.sh/htm@3.1.1/preact?external=preact",
"dom-align": "https://cdn.jsdelivr.net/npm/dom-align@1.12.4/+esm",
"~/": "~~/static/js/"
}
</script>
</head>
<body>
<!-- 应用容器 -->
<srdb-app></srdb-app>
}
</script>
</head>
<body>
<div id="app"></div>
<!-- Modal -->
<srdb-modal-dialog></srdb-modal-dialog>
<!-- 加载 Lit 组件 -->
<script type="module" src="/static/js/main.js"></script>
</body>
<!-- Load Preact App -->
<script type="module" src="~~/static/js/main.js"></script>
</body>
</html>

View File

@@ -1,174 +0,0 @@
/**
* API 请求管理模块
* 统一管理所有后端接口请求
*/
const API_BASE = '/api';
/**
* 通用请求处理函数
* @param {string} url - 请求 URL
* @param {RequestInit} options - fetch 选项
* @returns {Promise<any>}
*/
async function request(url, options = {}) {
try {
const response = await fetch(url, {
headers: {
'Content-Type': 'application/json',
...options.headers,
},
...options,
});
if (!response.ok) {
const error = new Error(`HTTP ${response.status}: ${response.statusText}`);
error.status = response.status;
error.response = response;
throw error;
}
return await response.json();
} catch (error) {
console.error('API request failed:', url, error);
throw error;
}
}
/**
* 表相关 API
*/
export const tableAPI = {
/**
* 获取所有表列表
* @returns {Promise<Array>}
*/
async list() {
return request(`${API_BASE}/tables`);
},
/**
* 获取表的 Schema
* @param {string} tableName - 表名
* @returns {Promise<Object>}
*/
async getSchema(tableName) {
return request(`${API_BASE}/tables/${tableName}/schema`);
},
/**
* 获取表数据(分页)
* @param {string} tableName - 表名
* @param {Object} params - 查询参数
* @param {number} params.page - 页码
* @param {number} params.pageSize - 每页大小
* @param {string} params.select - 选择的列(逗号分隔)
* @returns {Promise<Object>}
*/
async getData(tableName, { page = 1, pageSize = 20, select = '' } = {}) {
const params = new URLSearchParams({
page: page.toString(),
pageSize: pageSize.toString(),
});
if (select) {
params.append('select', select);
}
return request(`${API_BASE}/tables/${tableName}/data?${params}`);
},
/**
* 获取单行数据详情
* @param {string} tableName - 表名
* @param {number} seq - 序列号
* @returns {Promise<Object>}
*/
async getRow(tableName, seq) {
return request(`${API_BASE}/tables/${tableName}/data/${seq}`);
},
/**
* 获取表的 Manifest 信息
* @param {string} tableName - 表名
* @returns {Promise<Object>}
*/
async getManifest(tableName) {
return request(`${API_BASE}/tables/${tableName}/manifest`);
},
/**
* 插入数据
* @param {string} tableName - 表名
* @param {Object} data - 数据对象
* @returns {Promise<Object>}
*/
async insert(tableName, data) {
return request(`${API_BASE}/tables/${tableName}/data`, {
method: 'POST',
body: JSON.stringify(data),
});
},
/**
* 批量插入数据
* @param {string} tableName - 表名
* @param {Array<Object>} data - 数据数组
* @returns {Promise<Object>}
*/
async batchInsert(tableName, data) {
return request(`${API_BASE}/tables/${tableName}/data/batch`, {
method: 'POST',
body: JSON.stringify(data),
});
},
/**
* 删除表
* @param {string} tableName - 表名
* @returns {Promise<Object>}
*/
async delete(tableName) {
return request(`${API_BASE}/tables/${tableName}`, {
method: 'DELETE',
});
},
/**
* 获取表统计信息
* @param {string} tableName - 表名
* @returns {Promise<Object>}
*/
async getStats(tableName) {
return request(`${API_BASE}/tables/${tableName}/stats`);
},
};
/**
* 数据库相关 API
*/
export const databaseAPI = {
/**
* 获取数据库信息
* @returns {Promise<Object>}
*/
async getInfo() {
return request(`${API_BASE}/database/info`);
},
/**
* 获取数据库统计信息
* @returns {Promise<Object>}
*/
async getStats() {
return request(`${API_BASE}/database/stats`);
},
};
/**
* 导出默认 API 对象
*/
export default {
table: tableAPI,
database: databaseAPI,
};

View File

@@ -1,89 +0,0 @@
import { css } from 'lit';
// 共享的基础样式
export const sharedStyles = css`
* {
box-sizing: border-box;
margin: 0;
padding: 0;
}
/* 自定义滚动条样式 */
*::-webkit-scrollbar {
width: 8px;
height: 8px;
}
*::-webkit-scrollbar-track {
background: transparent;
}
*::-webkit-scrollbar-thumb {
background: rgba(255, 255, 255, 0.1);
border-radius: 4px;
}
*::-webkit-scrollbar-thumb:hover {
background: rgba(255, 255, 255, 0.15);
}
/* 通用状态样式 */
.empty {
text-align: center;
padding: 60px 20px;
color: var(--text-secondary);
}
.loading {
text-align: center;
padding: 60px 20px;
color: var(--text-secondary);
}
`;
// CSS 变量(可以在组件中使用,优先使用外部定义的变量)
export const cssVariables = css`
:host {
/* 主色调 - 优雅的紫蓝色 */
--primary: var(--srdb-primary, #6366f1);
--primary-dark: var(--srdb-primary-dark, #4f46e5);
--primary-light: var(--srdb-primary-light, #818cf8);
--primary-bg: var(--srdb-primary-bg, rgba(99, 102, 241, 0.1));
/* 背景色 */
--bg-main: var(--srdb-bg-main, #0f0f1a);
--bg-surface: var(--srdb-bg-surface, #1a1a2e);
--bg-elevated: var(--srdb-bg-elevated, #222236);
--bg-hover: var(--srdb-bg-hover, #2a2a3e);
/* 文字颜色 */
--text-primary: var(--srdb-text-primary, #ffffff);
--text-secondary: var(--srdb-text-secondary, #a0a0b0);
--text-tertiary: var(--srdb-text-tertiary, #6b6b7b);
/* 边框和分隔线 */
--border-color: var(--srdb-border-color, rgba(255, 255, 255, 0.1));
--border-hover: var(--srdb-border-hover, rgba(255, 255, 255, 0.2));
/* 状态颜色 */
--success: var(--srdb-success, #10b981);
--warning: var(--srdb-warning, #f59e0b);
--danger: var(--srdb-danger, #ef4444);
--info: var(--srdb-info, #3b82f6);
/* 阴影 */
--shadow-sm: var(--srdb-shadow-sm, 0 1px 2px 0 rgba(0, 0, 0, 0.3));
--shadow-md: var(--srdb-shadow-md, 0 4px 6px -1px rgba(0, 0, 0, 0.4), 0 2px 4px -1px rgba(0, 0, 0, 0.3));
--shadow-lg: var(--srdb-shadow-lg, 0 10px 15px -3px rgba(0, 0, 0, 0.5), 0 4px 6px -2px rgba(0, 0, 0, 0.3));
--shadow-xl: var(--srdb-shadow-xl, 0 20px 25px -5px rgba(0, 0, 0, 0.5), 0 10px 10px -5px rgba(0, 0, 0, 0.3));
/* 圆角 */
--radius-sm: var(--srdb-radius-sm, 6px);
--radius-md: var(--srdb-radius-md, 8px);
--radius-lg: var(--srdb-radius-lg, 12px);
--radius-xl: var(--srdb-radius-xl, 16px);
/* 过渡 */
--transition: var(--srdb-transition, all 0.2s cubic-bezier(0.4, 0, 0.2, 1));
}
`;

View File

@@ -0,0 +1,146 @@
import { html } from 'htm/preact';
import { useState, useEffect } from 'preact/hooks';
import { Sidebar } from '~/components/Sidebar.js';
import { TableView } from '~/components/TableView.js';
import { getTables } from '~/utils/api.js';
export function App() {
const [theme, setTheme] = useState('dark');
const [tables, setTables] = useState([]);
const [selectedTable, setSelectedTable] = useState(null);
const [loading, setLoading] = useState(true);
// 初始化主题
useEffect(() => {
const savedTheme = localStorage.getItem('srdb_theme') || 'dark';
setTheme(savedTheme);
if (savedTheme === 'light') {
document.documentElement.setAttribute('data-theme', 'light');
}
}, []);
// 加载表列表
useEffect(() => {
fetchTables();
}, []);
const fetchTables = async () => {
try {
setLoading(true);
const data = await getTables();
setTables(data.tables || []);
if (data.tables && data.tables.length > 0) {
setSelectedTable(data.tables[0].name);
}
} catch (error) {
console.error('Failed to fetch tables:', error);
} finally {
setLoading(false);
}
};
const toggleTheme = () => {
const newTheme = theme === 'dark' ? 'light' : 'dark';
setTheme(newTheme);
localStorage.setItem('srdb_theme', newTheme);
if (newTheme === 'light') {
document.documentElement.setAttribute('data-theme', 'light');
} else {
document.documentElement.removeAttribute('data-theme');
}
};
const styles = {
container: {
display: 'flex',
height: '100vh',
overflow: 'hidden'
},
sidebar: {
width: '280px',
background: 'var(--bg-surface)',
borderRight: '1px solid var(--border-color)',
overflowY: 'auto',
overflowX: 'hidden',
padding: '16px 12px',
display: 'flex',
flexDirection: 'column',
gap: '8px'
},
sidebarHeader: {
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
marginBottom: '4px'
},
sidebarTitle: {
fontSize: '18px',
fontWeight: 700,
letterSpacing: '-0.02em',
background: 'linear-gradient(135deg, #667eea, #764ba2)',
WebkitBackgroundClip: 'text',
WebkitTextFillColor: 'transparent',
margin: 0
},
themeToggle: {
width: '32px',
height: '32px',
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)',
cursor: 'pointer',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
fontSize: '16px',
transition: 'var(--transition)'
},
main: {
flex: 1,
overflowY: 'auto',
overflowX: 'hidden',
background: 'var(--bg-main)',
padding: '24px'
}
};
return html`
<div style=${styles.container}>
<!-- 左侧侧边栏 -->
<div style=${styles.sidebar}>
<div style=${styles.sidebarHeader}>
<h1 style=${styles.sidebarTitle}>SRDB Tables</h1>
<button
style=${styles.themeToggle}
onClick=${toggleTheme}
onMouseEnter=${(e) => e.target.style.background = 'var(--bg-hover)'}
onMouseLeave=${(e) => e.target.style.background = 'var(--bg-elevated)'}
title=${theme === 'dark' ? '切换到浅色主题' : '切换到深色主题'}
>
${theme === 'dark' ? '☀️' : '🌙'}
</button>
</div>
<${Sidebar}
tables=${tables}
selectedTable=${selectedTable}
onSelectTable=${setSelectedTable}
loading=${loading}
/>
</div>
<!-- 右侧主内容区 -->
<div style=${styles.main}>
${!selectedTable && html`
<div class="empty">
<p>Select a table from the sidebar</p>
</div>
`}
${selectedTable && html`
<${TableView} tableName=${selectedTable} key=${selectedTable} />
`}
</div>
</div>
`;
}

View File

@@ -0,0 +1,186 @@
import { html } from 'htm/preact';
import { useState, useRef, useEffect } from 'preact/hooks';
const styles = {
container: {
position: 'relative',
display: 'inline-block'
},
button: {
display: 'flex',
alignItems: 'center',
gap: '8px',
padding: '8px 16px',
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)',
color: 'var(--text-primary)',
fontSize: '14px',
fontWeight: 500,
cursor: 'pointer',
transition: 'var(--transition)'
},
dropdown: (isOpen) => ({
position: 'absolute',
top: 'calc(100% + 4px)',
right: 0,
minWidth: '240px',
maxHeight: '400px',
overflowY: 'auto',
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)',
boxShadow: 'var(--shadow-lg)',
zIndex: 100,
display: isOpen ? 'block' : 'none'
}),
menuItem: (isSelected) => ({
display: 'flex',
alignItems: 'center',
gap: '12px',
padding: '10px 16px',
cursor: 'pointer',
transition: 'var(--transition)',
background: isSelected ? 'var(--primary-bg)' : 'transparent',
color: 'var(--text-primary)',
fontSize: '14px'
}),
checkbox: (isSelected) => ({
width: '18px',
height: '18px',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
flexShrink: 0,
color: isSelected ? 'var(--primary)' : 'transparent'
}),
fieldInfo: {
flex: 1,
minWidth: 0,
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
gap: '8px'
},
fieldName: {
display: 'flex',
alignItems: 'center',
gap: '6px',
flex: 1,
minWidth: 0,
overflow: 'hidden',
textOverflow: 'ellipsis',
whiteSpace: 'nowrap'
},
fieldType: {
fontSize: '10px',
color: 'var(--primary)',
background: 'var(--primary-bg)',
padding: '2px 5px',
borderRadius: 'var(--radius-sm)',
fontFamily: '"Courier New", monospace',
flexShrink: 0
}
};
export function ColumnSelector({ fields, selectedColumns, onToggle }) {
const [isOpen, setIsOpen] = useState(false);
const dropdownRef = useRef(null);
const buttonRef = useRef(null);
useEffect(() => {
const handleClickOutside = (event) => {
if (
dropdownRef.current &&
buttonRef.current &&
!dropdownRef.current.contains(event.target) &&
!buttonRef.current.contains(event.target)
) {
setIsOpen(false);
}
};
if (isOpen) {
document.addEventListener('mousedown', handleClickOutside);
}
return () => {
document.removeEventListener('mousedown', handleClickOutside);
};
}, [isOpen]);
const selectedCount = selectedColumns.length;
const totalCount = fields?.length || 0;
return html`
<div style=${styles.container}>
<button
ref=${buttonRef}
style=${styles.button}
onClick=${() => setIsOpen(!isOpen)}
onMouseEnter=${(e) => {
e.target.style.background = 'var(--bg-hover)';
e.target.style.borderColor = 'var(--border-hover)';
}}
onMouseLeave=${(e) => {
e.target.style.background = 'var(--bg-elevated)';
e.target.style.borderColor = 'var(--border-color)';
}}
>
<span>Columns</span>
${selectedCount > 0 && html`
<span style=${{
background: 'var(--primary)',
color: '#fff',
padding: '2px 6px',
borderRadius: 'var(--radius-sm)',
fontSize: '11px',
fontWeight: 600
}}>
${selectedCount}
</span>
`}
<span style=${{
fontSize: '12px',
color: 'var(--text-secondary)',
transition: 'transform 0.2s ease',
transform: isOpen ? 'rotate(180deg)' : 'rotate(0deg)'
}}>▼</span>
</button>
<div ref=${dropdownRef} style=${styles.dropdown(isOpen)}>
${fields?.map(field => {
const isSelected = selectedColumns.includes(field.name);
return html`
<div
key=${field.name}
style=${styles.menuItem(isSelected)}
onClick=${() => onToggle(field.name)}
onMouseEnter=${(e) => {
if (!isSelected) {
e.currentTarget.style.background = 'var(--bg-hover)';
}
}}
onMouseLeave=${(e) => {
if (!isSelected) {
e.currentTarget.style.background = 'transparent';
}
}}
>
<div style=${styles.checkbox(isSelected)}>
${isSelected ? '✓' : ''}
</div>
<div style=${styles.fieldInfo}>
<div style=${styles.fieldName}>
<span>${field.name}</span>
${field.indexed && html`<span style=${{ fontSize: '12px' }}>🔍</span>`}
</div>
<div style=${styles.fieldType}>${field.type}</div>
</div>
</div>
`;
})}
</div>
</div>
`;
}

View File

@@ -0,0 +1,52 @@
import { html } from 'htm/preact';
const styles = {
compactionStats: {
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-lg)',
padding: '16px'
},
compactionTitle: {
fontSize: '15px',
fontWeight: 600,
color: 'var(--text-primary)',
marginBottom: '12px'
},
compactionGrid: {
display: 'grid',
gridTemplateColumns: 'repeat(auto-fit, minmax(150px, 1fr))',
gap: '8px'
},
compactionItem: {
display: 'flex',
alignItems: 'center',
padding: '6px 0',
fontSize: '13px',
gap: "24px"
}
};
function formatNumber(num) {
return num.toLocaleString('zh-CN');
}
export function CompactionStats({ stats }) {
if (!stats) return null;
return html`
<div style=${styles.compactionStats}>
<div style=${styles.compactionTitle}>Compaction 统计</div>
<div style=${styles.compactionGrid}>
${Object.entries(stats).map(([key, value]) => html`
<div key=${key} style=${styles.compactionItem}>
<span style=${{ color: 'var(--text-secondary)' }}>${key}</span>
<span style=${{ color: 'var(--text-primary)', fontWeight: 500 }}>
${typeof value === 'number' ? formatNumber(value) : value}
</span>
</div>
`)}
</div>
</div>
`;
}

View File

@@ -0,0 +1,201 @@
import { html } from 'htm/preact';
import { useState, useEffect } from 'preact/hooks';
import { RowDetailModal } from '~/components/RowDetailModal.js';
import { Pagination } from '~/components/Pagination.js';
import { TableRow } from '~/components/TableRow.js';
import { useCellPopover } from '~/hooks/useCellPopover.js';
import { useTooltip } from '~/hooks/useTooltip.js';
import { getTableData } from '~/utils/api.js';
const styles = {
container: {
display: 'flex',
flexDirection: 'column',
gap: '12px',
position: 'relative'
},
loadingBar: {
position: 'absolute',
top: 0,
left: 0,
right: 0,
height: '3px',
background: 'var(--primary)',
zIndex: 100,
animation: 'loading-slide 1.5s ease-in-out infinite'
},
loadingOverlay: {
position: 'fixed',
top: '16px',
left: '50%',
transform: 'translateX(-50%)',
padding: '10px 20px',
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)',
boxShadow: 'var(--shadow-lg)',
zIndex: 999,
display: 'flex',
alignItems: 'center',
gap: '10px',
fontSize: '14px',
color: 'var(--text-primary)',
fontWeight: 500
},
tableWrapper: {
overflowX: 'auto',
background: 'var(--bg-surface)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)'
},
table: {
width: '100%',
borderCollapse: 'collapse',
fontSize: '13px'
},
th: {
background: 'var(--bg-elevated)',
color: 'var(--text-secondary)',
fontWeight: 600,
textAlign: 'left',
padding: '12px',
borderBottom: '1px solid var(--border-color)',
position: 'sticky',
top: 0,
zIndex: 1
}
};
export function DataTable({ schema, tableName, totalRows, selectedColumns = [] }) {
const [page, setPage] = useState(0);
const [pageSize, setPageSize] = useState(20);
const [data, setData] = useState([]);
const [loading, setLoading] = useState(false);
const [selectedSeq, setSelectedSeq] = useState(null);
const { showPopover, hidePopover } = useCellPopover();
const { showTooltip, hideTooltip } = useTooltip();
useEffect(() => {
fetchData();
}, [tableName, page, pageSize]);
const fetchData = async () => {
try {
setLoading(true);
const offset = page * pageSize;
const result = await getTableData(tableName, { limit: pageSize, offset });
setData(result.data || []);
} catch (error) {
console.error('Failed to fetch data:', error);
} finally {
setLoading(false);
}
};
const getColumns = () => {
let columns = [];
if (selectedColumns && selectedColumns.length > 0) {
// 使用选中的列
columns = [...selectedColumns];
} else if (schema && schema.fields) {
// 没有选择时,显示所有字段
columns = schema.fields.map(f => f.name);
} else {
return ['_seq', '_time'];
}
// 过滤掉 _seq 和 _time它们会被固定放到特定位置
const filtered = columns.filter(c => c !== '_seq' && c !== '_time');
// _seq 在开头其他字段在中间_time 在倒数第二Actions 列之前)
return ['_seq', ...filtered, '_time'];
};
const handleViewDetail = (seq) => {
setSelectedSeq(seq);
};
const handlePageSizeChange = (newPageSize) => {
setPageSize(newPageSize);
setPage(0);
};
const getFieldComment = (fieldName) => {
if (!schema || !schema.fields) return '';
const field = schema.fields.find(f => f.name === fieldName);
return field?.comment || '';
};
const columns = getColumns();
if (!data || data.length === 0) {
return html`<div class="empty"><p>暂无数据</p></div>`;
}
return html`
<div style=${styles.container}>
${loading && html`
<div style=${styles.loadingOverlay}>
<span style=${{ fontSize: '16px' }}>⏳</span>
<span>加载中...</span>
</div>
`}
<div style=${styles.tableWrapper}>
<table style=${styles.table}>
<thead>
<tr>
${columns.map(col => {
const comment = getFieldComment(col);
return html`
<th
key=${col}
style=${styles.th}
onMouseEnter=${(e) => comment && showTooltip(e.currentTarget, comment)}
onMouseLeave=${hideTooltip}
>
${col}
</th>
`;
})}
<th style=${{ ...styles.th, textAlign: 'center' }}>操作</th>
</tr>
</thead>
<tbody>
${data.map((row, idx) => html`
<${TableRow}
key=${row._seq || idx}
row=${row}
columns=${columns}
onViewDetail=${handleViewDetail}
onShowPopover=${showPopover}
onHidePopover=${hidePopover}
/>
`)}
</tbody>
</table>
</div>
<!-- 分页控件 -->
<${Pagination}
page=${page}
pageSize=${pageSize}
totalRows=${totalRows}
onPageChange=${setPage}
onPageSizeChange=${handlePageSizeChange}
onJumpToPage=${setPage}
/>
<!-- 详情模态框 -->
${selectedSeq !== null && html`
<${RowDetailModal}
tableName=${tableName}
seq=${selectedSeq}
onClose=${() => setSelectedSeq(null)}
/>
`}
</div>
`;
}

View File

@@ -0,0 +1,105 @@
import { html } from 'htm/preact';
const styles = {
schemaFields: {
borderTop: '1px solid var(--border-color)',
position: 'relative',
transition: 'var(--transition)'
},
verticalLine: {
content: '""',
position: 'absolute',
left: '16px',
top: 0,
bottom: '14px',
width: '1px',
background: 'var(--border-color)',
zIndex: 2
},
fieldItem: {
zIndex: 1,
display: 'flex',
alignItems: 'center',
gap: '8px',
padding: '4px 12px 4px 16px',
fontSize: '12px',
transition: 'var(--transition)',
position: 'relative'
},
horizontalLine: {
width: '8px',
height: '1px',
background: 'var(--border-color)'
},
fieldIndexIcon: {
display: 'inline-flex',
alignItems: 'center',
justifyContent: 'center',
fontSize: '12px',
flexShrink: 0
},
fieldName: {
fontWeight: 500,
color: 'var(--text-secondary)',
flex: 1,
display: 'flex',
alignItems: 'center',
gap: '4px'
},
fieldType: {
fontFamily: '"Courier New", monospace',
fontSize: '11px',
padding: '2px 6px',
background: 'var(--primary-bg)',
color: 'var(--primary)',
borderRadius: 'var(--radius-sm)',
fontWeight: 600
}
};
export function FieldList({ fields }) {
if (!fields || fields.length === 0) {
return null;
}
return html`
<div style=${styles.schemaFields}>
<!-- 垂直连接线 -->
<div style=${styles.verticalLine}></div>
${fields.map((field) => {
return html`
<div
key=${field.name}
style=${styles.fieldItem}
onMouseEnter=${(e) => {
e.currentTarget.style.background = 'var(--bg-hover)';
}}
onMouseLeave=${(e) => {
e.currentTarget.style.background = 'transparent';
}}
>
<!-- 水平连接线 -->
<div style=${styles.horizontalLine}></div>
<!-- 字段名称和索引图标 -->
<span style=${styles.fieldName}>
${field.name}
${field.indexed && html`
<span
style=${styles.fieldIndexIcon}
title="Indexed field"
>
🔍
</span>
`}
</span>
<!-- 字段类型 -->
<span style=${styles.fieldType}>${field.type}</span>
</div>
`;
})}
</div>
`;
}

View File

@@ -0,0 +1,90 @@
import { html } from 'htm/preact';
const styles = {
fileCard: {
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-sm)',
padding: '10px',
transition: 'var(--transition)'
},
fileName: {
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
marginBottom: '6px'
},
fileNameText: {
fontFamily: '"Courier New", monospace',
fontSize: '13px',
color: 'var(--text-primary)',
fontWeight: 500
},
fileLevelBadge: {
fontSize: '11px',
padding: '2px 8px',
background: 'var(--primary-bg)',
color: 'var(--primary)',
borderRadius: 'var(--radius-sm)',
fontWeight: 600,
fontFamily: '"Courier New", monospace'
},
fileDetail: {
display: 'flex',
flexDirection: 'column',
gap: '3px',
fontSize: '12px',
color: 'var(--text-secondary)'
},
fileDetailRow: {
display: 'flex',
justifyContent: 'space-between'
}
};
function formatBytes(bytes) {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return (bytes / Math.pow(k, i)).toFixed(2) + ' ' + sizes[i];
}
function formatNumber(num) {
return num.toLocaleString('zh-CN');
}
export function FileCard({ file }) {
return html`
<div
style=${styles.fileCard}
onMouseEnter=${(e) => {
e.currentTarget.style.borderColor = 'var(--primary)';
e.currentTarget.style.boxShadow = '0 2px 8px rgba(0, 0, 0, 0.1)';
}}
onMouseLeave=${(e) => {
e.currentTarget.style.borderColor = 'var(--border-color)';
e.currentTarget.style.boxShadow = 'none';
}}
>
<div style=${styles.fileName}>
<span style=${styles.fileNameText}>${String(file.file_number).padStart(6, '0')}.sst</span>
<span style=${styles.fileLevelBadge}>L${file.level}</span>
</div>
<div style=${styles.fileDetail}>
<div style=${styles.fileDetailRow}>
<span>Size:</span>
<span>${formatBytes(file.file_size)}</span>
</div>
<div style=${styles.fileDetailRow}>
<span>Rows:</span>
<span>${formatNumber(file.row_count)}</span>
</div>
<div style=${styles.fileDetailRow}>
<span>Seq Range:</span>
<span>${formatNumber(file.min_key)} - ${formatNumber(file.max_key)}</span>
</div>
</div>
</div>
`;
}

View File

@@ -0,0 +1,139 @@
import { html } from 'htm/preact';
import { FileCard } from '~/components/FileCard.js';
const styles = {
levelSection: {
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-lg)',
marginBottom: '8px',
overflow: 'hidden'
},
levelHeader: {
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
cursor: 'pointer',
padding: '14px 16px',
transition: 'var(--transition)',
},
levelHeaderLeft: {
display: 'flex',
alignItems: 'center',
gap: '12px',
flex: 1
},
expandIcon: (isExpanded) => ({
fontSize: '12px',
color: 'var(--text-secondary)',
transition: 'transform 0.2s ease',
userSelect: 'none',
transform: isExpanded ? 'rotate(90deg)' : 'rotate(0deg)'
}),
levelTitle: {
fontSize: '16px',
fontWeight: 600,
color: 'var(--text-primary)',
display: 'flex',
alignItems: 'center',
gap: '12px'
},
levelBadge: (level) => {
const colors = ['#667eea', '#764ba2', '#f093fb', '#4facfe'];
const bgColor = colors[level] || '#667eea';
return {
background: bgColor,
color: '#fff',
padding: '4px 10px',
borderRadius: 'var(--radius-sm)',
fontSize: '12px',
fontWeight: 600,
fontFamily: '"Courier New", monospace'
};
},
levelStats: {
display: 'flex',
gap: '20px',
fontSize: '13px',
color: 'var(--text-secondary)'
},
scoreBadge: (score) => ({
padding: '4px 8px',
borderRadius: 'var(--radius-sm)',
fontSize: '12px',
fontWeight: 600,
fontFamily: '"Courier New", monospace',
background: score > 1 ? '#fed7d7' : score > 0.8 ? '#feebc8' : '#c6f6d5',
color: score > 1 ? '#c53030' : score > 0.8 ? '#c05621' : '#22543d'
}),
filesContainer: (isExpanded) => ({
display: isExpanded ? 'block' : 'none',
padding: '12px',
background: 'var(--bg-surface)',
borderTop: '1px solid var(--border-color)',
borderRadius: '0 0 var(--radius-lg) var(--radius-lg)'
}),
filesGrid: {
display: 'grid',
gridTemplateColumns: 'repeat(auto-fill, minmax(280px, 1fr))',
gap: '8px'
}
};
function formatBytes(bytes) {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return (bytes / Math.pow(k, i)).toFixed(2) + ' ' + sizes[i];
}
export function LevelCard({ level, isExpanded, onToggle }) {
return html`
<div style=${styles.levelSection}>
<div
style=${styles.levelHeader}
onClick=${onToggle}
onMouseEnter=${(e) => {
e.currentTarget.style.background = 'var(--bg-hover)';
}}
onMouseLeave=${(e) => {
e.currentTarget.style.background = 'transparent';
}}
>
<div style=${styles.levelHeaderLeft}>
<span style=${styles.expandIcon(isExpanded)}>▶</span>
<div style=${styles.levelTitle}>
<span style=${styles.levelBadge(level.level)}>L${level.level}</span>
<span>Level ${level.level}</span>
</div>
</div>
<div style=${styles.levelStats}>
<span>文件: ${level.file_count}</span>
<span>大小: ${formatBytes(level.total_size)}</span>
${level.score > 0 && html`
<span style=${styles.scoreBadge(level.score)}>
Score: ${(level.score * 100).toFixed(0)}%
</span>
`}
</div>
</div>
${level.files && level.files.length > 0 && html`
<div style=${styles.filesContainer(isExpanded)}>
<div style=${styles.filesGrid}>
${level.files.map(file => html`
<${FileCard} key=${file.file_number} file=${file} />
`)}
</div>
</div>
`}
${(!level.files || level.files.length === 0) && html`
<div class="empty" style=${{ padding: '20px', textAlign: 'center' }}>
<p style=${{ color: 'var(--text-tertiary)', fontSize: '13px' }}>此层级暂无文件</p>
</div>
`}
</div>
`;
}

View File

@@ -0,0 +1,118 @@
import { html } from 'htm/preact';
import { useEffect } from 'preact/hooks';
import { ManifestView } from '~/components/ManifestView.js';
const styles = {
overlay: {
position: 'fixed',
top: 0,
left: 0,
right: 0,
bottom: 0,
background: 'rgba(0, 0, 0, 0.6)',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
zIndex: 1000,
padding: '20px'
},
modal: {
background: 'var(--bg-elevated)',
borderRadius: 'var(--radius-lg)',
boxShadow: 'var(--shadow-xl)',
width: '90vw',
maxWidth: '1200px',
maxHeight: '85vh',
display: 'flex',
flexDirection: 'column',
overflow: 'hidden'
},
header: {
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
padding: '20px 24px',
borderBottom: '1px solid var(--border-color)'
},
title: {
fontSize: '20px',
fontWeight: 600,
color: 'var(--text-primary)',
margin: 0
},
closeButton: {
width: '32px',
height: '32px',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
background: 'transparent',
border: 'none',
borderRadius: 'var(--radius-sm)',
cursor: 'pointer',
fontSize: '20px',
color: 'var(--text-secondary)',
transition: 'var(--transition)'
},
content: {
flex: 1,
overflowY: 'auto',
padding: '20px 24px'
}
};
export function ManifestModal({ tableName, onClose }) {
// ESC 键关闭
useEffect(() => {
const handleEscape = (e) => {
if (e.key === 'Escape') {
onClose();
}
};
document.addEventListener('keydown', handleEscape);
return () => document.removeEventListener('keydown', handleEscape);
}, [onClose]);
// 阻止背景滚动
useEffect(() => {
document.body.style.overflow = 'hidden';
return () => {
document.body.style.overflow = '';
};
}, []);
const handleOverlayClick = (e) => {
if (e.target === e.currentTarget) {
onClose();
}
};
return html`
<div style=${styles.overlay} onClick=${handleOverlayClick}>
<div style=${styles.modal}>
<div style=${styles.header}>
<h2 style=${styles.title}>Manifest - ${tableName}</h2>
<button
style=${styles.closeButton}
onClick=${onClose}
onMouseEnter=${(e) => {
e.target.style.background = 'var(--bg-hover)';
e.target.style.color = 'var(--text-primary)';
}}
onMouseLeave=${(e) => {
e.target.style.background = 'transparent';
e.target.style.color = 'var(--text-secondary)';
}}
title="关闭"
>
</button>
</div>
<div style=${styles.content}>
<${ManifestView} tableName=${tableName} />
</div>
</div>
</div>
`;
}

View File

@@ -0,0 +1,100 @@
import { html } from 'htm/preact';
import { useState, useEffect } from 'preact/hooks';
import { LevelCard } from '~/components/LevelCard.js';
import { StatCard } from '~/components/StatCard.js';
import { CompactionStats } from '~/components/CompactionStats.js';
import { getTableManifest } from '~/utils/api.js';
const styles = {
container: {
display: 'flex',
flexDirection: 'column',
gap: '12px'
},
statsGrid: {
display: 'grid',
gridTemplateColumns: 'repeat(auto-fit, minmax(200px, 1fr))',
gap: '8px'
}
};
function formatBytes(bytes) {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return (bytes / Math.pow(k, i)).toFixed(2) + ' ' + sizes[i];
}
function formatNumber(num) {
return num.toLocaleString('zh-CN');
}
export function ManifestView({ tableName }) {
const [manifest, setManifest] = useState(null);
const [loading, setLoading] = useState(true);
const [expandedLevels, setExpandedLevels] = useState(new Set());
useEffect(() => {
fetchManifest();
const interval = setInterval(fetchManifest, 5000); // 每5秒刷新
return () => clearInterval(interval);
}, [tableName]);
const fetchManifest = async () => {
try {
const data = await getTableManifest(tableName);
setManifest(data);
} catch (error) {
console.error('Failed to fetch manifest:', error);
} finally {
setLoading(false);
}
};
const toggleLevel = (levelNum) => {
const newExpanded = new Set(expandedLevels);
if (newExpanded.has(levelNum)) {
newExpanded.delete(levelNum);
} else {
newExpanded.add(levelNum);
}
setExpandedLevels(newExpanded);
};
if (loading) {
return html`<div class="loading"><p>加载中...</p></div>`;
}
if (!manifest) {
return html`<div class="empty"><p>无法加载 Manifest 数据</p></div>`;
}
const totalFiles = manifest.levels?.reduce((sum, level) => sum + level.file_count, 0) || 0;
const totalSize = manifest.levels?.reduce((sum, level) => sum + level.total_size, 0) || 0;
return html`
<div style=${styles.container}>
<!-- 统计信息 -->
<div style=${styles.statsGrid}>
<${StatCard} label="总文件数" value=${formatNumber(totalFiles)} />
<${StatCard} label="总大小" value=${formatBytes(totalSize)} />
<${StatCard} label="下一个文件号" value=${formatNumber(manifest.next_file_number || 0)} />
<${StatCard} label="最后序列号" value=${formatNumber(manifest.last_sequence || 0)} />
</div>
<!-- Compaction 统计 -->
<${CompactionStats} stats=${manifest.compaction_stats} />
<!-- 各层级详情 -->
${manifest.levels?.filter(level => level.file_count > 0).map(level => html`
<${LevelCard}
key=${level.level}
level=${level}
isExpanded=${expandedLevels.has(level.level)}
onToggle=${() => toggleLevel(level.level)}
/>
`)}
</div>
`;
}

View File

@@ -0,0 +1,71 @@
import { html } from 'htm/preact';
import { Fragment } from 'preact';
import { useState } from 'preact/hooks';
const styles = {
jumpInput: {
width: '80px',
padding: '6px 10px',
background: 'var(--bg-elevated)',
color: 'var(--text-primary)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-sm)',
fontSize: '13px'
},
jumpButton: {
padding: '6px 12px',
background: 'var(--bg-elevated)',
color: 'var(--text-primary)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-sm)',
cursor: 'pointer',
fontSize: '13px',
transition: 'var(--transition)',
fontWeight: 500
}
};
export function PageJumper({ totalPages, onJump }) {
const [inputValue, setInputValue] = useState('');
const handleJump = () => {
const num = parseInt(inputValue);
if (num >= 1 && num <= totalPages) {
onJump(num - 1);
setInputValue('');
}
};
const handleKeyDown = (e) => {
if (e.key === 'Enter') {
handleJump();
}
};
return html`
<${Fragment}>
<input
type="number"
min="1"
max=${totalPages}
placeholder="跳转"
value=${inputValue}
onInput=${(e) => setInputValue(e.target.value)}
onKeyDown=${handleKeyDown}
style=${styles.jumpInput}
/>
<button
style=${styles.jumpButton}
onClick=${handleJump}
onMouseEnter=${(e) => {
e.target.style.background = 'var(--bg-hover)';
}}
onMouseLeave=${(e) => {
e.target.style.background = 'var(--bg-elevated)';
}}
>
跳转
</button>
<//>
`;
}

View File

@@ -0,0 +1,122 @@
import { html } from 'htm/preact';
import { PageJumper } from './PageJumper.js';
const styles = {
pagination: {
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
padding: '12px',
background: 'var(--bg-surface)',
borderRadius: 'var(--radius-md)',
border: '1px solid var(--border-color)',
position: 'sticky',
bottom: 0,
zIndex: 10,
boxShadow: 'var(--shadow-md)'
},
paginationInfo: {
fontSize: '13px',
color: 'var(--text-secondary)'
},
paginationButtons: {
display: 'flex',
gap: '8px',
alignItems: 'center'
},
pageButton: (disabled) => ({
padding: '6px 12px',
background: disabled ? 'var(--bg-surface)' : 'var(--bg-elevated)',
color: disabled ? 'var(--text-tertiary)' : 'var(--text-primary)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-sm)',
cursor: disabled ? 'not-allowed' : 'pointer',
fontSize: '13px',
transition: 'var(--transition)',
fontWeight: 500
}),
pageSizeSelect: {
padding: '6px 10px',
background: 'var(--bg-elevated)',
color: 'var(--text-primary)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-sm)',
fontSize: '13px',
cursor: 'pointer'
}
};
export function Pagination({
page,
pageSize,
totalRows,
onPageChange,
onPageSizeChange,
onJumpToPage
}) {
const totalPages = Math.ceil(totalRows / pageSize);
const currentPage = page + 1;
const startRow = page * pageSize + 1;
const endRow = Math.min((page + 1) * pageSize, totalRows);
const handlePrevPage = () => {
if (page > 0) {
onPageChange(page - 1);
}
};
const handleNextPage = () => {
if (page < totalPages - 1) {
onPageChange(page + 1);
}
};
const handlePageSizeChange = (e) => {
onPageSizeChange(Number(e.target.value));
};
return html`
<div style=${styles.pagination}>
<div style=${styles.paginationInfo}>
显示 ${startRow}-${endRow} / 共 ${totalRows}
</div>
<div style=${styles.paginationButtons}>
<select
value=${pageSize}
onChange=${handlePageSizeChange}
style=${styles.pageSizeSelect}
>
<option value="10">10 / 页</option>
<option value="20">20 / 页</option>
<option value="50">50 / 页</option>
<option value="100">100 / 页</option>
</select>
<button
style=${styles.pageButton(page === 0)}
onClick=${handlePrevPage}
disabled=${page === 0}
onMouseEnter=${(e) => !e.target.disabled && (e.target.style.background = 'var(--bg-hover)')}
onMouseLeave=${(e) => !e.target.disabled && (e.target.style.background = 'var(--bg-elevated)')}
>
上一页
</button>
<span style=${{ fontSize: '13px', color: 'var(--text-secondary)' }}>
${currentPage} / ${totalPages}
</span>
<${PageJumper}
totalPages=${totalPages}
onJump=${onJumpToPage}
/>
<button
style=${styles.pageButton(page >= totalPages - 1)}
onClick=${handleNextPage}
disabled=${page >= totalPages - 1}
onMouseEnter=${(e) => !e.target.disabled && (e.target.style.background = 'var(--bg-hover)')}
onMouseLeave=${(e) => !e.target.disabled && (e.target.style.background = 'var(--bg-elevated)')}
>
下一页
</button>
</div>
</div>
`;
}

View File

@@ -0,0 +1,222 @@
import { html } from 'htm/preact';
import { useState, useEffect, useRef } from 'preact/hooks';
import { getRowBySeq } from '~/utils/api.js';
const styles = {
overlay: {
position: 'fixed',
top: 0,
left: 0,
right: 0,
bottom: 0,
background: 'rgba(0, 0, 0, 0.5)',
backdropFilter: 'blur(4px)',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
zIndex: 10000,
padding: '20px'
},
modal: {
background: 'var(--bg-elevated)',
borderRadius: 'var(--radius-lg)',
boxShadow: 'var(--shadow-xl)',
maxWidth: '800px',
width: '100%',
maxHeight: '90vh',
display: 'flex',
flexDirection: 'column',
border: '1px solid var(--border-color)'
},
header: {
padding: '20px',
borderBottom: '1px solid var(--border-color)',
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between'
},
title: {
fontSize: '18px',
fontWeight: 600,
color: 'var(--text-primary)',
margin: 0
},
closeButton: {
background: 'none',
border: 'none',
fontSize: '24px',
color: 'var(--text-secondary)',
cursor: 'pointer',
padding: '4px 8px',
borderRadius: 'var(--radius-sm)',
transition: 'var(--transition)',
lineHeight: 1
},
content: {
padding: '20px',
overflowY: 'auto',
flex: 1
},
fieldGroup: {
marginBottom: '16px',
padding: '12px',
background: 'var(--bg-surface)',
borderRadius: 'var(--radius-md)',
border: '1px solid var(--border-color)'
},
fieldLabel: {
fontSize: '12px',
fontWeight: 600,
color: 'var(--text-secondary)',
textTransform: 'uppercase',
marginBottom: '4px',
display: 'flex',
alignItems: 'center',
gap: '6px'
},
fieldValue: {
fontSize: '14px',
color: 'var(--text-primary)',
wordBreak: 'break-word',
fontFamily: '"Courier New", monospace',
whiteSpace: 'pre-wrap',
lineHeight: 1.5
},
metaTag: {
display: 'inline-block',
padding: '2px 6px',
fontSize: '10px',
fontWeight: 600,
borderRadius: 'var(--radius-sm)',
background: 'var(--primary-bg)',
color: 'var(--primary)',
marginLeft: '6px'
}
};
export function RowDetailModal({ tableName, seq, onClose }) {
const overlayRef = useRef(null);
const [rowData, setRowData] = useState(null);
const [loading, setLoading] = useState(true);
useEffect(() => {
fetchRowData();
}, [tableName, seq]);
// ESC 键关闭
useEffect(() => {
const handleKeyDown = (e) => {
if (e.key === 'Escape') {
onClose();
}
};
document.addEventListener('keydown', handleKeyDown);
return () => document.removeEventListener('keydown', handleKeyDown);
}, [onClose]);
const fetchRowData = async () => {
try {
setLoading(true);
const data = await getRowBySeq(tableName, seq);
setRowData(data);
} catch (error) {
console.error('Failed to fetch row data:', error);
} finally {
setLoading(false);
}
};
const handleOverlayClick = (e) => {
if (e.target === overlayRef.current) {
onClose();
}
};
const formatTime = (nanoTime) => {
if (!nanoTime) return '';
const date = new Date(nanoTime / 1000000);
return date.toLocaleString('zh-CN', {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
hour12: false
});
};
const formatValue = (value, key) => {
if (value === null) return 'null';
if (value === undefined) return 'undefined';
if (key === '_time') {
return formatTime(value);
}
if (typeof value === 'object') {
try {
return JSON.stringify(value, null, 2);
} catch (e) {
return '[Object]';
}
}
return String(value);
};
const renderField = (key, value) => {
const isMeta = key === '_seq' || key === '_time';
return html`
<div key=${key} style=${styles.fieldGroup}>
<div style=${styles.fieldLabel}>
${key}
${isMeta && html`<span style=${styles.metaTag}>系统字段</span>`}
</div>
<div style=${styles.fieldValue}>
${formatValue(value, key)}
</div>
</div>
`;
};
return html`
<div
ref=${overlayRef}
style=${styles.overlay}
onClick=${handleOverlayClick}
>
<div style=${styles.modal}>
<div style=${styles.header}>
<h3 style=${styles.title}>记录详情 - ${tableName}</h3>
<button
style=${styles.closeButton}
onClick=${onClose}
onMouseEnter=${(e) => e.target.style.background = 'var(--bg-hover)'}
onMouseLeave=${(e) => e.target.style.background = 'none'}
>
×
</button>
</div>
<div style=${styles.content}>
${loading && html`
<div class="loading" style=${{ textAlign: 'center', padding: '40px' }}>
<p>加载中...</p>
</div>
`}
${!loading && rowData && html`
<div>
${Object.entries(rowData).map(([key, value]) => renderField(key, value))}
</div>
`}
${!loading && !rowData && html`
<div class="empty" style=${{ textAlign: 'center', padding: '40px' }}>
<p>未找到数据</p>
</div>
`}
</div>
</div>
</div>
`;
}

View File

@@ -0,0 +1,31 @@
import { html } from 'htm/preact';
import { TableItem } from '~/components/TableItem.js';
export function Sidebar({ tables, selectedTable, onSelectTable, loading }) {
if (loading) {
return html`
<div class="loading">
<p>加载中...</p>
</div>
`;
}
if (tables.length === 0) {
return html`
<div class="empty">
<p>暂无数据表</p>
</div>
`;
}
return html`
${tables.map(table => html`
<${TableItem}
key=${table.name}
table=${table}
isSelected=${selectedTable === table.name}
onSelect=${() => onSelectTable(table.name)}
/>
`)}
`;
}

View File

@@ -0,0 +1,33 @@
import { html } from 'htm/preact';
const styles = {
statCard: {
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)',
padding: '12px',
transition: 'var(--transition)'
},
statLabel: {
fontSize: '12px',
color: 'var(--text-secondary)',
marginBottom: '6px',
textTransform: 'uppercase',
fontWeight: 600
},
statValue: {
fontSize: '22px',
fontWeight: 600,
color: 'var(--primary)',
fontFamily: '"Courier New", monospace'
}
};
export function StatCard({ label, value }) {
return html`
<div style=${styles.statCard}>
<div style=${styles.statLabel}>${label}</div>
<div style=${styles.statValue}>${value}</div>
</div>
`;
}

View File

@@ -0,0 +1,72 @@
import { html } from 'htm/preact';
const styles = {
td: {
padding: '10px 12px',
borderBottom: '1px solid var(--border-color)',
color: 'var(--text-primary)',
maxWidth: '300px',
overflow: 'hidden',
textOverflow: 'ellipsis',
whiteSpace: 'nowrap',
cursor: 'pointer'
}
};
function formatValue(value, col) {
if (col === '_time') {
return formatTime(value);
}
if (value === null) return 'null';
if (value === undefined) return 'undefined';
if (typeof value === 'object') {
try {
return JSON.stringify(value, null, 2);
} catch (e) {
return '[Object]';
}
}
return String(value);
}
function formatTime(nanoTime) {
if (!nanoTime) return '';
const date = new Date(nanoTime / 1000000);
const year = date.getFullYear();
const month = String(date.getMonth() + 1).padStart(2, '0');
const day = String(date.getDate()).padStart(2, '0');
const hours = String(date.getHours()).padStart(2, '0');
const minutes = String(date.getMinutes()).padStart(2, '0');
const seconds = String(date.getSeconds()).padStart(2, '0');
return `${year}-${month}-${day} ${hours}:${minutes}:${seconds}`;
}
export function TableCell({ value, column, isTruncated, onShowPopover, onHidePopover }) {
const formattedValue = formatValue(value, column);
const handleMouseEnter = (e) => {
if (onShowPopover) {
onShowPopover(e, formattedValue);
}
};
const handleMouseLeave = () => {
if (onHidePopover) {
onHidePopover();
}
};
return html`
<td
style=${styles.td}
onMouseEnter=${handleMouseEnter}
onMouseLeave=${handleMouseLeave}
>
${formattedValue}
${isTruncated && html`<span style=${{ marginLeft: '4px', fontSize: '10px' }}>✂️</span>`}
</td>
`;
}

View File

@@ -0,0 +1,130 @@
import { html } from 'htm/preact';
import { useState } from 'preact/hooks';
import { FieldList } from '~/components/FieldList.js';
const styles = {
tableItem: (isSelected, isExpanded) => ({
background: 'var(--bg-elevated)',
border: isSelected ? '1px solid var(--primary)' : '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)',
overflow: 'hidden',
transition: 'var(--transition)',
boxShadow: isSelected ? '0 0 0 1px var(--primary)' : 'none'
}),
tableHeader: (isSelected) => ({
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
paddingLeft: '4px',
paddingRight: '12px',
cursor: 'pointer',
transition: 'var(--transition)',
background: isSelected ? 'var(--primary-bg)' : 'transparent'
}),
tableHeaderLeft: {
display: 'flex',
alignItems: 'center',
gap: '8px',
flex: 1,
minWidth: 0
},
expandIcon: (isExpanded) => ({
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
width: '32px',
height: '32px',
fontSize: '12px',
transition: 'var(--transition)',
flexShrink: 0,
borderRadius: 'var(--radius-sm)',
cursor: 'pointer',
color: isExpanded ? 'var(--primary)' : 'var(--text-secondary)',
transform: isExpanded ? 'rotate(90deg)' : 'rotate(0deg)',
marginLeft: '-4px'
}),
tableName: {
fontWeight: 500,
color: 'var(--text-primary)',
whiteSpace: 'nowrap',
overflow: 'hidden',
textOverflow: 'ellipsis'
},
tableCount: {
fontSize: '12px',
color: 'var(--text-tertiary)',
whiteSpace: 'nowrap',
flexShrink: 0
}
};
export function TableItem({ table, isSelected, onSelect }) {
const [isExpanded, setIsExpanded] = useState(false);
const toggleExpand = (e) => {
e.stopPropagation();
setIsExpanded(!isExpanded);
};
return html`
<div
style=${styles.tableItem(isSelected, isExpanded)}
onMouseEnter=${(e) => {
if (!isSelected) {
e.currentTarget.style.borderColor = 'var(--border-hover)';
}
}}
onMouseLeave=${(e) => {
if (!isSelected) {
e.currentTarget.style.borderColor = 'var(--border-color)';
}
}}
>
<!-- 表头 -->
<div
style=${styles.tableHeader(isSelected)}
onClick=${onSelect}
onMouseEnter=${(e) => {
if (!isSelected) {
e.currentTarget.style.background = 'var(--bg-hover)';
}
}}
onMouseLeave=${(e) => {
if (!isSelected) {
e.currentTarget.style.background = 'transparent';
}
}}
>
<div style=${styles.tableHeaderLeft}>
<span
style=${styles.expandIcon(isExpanded)}
onClick=${toggleExpand}
onMouseEnter=${(e) => {
e.currentTarget.style.background = 'var(--bg-hover)';
if (!isExpanded) {
e.currentTarget.style.color = 'var(--text-primary)';
}
}}
onMouseLeave=${(e) => {
e.currentTarget.style.background = 'transparent';
if (!isExpanded) {
e.currentTarget.style.color = 'var(--text-secondary)';
}
}}
>
</span>
<span style=${styles.tableName}>${table.name}</span>
</div>
<span style=${styles.tableCount}>
${table.fields?.length || 0} fields
</span>
</div>
<!-- 字段列表 -->
${isExpanded && html`
<${FieldList} fields=${table.fields} />
`}
</div>
`;
}

View File

@@ -0,0 +1,68 @@
import { html } from 'htm/preact';
import { useState } from 'preact/hooks';
import { TableCell } from '~/components/TableCell.js';
const styles = {
td: {
padding: '10px 12px',
borderBottom: '1px solid var(--border-color)',
color: 'var(--text-primary)',
textAlign: 'center'
},
iconButton: (isRowHovered, isButtonHovered) => ({
width: '28px',
height: '28px',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
background: isButtonHovered ? 'var(--primary)' : 'var(--bg-elevated)',
color: isButtonHovered ? '#fff' : 'var(--primary)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-sm)',
cursor: 'pointer',
fontSize: '14px',
transition: 'all 0.2s ease',
opacity: isRowHovered ? 1 : 0,
pointerEvents: isRowHovered ? 'auto' : 'none'
})
};
export function TableRow({ row, columns, onViewDetail, onShowPopover, onHidePopover }) {
const [isRowHovered, setIsRowHovered] = useState(false);
const [isButtonHovered, setIsButtonHovered] = useState(false);
return html`
<tr
onMouseEnter=${(e) => {
e.currentTarget.style.background = 'var(--bg-hover)';
setIsRowHovered(true);
}}
onMouseLeave=${(e) => {
e.currentTarget.style.background = 'transparent';
setIsRowHovered(false);
}}
>
${columns.map(col => html`
<${TableCell}
key=${col}
value=${row[col]}
column=${col}
isTruncated=${row[col + '_truncated']}
onShowPopover=${onShowPopover}
onHidePopover=${onHidePopover}
/>
`)}
<td style=${styles.td}>
<button
style=${styles.iconButton(isRowHovered, isButtonHovered)}
onClick=${() => onViewDetail(row._seq)}
onMouseEnter=${() => setIsButtonHovered(true)}
onMouseLeave=${() => setIsButtonHovered(false)}
title="查看详情"
>
👁
</button>
</td>
</tr>
`;
}

View File

@@ -0,0 +1,178 @@
import { html } from 'htm/preact';
import { useState, useEffect } from 'preact/hooks';
import { DataTable } from '~/components/DataTable.js';
import { ColumnSelector } from '~/components/ColumnSelector.js';
import { ManifestModal } from '~/components/ManifestModal.js';
import { useTooltip } from '~/hooks/useTooltip.js';
import { getTableSchema, getTableData } from '~/utils/api.js';
const styles = {
container: {
display: 'flex',
flexDirection: 'column',
gap: '20px'
},
section: {
display: 'flex',
flexDirection: 'column',
gap: '16px'
},
sectionTitle: {
fontSize: '16px',
fontWeight: 600,
color: 'var(--text-primary)'
},
manifestButton: {
display: 'flex',
alignItems: 'center',
gap: '6px',
padding: '8px 16px',
background: 'var(--bg-elevated)',
border: '1px solid var(--border-color)',
borderRadius: 'var(--radius-md)',
color: 'var(--text-primary)',
fontSize: '14px',
fontWeight: 500,
cursor: 'pointer',
transition: 'var(--transition)'
}
};
export function TableView({ tableName }) {
const [schema, setSchema] = useState(null);
const [totalRows, setTotalRows] = useState(0);
const [loading, setLoading] = useState(true);
const [selectedColumns, setSelectedColumns] = useState([]);
const [showManifest, setShowManifest] = useState(false);
const { showTooltip, hideTooltip } = useTooltip();
useEffect(() => {
fetchTableInfo();
}, [tableName]);
useEffect(() => {
// 加载保存的列选择
if (tableName && schema) {
const saved = loadSelectedColumns();
if (saved && saved.length > 0) {
const validColumns = saved.filter(col =>
schema.fields.some(field => field.name === col)
);
if (validColumns.length > 0) {
setSelectedColumns(validColumns);
}
}
}
}, [tableName, schema]);
const fetchTableInfo = async () => {
try {
setLoading(true);
// 获取 Schema
const schemaData = await getTableSchema(tableName);
setSchema(schemaData);
// 获取数据行数(通过一次小查询)
const data = await getTableData(tableName, { limit: 1, offset: 0 });
setTotalRows(data.totalRows || 0);
} catch (error) {
console.error('Failed to fetch table info:', error);
} finally {
setLoading(false);
}
};
const toggleColumn = (columnName) => {
const index = selectedColumns.indexOf(columnName);
let newSelection;
if (index > -1) {
newSelection = selectedColumns.filter(c => c !== columnName);
} else {
newSelection = [...selectedColumns, columnName];
}
setSelectedColumns(newSelection);
saveSelectedColumns(newSelection);
};
const saveSelectedColumns = (columns) => {
if (!tableName) return;
const key = `srdb_columns_${tableName}`;
localStorage.setItem(key, JSON.stringify(columns));
};
const loadSelectedColumns = () => {
if (!tableName) return null;
const key = `srdb_columns_${tableName}`;
const saved = localStorage.getItem(key);
return saved ? JSON.parse(saved) : null;
};
if (loading) {
return html`<div class="loading"><p>加载中...</p></div>`;
}
return html`
<div style=${styles.container}>
<div style=${styles.section}>
<div style=${{ ...styles.sectionTitle, display: 'flex', alignItems: 'center', justifyContent: 'space-between' }}>
<div>
<span
style=${{ cursor: schema?.comment ? 'help' : 'default' }}
onMouseEnter=${(e) => schema?.comment && showTooltip(e.currentTarget, schema.comment)}
onMouseLeave=${hideTooltip}
>
${tableName}
</span>
<span style=${{ fontSize: '12px', fontWeight: 400, color: 'var(--text-secondary)', marginLeft: '8px' }}>
(共 ${formatCount(totalRows)} 行)
</span>
</div>
<div style=${{ display: 'flex', gap: '8px' }}>
<button
style=${styles.manifestButton}
onClick=${() => setShowManifest(true)}
onMouseEnter=${(e) => {
e.target.style.background = 'var(--bg-hover)';
e.target.style.borderColor = 'var(--border-hover)';
}}
onMouseLeave=${(e) => {
e.target.style.background = 'var(--bg-elevated)';
e.target.style.borderColor = 'var(--border-color)';
}}
>
📊 Manifest
</button>
${schema && html`
<${ColumnSelector}
fields=${schema.fields}
selectedColumns=${selectedColumns}
onToggle=${toggleColumn}
/>
`}
</div>
</div>
<${DataTable}
schema=${schema}
tableName=${tableName}
totalRows=${totalRows}
selectedColumns=${selectedColumns}
/>
</div>
${showManifest && html`
<${ManifestModal}
tableName=${tableName}
onClose=${() => setShowManifest(false)}
/>
`}
</div>
`;
}
function formatCount(count) {
if (count >= 1000000) return (count / 1000000).toFixed(1) + 'M';
if (count >= 1000) return (count / 1000).toFixed(1) + 'K';
return count.toString();
}

View File

@@ -1,145 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
export class AppContainer extends LitElement {
static properties = {
mobileMenuOpen: { type: Boolean }
};
constructor() {
super();
this.mobileMenuOpen = false;
}
toggleMobileMenu() {
this.mobileMenuOpen = !this.mobileMenuOpen;
}
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: block;
height: 100vh;
}
.container {
display: flex;
height: 100%;
overflow: hidden;
}
.sidebar {
width: 280px;
background: var(--bg-surface);
border-right: 1px solid var(--border-color);
overflow-y: auto;
overflow-x: hidden;
padding: 16px 12px;
display: flex;
flex-direction: column;
gap: 8px;
}
.sidebar::-webkit-scrollbar {
width: 6px;
}
.sidebar-header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 4px;
}
.sidebar h1 {
font-size: 18px;
font-weight: 700;
letter-spacing: -0.02em;
background: linear-gradient(135deg, var(--primary-light), var(--primary));
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
}
.main {
flex: 1;
overflow-y: auto;
overflow-x: hidden;
background: var(--bg-main);
display: flex;
flex-direction: column;
}
/* 移动端遮罩 */
.mobile-overlay {
display: none;
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.5);
z-index: 999;
}
@media (max-width: 768px) {
.mobile-overlay.show {
display: block;
}
.container {
flex-direction: column;
}
.sidebar {
position: fixed;
top: 0;
left: 0;
width: 280px;
height: 100vh;
border-right: 1px solid var(--border-color);
border-bottom: none;
transform: translateX(-100%);
transition: transform 0.3s ease;
z-index: 1000;
}
.sidebar.open {
transform: translateX(0);
}
.main {
padding-top: 0;
}
}
`
];
render() {
return html`
<!-- 移动端遮罩 -->
<div class="mobile-overlay ${this.mobileMenuOpen ? 'show' : ''}" @click=${this.toggleMobileMenu}></div>
<div class="container">
<!-- 左侧表列表 -->
<div class="sidebar ${this.mobileMenuOpen ? 'open' : ''}">
<div class="sidebar-header">
<h1>SRDB Tables</h1>
<srdb-theme-toggle></srdb-theme-toggle>
</div>
<srdb-table-list @table-selected=${this.toggleMobileMenu}></srdb-table-list>
</div>
<!-- 右侧主内容区 -->
<div class="main">
<srdb-page-header @toggle-mobile-menu=${this.toggleMobileMenu}></srdb-page-header>
<srdb-table-view></srdb-table-view>
</div>
</div>
`;
}
}
customElements.define('srdb-app', AppContainer);

View File

@@ -1,106 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
export class Badge extends LitElement {
static properties = {
variant: { type: String }, // 'primary', 'success', 'warning', 'danger', 'info'
icon: { type: String },
size: { type: String } // 'sm', 'md'
};
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: inline-flex;
}
.badge {
position: relative;
display: inline-flex;
align-items: center;
gap: 4px;
padding: 2px 8px;
font-size: 11px;
font-weight: 600;
border-radius: var(--radius-sm);
white-space: nowrap;
}
.badge.size-md {
padding: 4px 10px;
font-size: 12px;
}
/* Primary variant */
.badge.variant-primary {
--badge-border-color: rgba(99, 102, 241, 0.2);
background: rgba(99, 102, 241, 0.15);
color: var(--primary);
}
/* Success variant */
.badge.variant-success {
--badge-border-color: rgba(16, 185, 129, 0.2);
background: rgba(16, 185, 129, 0.15);
color: var(--success);
}
/* Warning variant */
.badge.variant-warning {
--badge-border-color: rgba(245, 158, 11, 0.2);
background: rgba(245, 158, 11, 0.15);
color: var(--warning);
}
/* Danger variant */
.badge.variant-danger {
--badge-border-color: rgba(239, 68, 68, 0.2);
background: rgba(239, 68, 68, 0.15);
color: var(--danger);
}
/* Info variant */
.badge.variant-info {
--badge-border-color: rgba(59, 130, 246, 0.2);
background: rgba(59, 130, 246, 0.15);
color: var(--info);
}
/* Secondary variant */
.badge.variant-secondary {
--badge-border-color: rgba(160, 160, 176, 0.2);
background: rgba(160, 160, 176, 0.15);
color: var(--text-secondary);
}
.icon {
font-size: 12px;
line-height: 1;
}
.badge.size-md .icon {
font-size: 14px;
}
`
];
constructor() {
super();
this.variant = 'primary';
this.icon = '';
this.size = 'sm';
}
render() {
return html`
<span class="badge variant-${this.variant} size-${this.size}">
${this.icon ? html`<span class="icon">${this.icon}</span>` : ''}
<slot></slot>
</span>
`;
}
}
customElements.define('srdb-badge', Badge);

View File

@@ -1,351 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
export class DataView extends LitElement {
static properties = {
tableName: { type: String },
schema: { type: Object },
tableData: { type: Object },
selectedColumns: { type: Array },
loading: { type: Boolean }
};
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: block;
}
h3 {
font-size: 16px;
font-weight: 600;
margin: 20px 0 12px 0;
color: var(--text-primary);
}
.schema-section {
margin-bottom: 24px;
}
.schema-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(200px, 1fr));
gap: 12px;
}
.schema-field-card {
padding: 12px;
background: var(--bg-elevated);
border: 1px solid var(--border-color);
border-radius: var(--radius-md);
cursor: pointer;
transition: var(--transition);
}
.schema-field-card:hover {
background: var(--bg-hover);
}
.schema-field-card.selected {
border-color: var(--primary);
background: var(--primary-bg);
}
.field-item {
display: flex;
flex-direction: column;
gap: 4px;
}
.field-item-row {
display: flex;
align-items: center;
gap: 8px;
}
.field-index-icon {
font-size: 14px;
flex-shrink: 0;
}
.field-name {
font-weight: 500;
color: var(--text-primary);
font-size: 13px;
flex: 1;
}
.field-type {
font-family: 'Courier New', monospace;
}
.field-comment {
color: var(--text-tertiary);
font-size: 11px;
margin-top: 4px;
font-style: italic;
min-height: 16px;
}
.field-comment:empty::before {
content: "No comment";
opacity: 0.5;
}
.table-wrapper {
overflow-x: auto;
background: var(--bg-elevated);
border-radius: var(--radius-md);
border: 1px solid var(--border-color);
}
.data-table {
width: 100%;
border-collapse: collapse;
font-size: 13px;
}
.data-table th {
background: var(--bg-surface);
color: var(--text-secondary);
font-weight: 600;
text-align: left;
padding: 12px;
border-bottom: 1px solid var(--border-color);
position: sticky;
top: 0;
z-index: 1;
}
.data-table td {
padding: 10px 12px;
border-bottom: 1px solid var(--border-color);
color: var(--text-primary);
max-width: 300px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.data-table td:hover {
white-space: normal;
word-break: break-all;
}
.data-table tr:hover {
background: var(--bg-hover);
}
.truncated-icon {
margin-left: 4px;
font-size: 10px;
}
.row-detail-btn {
padding: 4px 12px;
background: var(--primary);
color: white;
border: none;
border-radius: var(--radius-sm);
cursor: pointer;
font-size: 12px;
transition: var(--transition);
}
.row-detail-btn:hover {
background: var(--primary-dark);
}
`
];
constructor() {
super();
this.tableName = '';
this.schema = null;
this.tableData = null;
this.selectedColumns = [];
this.loading = false;
}
updated(changedProperties) {
// 当 tableName 或 schema 改变时,尝试加载保存的列选择
if ((changedProperties.has('tableName') || changedProperties.has('schema')) && this.tableName && this.schema) {
const saved = this.loadSelectedColumns();
if (saved && saved.length > 0) {
// 验证保存的列是否仍然存在于当前 schema 中
const validColumns = saved.filter(col =>
this.schema.fields.some(field => field.name === col)
);
if (validColumns.length > 0) {
this.selectedColumns = validColumns;
}
}
}
}
toggleColumn(columnName) {
const index = this.selectedColumns.indexOf(columnName);
if (index > -1) {
this.selectedColumns = this.selectedColumns.filter(c => c !== columnName);
} else {
this.selectedColumns = [...this.selectedColumns, columnName];
}
// 持久化到 localStorage
this.saveSelectedColumns();
this.dispatchEvent(new CustomEvent('columns-changed', {
detail: { columns: this.selectedColumns },
bubbles: true,
composed: true
}));
}
saveSelectedColumns() {
if (!this.tableName) return;
const key = `srdb_columns_${this.tableName}`;
localStorage.setItem(key, JSON.stringify(this.selectedColumns));
}
loadSelectedColumns() {
if (!this.tableName) return null;
const key = `srdb_columns_${this.tableName}`;
const saved = localStorage.getItem(key);
return saved ? JSON.parse(saved) : null;
}
showRowDetail(seq) {
this.dispatchEvent(new CustomEvent('show-row-detail', {
detail: { tableName: this.tableName, seq },
bubbles: true,
composed: true
}));
}
formatCount(count) {
if (count >= 1000000) return (count / 1000000).toFixed(1) + 'M';
if (count >= 1000) return (count / 1000).toFixed(1) + 'K';
return count.toString();
}
formatTime(nanoTime) {
if (!nanoTime) return '';
// 将纳秒转换为毫秒
const date = new Date(nanoTime / 1000000);
// 格式化为 YYYY-MM-DD HH:mm:ss
const year = date.getFullYear();
const month = String(date.getMonth() + 1).padStart(2, '0');
const day = String(date.getDate()).padStart(2, '0');
const hours = String(date.getHours()).padStart(2, '0');
const minutes = String(date.getMinutes()).padStart(2, '0');
const seconds = String(date.getSeconds()).padStart(2, '0');
return `${year}-${month}-${day} ${hours}:${minutes}:${seconds}`;
}
getColumns() {
let columns = [];
if (this.selectedColumns.length > 0) {
columns = [...this.selectedColumns];
} else {
columns = this.schema?.fields?.map(f => f.name) || [];
}
// 确保系统字段的顺序_seq 在开头_time 在倒数第二
const filtered = columns.filter(c => c !== '_seq' && c !== '_time');
// _seq 放开头
const result = ['_seq', ...filtered];
// _time 放倒数第二Actions 列之前)
result.push('_time');
return result;
}
render() {
if (this.loading || !this.schema || !this.tableData) {
return html`<div class="loading">Loading data...</div>`;
}
const columns = this.getColumns();
return html`
${this.renderSchemaSection()}
<h3>Data (${this.formatCount(this.tableData.totalRows)} rows)</h3>
${this.tableData.data.length === 0 ? html`
<div class="empty"><p>No data available</p></div>
` : html`
<div class="table-wrapper">
<table class="data-table">
<thead>
<tr>
${columns.map(col => html`<th>${col}</th>`)}
<th style="text-align: center;">Actions</th>
</tr>
</thead>
<tbody>
${this.tableData.data.map(row => html`
<tr>
${columns.map(col => html`
<td>
${col === '_time' ? this.formatTime(row[col]) : row[col]}
${row[col + '_truncated'] ? html`<span class="truncated-icon">✂️</span>` : ''}
</td>
`)}
<td style="text-align: center;">
<button
class="row-detail-btn"
@click=${() => this.showRowDetail(row._seq)}
>
Detail
</button>
</td>
</tr>
`)}
</tbody>
</table>
</div>
`}
`;
}
renderSchemaSection() {
if (!this.schema || !this.schema.fields) return '';
return html`
<div class="schema-section">
<h3>Schema <span style="font-size: 12px; font-weight: 400; color: var(--text-secondary);">(点击字段卡片选择要显示的列)</span></h3>
<div class="schema-grid">
${this.schema.fields.map(field => html`
<div
class="schema-field-card ${this.selectedColumns.includes(field.name) ? 'selected' : ''}"
@click=${() => this.toggleColumn(field.name)}
>
<div class="field-item">
<div class="field-item-row">
<srdb-field-icon
?indexed=${field.indexed}
class="field-index-icon"
title="${field.indexed ? 'Indexed field (fast)' : 'Not indexed (slow)'}"
></srdb-field-icon>
<span class="field-name">${field.name}</span>
<srdb-badge variant="primary" class="field-type">
${field.type}
</srdb-badge>
</div>
<div class="field-comment">${field.comment || ''}</div>
</div>
</div>
`)}
</div>
</div>
`;
}
}
customElements.define('srdb-data-view', DataView);

View File

@@ -1,58 +0,0 @@
import { LitElement, html, css } from 'lit';
export class FieldIcon extends LitElement {
static properties = {
indexed: { type: Boolean }
};
static styles = css`
:host {
display: inline-flex;
align-items: center;
justify-content: center;
width: 20px;
height: 20px;
}
svg {
width: 16px;
height: 16px;
}
.indexed {
fill: var(--success);
color: var(--success);
opacity: 1;
}
.not-indexed {
fill: var(--text-secondary);
color: var(--text-secondary);
opacity: 0.6;
}
`;
constructor() {
super();
this.indexed = false;
}
render() {
if (this.indexed) {
// 闪电图标 - 已索引(快速)
return html`
<svg viewBox="0 0 24 24" class="indexed" xmlns="http://www.w3.org/2000/svg" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<polygon points="13 2 3 14 12 14 11 22 21 10 12 10 13 2" fill="currentColor" stroke="none"/>
</svg>
`;
} else {
// 圆点图标 - 未索引
return html`
<svg viewBox="0 0 24 24" class="not-indexed" xmlns="http://www.w3.org/2000/svg">
<circle cx="12" cy="12" r="4" fill="currentColor"/>
</svg>
`;
}
}
}
customElements.define('srdb-field-icon', FieldIcon);

View File

@@ -1,320 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
export class ManifestView extends LitElement {
static properties = {
manifestData: { type: Object },
loading: { type: Boolean },
expandedLevels: { type: Set }
};
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: block;
}
h3 {
font-size: 16px;
font-weight: 600;
margin: 20px 0 12px 0;
color: var(--text-primary);
}
.manifest-stats {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 16px;
margin-bottom: 24px;
}
.stat-card {
padding: 16px;
background: var(--bg-elevated);
border: 1px solid var(--border-color);
border-radius: var(--radius-md);
}
.stat-label {
font-size: 12px;
color: var(--text-tertiary);
margin-bottom: 8px;
}
.stat-value {
font-size: 20px;
font-weight: 600;
color: var(--text-primary);
}
.level-card {
margin-bottom: 12px;
background: var(--bg-elevated);
border: 1px solid var(--border-color);
border-radius: var(--radius-md);
overflow: hidden;
}
.level-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 16px;
cursor: pointer;
transition: var(--transition);
}
.level-header:hover {
background: var(--bg-hover);
}
.level-header-left {
display: flex;
align-items: center;
gap: 12px;
flex: 1;
}
.expand-icon {
font-size: 12px;
color: var(--text-secondary);
transition: transform 0.2s ease;
user-select: none;
}
.expand-icon.expanded {
transform: rotate(90deg);
}
.level-title {
font-size: 16px;
font-weight: 600;
color: var(--text-primary);
}
.level-stats {
display: flex;
gap: 16px;
font-size: 13px;
color: var(--text-secondary);
}
.level-files {
padding: 16px;
background: var(--bg-surface);
border-top: 1px solid var(--border-color);
display: none;
}
.level-files.expanded {
display: block;
}
.file-list {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
gap: 12px;
}
.file-item {
padding: 12px;
background: var(--bg-elevated);
border: 1px solid var(--border-color);
border-radius: var(--radius-sm);
transition: var(--transition);
}
.file-item:hover {
border-color: var(--primary);
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
}
.file-name {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 8px;
}
.file-name-text {
font-family: 'Courier New', monospace;
font-size: 13px;
color: var(--text-primary);
font-weight: 500;
}
.file-level-badge {
font-size: 11px;
padding: 2px 8px;
background: var(--primary-bg);
color: var(--primary);
border-radius: var(--radius-sm);
font-weight: 600;
font-family: 'Courier New', monospace;
}
.file-detail {
display: flex;
flex-direction: column;
gap: 4px;
font-size: 12px;
color: var(--text-secondary);
}
.file-detail-row {
display: flex;
justify-content: space-between;
}
@media (max-width: 768px) {
.file-list {
grid-template-columns: 1fr;
}
}
.empty {
background: var(--bg-elevated);
border-radius: var(--radius-md);
border: 1px dashed var(--border-color);
margin-top: 24px;
}
.empty p {
margin: 0;
}
`
];
constructor() {
super();
this.manifestData = null;
this.loading = false;
this.expandedLevels = new Set();
}
toggleLevel(levelNum) {
if (this.expandedLevels.has(levelNum)) {
this.expandedLevels.delete(levelNum);
} else {
this.expandedLevels.add(levelNum);
}
this.requestUpdate();
}
formatSize(bytes) {
if (bytes >= 1024 * 1024 * 1024) return (bytes / (1024 * 1024 * 1024)).toFixed(2) + ' GB';
if (bytes >= 1024 * 1024) return (bytes / (1024 * 1024)).toFixed(2) + ' MB';
if (bytes >= 1024) return (bytes / 1024).toFixed(2) + ' KB';
return bytes + ' B';
}
getScoreVariant(score) {
if (score >= 0.8) return 'danger'; // 高分 = 需要紧急 compaction
if (score >= 0.5) return 'warning'; // 中分 = 需要关注
return 'success'; // 低分 = 健康状态
}
render() {
if (this.loading || !this.manifestData) {
return html`<div class="loading">Loading manifest...</div>`;
}
const totalFiles = this.manifestData.levels.reduce((sum, l) => sum + l.file_count, 0);
const totalSize = this.manifestData.levels.reduce((sum, l) => sum + l.total_size, 0);
const totalCompactions = this.manifestData.compaction_stats?.total_compactions || 0;
return html`
<h3>LSM-Tree Structure</h3>
<div class="manifest-stats">
<div class="stat-card">
<div class="stat-label">Active Levels</div>
<div class="stat-value">${this.manifestData.levels.filter(l => l.file_count > 0).length}</div>
</div>
<div class="stat-card">
<div class="stat-label">Total Files</div>
<div class="stat-value">${totalFiles}</div>
</div>
<div class="stat-card">
<div class="stat-label">Total Size</div>
<div class="stat-value">${this.formatSize(totalSize)}</div>
</div>
<div class="stat-card">
<div class="stat-label">Compactions</div>
<div class="stat-value">${totalCompactions}</div>
</div>
</div>
${this.manifestData.levels && this.manifestData.levels.length > 0
? this.manifestData.levels.map(level => this.renderLevelCard(level))
: html`
<div class="empty">
<p>No SSTable files in this table yet.</p>
<p style="font-size: 14px; margin-top: 8px;">Insert some data to see the LSM-Tree structure.</p>
</div>
`
}
`;
}
renderLevelCard(level) {
if (level.file_count === 0) return '';
const isExpanded = this.expandedLevels.has(level.level);
return html`
<div class="level-card">
<div class="level-header" @click=${() => this.toggleLevel(level.level)}>
<div class="level-header-left">
<span class="expand-icon ${isExpanded ? 'expanded' : ''}">▶</span>
<div>
<div class="level-title">Level ${level.level}</div>
<div class="level-stats">
<span>${level.file_count} files</span>
<span>${this.formatSize(level.total_size)}</span>
${level.score !== undefined ? html`
<srdb-badge variant="${this.getScoreVariant(level.score)}">
Score: ${(level.score * 100).toFixed(0)}%
</srdb-badge>
` : ''}
</div>
</div>
</div>
</div>
${level.files && level.files.length > 0 ? html`
<div class="level-files ${isExpanded ? 'expanded' : ''}">
<div class="file-list">
${level.files.map(file => html`
<div class="file-item">
<div class="file-name">
<span class="file-name-text">${file.file_number}.sst</span>
<span class="file-level-badge">L${level.level}</span>
</div>
<div class="file-detail">
<div class="file-detail-row">
<span>Size:</span>
<span>${this.formatSize(file.file_size)}</span>
</div>
<div class="file-detail-row">
<span>Rows:</span>
<span>${file.row_count || 0}</span>
</div>
<div class="file-detail-row">
<span>Seq Range:</span>
<span>${file.min_key} - ${file.max_key}</span>
</div>
</div>
</div>
`)}
</div>
</div>
` : ''}
</div>
`;
}
}
customElements.define('srdb-manifest-view', ManifestView);

View File

@@ -1,179 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
export class ModalDialog extends LitElement {
static properties = {
open: { type: Boolean },
title: { type: String },
content: { type: String }
};
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: none;
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.7);
z-index: 1000;
align-items: center;
justify-content: center;
}
:host([open]) {
display: flex;
}
.modal-content {
background: var(--bg-surface);
border-radius: var(--radius-lg);
box-shadow: var(--shadow-xl);
min-width: 324px;
max-width: 90vw;
max-height: 80vh;
width: fit-content;
display: flex;
flex-direction: column;
border: 1px solid var(--border-color);
}
@media (max-width: 768px) {
.modal-content {
min-width: 300px;
max-width: 95vw;
}
}
.modal-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 20px 24px;
border-bottom: 1px solid var(--border-color);
}
.modal-header h3 {
margin: 0;
font-size: 18px;
font-weight: 600;
color: var(--text-primary);
}
.modal-close {
background: transparent;
border: none;
color: var(--text-secondary);
cursor: pointer;
padding: 4px;
display: flex;
align-items: center;
justify-content: center;
border-radius: var(--radius-sm);
transition: var(--transition);
}
.modal-close:hover {
background: var(--bg-hover);
color: var(--text-primary);
}
.modal-body {
padding: 24px;
overflow-y: auto;
flex: 1;
}
pre {
/*background: var(--bg-elevated);*/
padding: 16px;
border-radius: var(--radius-md);
overflow-x: auto;
color: var(--text-primary);
font-family: 'Courier New', monospace;
font-size: 13px;
line-height: 1.5;
margin: 0;
white-space: pre-wrap;
word-wrap: break-word;
}
`
];
constructor() {
super();
this.open = false;
this.title = 'Content';
this.content = '';
this._handleKeyDown = this._handleKeyDown.bind(this);
}
connectedCallback() {
super.connectedCallback();
document.addEventListener('keydown', this._handleKeyDown);
}
disconnectedCallback() {
super.disconnectedCallback();
document.removeEventListener('keydown', this._handleKeyDown);
}
updated(changedProperties) {
if (changedProperties.has('open')) {
if (this.open) {
this.setAttribute('open', '');
} else {
this.removeAttribute('open');
}
}
}
_handleKeyDown(e) {
if (this.open && e.key === 'Escape') {
this.close();
}
}
close() {
this.open = false;
this.dispatchEvent(new CustomEvent('modal-close', {
bubbles: true,
composed: true
}));
}
render() {
return html`
<div class="modal-content" @click=${(e) => e.stopPropagation()}>
<div class="modal-header">
<h3>${this.title}</h3>
<button class="modal-close" @click=${this.close}>
<svg
xmlns="http://www.w3.org/2000/svg"
width="20"
height="20"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<line x1="18" y1="6" x2="6" y2="18"></line>
<line x1="6" y1="6" x2="18" y2="18"></line>
</svg>
</button>
</div>
<div class="modal-body">
<pre>${this.content}</pre>
</div>
</div>
`;
}
}
customElements.define('srdb-modal-dialog', ModalDialog);

View File

@@ -1,267 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
export class PageHeader extends LitElement {
static properties = {
tableName: { type: String },
view: { type: String }
};
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: block;
background: var(--bg-surface);
border-bottom: 1px solid var(--border-color);
position: sticky;
top: 0;
z-index: 10;
}
.header-content {
padding: 16px 24px;
}
.header-top {
display: flex;
align-items: center;
gap: 16px;
margin-bottom: 12px;
}
.mobile-menu-btn {
display: none;
width: 40px;
height: 40px;
background: var(--primary);
border: none;
border-radius: var(--radius-md);
color: white;
cursor: pointer;
flex-shrink: 0;
transition: var(--transition);
}
.mobile-menu-btn:hover {
background: var(--primary-dark);
}
.mobile-menu-btn svg {
width: 20px;
height: 20px;
}
h2 {
font-size: 24px;
font-weight: 600;
margin: 0;
color: var(--text-primary);
}
.empty-state {
text-align: center;
padding: 20px;
color: var(--text-secondary);
}
.view-tabs {
display: flex;
align-items: center;
gap: 8px;
border-bottom: 1px solid var(--border-color);
margin: 0 -12px;
padding: 0 24px;
}
.refresh-btn {
margin-left: auto;
padding: 8px 12px;
background: transparent;
border: 1px solid var(--border-color);
border-radius: var(--radius-sm);
color: var(--text-secondary);
cursor: pointer;
transition: var(--transition);
display: flex;
align-items: center;
gap: 6px;
font-size: 13px;
}
.refresh-btn:hover {
background: var(--bg-hover);
border-color: var(--border-hover);
color: var(--text-primary);
}
.refresh-btn svg {
width: 16px;
height: 16px;
}
.view-tab {
position: relative;
padding: 16px 20px;
background: transparent;
border: none;
color: var(--text-secondary);
cursor: pointer;
font-size: 14px;
font-weight: 500;
transition: var(--transition);
}
.view-tab:hover {
color: var(--text-primary);
/*background: var(--bg-hover);*/
}
.view-tab.active {
color: var(--primary);
}
.view-tab::before,
.view-tab::after {
position: absolute;
display: block;
content: "";
z-index: 0;
opacity: 0;
inset-inline: 8px;
}
.view-tab::before {
inset-block: 8px;
border-radius: var(--radius-md);
background: var(--bg-elevated);
}
.view-tab::after {
border-radius: var(--radius-md);
background: var(--primary);
bottom: -2px;
height: 4px;
}
.view-tab.active::before,
.view-tab.active::after,
.view-tab:hover::before {
opacity: 1;
}
.view-tab span {
position: relative;
z-index: 1;
}
@media (max-width: 768px) {
.mobile-menu-btn {
display: flex;
align-items: center;
justify-content: center;
}
.header-content {
padding: 12px 16px;
}
h2 {
font-size: 18px;
flex: 1;
}
.view-tabs {
margin: 0 -16px;
padding: 0 16px;
}
.view-tab {
padding: 8px 16px;
font-size: 13px;
}
}
`
];
constructor() {
super();
this.tableName = '';
this.view = 'data';
}
switchView(newView) {
this.view = newView;
this.dispatchEvent(new CustomEvent('view-changed', {
detail: { view: newView },
bubbles: true,
composed: true
}));
}
toggleMobileMenu() {
this.dispatchEvent(new CustomEvent('toggle-mobile-menu', {
bubbles: true,
composed: true
}));
}
refreshView() {
this.dispatchEvent(new CustomEvent('refresh-view', {
detail: { view: this.view },
bubbles: true,
composed: true
}));
}
render() {
if (!this.tableName) {
return html`
<div class="header-content">
<div class="empty-state">
<p>Select a table from the sidebar</p>
</div>
</div>
`;
}
return html`
<div class="header-content">
<div class="header-top">
<button class="mobile-menu-btn" @click=${this.toggleMobileMenu}>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<line x1="3" y1="12" x2="21" y2="12"></line>
<line x1="3" y1="6" x2="21" y2="6"></line>
<line x1="3" y1="18" x2="21" y2="18"></line>
</svg>
</button>
<h2>${this.tableName}</h2>
</div>
</div>
<div class="view-tabs">
<button
class="view-tab ${this.view === 'data' ? 'active' : ''}"
@click=${() => this.switchView('data')}
>
<span>Data</span>
</button>
<button
class="view-tab ${this.view === 'manifest' ? 'active' : ''}"
@click=${() => this.switchView('manifest')}
>
<span>Manifest / Storage Layers</span>
</button>
<button class="refresh-btn" @click=${this.refreshView} title="Refresh current view">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M21.5 2v6h-6M2.5 22v-6h6M2 11.5a10 10 0 0 1 18.8-4.3M22 12.5a10 10 0 0 1-18.8 4.2"/>
</svg>
<span>Refresh</span>
</button>
</div>
`;
}
}
customElements.define('srdb-page-header', PageHeader);

View File

@@ -1,283 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
import { tableAPI } from '~/common/api.js';
export class TableList extends LitElement {
static properties = {
tables: { type: Array },
selectedTable: { type: String },
expandedTables: { type: Set }
};
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: block;
width: 100%;
}
.table-item {
margin-bottom: 8px;
background: var(--bg-elevated);
border: 1px solid var(--border-color);
border-radius: var(--radius-md);
overflow: hidden;
transition: var(--transition);
}
.table-item:hover {
border-color: var(--border-hover);
}
.table-item.selected {
border-color: var(--primary);
box-shadow: 0 0 0 1px var(--primary);
}
.table-item.selected .table-header {
background: var(--primary-bg);
}
.table-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 10px 12px;
cursor: pointer;
transition: var(--transition);
}
.table-header:hover {
background: var(--bg-hover);
}
.table-item.has-expanded .table-header {
border-bottom-color: var(--border-color);
}
.table-header-left {
display: flex;
align-items: center;
gap: 8px;
flex: 1;
min-width: 0;
}
.expand-icon {
display: flex;
align-items: center;
justify-content: center;
width: 24px;
height: 24px;
font-size: 12px;
transition: var(--transition);
flex-shrink: 0;
border-radius: var(--radius-sm);
cursor: pointer;
color: var(--text-secondary);
}
.expand-icon:hover {
background: var(--bg-hover);
color: var(--text-primary);
}
.expand-icon.expanded {
transform: rotate(90deg);
color: var(--primary);
}
.table-name {
font-weight: 500;
color: var(--text-primary);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.table-count {
font-size: 12px;
color: var(--text-tertiary);
white-space: nowrap;
flex-shrink: 0;
}
/* Schema 字段列表 */
.schema-fields {
display: none;
flex-direction: column;
border-top: 1px solid transparent;
transition: var(--transition);
position: relative;
}
.schema-fields.expanded {
display: block;
border-top-color: var(--border-color);
}
/* 共享的垂直线 */
.schema-fields.expanded::before {
z-index: 2;
content: "";
position: absolute;
left: 24px;
top: 0;
bottom: 24px;
width: 1px;
background: var(--border-color);
}
.field-item {
z-index: 1;
display: flex;
align-items: center;
gap: 8px;
padding: 8px 12px 8px 24px;
font-size: 12px;
transition: var(--transition);
position: relative;
}
/* 每个字段的水平线 */
.field-item::before {
content: "";
width: 8px;
height: 1px;
background: var(--border-color);
}
.field-item:hover {
background: var(--bg-hover);
}
.field-item:last-child {
padding-bottom: 12px;
}
.field-index-icon {
display: flex;
align-items: center;
justify-content: center;
width: 20px;
height: 20px;
font-size: 14px;
flex-shrink: 0;
}
.field-index-icon.indexed {
color: var(--success);
}
.field-index-icon.not-indexed {
color: var(--text-tertiary);
opacity: 0.5;
}
.field-name {
font-weight: 500;
color: var(--text-secondary);
flex: 1;
}
.field-type {
font-family: 'Courier New', monospace;
}
.loading {
padding: 20px;
}
.error {
text-align: center;
padding: 20px;
color: var(--danger);
}
`
];
constructor() {
super();
this.tables = [];
this.selectedTable = '';
this.expandedTables = new Set();
}
connectedCallback() {
super.connectedCallback();
this.loadTables();
}
async loadTables() {
try {
this.tables = await tableAPI.list();
} catch (error) {
console.error('Error loading tables:', error);
}
}
toggleExpand(tableName, event) {
event.stopPropagation();
if (this.expandedTables.has(tableName)) {
this.expandedTables.delete(tableName);
} else {
this.expandedTables.add(tableName);
}
this.requestUpdate();
}
selectTable(tableName) {
this.selectedTable = tableName;
this.dispatchEvent(new CustomEvent('table-selected', {
detail: { tableName },
bubbles: true,
composed: true
}));
}
render() {
if (this.tables.length === 0) {
return html`<div class="loading">Loading tables...</div>`;
}
return html`
${this.tables.map(table => html`
<div class="table-item ${this.expandedTables.has(table.name) ? 'has-expanded' : ''} ${this.selectedTable === table.name ? 'selected' : ''}">
<div
class="table-header"
@click=${() => this.selectTable(table.name)}
>
<div class="table-header-left">
<span
class="expand-icon ${this.expandedTables.has(table.name) ? 'expanded' : ''}"
@click=${(e) => this.toggleExpand(table.name, e)}
>
</span>
<span class="table-name">${table.name}</span>
</div>
<span class="table-count">${table.fields.length} fields</span>
</div>
<div class="schema-fields ${this.expandedTables.has(table.name) ? 'expanded' : ''}">
${table.fields.map(field => html`
<div class="field-item">
<srdb-field-icon
?indexed=${field.indexed}
class="field-index-icon"
title="${field.indexed ? 'Indexed field (fast)' : 'Not indexed (slow)'}"
></srdb-field-icon>
<span class="field-name">${field.name}</span>
<srdb-badge variant="primary" class="field-type">
${field.type}
</srdb-badge>
</div>
`)}
</div>
</div>
`)}
`;
}
}
customElements.define('srdb-table-list', TableList);

View File

@@ -1,376 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles, cssVariables } from '~/common/shared-styles.js';
import { tableAPI } from '~/common/api.js';
export class TableView extends LitElement {
static properties = {
tableName: { type: String },
view: { type: String }, // 'data' or 'manifest'
schema: { type: Object },
tableData: { type: Object },
manifestData: { type: Object },
selectedColumns: { type: Array },
page: { type: Number },
pageSize: { type: Number },
loading: { type: Boolean }
};
static styles = [
sharedStyles,
cssVariables,
css`
:host {
display: flex;
flex-direction: column;
width: 100%;
flex: 1;
position: relative;
overflow: hidden;
}
.content-wrapper {
flex: 1;
overflow-y: auto;
padding: 24px;
padding-bottom: 80px;
}
.pagination {
position: fixed;
bottom: 0;
left: 280px;
right: 0;
display: flex;
align-items: center;
justify-content: center;
gap: 12px;
padding: 16px 24px;
background: var(--bg-elevated);
border-top: 1px solid var(--border-color);
z-index: 10;
}
@media (max-width: 768px) {
.pagination {
left: 0;
}
}
.pagination button {
padding: 8px 16px;
background: var(--bg-surface);
color: var(--text-primary);
border: 1px solid var(--border-color);
border-radius: var(--radius-sm);
cursor: pointer;
transition: var(--transition);
}
.pagination button:hover:not(:disabled) {
background: var(--bg-hover);
border-color: var(--border-hover);
}
.pagination button:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.pagination select,
.pagination input {
padding: 8px 12px;
background: var(--bg-surface);
color: var(--text-primary);
border: 1px solid var(--border-color);
border-radius: var(--radius-sm);
}
.pagination input {
width: 80px;
}
.pagination span {
color: var(--text-primary);
font-size: 14px;
}
`
];
constructor() {
super();
this.tableName = '';
this.view = 'data';
this.schema = null;
this.tableData = null;
this.manifestData = null;
this.selectedColumns = [];
this.page = 1;
this.pageSize = 20;
this.loading = false;
}
updated(changedProperties) {
if (changedProperties.has('tableName') && this.tableName) {
// 切换表时重置选中的列
this.selectedColumns = [];
this.page = 1;
this.loadData();
}
if (changedProperties.has('view') && this.tableName) {
this.loadData();
}
}
async loadData() {
if (!this.tableName) return;
this.loading = true;
try {
// Load schema
this.schema = await tableAPI.getSchema(this.tableName);
// Initialize selected columns from localStorage or all by default
if (this.schema.fields) {
const saved = this.loadSelectedColumns();
if (saved && saved.length > 0) {
// 验证保存的列是否仍然存在于当前 schema 中
const validColumns = saved.filter(col =>
this.schema.fields.some(field => field.name === col)
);
this.selectedColumns = validColumns.length > 0 ? validColumns : this.schema.fields.map(f => f.name);
} else {
this.selectedColumns = this.schema.fields.map(f => f.name);
}
}
if (this.view === 'data') {
await this.loadTableData();
} else if (this.view === 'manifest') {
await this.loadManifestData();
}
} catch (error) {
console.error('Error loading data:', error);
} finally {
this.loading = false;
}
}
async loadTableData() {
this.tableData = await tableAPI.getData(this.tableName, {
page: this.page,
pageSize: this.pageSize,
select: this.selectedColumns.join(',')
});
}
async loadManifestData() {
this.manifestData = await tableAPI.getManifest(this.tableName);
}
switchView(newView) {
this.view = newView;
}
loadSelectedColumns() {
if (!this.tableName) return null;
const key = `srdb_columns_${this.tableName}`;
const saved = localStorage.getItem(key);
return saved ? JSON.parse(saved) : null;
}
toggleColumn(columnName) {
const index = this.selectedColumns.indexOf(columnName);
if (index > -1) {
this.selectedColumns = this.selectedColumns.filter(c => c !== columnName);
} else {
this.selectedColumns = [...this.selectedColumns, columnName];
}
this.loadTableData();
}
changePage(delta) {
this.page = Math.max(1, this.page + delta);
this.loadTableData();
}
changePageSize(newSize) {
this.pageSize = parseInt(newSize);
this.page = 1;
this.loadTableData();
}
jumpToPage(pageNum) {
const num = parseInt(pageNum);
if (num > 0 && this.tableData && num <= this.tableData.totalPages) {
this.page = num;
this.loadTableData();
}
}
showRowDetail(seq) {
this.dispatchEvent(new CustomEvent('show-row-detail', {
detail: { tableName: this.tableName, seq },
bubbles: true,
composed: true
}));
}
toggleLevel(level) {
const levelCard = this.shadowRoot.querySelector(`[data-level="${level}"]`);
if (levelCard) {
const fileList = levelCard.querySelector('.file-list');
const icon = levelCard.querySelector('.expand-icon');
if (fileList.classList.contains('expanded')) {
fileList.classList.remove('expanded');
icon.style.transform = 'rotate(0deg)';
} else {
fileList.classList.add('expanded');
icon.style.transform = 'rotate(90deg)';
}
}
}
formatBytes(bytes) {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return (bytes / Math.pow(k, i)).toFixed(2) + ' ' + sizes[i];
}
formatCount(count) {
if (count >= 1000000) return (count / 1000000).toFixed(1) + 'M';
if (count >= 1000) return (count / 1000).toFixed(1) + 'K';
return count.toString();
}
render() {
if (!this.tableName) {
return html`
<div class="empty">
<h2>Select a table to view data</h2>
<p>Choose a table from the sidebar to get started</p>
</div>
`;
}
if (this.loading) {
return html`<div class="loading">Loading...</div>`;
}
return html`
<div class="content-wrapper">
${this.view === 'data' ? html`
<srdb-data-view
.tableName=${this.tableName}
.schema=${this.schema}
.tableData=${this.tableData}
.selectedColumns=${this.selectedColumns}
.loading=${this.loading}
@columns-changed=${(e) => {
this.selectedColumns = e.detail.columns;
this.loadTableData();
}}
@show-row-detail=${(e) => this.showRowDetail(e.detail.seq)}
></srdb-data-view>
` : html`
<srdb-manifest-view
.manifestData=${this.manifestData}
.loading=${this.loading}
></srdb-manifest-view>
`}
</div>
${this.view === 'data' && this.tableData ? this.renderPagination() : ''}
`;
}
renderPagination() {
return html`
<div class="pagination">
<select @change=${(e) => this.changePageSize(e.target.value)}>
${[10, 20, 50, 100].map(size => html`
<option value="${size}" ?selected=${size === this.pageSize}>
${size} / page
</option>
`)}
</select>
<button
@click=${() => this.changePage(-1)}
?disabled=${this.page <= 1}
>
Previous
</button>
<span>
Page ${this.page} of ${this.tableData.totalPages}
(${this.formatCount(this.tableData.totalRows)} rows)
</span>
<input
type="number"
min="1"
max="${this.tableData.totalPages}"
placeholder="Jump to"
@keydown=${(e) => e.key === 'Enter' && this.jumpToPage(e.target.value)}
/>
<button @click=${(e) => this.jumpToPage(e.target.previousElementSibling.value)}>
Go
</button>
<button
@click=${() => this.changePage(1)}
?disabled=${this.page >= this.tableData.totalPages}
>
Next
</button>
</div>
`;
}
formatCount(count) {
if (count >= 1000000) return (count / 1000000).toFixed(1) + 'M';
if (count >= 1000) return (count / 1000).toFixed(1) + 'K';
return count.toString();
}
changePage(delta) {
this.page = Math.max(1, this.page + delta);
this.loadTableData();
}
changePageSize(newSize) {
this.pageSize = parseInt(newSize);
this.page = 1;
this.loadTableData();
}
jumpToPage(pageNum) {
const num = parseInt(pageNum);
if (num > 0 && this.tableData && num <= this.tableData.totalPages) {
this.page = num;
this.loadTableData();
}
}
showRowDetail(seq) {
this.dispatchEvent(new CustomEvent('show-row-detail', {
detail: { tableName: this.tableName, seq },
bubbles: true,
composed: true
}));
}
showCellContent(content) {
this.dispatchEvent(new CustomEvent('show-cell-content', {
detail: { content },
bubbles: true,
composed: true
}));
}
}
customElements.define('srdb-table-view', TableView);

View File

@@ -1,123 +0,0 @@
import { LitElement, html, css } from 'lit';
import { sharedStyles } from '~/common/shared-styles.js';
export class ThemeToggle extends LitElement {
static properties = {
theme: { type: String }
};
static styles = [
sharedStyles,
css`
:host {
display: inline-block;
}
.theme-toggle {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 12px;
background: var(--bg-elevated);
border: 1px solid var(--border-color);
border-radius: var(--radius-md);
cursor: pointer;
transition: var(--transition);
font-size: 14px;
color: var(--text-primary);
}
.theme-toggle:hover {
background: var(--bg-hover);
border-color: var(--border-hover);
}
.icon {
font-size: 18px;
display: flex;
align-items: center;
}
.label {
font-weight: 500;
}
`
];
constructor() {
super();
// 从 localStorage 读取主题,默认为 dark
this.theme = localStorage.getItem('srdb-theme') || 'dark';
this.applyTheme();
}
toggleTheme() {
this.theme = this.theme === 'dark' ? 'light' : 'dark';
localStorage.setItem('srdb-theme', this.theme);
this.applyTheme();
// 触发主题变化事件
this.dispatchEvent(new CustomEvent('theme-changed', {
detail: { theme: this.theme },
bubbles: true,
composed: true
}));
}
applyTheme() {
const root = document.documentElement;
if (this.theme === 'light') {
// 浅色主题
root.style.setProperty('--srdb-bg-main', '#ffffff');
root.style.setProperty('--srdb-bg-surface', '#f5f5f5');
root.style.setProperty('--srdb-bg-elevated', '#e5e5e5');
root.style.setProperty('--srdb-bg-hover', '#d4d4d4');
root.style.setProperty('--srdb-text-primary', '#1a1a1a');
root.style.setProperty('--srdb-text-secondary', '#666666');
root.style.setProperty('--srdb-text-tertiary', '#999999');
root.style.setProperty('--srdb-border-color', 'rgba(0, 0, 0, 0.1)');
root.style.setProperty('--srdb-border-hover', 'rgba(0, 0, 0, 0.2)');
root.style.setProperty('--srdb-shadow-sm', '0 1px 2px 0 rgba(0, 0, 0, 0.05)');
root.style.setProperty('--srdb-shadow-md', '0 4px 6px -1px rgba(0, 0, 0, 0.1), 0 2px 4px -1px rgba(0, 0, 0, 0.06)');
root.style.setProperty('--srdb-shadow-lg', '0 10px 15px -3px rgba(0, 0, 0, 0.1), 0 4px 6px -2px rgba(0, 0, 0, 0.05)');
root.style.setProperty('--srdb-shadow-xl', '0 20px 25px -5px rgba(0, 0, 0, 0.1), 0 10px 10px -5px rgba(0, 0, 0, 0.04)');
} else {
// 深色主题(默认值)
root.style.setProperty('--srdb-bg-main', '#0f0f1a');
root.style.setProperty('--srdb-bg-surface', '#1a1a2e');
root.style.setProperty('--srdb-bg-elevated', '#222236');
root.style.setProperty('--srdb-bg-hover', '#2a2a3e');
root.style.setProperty('--srdb-text-primary', '#ffffff');
root.style.setProperty('--srdb-text-secondary', '#a0a0b0');
root.style.setProperty('--srdb-text-tertiary', '#6b6b7b');
root.style.setProperty('--srdb-border-color', 'rgba(255, 255, 255, 0.1)');
root.style.setProperty('--srdb-border-hover', 'rgba(255, 255, 255, 0.2)');
root.style.setProperty('--srdb-shadow-sm', '0 1px 2px 0 rgba(0, 0, 0, 0.3)');
root.style.setProperty('--srdb-shadow-md', '0 4px 6px -1px rgba(0, 0, 0, 0.4), 0 2px 4px -1px rgba(0, 0, 0, 0.3)');
root.style.setProperty('--srdb-shadow-lg', '0 10px 15px -3px rgba(0, 0, 0, 0.5), 0 4px 6px -2px rgba(0, 0, 0, 0.3)');
root.style.setProperty('--srdb-shadow-xl', '0 20px 25px -5px rgba(0, 0, 0, 0.5), 0 10px 10px -5px rgba(0, 0, 0, 0.3)');
}
}
render() {
return html`
<button class="theme-toggle" @click=${this.toggleTheme}>
<span class="icon">
${this.theme === 'dark' ? '🌙' : '☀️'}
</span>
<span class="label">
${this.theme === 'dark' ? 'Dark' : 'Light'}
</span>
</button>
`;
}
}
customElements.define('srdb-theme-toggle', ThemeToggle);

View File

@@ -0,0 +1,155 @@
import { useEffect, useRef } from 'preact/hooks';
import domAlign from 'dom-align';
export function useCellPopover() {
const popoverRef = useRef(null);
const hideTimeoutRef = useRef(null);
const targetCellRef = useRef(null);
useEffect(() => {
// 创建 popover 元素
const popover = document.createElement('div');
popover.className = 'srdb-popover';
updatePopoverStyles(popover);
document.body.appendChild(popover);
popoverRef.current = popover;
// 添加样式
addPopoverStyles();
// 监听主题变化
const observer = new MutationObserver(() => {
updatePopoverStyles(popover);
});
observer.observe(document.documentElement, {
attributes: true,
attributeFilter: ['data-theme']
});
// 清理
return () => {
if (popover) {
popover.remove();
}
observer.disconnect();
};
}, []);
const updatePopoverStyles = (popover) => {
const rootStyles = getComputedStyle(document.documentElement);
const bgElevated = rootStyles.getPropertyValue('--bg-elevated').trim();
const textPrimary = rootStyles.getPropertyValue('--text-primary').trim();
const borderColor = rootStyles.getPropertyValue('--border-color').trim();
const shadowMd = rootStyles.getPropertyValue('--shadow-md').trim();
popover.style.cssText = `
position: fixed;
z-index: 9999;
background: ${bgElevated};
border: 1px solid ${borderColor};
border-radius: 8px;
box-shadow: ${shadowMd};
padding: 12px;
max-width: 500px;
max-height: 400px;
overflow: auto;
font-size: 13px;
color: ${textPrimary};
white-space: pre-wrap;
word-break: break-word;
font-family: 'Courier New', monospace;
opacity: 0;
transition: opacity 0.15s ease-in-out, background 0.3s ease, color 0.3s ease;
display: none;
pointer-events: auto;
`;
};
const addPopoverStyles = () => {
if (document.getElementById('srdb-popover-styles')) return;
const style = document.createElement('style');
style.id = 'srdb-popover-styles';
style.textContent = `
.srdb-popover::-webkit-scrollbar {
width: 8px;
height: 8px;
}
.srdb-popover::-webkit-scrollbar-track {
background: var(--bg-surface);
border-radius: 4px;
}
.srdb-popover::-webkit-scrollbar-thumb {
background: var(--border-color);
border-radius: 4px;
}
.srdb-popover::-webkit-scrollbar-thumb:hover {
background: var(--border-hover);
}
`;
document.head.appendChild(style);
};
const showPopover = (e, content) => {
if (!popoverRef.current) return;
// 清除隐藏定时器
if (hideTimeoutRef.current) {
clearTimeout(hideTimeoutRef.current);
hideTimeoutRef.current = null;
}
// 只在内容较长时显示
if (content.length < 50) {
return;
}
// 更新 popover
popoverRef.current.textContent = content;
popoverRef.current.style.display = 'block';
targetCellRef.current = e.target;
// 使用 dom-align 定位
domAlign(popoverRef.current, e.target, {
points: ['tl', 'tr'],
offset: [2, 0],
overflow: { adjustX: true, adjustY: true }
});
// 显示动画
setTimeout(() => {
if (popoverRef.current) {
popoverRef.current.style.opacity = '1';
}
}, 10);
// 添加鼠标事件监听
popoverRef.current.addEventListener('mouseenter', keepPopover);
popoverRef.current.addEventListener('mouseleave', hidePopover);
};
const hidePopover = () => {
hideTimeoutRef.current = setTimeout(() => {
if (popoverRef.current) {
popoverRef.current.style.opacity = '0';
setTimeout(() => {
if (popoverRef.current) {
popoverRef.current.style.display = 'none';
}
}, 150);
}
}, 300);
};
const keepPopover = () => {
if (hideTimeoutRef.current) {
clearTimeout(hideTimeoutRef.current);
hideTimeoutRef.current = null;
}
};
return {
showPopover,
hidePopover
};
}

View File

@@ -0,0 +1,76 @@
import { useRef, useEffect } from 'preact/hooks';
import align from 'dom-align';
export function useTooltip() {
const tooltipRef = useRef(null);
const hideTimeoutRef = useRef(null);
useEffect(() => {
// 创建 tooltip 元素
const tooltip = document.createElement('div');
tooltip.className = 'srdb-tooltip';
tooltip.style.cssText = `
position: absolute;
z-index: 1000;
background: var(--bg-elevated);
border: 1px solid var(--border-color);
border-radius: var(--radius-sm);
padding: 8px 12px;
font-size: 13px;
color: var(--text-primary);
box-shadow: var(--shadow-lg);
max-width: 300px;
word-wrap: break-word;
display: none;
pointer-events: none;
`;
document.body.appendChild(tooltip);
tooltipRef.current = tooltip;
return () => {
if (tooltipRef.current) {
document.body.removeChild(tooltipRef.current);
}
if (hideTimeoutRef.current) {
clearTimeout(hideTimeoutRef.current);
}
};
}, []);
const showTooltip = (target, comment) => {
if (!tooltipRef.current || !comment) return;
// 清除之前的隐藏定时器
if (hideTimeoutRef.current) {
clearTimeout(hideTimeoutRef.current);
hideTimeoutRef.current = null;
}
const tooltip = tooltipRef.current;
tooltip.textContent = comment;
tooltip.style.display = 'block';
// 使用 dom-align 对齐到目标元素下方
align(tooltip, target, {
points: ['tc', 'bc'],
offset: [0, -8],
overflow: {
adjustX: true,
adjustY: true
}
});
};
const hideTooltip = () => {
if (!tooltipRef.current) return;
// 延迟隐藏,避免闪烁
hideTimeoutRef.current = setTimeout(() => {
if (tooltipRef.current) {
tooltipRef.current.style.display = 'none';
}
}, 100);
};
return { showTooltip, hideTooltip };
}

View File

@@ -1,108 +1,6 @@
import '~/components/app.js';
import '~/components/table-list.js';
import '~/components/table-view.js';
import '~/components/modal-dialog.js';
import '~/components/theme-toggle.js';
import '~/components/badge.js';
import '~/components/field-icon.js';
import '~/components/data-view.js';
import '~/components/manifest-view.js';
import '~/components/page-header.js';
import { tableAPI } from '~/common/api.js';
import { render } from 'preact';
import { html } from 'htm/preact';
import { App } from '~/components/App.js';
class App {
constructor() {
// 等待 srdb-app 组件渲染完成
this.appContainer = document.querySelector('srdb-app');
this.modal = document.querySelector('srdb-modal-dialog');
// 等待组件初始化
if (this.appContainer) {
// 使用 updateComplete 等待组件渲染完成
this.appContainer.updateComplete.then(() => {
this.tableList = this.appContainer.shadowRoot.querySelector('srdb-table-list');
this.tableView = this.appContainer.shadowRoot.querySelector('srdb-table-view');
this.pageHeader = this.appContainer.shadowRoot.querySelector('srdb-page-header');
this.setupEventListeners();
});
} else {
// 如果组件还未定义,等待它被定义
customElements.whenDefined('srdb-app').then(() => {
this.appContainer = document.querySelector('srdb-app');
this.appContainer.updateComplete.then(() => {
this.tableList = this.appContainer.shadowRoot.querySelector('srdb-table-list');
this.tableView = this.appContainer.shadowRoot.querySelector('srdb-table-view');
this.pageHeader = this.appContainer.shadowRoot.querySelector('srdb-page-header');
this.setupEventListeners();
});
});
}
}
setupEventListeners() {
// Listen for table selection
document.addEventListener('table-selected', (e) => {
const tableName = e.detail.tableName;
this.pageHeader.tableName = tableName;
this.pageHeader.view = 'data';
this.tableView.tableName = tableName;
this.tableView.view = 'data';
this.tableView.page = 1;
});
// Listen for view change from page-header
document.addEventListener('view-changed', (e) => {
this.tableView.view = e.detail.view;
});
// Listen for refresh request from page-header
document.addEventListener('refresh-view', (e) => {
this.tableView.loadData();
});
// Listen for row detail request
document.addEventListener('show-row-detail', async (e) => {
const { tableName, seq } = e.detail;
await this.showRowDetail(tableName, seq);
});
// Listen for cell content request
document.addEventListener('show-cell-content', (e) => {
this.showCellContent(e.detail.content);
});
// Close modal on backdrop click
this.modal.addEventListener('click', () => {
this.modal.open = false;
});
}
async showRowDetail(tableName, seq) {
try {
const data = await tableAPI.getRow(tableName, seq);
const content = JSON.stringify(data, null, 2);
this.modal.title = `Row Detail - Seq: ${seq}`;
this.modal.content = content;
this.modal.open = true;
} catch (error) {
console.error('Error loading row detail:', error);
this.modal.title = 'Error';
this.modal.content = `Failed to load row detail: ${error.message}`;
this.modal.open = true;
}
}
showCellContent(content) {
this.modal.title = 'Cell Content';
this.modal.content = String(content);
this.modal.open = true;
}
}
// Initialize app when DOM is ready
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', () => new App());
} else {
new App();
}
// 渲染应用
render(html`<${App} />`, document.getElementById('app'));

View File

@@ -0,0 +1,110 @@
// api.js - 统一的 API 服务层
/**
* 获取 API 基础路径
* 从全局变量 window.API_BASE 读取,由服务端渲染时注入
*/
function getBasePath() {
return window.API_BASE || '';
}
/**
* 构建完整的 API URL
* @param {string} path - API 路径,如 '/tables' 或 '/tables/users/schema'
* @returns {string} 完整 URL
*/
function buildApiUrl(path) {
const basePath = getBasePath();
const normalizedPath = path.startsWith('/') ? path : '/' + path;
const fullPath = basePath ? `${basePath}/api${normalizedPath}` : `/api${normalizedPath}`;
console.log('[API] Request:', fullPath);
return fullPath;
}
/**
* 统一的 fetch 封装
* @param {string} path - API 路径
* @param {object} options - fetch 选项
* @returns {Promise<Response>}
*/
async function apiFetch(path, options = {}) {
const url = buildApiUrl(path);
const response = await fetch(url, {
...options,
headers: {
'Content-Type': 'application/json',
...options.headers
}
});
if (!response.ok) {
console.error('[API] Error:', response.status, url);
}
return response;
}
// ============ 表相关 API ============
/**
* 获取所有表列表
* @returns {Promise<{tables: Array}>}
*/
export async function getTables() {
const response = await apiFetch('/tables');
return response.json();
}
/**
* 获取表的 Schema
* @param {string} tableName - 表名
* @returns {Promise<{name: string, fields: Array}>}
*/
export async function getTableSchema(tableName) {
const response = await apiFetch(`/tables/${tableName}/schema`);
return response.json();
}
/**
* 获取表数据(分页)
* @param {string} tableName - 表名
* @param {object} params - 查询参数
* @param {number} params.limit - 每页数量
* @param {number} params.offset - 偏移量
* @param {string} params.select - 选择的字段(逗号分隔)
* @returns {Promise<{data: Array, totalRows: number}>}
*/
export async function getTableData(tableName, params = {}) {
const { limit = 20, offset = 0, select } = params;
const queryParams = new URLSearchParams();
queryParams.append('limit', limit);
queryParams.append('offset', offset);
if (select) {
queryParams.append('select', select);
}
const response = await apiFetch(`/tables/${tableName}/data?${queryParams}`);
return response.json();
}
/**
* 根据序列号获取单条数据
* @param {string} tableName - 表名
* @param {number} seq - 序列号
* @returns {Promise<object>}
*/
export async function getRowBySeq(tableName, seq) {
const response = await apiFetch(`/tables/${tableName}/data/${seq}`);
return response.json();
}
/**
* 获取表的 Manifest 信息
* @param {string} tableName - 表名
* @returns {Promise<object>}
*/
export async function getTableManifest(tableName) {
const response = await apiFetch(`/tables/${tableName}/manifest`);
return response.json();
}

View File

@@ -17,15 +17,21 @@ import (
//go:embed static
var staticFS embed.FS
// WebUI Web 界面处理器
// WebUI Web 界面处理器 v2 (Preact)
type WebUI struct {
db *srdb.Database
basePath string
handler http.Handler
}
// NewWebUI 创建 WebUI 实例
func NewWebUI(db *srdb.Database) *WebUI {
ui := &WebUI{db: db}
// NewWebUI 创建 WebUI v2 实例
// basePath 是可选的 URL 前缀路径(例如 "/debug"),如果为空则使用根路径 "/"
func NewWebUI(db *srdb.Database, basePath ...string) *WebUI {
bp := ""
if len(basePath) > 0 && basePath[0] != "" {
bp = strings.TrimSuffix(basePath[0], "/") // 移除尾部斜杠
}
ui := &WebUI{db: db, basePath: bp}
ui.handler = ui.setupHandler()
return ui
}
@@ -34,20 +40,33 @@ func NewWebUI(db *srdb.Database) *WebUI {
func (ui *WebUI) setupHandler() http.Handler {
mux := http.NewServeMux()
// API endpoints - 纯 JSON API
mux.HandleFunc("/api/tables", ui.handleListTables)
mux.HandleFunc("/api/tables/", ui.handleTableAPI)
// API endpoints - 纯 JSON API(与 v1 共享)
mux.HandleFunc(ui.path("/api/tables"), ui.handleListTables)
mux.HandleFunc(ui.path("/api/tables/"), ui.handleTableAPI)
// 静态文件服务
staticFiles, _ := fs.Sub(staticFS, "static")
mux.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.FS(staticFiles))))
staticPath := ui.path("/static/")
mux.Handle(staticPath, http.StripPrefix(staticPath, http.FileServer(http.FS(staticFiles))))
// 首页
mux.HandleFunc("/", ui.handleIndex)
mux.HandleFunc(ui.path("/"), ui.handleIndex)
return mux
}
// path 返回带 basePath 前缀的路径
func (ui *WebUI) path(p string) string {
if ui.basePath == "" {
return p
}
// 确保路径以 / 开头
if !strings.HasPrefix(p, "/") {
p = "/" + p
}
return ui.basePath + p
}
// Handler 返回 HTTP Handler
func (ui *WebUI) Handler() http.Handler {
return ui.handler
@@ -74,6 +93,7 @@ func (ui *WebUI) handleListTables(w http.ResponseWriter, r *http.Request) {
type TableListItem struct {
Name string `json:"name"`
RowCount int64 `json:"row_count,omitempty"`
CreatedAt int64 `json:"created_at"`
Fields []FieldInfo `json:"fields"`
}
@@ -92,8 +112,18 @@ func (ui *WebUI) handleListTables(w http.ResponseWriter, r *http.Request) {
})
}
// 尝试获取行数(通过快速查询)
rowCount := int64(0)
if rows, err := table.Query().Rows(); err == nil {
for rows.Next() {
rowCount++
}
rows.Close()
}
tables = append(tables, TableListItem{
Name: name,
RowCount: rowCount,
CreatedAt: 0, // TODO: Table 不再有 createdAt 字段
Fields: fields,
})
@@ -105,13 +135,20 @@ func (ui *WebUI) handleListTables(w http.ResponseWriter, r *http.Request) {
})
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(tables)
json.NewEncoder(w).Encode(map[string]any{
"tables": tables,
})
}
// handleTableAPI 处理表相关的 API 请求
func (ui *WebUI) handleTableAPI(w http.ResponseWriter, r *http.Request) {
// 解析路径: /api/tables/{name}/schema 或 /api/tables/{name}/data
path := strings.TrimPrefix(r.URL.Path, "/api/tables/")
// 需要先移除 basePath再移除 /api/tables/ 前缀
path := r.URL.Path
if ui.basePath != "" {
path = strings.TrimPrefix(path, ui.basePath)
}
path = strings.TrimPrefix(path, "/api/tables/")
parts := strings.Split(path, "/")
if len(parts) < 2 {
@@ -214,12 +251,12 @@ func (ui *WebUI) handleTableManifest(w http.ResponseWriter, r *http.Request, tab
Files []FileInfo `json:"files"`
}
// 获取 Compaction Manager 和 Picker
// 获取 Compaction Manager
compactionMgr := table.GetCompactionManager()
picker := compactionMgr.GetPicker()
levels := make([]LevelInfo, 0)
for level := range 7 {
// 只显示 L0-L3 层
levels := make([]LevelInfo, 0, 4)
for level := range 4 {
files := version.GetLevel(level)
totalSize := int64(0)
@@ -237,8 +274,11 @@ func (ui *WebUI) handleTableManifest(w http.ResponseWriter, r *http.Request, tab
}
score := 0.0
if len(files) > 0 {
score = picker.GetLevelScore(version, level)
if len(files) > 0 && level < 3 {
nextLevelLimit := compactionMgr.GetLevelSizeLimit(level + 1)
if nextLevelLimit > 0 {
score = float64(totalSize) / float64(nextLevelLimit)
}
}
levels = append(levels, LevelInfo{
@@ -314,23 +354,23 @@ func (ui *WebUI) handleTableData(w http.ResponseWriter, r *http.Request, tableNa
return
}
// 解析分页参数
pageStr := r.URL.Query().Get("page")
pageSizeStr := r.URL.Query().Get("pageSize")
// 解析查询参数
limitStr := r.URL.Query().Get("limit")
offsetStr := r.URL.Query().Get("offset")
selectParam := r.URL.Query().Get("select") // 要选择的字段,逗号分隔
page := 1
pageSize := 20
limit := 100
offset := 0
if pageStr != "" {
if p, err := strconv.Atoi(pageStr); err == nil && p > 0 {
page = p
if limitStr != "" {
if l, err := strconv.Atoi(limitStr); err == nil && l > 0 && l <= 1000 {
limit = l
}
}
if pageSizeStr != "" {
if ps, err := strconv.Atoi(pageSizeStr); err == nil && ps > 0 && ps <= 1000 {
pageSize = ps
if offsetStr != "" {
if o, err := strconv.Atoi(offsetStr); err == nil && o >= 0 {
offset = o
}
}
@@ -338,19 +378,17 @@ func (ui *WebUI) handleTableData(w http.ResponseWriter, r *http.Request, tableNa
var selectedFields []string
if selectParam != "" {
selectedFields = strings.Split(selectParam, ",")
// 清理字段名(去除空格)
for i := range selectedFields {
selectedFields[i] = strings.TrimSpace(selectedFields[i])
}
}
// 获取 schema 用于字段类型判断
// 获取 schema
tableSchema := table.GetSchema()
// 使用 Query API 获取数据,如果指定了字段则只查询指定字段(按字段压缩优化)
// 使用 Query API 获取数据
queryBuilder := table.Query()
if len(selectedFields) > 0 {
// 确保 _seq 和 _time 总是被查询(用于构造响应)
fieldsWithMeta := make([]string, 0, len(selectedFields)+2)
hasSeq := false
hasTime := false
@@ -372,6 +410,7 @@ func (ui *WebUI) handleTableData(w http.ResponseWriter, r *http.Request, tableNa
queryBuilder = queryBuilder.Select(fieldsWithMeta...)
}
queryRows, err := queryBuilder.Rows()
if err != nil {
http.Error(w, fmt.Sprintf("Failed to query table: %v", err), http.StatusInternalServerError)
@@ -379,26 +418,23 @@ func (ui *WebUI) handleTableData(w http.ResponseWriter, r *http.Request, tableNa
}
defer queryRows.Close()
// 计算分页范围
offset := (page - 1) * pageSize
// 收集数据
const maxStringLength = 100
data := make([]map[string]any, 0, limit)
currentIndex := 0
// 直接在遍历时进行分页和字段处理
const maxStringLength = 100 // 最大字符串长度
data := make([]map[string]any, 0, pageSize)
totalRows := int64(0)
for queryRows.Next() {
totalRows++
// 跳过不在当前页的数据
// 跳过 offset 之前的数据
if currentIndex < offset {
currentIndex++
continue
}
// 已经收集够当前页的数据
if len(data) >= pageSize {
// 已经收集够数据
if len(data) >= limit {
continue
}
@@ -434,11 +470,10 @@ func (ui *WebUI) handleTableData(w http.ResponseWriter, r *http.Request, tableNa
}
response := map[string]any{
"data": data,
"page": page,
"pageSize": pageSize,
"totalRows": totalRows,
"totalPages": (totalRows + int64(pageSize) - 1) / int64(pageSize),
"data": data,
"limit": limit,
"offset": offset,
"totalRows": totalRows,
}
w.Header().Set("Content-Type", "application/json")
@@ -447,7 +482,9 @@ func (ui *WebUI) handleTableData(w http.ResponseWriter, r *http.Request, tableNa
// handleIndex 处理首页请求
func (ui *WebUI) handleIndex(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
// 检查路径是否匹配(支持 basePath
expectedPath := ui.path("/")
if r.URL.Path != expectedPath {
http.NotFound(w, r)
return
}
@@ -460,6 +497,25 @@ func (ui *WebUI) handleIndex(w http.ResponseWriter, r *http.Request) {
return
}
// 替换路径占位符 ~~ 为 basePath
// 如果 basePath 为空,则 ~~ → ""
// 如果 basePath 为 "/debug",则 ~~ → "/debug"
// 示例:
// - 无 basePath: href="~~/static/css/styles.css" → href="/static/css/styles.css"
// - 有 basePath: href="~~/static/css/styles.css" → href="/debug/static/css/styles.css"
htmlContent := string(content)
if ui.basePath == "" {
// 无 basePath 时,直接替换为空
htmlContent = strings.ReplaceAll(htmlContent, `"~~"`, `""`)
htmlContent = strings.ReplaceAll(htmlContent, `"~~/`, `"/`)
htmlContent = strings.ReplaceAll(htmlContent, `'~~/`, `'/`)
} else {
// 有 basePath 时,替换为实际的 basePath
htmlContent = strings.ReplaceAll(htmlContent, `"~~"`, fmt.Sprintf(`"%s"`, ui.basePath))
htmlContent = strings.ReplaceAll(htmlContent, `"~~/`, fmt.Sprintf(`"%s/`, ui.basePath))
htmlContent = strings.ReplaceAll(htmlContent, `'~~/`, fmt.Sprintf(`'%s/`, ui.basePath))
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.Write(content)
w.Write([]byte(htmlContent))
}