grogudb: 一个为高频 Put/Has/Del/Range 操作而设计的持久化 KV 数据库

2023-04-12 11:12:24 +08:00
 chenjiandongx

项目地址: https://github.com/chenjiandongx/grogudb

设计文档

特性

  • 纯 Go 实现,可内嵌进程序中。
  • 高效的 Put/Has/Del/Range 操作。
  • 线程安全。
  • 允许存储超过物理内存的数据。
  • 简洁的 API 。

压测

Iter: 10k, Bucket: 100 => 1M key

Storage: grogudb    Op: PutUnique       Elapsed: 1.587680726s   Ops: 629849.555785/s
Storage: leveldb    Op: PutUnique       Elapsed: 3.207660071s   Ops: 311753.732586/s
Storage: badger     Op: PutUnique       Elapsed: 3.713279852s   Ops: 269303.699117/s
Storage: grogudb    Op: PutDuplicate    Elapsed: 809.645321ms   Ops: 1235108.724849/s
Storage: leveldb    Op: PutDuplicate    Elapsed: 3.344680804s   Ops: 298982.192502/s
Storage: badger     Op: PutDuplicate    Elapsed: 3.809289718s   Ops: 262516.131360/s
Storage: grogudb    Op: PutIf           Elapsed: 179.746041ms   Ops: 5563404.870764/s
Storage: leveldb    Op: PutIf           Elapsed: 512.160806ms   Ops: 1952511.766392/s
Storage: badger     Op: PutIf           Elapsed: 1.063730519s   Ops: 940087.721597/s
Storage: grogudb    Op: Has             Elapsed: 79.718185ms    Ops: 12544189.258699/s
Storage: leveldb    Op: Has             Elapsed: 1.188825549s   Ops: 841166.309759/s
Storage: badger     Op: Has             Elapsed: 1.443558895s   Ops: 692732.387618/s
Storage: grogudb    Op: Del             Elapsed: 1.25951208s    Ops: 793958.244529/s
Storage: leveldb    Op: Del             Elapsed: 3.471029382s   Ops: 288098.972940/s`
Storage: badger     Op: Del             Elapsed: 4.524956978s   Ops: 220996.576291/s
Storage: grogudb    Op: Range           Elapsed: 81.139301ms    Ops: 12.324484/s
Storage: leveldb    Op: Range           Elapsed: 71.821588ms    Ops: 13.923390/s
Storage: badger     Op: Range           Elapsed: 295.666737ms   Ops: 3.382186/s
Storage: grogudb    Op: Get             Elapsed: 26.561270284s  Ops: 37648.801782/s
Storage: leveldb    Op: Get             Elapsed: 1.080395935s   Ops: 925586.599879/s
Storage: badger     Op: Get             Elapsed: 1.423728937s   Ops: 702380.891483/s

Iter: 100k, Bucket: 100 => 10M key (Without grogudb)

Storage: grogudb    Op: PutUnique       Elapsed: 16.423032579s  Ops: 608900.941522/s
Storage: leveldb    Op: PutUnique       Elapsed: 51.516953146s  Ops: 194110.858452/s
Storage: badger     Op: PutUnique       Elapsed: 42.421363992s  Ops: 235730.279721/s
Storage: grogudb    Op: PutDuplicate    Elapsed: 8.815478924s   Ops: 1134368.318070/s
Storage: leveldb    Op: PutDuplicate    Elapsed: 39.615313747s  Ops: 252427.636037/s
Storage: badger     Op: PutDuplicate    Elapsed: 47.31107471s   Ops: 211367.001517/s
Storage: grogudb    Op: PutIf           Elapsed: 2.299923889s   Ops: 4347969.968844/s
Storage: leveldb    Op: PutIf           Elapsed: 5.870490731s   Ops: 1703435.105892/s
Storage: badger     Op: PutIf           Elapsed: 15.958825217s  Ops: 626612.539709/s
Storage: grogudb    Op: Has             Elapsed: 850.056456ms   Ops: 11763924.536325/s
Storage: leveldb    Op: Has             Elapsed: 19.188154981s  Ops: 521154.848390/s
Storage: badger     Op: Has             Elapsed: 22.721393642s  Ops: 440113.848541/s
Storage: grogudb    Op: Del             Elapsed: 12.924122561s  Ops: 773746.918044/s
Storage: leveldb    Op: Del             Elapsed: 44.487984603s  Ops: 224779.793673/s
Storage: badger     Op: Del             Elapsed: 44.315291044s  Ops: 225655.744652/s
Storage: grogudb    Op: Range           Elapsed: 524.850653ms   Ops: 1.905304/s
Storage: leveldb    Op: Range           Elapsed: 1.177148523s   Ops: 0.849510/s
Storage: badger     Op: Range           Elapsed: 3.15658723s    Ops: 0.316798/s
924 次点击
所在节点    分享创造
5 条回复
Kould
2023-04-12 11:14:39 +08:00
好巧,我也在写存储引擎: https://github.com/KKould/KipDB
danbai
2023-04-17 18:26:49 +08:00
好巧,我刚好在看曼达洛人
chenjiandongx
2023-04-25 11:03:49 +08:00
@danbai This is the way
chenjiandongx
2023-04-25 11:04:20 +08:00
@Kould 👍🏻
danbai
2023-04-25 11:37:40 +08:00
@chenjiandongx This is the way

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/931851

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX