baoyinhe
V2EX  ›  问与答

现在存储向量数据库都用什么,哪种是未来的趋势?

  •  
  •   baoyinhe · May 6, 2023 · 2809 views
    This topic created in 1147 days ago, the information mentioned may be changed or developed.
    vivisidea
        1
    vivisidea  
       May 6, 2023   ❤️ 1
    刚好在隔壁看一篇关于用 openai embedding 来做检索的文章,里面有提到这个

    向量数据库选项包括:

    Pinecone ,完全托管的向量数据库
    Weaviate ,开源向量搜索引擎
    Redis ,向量数据库
    Qdrant ,向量搜索引擎
    Milvus ,用于可扩展相似性搜索的向量数据库
    Chroma ,开源的嵌入向量存储

    source
    https://zhuanlan.zhihu.com/p/619233637
    baoyinhe
        2
    baoyinhe  
    OP
       May 6, 2023
    @vivisidea #1 有用
    lchynn
        3
    lchynn  
       May 6, 2023
    好奇,redis 内存数据库来做向量数据存储,那么内存如果存到后面不够了,oom 了咋办? 生产业务允许矢量化的数据丢失么? 对应到 embedding 的 gpt 例子, 好比你要查找一堆文本有关相似性的业务数据都没了?
    vivisidea
        4
    vivisidea  
       May 6, 2023
    @lchynn 我没仔细看 redis 检索的实现原理,但在 kv 应用场景,一般可以控制 key 的驱逐策略,比如可以用 noeviction 策略,拒绝新的写入,或者 LRU 把最老的 evict 掉

    https://redis.io/docs/reference/eviction/

    The exact behavior Redis follows when the maxmemory limit is reached is configured using the maxmemory-policy configuration directive.

    The following policies are available:

    noeviction: New values aren’t saved when memory limit is reached. When a database uses replication, this applies to the primary database
    allkeys-lru: Keeps most recently used keys; removes least recently used (LRU) keys
    allkeys-lfu: Keeps frequently used keys; removes least frequently used (LFU) keys
    volatile-lru: Removes least recently used keys with the expire field set to true.
    volatile-lfu: Removes least frequently used keys with the expire field set to true.
    allkeys-random: Randomly removes keys to make space for the new data added.
    volatile-random: Randomly removes keys with expire field set to true.
    volatile-ttl: Removes keys with expire field set to true and the shortest remaining time-to-live (TTL) value.
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   1168 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 32ms · UTC 23:57 · PVG 07:57 · LAX 16:57 · JFK 19:57
    ♥ Do have faith in what you're doing.