Hi makers! I ve been obsessed with storage performance lately. We keep buying 7GB/s NVMe drives, but most databases still use architecture (like LSM-trees) designed for a time when disks were the bottleneck.
Background compactions, write-amplification spikes, and P99 latency jitters seem to be the "tax" we pay for general-purpose storage. Is it time we move toward memory-first, zero-compaction engines, even if it means trading off some RAM? Or is the LSM-tree still the king of the hill? Would love to hear from anyone fighting database latency at scale!
Hi Product Hunt! I’m Shubham, Co-founder of VeltrixDB.
The Problem: Managing huge key-value stores at scale is frustrating. You provision expensive, blazing-fast NVMe drives, but legacy LSM-tree databases (like RocksDB or Pebble) choke your hardware with heavy background compactions and massive write amplification. Write-stalls and latency spikes become inevitable.
The Solution: We wanted a storage engine that stops fighting the hardware. So, we threw out the LSM-tree entirely. VeltrixDB uses an in-memory Adaptive Radix Tree (ART) index paired with an append-only disk log. By utilising io_uring and O_DIRECT, we write straight to the NVMe at its physical limits—achieving zero background sorting, zero compaction stutter, and a strict ~1.0x write amplification.
The Trade-offs: I want to be brutally honest with developers—this isn't a drop-in SQL replacement.
It is memory-bound: The entire index lives in RAM (~80 bytes/key).
No range queries: It is built strictly for high-speed point lookups.
Bare-metal only: It bypasses OS caches, meaning it will crawl on standard cloud EBS volumes.
It is a hyper-specialised scalpel built for AdTech (RTB), High-Frequency Trading, and extreme session caching where microsecond point-lookup speed is everything.
For the tech-savvy folks, we've open-sourced our benchmarks here: https://github.com/VeltrixDB/veltrixdb-benchmark
I’d love to answer any deep architectural questions or hear about the database scaling bottlenecks you are currently dealing with! Fire away in the comments.