Imagine a team building a service that runs at the edge, with Go as its main language. Metadata, task cursors and a deduplication table all sit in one embedded store. Two years ago the team picked RocksDB for a plain reason: it is mature, it has a reputation for performance, and plenty of people use it. The service ran in x86 containers for more than a year without a single storage incident.
Then came a delivery. The customer wanted the same code running on an ARM board, with a statically linked binary and no package manager on site. The team started working through cgo cross-compilation: toolchain, system library versions, link order, one item at a time. Two weeks later the build still succeeded only on the development machine. Someone suggested swapping in a pure Go store. Someone else pointed out that LMDB is a pure C library, so static linking ought to be less trouble. A third person said leave it alone and grind the build problem down.
All three directions can work. What differs is the bill attached to each. This piece tries to lay those bills out in the open: first what RocksDB actually provides, then what Pebble and LMDB each trade away, and finally the question most often skipped, whether this is a change worth making at all.
What RocksDB gives you, and what comes with it
RocksDB is developed and maintained by Facebook’s database engineering team, built on top of Google’s LevelDB, which came out of work by Sanjay Ghemawat and Jeff Dean. It is an embedded, persistent key-value store with no separate server process, linked straight into the application. The public interface lives in the include/ directory, and internal headers may change without notice.
The storage structure is LSM, log-structured merge. Writes land in an in-memory structure and a log, then flush into sorted files, and background compaction merges those files level by level. The design lets you trade write amplification, read amplification and space amplification against one another. Compact often and reads touch fewer levels while writes move more data around. Compact lazily and writes get cheaper while reads have more levels to search. It supports multi-threaded compaction, is built to hold several terabytes in one database, and the project stresses that it suits flash storage in particular.
Those capabilities are paid for in complexity. There are many parameters, a compaction strategy to choose, a memory budget to tune, and when something goes wrong you need to read the level state of an LSM to understand it. For a service holding a few dozen gigabytes of metadata, with far more reads than writes, most of those knobs sit idle while the build burden stays: cgo, a C++ toolchain, platform-specific compile flags, and a fresh round of all of it every time the target platform changes.
Licensing, and where the worry usually lands
Legal asking about licensing is a familiar scene in embedded store selection, and the question usually lands in the wrong place. RocksDB ships under a dual license, GPLv2 and Apache 2.0, and the user picks one. That detail matters. Choose Apache 2.0 and you do not have to treat your own code under GPL’s copyleft terms. So if the only worry is whether RocksDB will contaminate closed source, the answer is usually no, provided the team explicitly chooses Apache 2.0 and writes that choice into its compliance record.
The harder items sit elsewhere. One is build and distribution. Once you link statically, a large block of C++ travels inside your binary, which adds size and adds dependencies to audit. The other is knowledge. A Go team has to maintain a set of LSM tuning habits, and that knowledge is hard to hand over when people leave.
Put licensing back where it belongs and the rest of the comparison starts to carry weight.
Pebble, the same file format in another language
Pebble is a Go key-value database written by the CockroachDB team, inspired by LevelDB and RocksDB, with a clear goal: performance, and serving CockroachDB’s own needs. It inherits RocksDB’s file format along with a few extensions, among them range deletion tombstones, table-level bloom filters and updates to the MANIFEST format.
It deliberately does not implement everything RocksDB does. What exists covers block-based tables, checkpoints, indexed batches, iterator options, level-based compaction, manual compaction, merge operators, prefix bloom filters, prefix iteration, range deletion, reverse iteration, SSTable import, single delete, snapshots and table-level bloom filters. What is missing includes backups, column families, range file deletion, FIFO compaction, forward and tailing iterators, the hash table format, memtable bloom filters, a persistent cache, pinning iterator keys and values, the plain table format, SSTable ingest-behind, sub-compactions, transactions and universal compaction.
That list deserves a line-by-line check against your own usage. If a team uses column families for logical isolation, or the backup interface for periodic snapshots, then what Pebble lacks is a load-bearing piece, not an optional convenience.
Pebble also improves on RocksDB in several places. The project lists a memtable skiplist with backward links for faster reverse iteration, a faster commit pipeline and better concurrency, indexed batches that merge with iteration, L0 sublevels with flush splitting, a copy-on-write B-tree holding file metadata, delete-only compaction, block-property collectors and filters, a range keys API, and a smaller, more readable codebase. Those improvements follow CockroachDB’s requirements, so the closer your workload resembles theirs, the more the list tells you.
As CockroachDB’s storage engine, Pebble arrived in v20.1, released in May 2020, and became the default in v20.2, released in November 2020. The project states it has been in production and stable. That history shows it can carry the write pressure of a distributed database, and it also shows that its feature boundary was drawn by one product’s requirements.
Compatibility is narrower than the name suggests
Reading that Pebble inherits the RocksDB file format, it is tempting to point it at an existing RocksDB data directory. Take that step with care.
Pebble v1 aims for forward compatibility with RocksDB 6.2.1. Version 2 and later no longer support opening a database written by RocksDB directly. You first have to run a format major version upgrade through an older Pebble. That upgrade is permanent, and once it happens there is no going back to an earlier format.
Even on a compatible version, the coverage is only the subset of features CockroachDB uses. The project warns about this in plain terms: if Pebble opens a RocksDB database that uses features it does not support, Pebble may silently corrupt the data or behave incorrectly. Silent corruption is the sharpest word in that sentence, because it means the error does not surface at open time. It shows up later as missing or scrambled data on some read.
One more limit: SSTable format versions 3 and 4 are not supported. If the existing database was written in either of those formats, the compatibility discussion ends there.
So a Pebble migration has to be run as a formal data migration, not as a library swap followed by a reopen. A sensible order is to confirm feature coverage on the old format, rehearse the format upgrade, compare data before and after with checksum tools, and only then move traffic. That stretch of work usually costs more than the code change.
LMDB, the whole database as mapped memory
Look from another direction. LMDB is a B-tree database management library whose API follows BerkeleyDB in rough outline but is much simplified. The part that sets it apart is that the entire database is mapped into memory. Reads return pointers into that mapped memory, with no malloc and no memcpy during the read, and the library keeps no page cache layer of its own.
That matters a great deal when reads dominate. Reasoning from the structure, skipping the copy from page cache into a user buffer should show up as lower per-call cost on a path of frequent small reads, and when the data sits in the page cache a read comes down to pointer arithmetic. This is an inference from how the library is built, not a benchmark result. For your own data size and access distribution, run a test.
LMDB provides full ACID transactions. The default is a read-only mapping, so a stray pointer write in application code cannot reach the database, which is a practical benefit in projects mixing C and C++. Open with MDB_WRITEMAP for a read-write mapping and writes get faster, at the cost of a stray pointer write being able to damage the database without warning.
Data pages are copy-on-write, so live pages are never overwritten and a crash needs no dedicated recovery procedure. There is no write-ahead log and no append-only design, which means no checkpoint and no compaction while the service runs. Free pages are tracked and reused by the library, and in normal use the file does not grow without bound. For a team that treats operating an LSM as a burden, that paragraph reads close to an ideal answer.
One writer, many readers, and a few traps
LMDB’s concurrency model is simple. Write transactions are fully serialized, with exactly one at a time, so writers cannot deadlock against each other. The multi-version structure keeps readers lock-free: a writer does not block readers and readers do not block a writer. The library is thread-safe and supports concurrent reads and writes across multiple processes and threads.
The cost lands on the write side. Writes cannot run in parallel, so single-machine write throughput is pinned by that one serialized writer. If a team’s load is a flood of small writes, say a constant stream of events hitting disk, the LSM habit of merging writes into large sequential IO fits better. If instead the load is mostly reads with the occasional metadata update, serialized writes are nowhere near the bottleneck, and LMDB’s structure removes the read jitter and background IO that compaction brings.
A few cautions are written into the official headers and worth knowing early. A stale read transaction, left behind by a process that died badly, can make later writes grow the database quickly, so check periodically with mdb_reader_check or mdb_stat. A thread may hold only one transaction at a time, and sub-transactions count as part of the same one. After a fork, do not use an MDB_env opened before the fork in the new child process. Do not open the same LMDB database twice inside one process. The configuration often reserves a large unused address space and a possible file size, and that reservation occupies neither real memory nor disk. There is usually no pure read-only mode, because readers need write permission on the lock file, with a read-only filesystem or MDB_NOLOCK as the exceptions.
That last point is easy to hit in container deployments: mount the data directory read-only and readers will fail to start.
On licensing, LMDB uses the OpenLDAP Public License 2.8, dated August 17, 2003. The short version: redistributing source requires keeping the copyright notice; redistributing binaries requires reproducing the applicable copyright notice, list of conditions and disclaimer in the documentation; redistribution must include a verbatim copy of the license; and the author’s name may not be used for advertising. It is a permissive license in the usual sense, with obligations concentrated on keeping notices intact, and it is easier to work with than GPL copyleft terms.
The three side by side
The differences described so far are easier to read in one table. Each cell holds a confirmed fact only, and anything not established is left as N/A rather than filled in from impression.
| Item | RocksDB | Pebble | LMDB |
|---|---|---|---|
| Implementation language | C++ | Go | C |
| Storage structure | LSM | LSM, inherits the RocksDB file format | Copy-on-write B-tree, whole database memory-mapped |
| License | Dual GPLv2 and Apache 2.0, choose one | N/A | OpenLDAP Public License 2.8 |
| Compaction required | Yes, multi-threaded compaction | Yes, including level-based compaction | No, and no checkpoint either |
| Concurrency model | Multi-threaded compaction | Faster commit pipeline, better concurrency | Write transactions serialized, readers lock-free and non-blocking toward writers |
| Compatible with RocksDB data files | It is the RocksDB format | v1 aims at RocksDB 6.2.1; v2 and later cannot open it directly | No, the data files are entirely different |
The row easiest to skim past is the one about compaction. It looks like an implementation detail, but it decides the shape of operations. With compaction you get background IO, swings in space amplification, and parameters to tune. Without it you get none of those, and you pay on the write side with serialization. The other row worth pausing on is compatibility, which decides whether a migration means changing a connection string or moving the data.
Most of the migration bill sits outside the code
Pebble’s migration cost concentrates in two places. Feature coverage needs checking item by item, especially against the unimplemented list above. The data files need a format upgrade, and that upgrade is one way. If a database has been in use for years, holding data written across different periods, then before switching it is worth running the upgrade, the checks and the rollback plan on a copy. A rollback plan on the format upgrade path is thin by nature, since the new format cannot go back, so the real insurance is a snapshot taken before the upgrade.
LMDB’s migration cost is plainer. The file format is entirely different, there is no compatibility path, and the data has to be exported and imported. In between sits a full export, an import and a key-by-key comparison. The larger the dataset, the earlier that move needs a scheduled downtime window; a small dataset makes it light work. After the import, revisit the access patterns. Range queries that relied on LSM prefix scans usually translate fine to a B-tree, but if the code depends on column families or transaction semantics, the design is what has to change.
RocksDB has migration costs of its own, except this team has already paid them. Tuning experience has accumulated, and monitoring metrics, alert thresholds and incident playbooks were all built around it. Replace it and that work is rebuilt from scratch. This part is routinely undervalued during evaluation.
When staying on RocksDB is the right answer
Several situations make a switch hard to justify.
The database has reached several terabytes, write throughput has to be high, and the write pattern is a mass of small writes. This is where the LSM advantage of turning random writes into sequential ones is clearest, and a B-tree structure would face write amplification of a different shape.
The workload uses features Pebble does not implement, such as column families, the backup interface or transactions. In that case Pebble is out of the running and there is no performance comparison left to have.
A working operations practice already exists, with someone on the team who understands LSM tuning and failure analysis well. Swapping the store at that point is digging up a foundation that has already settled.
On the other side, if the dataset sits between a few dozen gigabytes and a few hundred, reads far outnumber writes, and the team writes Go while the C++ toolchain is pure overhead, the arithmetic starts to balance.
How to choose
Put the three paths together and the trade-offs line up.
Staying on RocksDB is usually about scale, write pressure and accumulated operational knowledge. It has the widest feature set, the dual license leaves room on compliance, and the price is build complexity plus tuning effort. If the immediate pain is only cross-compilation, fixing the build is usually far cheaper than replacing the store.
Pebble makes sense when the project is written in Go and the features in use fall inside its implemented list. It inherits RocksDB’s file format and part of its design, and its storage structure belongs to the same LSM family. Two things have to be accepted: the feature set is a subset, and data migration goes through a format upgrade that cannot be reversed. If the existing database depends on anything unimplemented, or the team must interoperate with the RocksDB tooling over the long term, it does not fit.
LMDB makes sense when reads dominate, writes can be serialized, the dataset fits inside what one memory mapping can cover, and the team accepts a C dependency. It shrinks the operational surface to almost nothing: no compaction, no checkpoint, no recovery procedure after a crash. What has to be accepted is serialized writes, usually no pure read-only mode, and those constraints around transactions and fork.
If a sequence has to be forced, start with data size and write pattern, which decides LSM or B-tree. Then check feature coverage, which decides whether Pebble is viable. Then weigh build and operational cost, which decides whether a switch is worth it. After those three steps there is usually one answer left standing.
As for performance, every statement in this piece about speed is reasoned from storage structure. None of it comes from a measured benchmark. To make a real decision, run your own data and access distribution through a test round; that beats any comparison table.
References
The RocksDB repository is at https://github.com/meta/rocksdb, Pebble at https://github.com/cockroachdb/pebble, and a read-only mirror of LMDB at https://github.com/LMDB/lmdb, with the official LMDB repository hosted at openldap.org. The descriptions above of feature lists, compatibility boundaries and license terms all come from those three public sources.
Related reading
- FastGPT Alternatives 2026: Dify, Langflow, RAGFlow for a Self-Hosted Knowledge Base Q&A Platform
- SmsForwarder Alternatives: Choosing Between Bark, ntfy and Pushover for Phone Notification Forwarding
- Hasura Alternatives: PostGraphile, Supabase, and Directus for Auto-Generated APIs
- Beyond Reflex: Choosing Between Streamlit, NiceGUI and Gradio for Pure-Python Web Apps
- Gitleaks Alternatives: How to Choose a Secret Scanner for Your Repos
Stay updated with our latest AI insights
