README.md raw

BadgerDB

Go Reference Go Report Card Sourcegraph ci-badger-tests ci-badger-bank-tests ci-badger-bank-tests-nightly

Badger mascot

BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go. It is the underlying database for Dgraph, a fast, distributed graph database. It's meant to be a performant alternative to non-Go-based key-value stores like RocksDB.

Project Status

Badger is stable and is being used to serve data sets worth hundreds of terabytes. Badger supports concurrent ACID transactions with serializable snapshot isolation (SSI) guarantees. A Jepsen-style bank test runs nightly for 8h, with --race flag and ensures the maintenance of transactional guarantees. Badger has also been tested to work with filesystem level anomalies, to ensure persistence and consistency. Badger is being used by a number of projects which includes Dgraph, Jaeger Tracing, UsenetExpress, and many more.

The list of projects using Badger can be found here.

Please consult the [Changelog] for more detailed information on releases.

[Changelog]: https://github.com/hypermodeinc/badger/blob/main/CHANGELOG.md

Table of Contents

- Project Status - Table of Contents - Getting Started - Installing - Installing Badger Command Line Tool - Choosing a version - Badger Documentation - Resources - Blog Posts - Design - Comparisons - Benchmarks - Projects Using Badger - Contributing - Contact

Getting Started

Installing

To start using Badger, install Go 1.21 or above. Badger v3 and above needs go modules. From your project, run the following command

go get github.com/dgraph-io/badger/v4

This will retrieve the library.

Installing Badger Command Line Tool

Badger provides a CLI tool which can perform certain operations like offline backup/restore. To install the Badger CLI, retrieve the repository and checkout the desired version. Then run

cd badger
go install .

This will install the badger command line utility into your $GOBIN path.

Badger Documentation

Badger Documentation is available at https://docs.hypermode.com/badger

Resources

Blog Posts

  1. Introducing Badger: A fast key-value store written natively in Go
  2. Make Badger crash resilient with ALICE
  3. Badger vs LMDB vs BoltDB: Benchmarking key-value databases in Go
  4. Concurrent ACID Transactions in Badger

Design

Badger was written with these design goals in mind:

Badger’s design is based on a paper titled _[WiscKey: Separating Keys from Values in SSD-conscious Storage][wisckey]_.

[wisckey]: https://www.usenix.org/system/files/conference/fast16/fast16-papers-lu.pdf

Comparisons

FeatureBadgerRocksDBBoltDB
DesignLSM tree with value logLSM tree onlyB+ tree
High Read throughputYesNoYes
High Write throughputYesYesNo
Designed for SSDsYes (with latest research <sup>1</sup>)Not specifically <sup>2</sup>No
EmbeddableYesYesYes
Sorted KV accessYesYesYes
Pure Go (no Cgo)YesNoYes
TransactionsYes, ACID, concurrent with SSI<sup>3</sup>Yes (but non-ACID)Yes, ACID
SnapshotsYesYesYes
TTL supportYesYesNo
3D access (key-value-version)Yes<sup>4</sup>NoNo

<sup>1</sup> The [WISCKEY paper][wisckey] (on which Badger is based) saw big wins with separating values from keys, significantly reducing the write amplification compared to a typical LSM tree.

<sup>2</sup> RocksDB is an SSD optimized version of LevelDB, which was designed specifically for rotating disks. As such RocksDB's design isn't aimed at SSDs.

<sup>3</sup> SSI: Serializable Snapshot Isolation. For more details, see the blog post Concurrent ACID Transactions in Badger

<sup>4</sup> Badger provides direct access to value versions via its Iterator API. Users can also specify how many versions to keep per key via Options.

Benchmarks

We have run comprehensive benchmarks against RocksDB, Bolt and LMDB. The benchmarking code, and the detailed logs for the benchmarks can be found in the [badger-bench] repo. More explanation, including graphs can be found the blog posts (linked above).

[badger-bench]: https://github.com/dgraph-io/badger-bench

Projects Using Badger

Below is a list of known projects that use Badger:

a new hypermedia distribution protocol.

broker with message storage, uses MQTT, gossip and badger.

transform data from multiple databases using SQL.

certificate authority for secure, automated certificate management.

persistent, time sorted message queue.

database.

Project.

application data analytics.

same redis protocol.

Hashicorp raft.

distributed applications.

cryptocurrency with a system of community-based governance integrated into its blockchain.

running, monitoring and managing honeypots.

network for IoT powered by scalability- and privacy-centric blockchains.

fasthttp.

applications.

database for Go projects.

types built on Badger

BadgerDB

built on Badger. Makes it easy bringing up a cluster of Badger with Raft consensus algorithm by hashicorp/raft.

server - storage powered by BadgerDB

built with BadgerDB

Recommendation tasks.

the badger.Iterator, simplifying from-to, and prefix mechanics.

database, exposing BadgerDB with key-range scanning semantics.

based on Badger for the storage. Compatible with all existing reverse-proxies.

with great transaction performance.

Gossamer, a Go client for the Polkadot Network.

simplest API interface.

storage project

for cluster coordination and Badger for data storage.

used in conjunction with pywb

built to support the next generation of games, apps and the digital assets that power them.

store and diff CSV.

and aggregator.

StableStore Interface of hashcorp/raft. it is used to store raft log and metadata of hashcorp/raft.

of data types. Originally created for large-scale brain reconstructions in Connectomics.

full structs into BadgerDB, using an ownership hierarchy model.

shorten links and track link usage. Support for BadgerDB and MongoDB. Improved performance by more than 30% when using BadgerDB

transforms your object storage to Git-like repositories. lakeFS uses BadgerDB for its underlying local metadata KV store implementation

Vulcan Optivum schedule program

and badger db

and high capability, fitting for a powerful concurrent map.

the Fortuna smart contract on the Cardano blockchain

If you are using Badger in a project please send a pull request to add it to the list.

Contributing

If you're interested in contributing to Badger see CONTRIBUTING.

Contact

requests.