bow

by zippoxer

zippoxer / bow

Bow - Minimal embedded database powered by Badger

206 Stars 14 Forks Last release: Not found MIT License 30 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Bow GoDoc

Bow is a minimal embedded database powered by Badger.

The mission of Bow is to provide a simple, fast and reliable way to persist structured data for projects that don't need an external database server such as PostgreSQL or MongoDB.

Bow is powered by BadgerDB, implementing buckets and serialization on top of it.

Table of Contents

Why Badger and not Bolt?

Badger is more actively maintained than bbolt, allows for key-only iteration and has some very interesting performance characteristics.

Getting Started

Installing

go get -u github.com/zippoxer/bow

Opening a database

// Open database under directory "test".
db, err := bow.Open("test")
if err != nil {
    log.Fatal(err)
}
defer db.Close()

With options:

db, err := bow.Open("test",
    bow.SetCodec(msgp.Codec{}),
    bow.SetBadgerOptions(badger.DefaultOptions}))
if err != nil {
    log.Fatal(err)
}

Defining a structure

Each record in the database has an explicit or implicit unique key.

If a structure doesn't define a key or has a zero-value key, Bow stores it with a randomly generated key.

type Page struct {
    Body     []byte
    Tags     []string
    Created  time.Time
}

Structures must define a key if they wish to manipulate it.

type Page struct {
    Id      string `bow:"key"`
    Body     []byte
    Tags     []string
    Created  time.Time
}

Keys must be a string, a byte slice, any built-in integer of at least 32 bits (int32, uint32 and above) or a type that implements

codec.Marshaler
and
codec.Unmarshaler
.

Randomly generated keys

Id
is a convenient placeholder for Bow's randomly generated keys.

type Page struct {
    Id bow.Id // Annotating with `bow:"key"` isn't necessary.
    // ...
}

Id.String()
returns a user-friendly representation of
Id
.

ParseId(string)
parses the user-friendly representation into an
Id
.

NewId()
generates a random Id. Only necessary when you need to know the inserted Id.

Persisting a structure

Put
persists a structure into the bucket. If a record with the same key already exists, then it will be updated.
page1 := Page{
    Id:      bow.NewId(),
    Body:    []byte("

Example Domain

"), Tags: []string{"example", "h1"}, Created: time.Now(), } err := db.Bucket("pages").Put(page1) if err != nil { log.Fatal(err) }

Retrieving a structure

Get
retrieves a structure by key from a bucket, returning ErrNotFound if it doesn't exist.
var page2 Page
err := db.Bucket("pages").Get(page1.Id, &page2)
if err != nil {
    log.Fatal(err)
}

Iterating a bucket

iter := db.Bucket("pages").Iter()
defer iter.Close()
var page Page
for iter.Next(&page) {
    log.Println(page.Id.String()) // User-friendly representation of bow.Id.
}
if iter.Err() != nil {
    log.Fatal(err)
}

Prefix iteration

Iterate over records whose key starts with a given prefix.

For example, let's define

Page
with URL as the key:
type Page struct {
    URL  string `bow:"key"`
    Body []byte
}

Finally, let's iterate over HTTPS pages:

iter := db.Bucket("pages").Prefix("https://")
var page Page
for iter.Next(&page) {
    log.Println(page.URL)
}

Serialization

By default, Bow serializes structures with

encoding/json
. You can change that behaviour by passing a type that implements
codec.Codec
via the
bow.SetCodec
option.

MessagePack with
tinylib/msgp

msgp is a code generation tool and serialization library for MessagePack, and it's nearly as fast as protocol buffers and about an order of magnitude faster than encoding/json (see benchmarks). Bow provides a

codec.Codec
implementation for msgp under the
codec/msgp
package. Here's how to use it:
  • Since msgp generates code to serialize structures, you must include the following directive in your Go file:
//go:generate msgp
  • Replace any use of

    bow.Id
    in your structures with
    string
    . Since
    bow.Id
    is a
    string
    , you can convert between the two without any cost.
  • Import

    github.com/zippoxer/bow/codec/msgp
    and open a database with
    msgp.Codec
    :
    go
    bow.Open("test", bow.SetCodec(msgp.Codec{}))
    

Read more about msgp and it's code generation settings at https://github.com/tinylib/msgp

Upcoming

Key-only iteration

Since Badger separates keys from values, key-only iteration should be orders of magnitude faster, at least in some cases, than it's equivalent with Bolt.

Bow's key-only iterator is a work in progress.

Transactions

Cross-bucket transactions are a work in progress. See branch tx.

Querying

Bow doesn't feature a querying mechanism yet. Instead, you must iterate records to query or filter them.

For example, let's say I want to perform the equivalent of

SELECT * FROM pages WHERE url LIKE '%/home%'

in Bow, I could iterate the pages bucket and filter pages with URLs containing '/home':

var matches []Page
iter := db.Bag("pages").Iter()
defer iter.Close()
var page Page
for iter.Next(&page) {
    if strings.Contains(strings.ToLower(page.URL), "/home") {
        matches = append(matches, page)
    }
}
return matches, iter.Err()

Meanwhile, you can try Storm if you want convenient querying.

Performance

Bow is nearly as fast as Badger, and in most cases faster than Storm. See Go Database Benchmarks.

Contributing

I welcome any feedback and contribution.

My priorities right now are tests, documentation and polish.

Bow lacks decent tests and documentation for types, functions and methods. Most of Bow's code was written before I had any plan to release it, and I think it needs polish.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.