dynamite

by Medium

Medium / dynamite

A promise-based DynamoDB client

196 Stars 26 Forks Last release: over 2 years ago (v0.9.5) Other 258 Commits 28 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Dynamite Build Status

Dynamite is a promise-based DynamoDB client. It was created to address performance issues in our previous DynamoDB client. Dynamite will almost always comply with the latest DynamoDB spec on Amazon.

Installation

$ npm install dynamite

Running Tests

Ensure that all of the required node modules are installed in the Dynamite directory by first running:

$ npm install

The tests will be run against a

LocalDynamo
service. Currently, there is no way to change the port without modifying the connection code in
test/utils/TestUtils.js
. To run the tests:
$ npm test

Creating a Client

var Dynamite = require('dynamite')

var options = { region: 'us-east-1', accessKeyId: 'xxx', secretAccessKey: 'xxx' }

var client = new Dynamite.Client(options)

Options requires all of:

  • region
  • accessKeyId
  • secretAccessKey

If a

region
key is not provided in the
options
hash but a
endpoint
key is present, Dynamite will try to infer the region from the
host
key.

Options can also optionally take a hash with a key

dbClient
which points to an object that implements the AWS SDK interface for node.js.

Optional Options Keys

  • sslEnabled
    : a boolean to turn ssl on or off for the connection.
  • endPoint
    : the address of the DynamoDB instance to try to communicate with.
  • retryHandler
    : a
    function(method, table, response)
    that will be triggered if Dynamite needs to retry a command.

Foreword: Kew and You

All functions return Kew promises on

execute()
. These functions will all then take the form:
client.fn(params)
  .execute()
  .then(function(){
    // handle success
  })
  .fail(function(e) {
    // handle failure
  })
  .fin(function() {
    // when all is said and done
  })

Therefore, these docs will focus more on function signatures and assume that the developer using those functions will comply with the Kew API in turn.

Tables

Creating a Table

Table creation is part of the database's concerns and thus doesn't have its own pretty API built into Dynamite. A snippet successfully creating a table that is compliant with the 2012 DynamoDB spec can be found in

test/utils/TestUtils.js
.

Describing a Table

Tables can have descriptions. Retrieve them with:

client.describeTable('table-name')

Conditions

Conditions ensure that certain properties of the item are either absent or equal to a certain value before allowing whatever operation to which they were supplied to mutate the item. They become very useful when items should only be updated if they are missing a field or are of the wrong value. There currently exist two kinds of conditions:

expectAttributeEquals
and
expectAttributeAbsent
. Every operation has particular behaviors when conditions are or are not met.

Adding conditions to an operation is fairly trivial:

var conditions = client.newConditionBuilder()
  .expectAttributeEquals('age', 29)

client.fn('some-table') .withCondition(conditions) .execute() .then(function () { // handle the operation output })

There is also a helper method for building conditions from a JSON object.

var conditions = client.conditions({age: 29})
client.fn('some-table')
  .withCondition(conditions)
  .execute()

If a condition fails, the promise will be rejected with a conditional error, which you can detect with the

isConditionalError
method
client.fn('some-table')
  .withCondition(client.conditions({age: 29})
  .execute()
  .fail(function (e) {
    if (!client.isConditionalError(e)) {
      throw new Error('Unexpected age; conditional check failed')
    } else {
      throw e
    }
  })

Catching all conditional errors is a common idiom, so there is a

throwUnlessConditionalError
helper method for this case.
client.fn('some-table')
  .withCondition(client.conditions({age: 29})
  .execute()
  .fail(client.throwUnlessConditionalError)

Getting an Item From a Table

client.getItem('user-table')
  .setHashKey('userId', 'userA')
  .setRangeKey('column', '@')
  .execute()
  .then(function(data) {
    // data.result: the resulting object
  })

If an item does not exist,

data.result
will be
undefined
.

Getting Select Attributes

client.getItem('user-table')
  .setHashKey('userId', 'userA')
  .setRangeKey('column', '@')
  .selectAttributes(['userId', 'column'])
  .execute()
  .then(function(data) {
    // data.result: the resulting object
    //              only the attributes passed into selectAttributes()
    //              appear as keys in data.result
  })

Batch Get

The batch get API allows you to request multiple items with specific primary keys, from different tables, in a single fetch.

client.newBatchGetBuilder()
  .requestItems('user', [{'userId': 'userA', 'column': '@'}, {'userId': 'userB', 'column': '@'}])
  .requestItems('phones', [{'userId': 'userA', 'column': 'phone1'}, {'userId': 'userB', 'column': 'phone1'}])
  .execute()

requestItems
can be called multiple times, with a table name and an array of objects representing primary keys, in the form
{hashKey: 123, rangeKey: 456}
.

Putting an Item Into a Table

Items are handled as JavaScript Objects by the client. These are then converted into an AWS specific format and sent off. The only accepted types of data that can be stored in DynamoDB are Strings, Numbers, and Sets (Arrays). Sets can contain either only Numbers or Strings.

client.putItem('user-table', {
  userId: 'userA',
  column: '@',
  age: 30,
  company: 'Medium',
  nickNames: ['Ev', 'Evan'],
  postIds: [1, 2, 3]
})

Overrides

If an item with the same hash and range keys as the one that is being inserted, the old item will be replaced with the item that is being put in its place.

// initialData = [{userId: 'userA', column: '@', age: 27]

client.putItem('user-table', { userId: 'userA', column: '@', height: 72 })

If the item above were to be retrieved from the table

user-table
, then age would be undefined and a new key
height
would be available.

Conditional Writes

expectAttributeEquals

The item will only be replaced if the field

field
in the item is equal to the param
value
. If the item does not exist in the table, or the condition is not met, the request will fail.

expectAttributeAbsent

The item will only be replaced if the field

field
is not set in the item in the table. If the item does not exist in the table, then the item will be written to the table. If the field
field
exists for the item in the table, the request will fail.

Deleting Items From a Table

If the hash key and range key match an item, it will be deleted. Upon success, the function returns the previous attributes and values of the deleted item.

client.deleteItem('user-table')
  .setHashKey('userId', 'userA')
  .setRangeKey('column', '@')
  .execute()
  .then(function (data) {
    // data.result will contain the origin item attributes and their corresponding values
  })

Conditional Deletes

expectAttributeEquals

If an item does not exist, then the request will fail. Otherwise, if the condition is met, the item will be deleted.

expectAttributeAbsent

If an item does not exist, then the request will fail. Otherwise, if the condition is met, the item will be deleted.

Updating an Item

There are three methods available to modify columns for items:

putAttribute(field, value)
,
deleteAttribute(field)
, and
addToAttribute(field, value)
.

If an item does not exist, the update query will create the item and update its attributes accordingly.

If a value is updated on an attribute that does not exist, the attribute will be added to the item and set to the

value
passed to
putsAttribute(field, value)
. If an attribute does not exist and it's value is incremented, that attribute will be added to the item and it's value will be set to the
value
passed to
addToAttribute(field, value)
. If an attribute is deleted and it does not exist, the operation becomes a nonsense operation and has no effect on the item.

Putting empty attributes causes the whole update query to fail.

// initialData = [{userId: 'userA', column: '@', age: 27, weight: 180]

client.newUpdateBuilder('user-table') .setHashKey('userId', 'userA') .setRangeKey('column', '@') .enableUpsert() .putAttribute('age', 30) .addToAttribute('age', 1) .deleteAttribute('weight') .putAttribute('height', 72) .execute() .then(function (data) { // data.result == {userId: 'userA', column: '@', age: 31, height: 72} })

Conditional Updates

Conditions should be added with

withCondition
before any update commands.
expectAttributeEquals

If the item does not exist, the update query will fail.

expectAttributeAbsent

If the item does not exist, the update query will create the item and update its attributes accordingly.

Querying a Table

Amazon features extensive documentation describing querying and scanning in great detail.

A Query operation searches only primary key attribute values and supports a subset of comparison operators on key attribute values to refine the search process. A query returns all of the item data for the matching primary keys (all of each item's attributes) up to 1 MB of data per query operation. A Query operation always returns results, but can return empty results.

A Query operation seeks the specified composite primary key, or range of keys, until one of the following events occur:

  • The result set is exhausted.
  • The number of items retrieved reaches the value of the Limit parameter, if specified.
  • The amount of data retrieved reaches the maximum result set size limit of 1 MB.

Usage

Our initial data set:

[
  {"postId": "post1", "column": "@", "title": "This is my post", "content": "And here is some content!", "tags": ['foo', 'bar']},
  {"postId": "post1", "column": "/comment/timestamp/002123", "comment": "this is slightly later"},
  {"postId": "post1", "column": "/comment/timestamp/010000", "comment": "where am I?"},
  {"postId": "post1", "column": "/comment/timestamp/001111", "comment": "HEYYOOOOO"},
  {"postId": "post1", "column": "/comment/timestamp/001112", "comment": "what's up?"},
  {"postId": "post1", "column": "/canEdit/user/AAA", "userId": "AAA"}
]

Querying all items whose postId is

post1
:
client.newQueryBuilder('comments-table')
  .setHashKey('postId', 'post1')
  .execute()
  .then(function (data) {
    // data.result is an array of posts whose hash key is `post1`
  })

The result of the query will be a DynamoResult object with a

result
property for the result set.

DynamoResult also has two methods:

  • hasNext(): boolean

Returns whether there are remaining results for this query.

  • next(): Promise.<DynamoResult>

Executes a new query that fetches the next page of results.

There are also a variety of methods that refine and restrict the returned set of results that operate on the indexed range key, which in our sample case is

column
.

getCount()

Get the count of the number of items, not the actual items themselves.

client.newQueryBuilder('comments-table')
  .setHashKey('postId', 'post1')
  .getCount()

scanForward()

Demand that items be returned in ascending ASCII or numerical value. This is the default.

scanBackward()

Demand that items be returned in descending ASCII or numerical value.

setStartKey(key)

Start the query at a specified hash key. Useful when your request is returned in chunks and subsequent chunks need to be retrieved after the current batch is processed.

When partial results are returned, the

LastEvaluatedKey
can be passed in as an argument to
setStartKey()
on the next query to get the next section of results.

In general, calling setStartKey directly is discouraged in favor of using the

next()
method described above.

setLimit(max)

Return at most

max
items. Note that if the response will be larger than 1mb, then at most 1mb of data is returned, and the next batch of items needs to be queried while specifying that the query start at the
LastEvaluatedKey
. That key is returned with the results of the current query.

indexBeginsWith(rangekey, keypart)

Return only items where the range key begins with

key_part
. For instance, retrieve all comments for posts with a, in our case unique, hash key of
post1
.
client.newQueryBuilder('comments-table')
  .setHashKey('postId', 'post1')
  .indexBeginsWith('column', '/comment/')

indexBetween(rangekey, keypartstart, keypart_end)

Return only items whose range key is "between" the start and end keys. The range key will be compared to the start and end keys in a lexicographic manner. So 'b' is "between" 'a' and 'c'.

Retrieve all comments for posts with the hash key

post1
up until the
009999
timestamp:
client.newQueryBuilder('comments-table')
  .setHashKey('postId', 'post1')
  .indexBetween('column', '/comment/', '/comment/timestamp/009999')

indexLessThan(range_key, value)

indexLessThanEqual(range_key, value)

indexGreaterThan(range_key, value)

indexGreaterThanEqual(range_key, value)

indexEqual(range_key, value)

Return all items whose range keys comply with the afore-listed operations.

selectAttributes(attributes[])

Returned items will be stripped of all attributes except their hash key, range key, and the provided array of strings

attributes
.

Scanning A Table

Amazon features extensive documentation describing querying and scanning in great detail.

A Scan operation examines every item in the table. You can specify filters to apply to the results to refine the values returned to you, after the scan has finished. Amazon DynamoDB puts a 1 MB limit on the scan (the limit applies before the results are filtered). A Scan can result in no table data meeting the filter criteria.

Scan supports a specific set of comparison operators. For information about each comparison operator available for scan operations, go to the API entry for Scan in the Amazon DynamoDB API Reference.

Usage

Our initial data set:

[
  {"userId": "c", "column": "@", "post": "3", "email": "[email protected]"},
  {"userId": "b", "column": "@", "post": "0", "address": "800 Market St. SF, CA"},
  {"userId": "a", "column": "@", "post": "5", "email": "[email protected]"},
  {"userId": "d", "column": "@", "post": "2", "twitter": "haha"},
  {"userId": "e", "column": "@", "post": "2", "twitter": "hoho"},
  {"userId": "f", "column": "@", "post": "4", "description": "Designer", "email": "[email protected]"},
  {"userId": "h", "column": "@", "post": "6", "tags": ['foo', 'bar']}
]

A simple scan looks like this:

client.newScanBuilder('user-table')
  .execute()
  .then(function (data) {
    // data.result contains all of the users
  })

If your dataset contains more than 1 MB of data, the

data
that is returned will contain a
LastEvaluatedKey
key that will tell you what the last evaluated key for the scan was, so you can start the next
scan
there by passing the
LastEvaluatedKey
to
setStartKey(key)
.

.filterAttributeEquals(field, value)

Include items whose

field
equals
value
.
client.newScanBuilder('user-table')
  .filterAttributeEquals('twitter', 'haha')
  .execute()
  .then(function (data) {
    // data.result #=> [{"userId": "d", "column": "@", "post": "2", "twitter": "haha"}]
  })

The other

filterAttribute*
functions are used in the exact same way.

.filterAttributeNotEquals(field, value)

Include items whose

field
does not equal
value
.

.filterAttributeLessThanEqual(field, value)

Include items whose

field
is less than or equal to
value
.

.filterAttributeLessThan(field, value)

Include items whose

field
is less than
value
.

.filterAttributeGreaterThanEqual(field, value)

Include items whose

field
is greater than or equal to
value
.

.filterAttributeGreaterThan(field, value)

Include items whose

field
is greater than
value
.

.filterAttributeNotNull(field)

Include items whose

field
is not
null
, or doesn't exist.

.filterAttributeContains(field, value)

Include items whose

field
contains
value
.

If an item's

field
attribute is a string,
filterAttributeContains
will search for
value
in that field's value. If an item's
field
attribute is a set,
filterAttributeContains
will search for
value
in that set.

.filterAttributeNotContains(field, value)

Include items whose

field
does not contain
value
. Essentially the inverse of
filterAttributeContains
.

.filterAttributeBeginsWith(field, value)

Include items whose

field
attribute begins with
value
.

.filterAttributeBetween(field, lower, upper)

Include items whose

field
attribute's value is between
lower
and
upper
, exclusive.

.filterAttributeIn(field, arrayofvalues)

Filter out rows where field is not one of the values in

array_of_values
.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.