Need help with sqlitedict?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

RaRe-Technologies
627 Stars 91 Forks Apache License 2.0 255 Commits 24 Opened issues

Description

Persistent dict, backed by sqlite3 and pickle, multithread-safe.

Services available

!
?

Need anything else?

Contributors list

=================================================================

sqlitedict -- persistent
dict
, backed-up by SQLite and pickle

|Travis|_ |License|_

.. |Travis| image:: https://travis-ci.org/RaRe-Technologies/sqlitedict.svg?branch=master .. |Downloads| image:: https://img.shields.io/pypi/dm/sqlitedict.svg .. |License| image:: https://img.shields.io/pypi/l/sqlitedict.svg .. _Travis: https://travis-ci.org/RaRe-Technologies/sqlitedict .. _Downloads: https://pypi.python.org/pypi/sqlitedict .. _License: https://pypi.python.org/pypi/sqlitedict

A lightweight wrapper around Python's sqlite3 database with a simple, Pythonic dict-like interface and support for multi-thread access:

.. code-block:: python

from sqlitedict import SqliteDict mydict = SqliteDict('./mydb.sqlite', autocommit=True) mydict['somekey'] = 'anypicklableobject' print(mydict['somekey']) # prints the new value anypicklableobject for key, value in mydict.iteritems(): ... print(key, value) somekey anypicklableobject print(len(mydict)) # etc... all dict functions work 1 mydict.close()

Pickle is used internally to (de)serialize the values. Keys are arbitrary strings, values arbitrary pickle-able objects.

If you don't use autocommit (default is no autocommit for performance), then don't forget to call

mydict.commit()
when done with a transaction:

.. code-block:: python

using SqliteDict as context manager works too (RECOMMENDED)

with SqliteDict('./mydb.sqlite') as mydict: # note no autocommit=True ... mydict['somekey'] = u"first value" ... mydict['anotherkey'] = range(10) ... mydict.commit() ... mydict['somekey'] = u"new value" ... # no explicit commit here with SqliteDict('./mydb.sqlite') as mydict: # re-open the same DB ... print(mydict['somekey']) # outputs 'first value', not 'new value' first value

Features

  • Values can be any picklable objects (uses
    cPickle
    with the highest protocol).
  • Support for multiple tables (=dicts) living in the same database file.
  • Support for access from multiple threads to the same connection (needed by e.g. Pyro). Vanilla sqlite3 gives you
    ProgrammingError: SQLite objects created in a thread can
    only be used in that same thread.

Concurrent requests are still serialized internally, so this "multithreaded support" doesn't give you any performance benefits. It is a work-around for sqlite limitations in Python.

  • Support for custom serialization or compression:

.. code-block:: python

# use JSON instead of pickle

import json mydict = SqliteDict('./my_db.sqlite', encode=json.dumps, decode=json.loads)

# apply zlib compression after pickling

import zlib, pickle, sqlite3 def myencode(obj): ... return sqlite3.Binary(zlib.compress(pickle.dumps(obj, pickle.HIGHESTPROTOCOL))) def mydecode(obj): ... return pickle.loads(zlib.decompress(bytes(obj))) mydict = SqliteDict('./mydb.sqlite', encode=myencode, decode=mydecode)

Installation

The module has no dependencies beyond Python itself. The minimum Python version is 2.5, continuously tested on Python 2.7, and above on

on Github Actions 
_.

Install or upgrade with::

pip install -U sqlitedict

or from the

source tar.gz 
_::
python setup.py install

Documentation

Standard Python document strings are inside the module:

.. code-block:: python

import sqlitedict help(sqlitedict)

(but it's just

dict
with a commit, really).

Beware: because of Python semantics,

sqlitedict
cannot know when a mutable SqliteDict-backed entry was modified in RAM. For example,
mydict.setdefault('new_key', []).append(1)
will leave
mydict['new_key']
equal to empty list, not
[1]
. You'll need to explicitly assign the mutated object back to SqliteDict to achieve the same effect:

.. code-block:: python

val = mydict.get('newkey', []) val.append(1) # sqlite DB not updated here! mydict['newkey'] = val # now updated

For developers

Install::

# pip install pytest coverage pytest-coverage

To perform all tests::

# mkdir -p tests/db
# pytest tests

To perform all tests with coverage::

# pytest tests --cov=sqlitedict

Comments, bug reports

sqlitedict
resides on
github 
_. You can file issues or pull requests there.

Housekeeping

.. code-block:: python

import os os.unlink('my_db.sqlite')


sqlitedict
is open source software released under the
Apache 2.0 license 
. Copyright (c) 2011-now
Radim Řehůřek 
and contributors.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.