Persistent dict, backed by sqlite3 and pickle, multithread-safe.
=================================================================
dict, backed-up by SQLite and pickle
|Travis|_ |License|_
.. |Travis| image:: https://travis-ci.org/RaRe-Technologies/sqlitedict.svg?branch=master .. |Downloads| image:: https://img.shields.io/pypi/dm/sqlitedict.svg .. |License| image:: https://img.shields.io/pypi/l/sqlitedict.svg .. _Travis: https://travis-ci.org/RaRe-Technologies/sqlitedict .. _Downloads: https://pypi.python.org/pypi/sqlitedict .. _License: https://pypi.python.org/pypi/sqlitedict
A lightweight wrapper around Python's sqlite3 database with a simple, Pythonic dict-like interface and support for multi-thread access:
.. code-block:: python
from sqlitedict import SqliteDict mydict = SqliteDict('./mydb.sqlite', autocommit=True) mydict['somekey'] = 'anypicklableobject' print(mydict['somekey']) # prints the new value anypicklableobject for key, value in mydict.iteritems(): ... print(key, value) somekey anypicklableobject print(len(mydict)) # etc... all dict functions work 1 mydict.close()
Pickle is used internally to (de)serialize the values. Keys are arbitrary strings, values arbitrary pickle-able objects.
If you don't use autocommit (default is no autocommit for performance), then don't forget to call
mydict.commit()when done with a transaction:
.. code-block:: python
using SqliteDict as context manager works too (RECOMMENDED)
with SqliteDict('./mydb.sqlite') as mydict: # note no autocommit=True ... mydict['somekey'] = u"first value" ... mydict['anotherkey'] = range(10) ... mydict.commit() ... mydict['somekey'] = u"new value" ... # no explicit commit here with SqliteDict('./mydb.sqlite') as mydict: # re-open the same DB ... print(mydict['somekey']) # outputs 'first value', not 'new value' first value
cPicklewith the highest protocol).
ProgrammingError: SQLite objects created in a thread can only be used in that same thread.
Concurrent requests are still serialized internally, so this "multithreaded support" doesn't give you any performance benefits. It is a work-around for sqlite limitations in Python.
.. code-block:: python
# use JSON instead of pickle
import json mydict = SqliteDict('./my_db.sqlite', encode=json.dumps, decode=json.loads)
# apply zlib compression after pickling
import zlib, pickle, sqlite3 def myencode(obj): ... return sqlite3.Binary(zlib.compress(pickle.dumps(obj, pickle.HIGHESTPROTOCOL))) def mydecode(obj): ... return pickle.loads(zlib.decompress(bytes(obj))) mydict = SqliteDict('./mydb.sqlite', encode=myencode, decode=mydecode)
The module has no dependencies beyond Python itself. The minimum Python version is 2.5, continuously tested on Python 2.7, and above on
on Github Actions_.
Install or upgrade with::
pip install -U sqlitedict
or from the
source tar.gz_::
python setup.py install
Standard Python document strings are inside the module:
.. code-block:: python
import sqlitedict help(sqlitedict)
(but it's just
dictwith a commit, really).
Beware: because of Python semantics,
sqlitedictcannot know when a mutable SqliteDict-backed entry was modified in RAM. For example,
mydict.setdefault('new_key', []).append(1)will leave
mydict['new_key']equal to empty list, not
[1]. You'll need to explicitly assign the mutated object back to SqliteDict to achieve the same effect:
.. code-block:: python
val = mydict.get('newkey', []) val.append(1) # sqlite DB not updated here! mydict['newkey'] = val # now updated
Install::
# pip install pytest coverage pytest-coverage
To perform all tests::
# mkdir -p tests/db # pytest tests
To perform all tests with coverage::
# pytest tests --cov=sqlitedict
sqlitedictresides on
github_. You can file issues or pull requests there.
.. code-block:: python
import os os.unlink('my_db.sqlite')
sqlitedictis open source software released under the
Apache 2.0 license. Copyright (c) 2011-now
Radim Řehůřekand contributors.