Need help with python-hashes?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

222 Stars 44 Forks MIT License 40 Commits 4 Opened issues


Interesting (non-cryptographic) hashes implemented in pure Python.

Services available


Need anything else?

Contributors list


Interesting (non-cryptographic) hashes implemented in pure Python. Included so far:

  • Bloom filters
  • Charikar similarity hashes
  • Nilsimsa signatures
  • geohashes

Each hash is implemented as its own type extended from the base class


Official repository and latest version:

To install the latest version, you can either do

easy_install python-hashes
pip install python-hashes
. You may need to use
, depending on your environment.


Charikar similarity is most useful for creating 'fingerprints' of documents or metadata so you can quickly find duplicates or cluster items. It operates on lists of strings, treating each word as its own token (order does not matter, as in the bag-of-words model).

Here is a quick example session showing off similarity hashes:

    >>> from hashes.simhash import simhash
    >>> hash1 = simhash('This is a test string one.')
    >>> hash2 = simhash('This is a test string TWO.')
    >>> hash1
    >>> print hash1, hash2
    10203485745788768176630988232 10749932022170787621889701832
    >>> hash1.hex()
    >>> hash1.similarity(hash2)
    0.875                   # % of bits in common (calculated via hamming distance)
    >>> long(hash1) - long(hash2)
    >>> hash1 < hash2       # Hashes of the same type can be compared
    >>> a_list = [hash2, hash1, 4.2]
    >>> for item in a_list: print item
    >>> a_list.sort()       # Because comparisons work, so does sorting
    >>> for item in a_list: print item

It can be extended to any bitlength using the

    >>> hash3 = simhash('this is yet another test', hashbits=8)
    >>> hash3.hex()
    >>> hash4 = simhash('extremely long hash bitlength', hashbits=2048)
    >>> hash4.hex()

But be careful; it only makes sense to compare equal-length hashes!

    >>> hash3.similarity(hash4)
    Traceback (most recent call last):
      File "", line 1, in 
      File "hashes/", line 63, in similarity
        raise Exception('Hashes must be of equal size to find similarity')
    Exception: Hashes must be of equal size to find similarity


The Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. False positives are possible, but false negatives are not. Elements can be added to the set but not removed.

Uses SHA-1 from Python's hashlib, but you can swap that out with any other 160-bit hash function. Also keep in mind that it starts off very sparse and become more dense (and false-positive-prone) as you add more elements.

Here is the basic use case:

    >>> from hashes.bloom import bloomfilter
    >>> hash1 = bloomfilter('test')
    >>> hash1.hashbits, hash1.num_hashes     # default values (see below)
    (28756, 7)
    >>> hash1.add('test string')
    >>> 'test string' in hash1
    >>> 'holy diver' in hash1
    >>> for word in 'these are some tokens to add to the filter'.split():
    ...     hash1.add(word)
    >>> 'these' in hash1

The hash length and number of internal hashes used for the digest are automatically determined using your input values

. The capacity is the upper bound on the number of items you wish to add. A lower false-positive rate will create a larger, but more accurate, filter.
    >>> hash2 = bloomfilter(capacity=100, false_positive_rate=0.01)
    >>> hash2.hashbits, hash2.num_hashes
    (959, 7)
    >>> hash3 = bloomfilter(capacity=1000000, false_positive_rate=0.01)
    >>> hash3.hashbits, hash3.num_hashes
    (9585059, 7)
    >>> hash4 = bloomfilter(capacity=1000000, false_positive_rate=0.0001)
    >>> hash4.hashbits, hash4.num_hashes
    (19170117, 14)

The hash grows in size to accommodate the number of items you wish to add, but remains sparse until you are done adding the projected number of items:

    >>> import zlib
    >>> len(hash4.hex())
    >>> len(zlib.compress(hash4.hex()))


Geohash is a latitude/longitude geocode system invented by Gustavo Niemeyer when writing the web service at, and put into the public domain.

It is a hierarchical spatial data structure which subdivides space into buckets of grid shape. Geohashes offer properties like arbitrary precision and the possibility of gradually removing characters from the end of the code to reduce its size (and gradually lose precision). As a consequence of the gradual precision degradation, nearby places will often (but not always) present similar prefixes. On the other side, the longer a shared prefix is, the closer the two places are. For this implementation, the default precision is 12 (base32) characters long.

It's very easy to use:

    >>> from hashes.geohash import geohash
    >>> here = geohash(33.0505, -1.024, precision=4)
    >>> there = geohash(34.5, -2.5, precision=4)
    >>> here.hash, there.hash
    ('evzs', 'eynk')
    >>> here.distance_in_miles(there)

&gt;&gt;&gt; # The longer the hash, the more accurate it is
&gt;&gt;&gt; here.encode(33.0505, -1.024, precision=8)
&gt;&gt;&gt; here.hash
&gt;&gt;&gt; here.decode()
(33.050565719604492, -1.0236167907714844)

&gt;&gt;&gt; # Now try with 20 characters
&gt;&gt;&gt; here.encode(33.0505, -1.024, precision=20)
&gt;&gt;&gt; here.hash
&gt;&gt;&gt; here.decode()
(33.050499999999936, -1.0239999999998339)


Most useful for filtering spam by creating signatures of documents to find near-duplicates. Charikar similarity hashes can be used on any datastream, whereas Nilsimsa is a digest ideal for documents (language doesn't matter) because it uses histograms of rolling trigraphs instead of the usual bag-of-words model where order doesn't matter.

Related paper and original reference.

The Nilsimsa hash does not output the same data as the reference implementation. Use at your own risk.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.