Need help with nanopq?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

matsui528
141 Stars 24 Forks MIT License 50 Commits 1 Opened issues

Description

Pure python implementation of product quantization for nearest neighbor search

Services available

!
?

Need anything else?

Contributors list

# 96,458
scikit-...
C++
Flask
aws-ec2
7 commits
# 203,931
HTML
Shell
Scala
voice-c...
2 commits

nanopq

Build Status Documentation Status PyPI version Downloads

Nano Product Quantization (nanopq): a vanilla implementation of Product Quantization (PQ) and Optimized Product Quantization (OPQ) written in pure python without any third party dependencies.

Installing

You can install the package via pip. This library works with Python 3.5+ on linux.

pip install nanopq

Documentation

Example

import nanopq
import numpy as np

N, Nt, D = 10000, 2000, 128 X = np.random.random((N, D)).astype(np.float32) # 10,000 128-dim vectors to be indexed Xt = np.random.random((Nt, D)).astype(np.float32) # 2,000 128-dim vectors for training query = np.random.random((D,)).astype(np.float32) # a 128-dim query vector

Instantiate with M=8 sub-spaces

pq = nanopq.PQ(M=8)

Train codewords

pq.fit(Xt)

Encode to PQ-codes

X_code = pq.encode(X) # (10000, 8) with dtype=np.uint8

Results: create a distance table online, and compute Asymmetric Distance to each PQ-code

dists = pq.dtable(query).adist(X_code) # (10000, )

Author

Reference

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.