Need help with redisearch-py?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

RediSearch
177 Stars 60 Forks BSD 2-Clause "Simplified" License 205 Commits 17 Opened issues

Description

RediSearch python client

Services available

!
?

Need anything else?

Contributors list

license PyPI version CircleCI GitHub issues Codecov Known Vulnerabilities Total alerts

RediSearch Python Client

Forum Discord

This is a Python search engine library that utilizes the RediSearch Redis Module API.

It is the "official" client of RediSearch, and should be regarded as its canonical client implementation.

Features

RediSearch is a source avaliable (RSAL), high performance search engine implemented as a Redis Module. It uses custom data types to allow fast, stable and feature rich full-text search inside Redis.

This client is a wrapper around the RediSearch API protocol, that allows you to utilize its features easily.

RediSearch's features include:

  • Full-Text indexing of multiple fields in documents.
  • Incremental indexing without performance loss.
  • Document ranking (provided manually by the user at index time) and field weights.
  • Auto-complete suggestions (with fuzzy prefix suggestions).
  • Exact Phrase Search.
  • Stemming based query expansion in many languages (using Snowball).
  • Limiting searches to specific document fields (up to 8 fields supported).
  • Numeric filters and ranges.
  • Automatically index existing HASH keys as documents.

For more details, visit http://redisearch.io

Examples

Creating a client instance

When you create a redisearch-py client instance, the only required argument is the name of the index.

from redisearch import Client

client = Client("my-index")

To connect with a username and/or password, pass those options to the client initializer.

client = Client("my-index", password="my-password")

Using core Redis commands

Every instance of

Client
contains an instance of the redis-py
Client
as well. Use this object to run core Redis commands.
import datetime

from redisearch import Client

START_TIME = datetime.datetime.now().strftime("%Y-%m-%d-%H:%M.%S")

client = Client("my-index")

client.redis.set("start-time", START_TIME)

Checking if a RediSearch index exists

To check if a RediSearch index exists, use the

FT.INFO
command and catch the
ResponseError
raised if the index does not exist.
from redis import ResponseError
from redisearch import Client

client = Client("my-index")

try: client.info() except ResponseError: # Index does not exist. We need to create it!

Defining a search index

Use an instance of

IndexDefinition
to define a search index. You only need to do this when you create an index.

RediSearch indexes follow Hashes in your Redis databases by watching key prefixes. If a Hash whose key starts with one of the search index's configured key prefixes is added, updated, or deleted from Redis, RediSearch will make those changes in the index. You configure a search index's key prefixes using the

prefix
parameter of the
IndexDefinition
initializer.

NOTE: Once you create an index, RediSearch will continuously index these keys when their Hashes change.

IndexDefinition
also takes a schema. The schema specifies which fields to index from within the Hashes that the index follows. The field types are:
  • TextField
  • TagField
  • NumericField
  • GeoField

For more information on what these field types mean, consult the RediSearch documentation on the

FT.CREATE
command.

With redisearch-py, the schema is an iterable of

Field
instances. Once you have an
IndexDefinition
instance, you can create the instance by passing a schema iterable to the
create_index()
method.
from redis import ResponseError
from redisearch import Client, IndexDefinition, TextField

SCHEMA = ( TextField("title", weight=5.0), TextField("body") )

client = Client("my-index")

definition = IndexDefinition(prefix=['blog:'])

try: client.info() except ResponseError: # Index does not exist. We need to create it! client.create_index(SCHEMA, definition=definition)

Indexing a document

A RediSearch 2.0 index continually follows Hashes with the key prefixes you defined, so if you want to add a document to the index, you only need to create a Hash with one of those prefixes.

# Indexing a document with RediSearch 2.0.
doc = {
    'title': 'RediSearch',
    'body': 'Redisearch adds querying, indexing, and full-text search to Redis'
}
client.redis.hset('doc:1', mapping=doc)

Past versions of RediSearch required that you call the

add_document()
method. This method is deprecated, but we include its usage here for reference.
# Indexing a document for RediSearch 1.x
client.add_document(
    "doc:2",
    title="RediSearch",
    body="Redisearch implements a search engine on top of redis",
)

Querying

Basic queries

Use the

search()
method to perform basic full-text and field-specific searches. This method doesn't take many of the options available to the RediSearch
FT.SEARCH
command -- read the section on building complex queries later in this document for information on how to use those.
res = client.search("evil wizards")

Result objects

Results are wrapped in a

Result
object that includes the number of results and a list of matching documents.
>>> print(res.total)
2
>>> print(res.docs[0].title)
"Wizard Story 2: Evil Wizards Strike Back"

Building complex queries

You can use the

Query
object to build complex queries:
q = Query("evil wizards").verbatim().no_content().with_scores().paging(0, 5)
res = client.search(q)

For an explanation of these options, see the RediSearch documentation for the

FT.SEARCH
command.

Query syntax

The default behavior of queries is to run a full-text search across all

TEXT
fields in the index for the intersection of all terms in the query.

So the example given in the "Basic queries" section of this README,

client.search("evil wizards")
, run a full-text search for the intersection of "evil" and "wizard" in all
TEXT
fields.

Many more types of queries are possible, however! The string you pass into the

search()
method or
Query()
initializer has the full range of query syntax available in RediSearch.

For example, a full-text search against a specific

TEXT
field in the index looks like this:
# Full-text search
res = client.search("@title:evil wizards")

Finding books published in 2020 or 2021 looks like this:

client.search("@published_year:[2020 2021]")

To learn more, see the RediSearch documentation on query syntax.

Aggregations

This library contains a programmatic interface to run aggregation queries with RediSearch.

Making an aggregation query

To make an aggregation query, pass an instance of the

AggregateRequest
class to the
search()
method of an instance of
Client
.

For example, here is what finding the most books published in a single year looks like:

from redisearch import Client
from redisearch import reducers
from redisearch.aggregation import AggregateRequest

client = Client('books-idx')

request = AggregateRequest('*').group_by( '@published_year', reducers.count().alias("num_published") ).group_by( [], reducers.max("@num_published").alias("max_books_published_per_year") )

result = client.aggregate(request)

A redis-cli equivalent query

The aggregation query just given is equivalent to the following

FT.AGGREGATE
command entered directly into the redis-cli:
FT.AGGREGATE books-idx *
    GROUPBY 1 @published_year
      REDUCE COUNT 0 AS num_published
    GROUPBY 0
      REDUCE MAX 1 @num_published AS max_books_published_per_year

The AggregateResult object

Aggregation queries return an

AggregateResult
object that contains the rows returned for the query and a cursor if you're using the cursor API.
from redisearch.aggregation import AggregateRequest, Asc

request = AggregateRequest('*').group_by( ['@published_year'], reducers.avg('average_rating').alias('average_rating_for_year') ).sort_by( Asc('@average_rating_for_year') ).limit( 0, 10 ).filter('@published_year > 0')

...

In [53]: resp = c.aggregate(request) In [54]: resp.rows Out[54]: [['published_year', '1914', 'average_rating_for_year', '0'], ['published_year', '2009', 'average_rating_for_year', '1.39166666667'], ['published_year', '2011', 'average_rating_for_year', '2.046'], ['published_year', '2010', 'average_rating_for_year', '3.125'], ['published_year', '2012', 'average_rating_for_year', '3.41'], ['published_year', '1967', 'average_rating_for_year', '3.603'], ['published_year', '1970', 'average_rating_for_year', '3.71875'], ['published_year', '1966', 'average_rating_for_year', '3.72666666667'], ['published_year', '1927', 'average_rating_for_year', '3.77']]

Reducer functions

Notice from the example that we used an object from the

reducers
module. See the RediSearch documentation for more examples of reducer functions you can use when grouping results.

Reducer functions include an

alias()
method that gives the result of the reducer a specific name. If you don't supply a name, RediSearch will generate one.

Grouping by zero, one, or multiple fields

The

group_by
statement can take a single field name as a string, or multiple field names as a list of strings.
AggregateRequest('*').group_by('@published_year', reducers.count())

AggregateRequest('*').group_by( ['@published_year', '@average_rating'], reducers.count())

To run a reducer function on every result from an aggregation query, pass an empty list to

group_by()
, which is equivalent to passing the option
GROUPBY 0
when writing an aggregation in the redis-cli.
AggregateRequest('*').group_by([], reducers.max("@num_published"))

NOTE: Aggregation queries require at least one

group_by()
method call.

Sorting and limiting

Using an

AggregateRequest
instance, you can sort with the
sort_by()
method and limit with the
limit()
method.

For example, finding the average rating of books published each year, sorting by the average rating for the year, and returning only the first ten results:

from redisearch import Client
from redisearch.aggregation import AggregateRequest, Asc

c = Client()

request = AggregateRequest('*').group_by( ['@published_year'], reducers.avg('average_rating').alias('average_rating_for_year') ).sort_by( Asc('@average_rating_for_year') ).limit(0, 10)

c.aggregate(request)

NOTE: The first option to

limit()
is a zero-based offset, and the second option is the number of results to return.

Filtering

Use filtering to reject results of an aggregation query after your reducer functions run. For example, calculating the average rating of books published each year and only returning years with an average rating higher than 3:

from redisearch.aggregation import AggregateRequest, Asc

req = AggregateRequest('*').group_by( ['@published_year'], reducers.avg('average_rating').alias('average_rating_for_year') ).sort_by( Asc('@average_rating_for_year') ).filter('@average_rating_for_year > 3')

Installing

  1. Install RediSearch
  2. Install the Python client:
$ pip install redisearch

Developing

  1. Create a virtualenv to manage your python dependencies, and ensure it's active.
    virtualenv -v venv
  2. Install pypoetry to manage your dependencies.
    pip install --user poetry
  3. Install dependencies.
    poetry install

Note: Due to an interaction between and python 3.10, you may need to run the following, if you receive a JSONError while installing packages.

poetry config experimental.new-installer false

Testing

Testing can easily be performed using using Docker. Run the following:

make -C test/docker test PYTHON_VER=3

(Replace

PYTHON_VER=3
with
PYTHON_VER=2
to test with Python 2.7.)

Alternatively, use the following procedure:

First, run:

PYTHON_VER=3 ./test/test-setup.sh

This will set up a Python virtual environment in

venv3
(or in
venv2
if
PYTHON_VER=2
is used).

Afterwards, run RediSearch in a container as a daemon:

docker run -d -p 6379:6379 redislabs/redisearch:2.0.0

Finally, invoke the virtual environment and run the tests:

. ./venv3/bin/activate
REDIS_PORT=6379 python test/test.py
REDIS_PORT=6379 python test/test_builder.py

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.