RediSearch Python Client

This is a Python search engine library that utilizes the RediSearch Redis Module API.

It is the "official" client of RediSearch, and should be regarded as its canonical client implementation.

Features

RediSearch is a source avaliable (RSAL), high performance search engine implemented as a Redis Module. It uses custom data types to allow fast, stable and feature rich full-text search inside Redis.

This client is a wrapper around the RediSearch API protocol, that allows you to utilize its features easily.

RediSearch's features include:

• Full-Text indexing of multiple fields in documents.
• Incremental indexing without performance loss.
• Document ranking (provided manually by the user at index time) and field weights.
• Auto-complete suggestions (with fuzzy prefix suggestions).
• Exact Phrase Search.
• Stemming based query expansion in many languages (using Snowball).
• Limiting searches to specific document fields (up to 8 fields supported).
• Numeric filters and ranges.
• Automatically index existing HASH keys as documents.

For more details, visit http://redisearch.io

Examples

Creating a client instance

When you create a redisearch-py client instance, the only required argument is the name of the index.

from redisearch import Client

client = Client("my-index")

To connect with a username and/or password, pass those options to the client initializer.

client = Client("my-index", password="my-password")

Using core Redis commands

Every instance of

Client

Client

import datetime

from redisearch import Client
START_TIME = datetime.datetime.now().strftime("%Y-%m-%d-%H:%M.%S")
client = Client("my-index")
client.redis.set("start-time", START_TIME)

Checking if a RediSearch index exists

To check if a RediSearch index exists, use the

FT.INFO

ResponseError

from redis import ResponseError
from redisearch import Client

client = Client("my-index")
try:
    client.info()
except ResponseError:
    # Index does not exist. We need to create it!

Defining a search index

Use an instance of

IndexDefinition

RediSearch indexes follow Hashes in your Redis databases by watching key prefixes. If a Hash whose key starts with one of the search index's configured key prefixes is added, updated, or deleted from Redis, RediSearch will make those changes in the index. You configure a search index's key prefixes using the

prefix

IndexDefinition

NOTE: Once you create an index, RediSearch will continuously index these keys when their Hashes change.

IndexDefinition

• TextField
• TagField
• NumericField
• GeoField

For more information on what these field types mean, consult the RediSearch documentation on the

FT.CREATE

With redisearch-py, the schema is an iterable of

Field

IndexDefinition

create_index()

from redis import ResponseError
from redisearch import Client, IndexDefinition, TextField

SCHEMA = (
    TextField("title", weight=5.0),
    TextField("body")
)
client = Client("my-index")
definition = IndexDefinition(prefix=['blog:'])
try:
    client.info()
except ResponseError:
    # Index does not exist. We need to create it!
    client.create_index(SCHEMA, definition=definition)

Indexing a document

A RediSearch 2.0 index continually follows Hashes with the key prefixes you defined, so if you want to add a document to the index, you only need to create a Hash with one of those prefixes.

# Indexing a document with RediSearch 2.0.
doc = {
    'title': 'RediSearch',
    'body': 'Redisearch adds querying, indexing, and full-text search to Redis'
}
client.redis.hset('doc:1', mapping=doc)

Past versions of RediSearch required that you call the

add_document()

# Indexing a document for RediSearch 1.x
client.add_document(
    "doc:2",
    title="RediSearch",
    body="Redisearch implements a search engine on top of redis",
)

Querying

Basic queries

Use the

search()

FT.SEARCH

res = client.search("evil wizards")

Result objects

Results are wrapped in a

Result

>>> print(res.total)
2
>>> print(res.docs[0].title)
"Wizard Story 2: Evil Wizards Strike Back"

Building complex queries

You can use the

Query

q = Query("evil wizards").verbatim().no_content().with_scores().paging(0, 5)
res = client.search(q)

For an explanation of these options, see the RediSearch documentation for the

FT.SEARCH

Query syntax

The default behavior of queries is to run a full-text search across all

TEXT

So the example given in the "Basic queries" section of this README,

client.search("evil wizards")

TEXT

Many more types of queries are possible, however! The string you pass into the

search()

Query()

For example, a full-text search against a specific

TEXT

# Full-text search
res = client.search("@title:evil wizards")

Finding books published in 2020 or 2021 looks like this:

client.search("@published_year:[2020 2021]")

To learn more, see the RediSearch documentation on query syntax.

Aggregations

This library contains a programmatic interface to run aggregation queries with RediSearch.

Making an aggregation query

To make an aggregation query, pass an instance of the

AggregateRequest

search()

Client

For example, here is what finding the most books published in a single year looks like:

from redisearch import Client
from redisearch import reducers
from redisearch.aggregation import AggregateRequest

client = Client('books-idx')
request = AggregateRequest('*').group_by(
    '@published_year', reducers.count().alias("num_published")
).group_by(
    [], reducers.max("@num_published").alias("max_books_published_per_year")
)
result = client.aggregate(request)

A redis-cli equivalent query

The aggregation query just given is equivalent to the following

FT.AGGREGATE

FT.AGGREGATE books-idx *
    GROUPBY 1 @published_year
      REDUCE COUNT 0 AS num_published
    GROUPBY 0
      REDUCE MAX 1 @num_published AS max_books_published_per_year

The AggregateResult object

Aggregation queries return an

AggregateResult

from redisearch.aggregation import AggregateRequest, Asc

request = AggregateRequest('*').group_by(
    ['@published_year'], reducers.avg('average_rating').alias('average_rating_for_year')
).sort_by(
    Asc('@average_rating_for_year')
).limit(
    0, 10
).filter('@published_year > 0')
...
In [53]: resp = c.aggregate(request)
In [54]: resp.rows
Out[54]:
[['published_year', '1914', 'average_rating_for_year', '0'],
 ['published_year', '2009', 'average_rating_for_year', '1.39166666667'],
 ['published_year', '2011', 'average_rating_for_year', '2.046'],
 ['published_year', '2010', 'average_rating_for_year', '3.125'],
 ['published_year', '2012', 'average_rating_for_year', '3.41'],
 ['published_year', '1967', 'average_rating_for_year', '3.603'],
 ['published_year', '1970', 'average_rating_for_year', '3.71875'],
 ['published_year', '1966', 'average_rating_for_year', '3.72666666667'],
 ['published_year', '1927', 'average_rating_for_year', '3.77']]

Reducer functions

Notice from the example that we used an object from the

reducers

Reducer functions include an

alias()

Grouping by zero, one, or multiple fields

The

group_by

AggregateRequest('*').group_by('@published_year', reducers.count())

AggregateRequest('*').group_by(
    ['@published_year', '@average_rating'],
    reducers.count())

To run a reducer function on every result from an aggregation query, pass an empty list to

group_by()

GROUPBY 0

AggregateRequest('*').group_by([], reducers.max("@num_published"))

NOTE: Aggregation queries require at least one

group_by()

Sorting and limiting

Using an

AggregateRequest

sort_by()

limit()

For example, finding the average rating of books published each year, sorting by the average rating for the year, and returning only the first ten results:

from redisearch import Client
from redisearch.aggregation import AggregateRequest, Asc

c = Client()
request = AggregateRequest('*').group_by(
    ['@published_year'], reducers.avg('average_rating').alias('average_rating_for_year')
).sort_by(
    Asc('@average_rating_for_year')
).limit(0, 10)
c.aggregate(request)

NOTE: The first option to

limit()

Filtering

Use filtering to reject results of an aggregation query after your reducer functions run. For example, calculating the average rating of books published each year and only returning years with an average rating higher than 3:

from redisearch.aggregation import AggregateRequest, Asc

req = AggregateRequest('*').group_by(
    ['@published_year'], reducers.avg('average_rating').alias('average_rating_for_year')
).sort_by(
    Asc('@average_rating_for_year')
).filter('@average_rating_for_year > 3')

Installing

1. Install RediSearch
2. Install the Python client:

$ pip install redisearch

Developing

1. Create a virtualenv to manage your python dependencies, and ensure it's active.

   virtualenv -v venv

2. Install pypoetry to manage your dependencies.

   pip install --user poetry

3. Install dependencies.

   poetry install

Note: Due to an interaction between and python 3.10, you may need to run the following, if you receive a JSONError while installing packages.

poetry config experimental.new-installer false

Testing

Testing can easily be performed using using Docker. Run the following:

make -C test/docker test PYTHON_VER=3

(Replace

PYTHON_VER=3

PYTHON_VER=2

Alternatively, use the following procedure:

First, run:

PYTHON_VER=3 ./test/test-setup.sh

This will set up a Python virtual environment in

venv3

venv2

PYTHON_VER=2

Afterwards, run RediSearch in a container as a daemon:

docker run -d -p 6379:6379 redislabs/redisearch:2.0.0

Finally, invoke the virtual environment and run the tests:

. ./venv3/bin/activate
REDIS_PORT=6379 python test/test.py
REDIS_PORT=6379 python test/test_builder.py