speech_recognition

by Uberi

Speech recognition module for Python, supporting several engines and APIs, online and offline.

5.1K Stars 1.8K Forks Last release: almost 3 years ago (3.8.1) Other 360 Commits 45 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

SpeechRecognition

.. image:: https://img.shields.io/pypi/v/SpeechRecognition.svg :target: https://pypi.python.org/pypi/SpeechRecognition/ :alt: Latest Version

.. image:: https://img.shields.io/pypi/status/SpeechRecognition.svg :target: https://pypi.python.org/pypi/SpeechRecognition/ :alt: Development Status

.. image:: https://img.shields.io/pypi/pyversions/SpeechRecognition.svg :target: https://pypi.python.org/pypi/SpeechRecognition/ :alt: Supported Python Versions

.. image:: https://img.shields.io/pypi/l/SpeechRecognition.svg :target: https://pypi.python.org/pypi/SpeechRecognition/ :alt: License

.. image:: https://api.travis-ci.org/Uberi/speechrecognition.svg?branch=master :target: https://travis-ci.org/Uberi/speechrecognition :alt: Continuous Integration Test Results

Library for performing speech recognition, with support for several engines and APIs, online and offline.

Speech recognition engine/API support:

  • CMU Sphinx 
    __ (works offline)
  • Google Speech Recognition
  • Google Cloud Speech API 
    __
  • Wit.ai 
    __
  • Microsoft Azure Speech 
    __
  • Microsoft Bing Voice Recognition (Deprecated) 
    __
  • Houndify API 
    __
  • IBM Speech to Text 
    __
  • Snowboy Hotword Detection 
    __ (works offline)

Quickstart:

pip install SpeechRecognition
. See the "Installing" section for more details.

To quickly try it out, run

python -m speech_recognition
after installing.

Project links:

  • PyPI 
    __
  • Source code 
    __
  • Issue tracker 
    __

Library Reference

The

library reference 
__ documents every publicly accessible object in the library. This document is also included under
reference/library-reference.rst
.

See

Notes on using PocketSphinx 
__ for information about installing languages, compiling PocketSphinx, and building language packs from online resources. This document is also included under
reference/pocketsphinx.rst
.

Examples

See the

examples/
directory 
__ in the repository root for usage examples:
  • Recognize speech input from the microphone 
    __
  • Transcribe an audio file 
    __
  • Save audio data to an audio file 
    __
  • Show extended recognition results 
    __
  • Calibrate the recognizer energy threshold for ambient noise levels 
    __ (see
    recognizer_instance.energy_threshold
    for details)
  • Listening to a microphone in the background 
    __
  • Various other useful recognizer features 
    __

Installing

First, make sure you have all the requirements listed in the "Requirements" section.

The easiest way to install this is using

pip install SpeechRecognition
.

Otherwise, download the source distribution from

PyPI 
__, and extract the archive.

In the folder, run

python setup.py install
.

Requirements

To use all of the functionality of the library, you should have:

  • Python 2.6, 2.7, or 3.3+ (required)
  • PyAudio 0.2.11+ (required only if you need to use microphone input,
    Microphone
    )
  • PocketSphinx (required only if you need to use the Sphinx recognizer,
    recognizer_instance.recognize_sphinx
    )
  • Google API Client Library for Python (required only if you need to use the Google Cloud Speech API,
    recognizer_instance.recognize_google_cloud
    )
  • FLAC encoder (required only if the system is not x86-based Windows/Linux/OS X)

The following requirements are optional, but can improve or extend functionality in some situations:

  • On Python 2, and only on Python 2, some functions (like
    recognizer_instance.recognize_bing
    ) will run slower if you do not have Monotonic for Python 2 installed.
  • If using CMU Sphinx, you may want to
    install additional language packs 
    __ to support languages like International French or Mandarin Chinese.

The following sections go over the details of each requirement.

Python ~~~~~~

The first software requirement is

Python 2.6, 2.7, or Python 3.3+ 
__. This is required to use the library.

PyAudio (for microphone users) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

PyAudio 
__ is required if and only if you want to use microphone input (
Microphone
). PyAudio version 0.2.11+ is required, as earlier versions have known memory management bugs when recording from microphones in certain situations.

If not installed, everything in the library will still work, except attempting to instantiate a

Microphone
object will raise an
AttributeError
.

The installation instructions on the PyAudio website are quite good - for convenience, they are summarized below:

  • On Windows, install PyAudio using
    Pip 
    __: execute
    pip install pyaudio
    in a terminal.
  • On Debian-derived Linux distributions (like Ubuntu and Mint), install PyAudio using
    APT 
    __: execute
    sudo apt-get install python-pyaudio python3-pyaudio
    in a terminal.
    • If the version in the repositories is too old, install the latest release using Pip: execute
      sudo apt-get install portaudio19-dev python-all-dev python3-all-dev && sudo pip install pyaudio
      (replace
      pip
      with
      pip3
      if using Python 3).
  • On OS X, install PortAudio using
    Homebrew 
    :
    brew install portaudio
    . Then, install PyAudio using
    Pip 
    :
    pip install pyaudio
    .
  • On other POSIX-based systems, install the
    portaudio19-dev
    and
    python-all-dev
    (or
    python3-all-dev
    if using Python 3) packages (or their closest equivalents) using a package manager of your choice, and then install PyAudio using
    Pip 
    __:
    pip install pyaudio
    (replace
    pip
    with
    pip3
    if using Python 3).

PyAudio

wheel packages 
__ for common 64-bit Python versions on Windows and Linux are included for convenience, under the
third-party/
directory 
__ in the repository root. To install, simply run
pip install wheel
followed by
pip install ./third-party/WHEEL_FILENAME
(replace
pip
with
pip3
if using Python 3) in the repository
root directory 
__.

PocketSphinx-Python (for Sphinx users) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

PocketSphinx-Python 
__ is required if and only if you want to use the Sphinx recognizer (
recognizer_instance.recognize_sphinx
).

PocketSphinx-Python

wheel packages 
__ for 64-bit Python 2.7, 3.4, and 3.5 on Windows are included for convenience, under the
third-party/
directory 
_. To install, simply run
pip install wheel
followed by ``pip install ./third-party/WHEEL
FILENAME
(replace
pip
with
pip3`` if using Python 3) in the SpeechRecognition folder.

On Linux and other POSIX systems (such as OS X), follow the instructions under "Building PocketSphinx-Python from source" in

Notes on using PocketSphinx 
__ for installation instructions.

Note that the versions available in most package repositories are outdated and will not work with the bundled language data. Using the bundled wheel packages or building from source is recommended.

See

Notes on using PocketSphinx 
__ for information about installing languages, compiling PocketSphinx, and building language packs from online resources. This document is also included under
reference/pocketsphinx.rst
.

Google Cloud Speech Library for Python (for Google Cloud Speech API users) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Google Cloud Speech library for Python 
__ is required if and only if you want to use the Google Cloud Speech API (
recognizer_instance.recognize_google_cloud
).

If not installed, everything in the library will still work, except calling

recognizer_instance.recognize_google_cloud
will raise an
RequestError
.

According to the

official installation instructions 
, the recommended way to install this is using
Pip 
: execute
pip install google-cloud-speech
(replace
pip
with
pip3
if using Python 3).

FLAC (for some systems) ~~~~~~~~~~~~~~~~~~~~~~~

A

FLAC encoder 
__ is required to encode the audio data to send to the API. If using Windows (x86 or x86-64), OS X (Intel Macs only, OS X 10.6 or higher), or Linux (x86 or x86-64), this is already bundled with this library - you do not need to install anything.

Otherwise, ensure that you have the

flac
command line tool, which is often available through the system package manager. For example, this would usually be
sudo apt-get install flac
on Debian-derivatives, or
brew install flac
on OS X with Homebrew.

Monotonic for Python 2 (for faster operations in some functions on Python 2) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

On Python 2, and only on Python 2, if you do not install the

Monotonic for Python 2 
__ library, some functions will run slower than they otherwise could (though everything will still work correctly).

On Python 3, that library's functionality is built into the Python standard library, which makes it unnecessary.

This is because monotonic time is necessary to handle cache expiry properly in the face of system time changes and other time-related issues. If monotonic time functionality is not available, then things like access token requests will not be cached.

To install, use

Pip 
__: execute
pip install monotonic
in a terminal.

Troubleshooting

The recognizer tries to recognize speech even when I'm not speaking, or after I'm done speaking. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Try increasing the

recognizer_instance.energy_threshold
property. This is basically how sensitive the recognizer is to when recognition should start. Higher values mean that it will be less sensitive, which is useful if you are in a loud room.

This value depends entirely on your microphone or audio data. There is no one-size-fits-all value, but good values typically range from 50 to 4000.

Also, check on your microphone volume settings. If it is too sensitive, the microphone may be picking up a lot of ambient noise. If it is too insensitive, the microphone may be rejecting speech as just noise.

The recognizer can't recognize speech right after it starts listening for the first time. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The

recognizer_instance.energy_threshold
property is probably set to a value that is too high to start off with, and then being adjusted lower automatically by dynamic energy threshold adjustment. Before it is at a good level, the energy threshold is so high that speech is just considered ambient noise.

The solution is to decrease this threshold, or call

recognizer_instance.adjust_for_ambient_noise
beforehand, which will set the threshold to a good value automatically.

The recognizer doesn't understand my particular language/dialect. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Try setting the recognition language to your language/dialect. To do this, see the documentation for

recognizer_instance.recognize_sphinx
,
recognizer_instance.recognize_google
,
recognizer_instance.recognize_wit
,
recognizer_instance.recognize_bing
,
recognizer_instance.recognize_api
,
recognizer_instance.recognize_houndify
, and
recognizer_instance.recognize_ibm
.

For example, if your language/dialect is British English, it is better to use

"en-GB"
as the language rather than
"en-US"
.

The recognizer hangs on

recognizer_instance.listen
; specifically, when it's calling
Microphone.MicrophoneStream.read
. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This usually happens when you're using a Raspberry Pi board, which doesn't have audio input capabilities by itself. This causes the default microphone used by PyAudio to simply block when we try to read it. If you happen to be using a Raspberry Pi, you'll need a USB sound card (or USB microphone).

Once you do this, change all instances of

Microphone()
to
Microphone(device_index=MICROPHONE_INDEX)
, where
MICROPHONE_INDEX
is the hardware-specific index of the microphone.

To figure out what the value of

MICROPHONE_INDEX
should be, run the following code:

.. code:: python

import speech_recognition as sr
for index, name in enumerate(sr.Microphone.list_microphone_names()):
    print("Microphone with name \"{1}\" found for `Microphone(device_index={0})`".format(index, name))

This will print out something like the following:

::

Microphone with name "HDA Intel HDMI: 0 (hw:0,3)" found for `Microphone(device_index=0)`
Microphone with name "HDA Intel HDMI: 1 (hw:0,7)" found for `Microphone(device_index=1)`
Microphone with name "HDA Intel HDMI: 2 (hw:0,8)" found for `Microphone(device_index=2)`
Microphone with name "Blue Snowball: USB Audio (hw:1,0)" found for `Microphone(device_index=3)`
Microphone with name "hdmi" found for `Microphone(device_index=4)`
Microphone with name "pulse" found for `Microphone(device_index=5)`
Microphone with name "default" found for `Microphone(device_index=6)`

Now, to use the Snowball microphone, you would change

Microphone()
to
Microphone(device_index=3)
.

Calling

Microphone()
gives the error
IOError: No Default Input Device Available
. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As the error says, the program doesn't know which microphone to use.

To proceed, either use

Microphone(device_index=MICROPHONE_INDEX, ...)
instead of
Microphone(...)
, or set a default microphone in your OS. You can obtain possible values of
MICROPHONE_INDEX
using the code in the troubleshooting entry right above this one.

The code examples raise

UnicodeEncodeError: 'ascii' codec can't encode character
when run. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When you're using Python 2, and your language uses non-ASCII characters, and the terminal or file-like object you're printing to only supports ASCII, an error is raised when trying to write non-ASCII characters.

This is because in Python 2,

recognizer_instance.recognize_sphinx
,
recognizer_instance.recognize_google
,
recognizer_instance.recognize_wit
,
recognizer_instance.recognize_bing
,
recognizer_instance.recognize_api
,
recognizer_instance.recognize_houndify
, and
recognizer_instance.recognize_ibm
return unicode strings (
u"something"
) rather than byte strings (
"something"
). In Python 3, all strings are unicode strings.

To make printing of unicode strings work in Python 2 as well, replace all print statements in your code of the following form:

.. code:: python

print SOME_UNICODE_STRING

With the following:

.. code:: python

print SOME_UNICODE_STRING.encode("utf8")

This change, however, will prevent the code from working in Python 3.

The program doesn't run when compiled with

PyInstaller 
__. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As of PyInstaller version 3.0, SpeechRecognition is supported out of the box. If you're getting weird issues when compiling your program using PyInstaller, simply update PyInstaller.

You can easily do this by running

pip install --upgrade pyinstaller
.

On Ubuntu/Debian, I get annoying output in the terminal saying things like "btaudioservice_open: [...] Connection refused" and various others. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The "btaudioservice_open" error means that you have a Bluetooth audio device, but as a physical device is not currently connected, we can't actually use it - if you're not using a Bluetooth microphone, then this can be safely ignored. If you are, and audio isn't working, then double check to make sure your microphone is actually connected. There does not seem to be a simple way to disable these messages.

For errors of the form "ALSA lib [...] Unknown PCM", see

this StackOverflow answer 
__. Basically, to get rid of an error of the form "Unknown PCM cards.pcm.rear", simply comment out
pcm.rear cards.pcm.rear
in
/usr/share/alsa/alsa.conf
,
~/.asoundrc
, and
/etc/asound.conf
.

For "jack server is not running or cannot be started" or "connect(2) call to /dev/shm/jack-1000/default/jack0 failed (err=No such file or directory)" or "attempt to connect to server failed", these are caused by ALSA trying to connect to JACK, and can be safely ignored. I'm not aware of any simple way to turn those messages off at this time, besides `entirely disabling printing while starting the microphone <https://github.com/Uberi/speechrecognition/issues/182#issuecomment-266256337>`__.

On OS X, I get a

ChildProcessError
saying that it couldn't find the system FLAC converter, even though it's installed. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Installing

FLAC for OS X 
__ directly from the source code will not work, since it doesn't correctly add the executables to the search path.

Installing FLAC using

Homebrew 
__ ensures that the search path is correctly updated. First, ensure you have Homebrew, then run
brew install flac
to install the necessary files.

Developing

To hack on this library, first make sure you have all the requirements listed in the "Requirements" section.

  • Most of the library code lives in
    speech_recognition/__init__.py
    .
  • Examples live under the
    examples/
    directory 
    , and the demo script lives in ``speech_recognition/main__.py``.
  • The FLAC encoder binaries are in the
    speech_recognition/
    directory 
    __.
  • Documentation can be found in the
    reference/
    directory 
    __.
  • Third-party libraries, utilities, and reference material are in the
    third-party/
    directory 
    __.

To install/reinstall the library locally, run

python setup.py install
in the project
root directory 
__.

Before a release, the version number is bumped in

README.rst
and
speech_recognition/__init__.py
. Version tags are then created using
git config gpg.program gpg2 && git config user.signingkey DB45F6C431DE7C2DCD99FF7904882258A4063489 && git tag -s VERSION_GOES_HERE -m "Version VERSION_GOES_HERE"
.

Releases are done by running

make-release.sh VERSION_GOES_HERE
to build the Python source packages, sign them, and upload them to PyPI.

Testing ~~~~~~~

To run all the tests:

.. code:: bash

python -m unittest discover --verbose

Testing is also done automatically by TravisCI, upon every push. To set up the environment for offline/local Travis-like testing on a Debian-like system:

.. code:: bash

sudo docker run --volume "$(pwd):/speech_recognition" --interactive --tty quay.io/travisci/travis-python:latest /bin/bash
su - travis && cd /speech_recognition
sudo apt-get update && sudo apt-get install swig libpulse-dev
pip install --user pocketsphinx monotonic && pip install --user flake8 rstcheck && pip install --user -e .
python -m unittest discover --verbose # run unit tests
python -m flake8 --ignore=E501,E701 speech_recognition tests examples setup.py # ignore errors for long lines and multi-statement lines
python -m rstcheck README.rst reference/*.rst # ensure RST is well-formed

FLAC Executables ~~~~~~~~~~~~~~~~

The included

flac-win32
executable is the
official FLAC 1.3.2 32-bit Windows binary 
__.

The included

flac-linux-x86
and
flac-linux-x86_64
executables are built from the
FLAC 1.3.2 source code 
__ with
Manylinux 
__ to ensure that it's compatible with a wide variety of distributions.

The built FLAC executables should be bit-for-bit reproducible. To rebuild them, run the following inside the project directory on a Debian-like system:

.. code:: bash

# download and extract the FLAC source code
cd third-party
sudo apt-get install --yes docker.io

build FLAC inside the Manylinux i686 Docker image

tar xf flac-1.3.2.tar.xz sudo docker run --tty --interactive --rm --volume "$(pwd):/root" quay.io/pypa/manylinux1_i686:latest bash cd /root/flac-1.3.2 ./configure LDFLAGS=-static # compiler flags to make a static build make exit cp flac-1.3.2/src/flac/flac ../speech_recognition/flac-linux-x86 && sudo rm -rf flac-1.3.2/

build FLAC inside the Manylinux x86_64 Docker image

tar xf flac-1.3.2.tar.xz sudo docker run --tty --interactive --rm --volume "$(pwd):/root" quay.io/pypa/manylinux1_x86_64:latest bash cd /root/flac-1.3.2 ./configure LDFLAGS=-static # compiler flags to make a static build make exit cp flac-1.3.2/src/flac/flac ../speech_recognition/flac-linux-x86_64 && sudo rm -r flac-1.3.2/

The included

flac-mac
executable is extracted from
xACT 2.39 
__, which is a frontend for FLAC 1.3.2 that conveniently includes binaries for all of its encoders. Specifically, it is a copy of
xACT 2.39/xACT.app/Contents/Resources/flac
in
xACT2.39.zip
.

Authors

::

Uberi  (Anthony Zhang)
bobsayshilol
arvindch  (Arvind Chembarpu)
kevinismith  (Kevin Smith)
haas85
DelightRun 
maverickagm
kamushadenes  (Kamus Hadenes)
sbraden  (Sarah Braden)
tb0hdan (Bohdan Turkynewych)
Thynix  (Steve Dougherty)
beeedy  (Broderick Carlin)

Please report bugs and suggestions at the

issue tracker 
__!

How to cite this library (APA style):

Zhang, A. (2017). Speech Recognition (Version 3.8) [Software]. Available from https://github.com/Uberi/speech_recognition#readme.

How to cite this library (Chicago style):

Zhang, Anthony. 2017. *Speech Recognition* (version 3.8).

Also check out the

Python Baidu Yuyin API 
, which is based on an older version of this project, and adds support for
Baidu Yuyin 
. Note that Baidu Yuyin is only available inside China.

License

Copyright 2014-2017

Anthony Zhang (Uberi) 
. The source code for this library is available online at
GitHub 
.

SpeechRecognition is made available under the 3-clause BSD license. See

LICENSE.txt
in the project's
root directory 
__ for more information.

For convenience, all the official distributions of SpeechRecognition already include a copy of the necessary copyright notices and licenses. In your project, you can simply say that licensing information for SpeechRecognition can be found within the SpeechRecognition README, and make sure SpeechRecognition is visible to users if they wish to see it.

SpeechRecognition distributes source code, binaries, and language files from

CMU Sphinx 
_. These files are BSD-licensed and redistributable as long as copyright notices are correctly retained. See ``speechrecognition/pocketsphinx-data//LICENSE.txt
and
third-party/LICENSE-Sphinx.txt`` for license details for individual parts.

SpeechRecognition distributes source code and binaries from

PyAudio 
__. These files are MIT-licensed and redistributable as long as copyright notices are correctly retained. See
third-party/LICENSE-PyAudio.txt
for license details.

SpeechRecognition distributes binaries from

FLAC 
__ -
speech_recognition/flac-win32.exe
,
speech_recognition/flac-linux-x86
, and
speech_recognition/flac-mac
. These files are GPLv2-licensed and redistributable, as long as the terms of the GPL are satisfied. The FLAC binaries are an
aggregate 
__ of
separate programs 
__, so these GPL restrictions do not apply to the library or your programs that use the library, only to FLAC itself. See
LICENSE-FLAC.txt
for license details.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.