a numpy-like fast vector module for micropython, circuitpython, and their derivatives
ulabis a
numpy-like array manipulation library for micropython and CircuitPython. The module is written in C, defines compact containers for numerical data of one to four dimensions, and is fast. The library is a software-only standard
micropythonuser module, i.e., it has no hardware dependencies, and can be compiled for any platform. The
floatimplementation of
micropython(
float, or
double) is automatically detected.
ulabimplements
numpy's
ndarraywith the
==,
!=,
<,
<=,
>,
>=,
+,
-,
/,
*,
**,
+=,
-=,
*=,
/=,
**=binary operators, and the
len,
~,
-,
+,
absunary operators that operate element-wise. Type-aware
ndarrays can be initialised from any
micropythoniterable, lists of iterables via the
arrayconstructor, or by means of the
arange,
concatenate,
diag,
eye,
frombuffer,
full,
linspace,
logspace,
ones, or
zerosfunctions.
ndarrays can be iterated on, and have a number of their own methods, such as
flatten,
itemsize,
reshape,
shape,
size,
strides,
tobytes, and
transpose.
In addition to the
ndarrayoperators and methods,
ulabdefines a great number of functions that can take
ndarrays or
micropythoniterables as their arguments. Most of the functions have been ported from
numpy, but several are re-implementations of
scipyfeatures. For a complete list, see micropython-ulab!
If flash space is a concern, unnecessary functions can be excluded from the compiled firmware with pre-processor switches. In addition,
ulabalso has options for trading execution speed for firmware size. A thorough discussion on how the firmware can be customised can be found in the corresponding section of the user manual.
It is also possible to extend the library with arbitrary user-defined functions operating on numerical arrays, and add them to the namespace, as explaind in the programming manual.
ulabsports a
numpy/scipy-compatible interface, which makes porting of
CPythoncode straightforward. The following snippet should run equally well in
micropython, or on a PC.
try: from ulab import numpy as np from ulab import scipy as spy except ImportError: import numpy as np import scipy as spyx = np.array([1, 2, 3]) spy.special.erf(x)
Documentation can be found on readthedocs under micropython-ulab, as well as at circuitpython-ulab. A number of practical examples are listed in Jeff Epler's excellent circuitpython-ulab overview.
Representative numbers on performance can be found under ulab samples.
Compiled firmware for many hardware platforms can be downloaded from Roberto Colistete's gitlab repository: for the pyboard, and for ESP8266. Since a number of features can be set in the firmware (threading, support for SD card, LEDs, user switch etc.), and it is impossible to create something that suits everyone, these releases should only be used for quick testing of
ulab. Otherwise, compilation from the source is required with the appropriate settings, which are usually defined in the
mpconfigboard.hfile of the port in question.
ulabis also included in the following compiled
micropythonvariants and derivatives:
CircuitPythonfor SAMD51 and nRF microcontrollers https://github.com/adafruit/circuitpython
MicroPython for K210https://github.com/loboris/MicroPythonK210LoBo
MaixPyhttps://github.com/sipeed/MaixPy
OpenMVhttps://github.com/openmv/openmv
pycomhttps://pycom.io/
If you want to try the latest version of
ulabon
micropythonor one of its forks, the firmware can be compiled from the source by following these steps:
Simply clone the
ulabrepository with
git clone https://github.com/v923z/micropython-ulab.git ulab
and then run
./build.sh
This command will clone
micropython, and build the
unixport automatically, as well as run the test scripts. If you want an interactive
unixsession, you can launch it in
ulab/micropython/ports/unix
First, you have to clone the
micropythonrepository by running
git clone https://github.com/micropython/micropython.git
on the command line. This will create a new repository with the name
micropython. Staying there, clone the
ulabrepository with
git clone https://github.com/v923z/micropython-ulab.git ulab
If you don't have the cross-compiler installed, your might want to do that now, for instance on Linux by executing
sudo apt-get install gcc-arm-none-eabi
If this step was successful, you can try to run the
makecommand in the port's directory as
make BOARD=PYBV11 USER_C_MODULES=../../../ulab all
which will prepare the firmware for pyboard.v.11. Similarly,
make BOARD=PYBD_SF6 USER_C_MODULES=../../../ulab all
will compile for the SF6 member of the PYBD series. If your target is
unix, you don't need to specify the
BOARDparameter.
Provided that you managed to compile the firmware, you would upload that by running either
dfu-util --alt 0 -D firmware.dfu
or
python pydfu.py -u firmware.dfu
In case you got stuck somewhere in the process, a bit more detailed instructions can be found under https://github.com/micropython/micropython/wiki/Getting-Started, and https://github.com/micropython/micropython/wiki/Pyboard-Firmware-Update.
cd $BUILD_DIR/micropython git checkout b137d064e9e0bfebd2a59a9b312935031252e742 # choose micropython version - note v1.12 is incompatible with ulab # and v1.13 is currently broken in some ways (on some platforms) https://github.com/BradenM/micropy-cli/issues/167 # - the patch is not live yet (should be in 1.14), but is at this commit git submodule update --init cd $BUILD_DIR/micropython/mpy-cross && make # build cross-compiler (required)cd $BUILD_DIR/micropython/ports/esp32 make ESPIDF= # will display supported ESP-IDF commit hashes
output should look like: """
...
Supported git hash (v3.3): 9e70825d1e1cbf7988cf36981774300066580ea7
Supported git hash (v4.0) (experimental): 4c81978a3e2220674a432a588292a4c860eef27b
Choose an ESPIDF version from one of the options printed by the previous command:
ESPIDF_VER=9e70825d1e1cbf7988cf36981774300066580ea7Download and prepare the SDK
git clone https://github.com/espressif/esp-idf.git $BUILD_DIR/esp-idf cd $BUILD_DIR/esp-idf git checkout $ESPIDF_VER git submodule update --init --recursive # get idf submodules pip install -r ./requirements.txt # install python reqs
Next, install the ESP32 compiler. If using an ESP-IDF version >= 4.x (chosen by
$ESPIDF_VERabove), this can be done by running
. $BUILD_DIR/esp-idf/install.sh. Otherwise, (for version 3.x) run:
cd $BUILD_DIRfor 64 bit linux
curl https://dl.espressif.com/dl/xtensa-esp32-elf-linux64-1.22.0-80-g6c4433a-5.2.0.tar.gz | tar xvz
for 32 bit
curl https://dl.espressif.com/dl/xtensa-esp32-elf-linux32-1.22.0-80-g6c4433a-5.2.0.tar.gz | tar xvz
don't worry about adding to path; we'll specify that later
also, see https://docs.espressif.com/projects/esp-idf/en/v3.3.2/get-started for more info
We can now clone the
ulabrepository
git clone https://github.com/v923z/micropython-ulab $BUILD_DIR/ulab
Finally, build the firmware:
cd $BUILD_DIR/micropython/ports/esp32 # temporarily add esp32 compiler to path export PATH=$BUILD_DIR/xtensa-esp32-elf/bin:$PATH export ESPIDF=$BUILD_DIR/esp-idf # req'd by Makefile export BOARD=GENERIC # options are dirs in ./boards export USER_C_MODULES=$BUILD_DIR/ulab # include ulab in firmwaremake submodules & make all
If it compiles without error, you can plug in your ESP32 via USB and then flash it with:
make erase && make deploy
If you find a problem with the code, please, raise an issue! An issue should address a single problem, and should contain a minimal code snippet that demonstrates the difference from the expected behaviour. Reducing a problem to the bare minimum significantly increases the chances of a quick fix.
Feature requests (porting a particular function from
numpyor
scipy) should also be posted at ulab issue.
Contributions of any kind are always welcome. If you feel like adding to the code, you can simply issue a pull request. If you do so, please, try to adhere to
micropython's coding conventions.
However, you can also contribute to the documentation (preferably via the jupyter notebooks, or improve the tests.
If you decide to lend a hand with testing, here are the steps:
ulab! This will clone the latest
micropython, compile the firmware for
unix, execute all scripts in the
ulab/tests, and compare the results to those in the expected results files, which are also in
ulab/tests, and have an extension
.exp. In case you have a new snippet, i.e., you have no expected results file, or if the results differ from those in the expected file, a new expected file will be generated in the root directory. You should inspect the contents of this file, and if they are satisfactory, then the file can be moved to the
ulab/testsfolder, alongside your snippet.