numpyml
Ever wish you had an inefficient but somewhat legible collection of machine
learning algorithms implemented exclusively in NumPy? No?
Installation
For rapid experimentation
To use this code as a starting point for ML prototyping / experimentation, just clone the repository, create a new virtualenv, and start hacking:
$ git clone https://github.com/ddbourgin/numpyml.git
$ cd numpyml && virtualenv npml && source npml/bin/activate
$ pip3 install r requirementsdev.txt
As a package
If you don't plan to modify the source, you can also install numpyml as a
Python package:
pip3 install u numpy_ml
.
The reinforcement learning agents train on environments defined in the OpenAI
gym. To install these alongside numpyml, you
can use
pip3 install u 'numpy_ml[rl]'
.
Documentation
For more details on the available models, see the project documentation.
Available models

Gaussian mixture model

Hidden Markov model
 Viterbi decoding
 Likelihood computation
 MLE parameter estimation via BaumWelch/forwardbackward algorithm

Latent Dirichlet allocation (topic model)
 Standard model with MLE parameter estimation via variational EM
 Smoothed model with MAP parameter estimation via MCMC

Neural networks
 Layers / Layerwise ops
 Add
 Flatten
 Multiply
 Softmax
 Fullyconnected/Dense
 Sparse evolutionary connections
 LSTM
 Elmanstyle RNN
 Max + average pooling
 Dotproduct attention
 Embedding layer
 Restricted Boltzmann machine (w. CDn training)
 2D deconvolution (w. padding and stride)
 2D convolution (w. padding, dilation, and stride)
 1D convolution (w. padding, dilation, stride, and causality)
 Modules
 Bidirectional LSTM
 ResNetstyle residual blocks (identity and convolution)
 WaveNetstyle residual blocks with dilated causal convolutions
 Transformerstyle multiheaded scaled dot product attention
 Regularizers
 Normalization
 Batch normalization (spatial and temporal)
 Layer normalization (spatial and temporal)
 Optimizers
 SGD w/ momentum
 AdaGrad
 RMSProp
 Adam
 Learning Rate Schedulers
 Constant
 Exponential
 Noam/Transformer
 Dlib scheduler
 Weight Initializers
 Glorot/Xavier uniform and normal
 He/Kaiming uniform and normal
 Standard and truncated normal
 Losses
 Cross entropy
 Squared error
 Bernoulli VAE loss
 Wasserstein loss with gradient penalty
 Noise contrastive estimation loss
 Activations
 ReLU
 Tanh
 Affine
 Sigmoid
 Leaky ReLU
 ELU
 SELU
 Exponential
 Hard Sigmoid
 Softplus
 Models
 Bernoulli variational autoencoder
 Wasserstein GAN with gradient penalty
 word2vec encoder with skipgram and CBOW architectures
 Utilities

col2im
(MATLAB port)

im2col
(MATLAB port)
conv1D
conv2D
deconv2D
minibatch

Treebased models
 Decision trees (CART)
 [Bagging] Random forests
 [Boosting] Gradientboosted decision trees

Linear models
 Ridge regression
 Logistic regression
 Ordinary least squares
 Bayesian linear regression w/ conjugate priors
 Unknown mean, known variance (Gaussian prior)
 Unknown mean, unknown variance (NormalGamma / NormalInverseWishart prior)

nGram sequence models
 Maximum likelihood scores
 Additive/Lidstone smoothing
 Simple GoodTuring smoothing

Multiarmed bandit models
 UCB1
 LinUCB
 Epsilongreedy
 Thompson sampling w/ conjugate priors
 LinUCB

Reinforcement learning models
 Crossentropy method agent
 First visit onpolicy Monte Carlo agent
 Weighted incremental importance sampling Monte Carlo agent
 Expected SARSA agent
 TD0 Qlearning agent
 DynaQ / DynaQ+ with prioritized sweeping

Nonparameteric models
 NadarayaWatson kernel regression
 kNearest neighbors classification and regression
 Gaussian process regression

Matrix factorization
 Regularized alternating leastsquares
 Nonnegative matrix factorization

Preprocessing
 Discrete Fourier transform (1D signals)
 Discrete cosine transform (typeII) (1D signals)
 Bilinear interpolation (2D signals)
 Nearest neighbor interpolation (1D and 2D signals)
 Autocorrelation (1D signals)
 Signal windowing
 Text tokenization
 Feature hashing
 Feature standardization
 Onehot encoding / decoding
 Huffman coding / decoding
 Term frequencyinverse document frequency (TFIDF) encoding
 MFCC encoding

Utilities
 Similarity kernels
 Distance metrics
 Priority queue
 Ball tree
 Discrete sampler
 Graph processing and generators
Contributing
Am I missing your favorite model? Is there something that could be cleaner /
less confusing? Did I mess something up? Submit a PR! The only requirement is
that your models are written with just the Python standard
library and NumPy. The
SciPy library is also permitted under special
circumstances ;)
See full contributing guidelines here.