Need help with neurocat?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

133 Stars 12 Forks Apache License 2.0 13 Commits 1 Opened issues


From neural networks to the Category of composable supervised learning algorithms in Scala with compile-time matrix checking based on singleton-types

Services available


Need anything else?

Contributors list

# 86,616
13 commits


Neurocat is an experimental toy library studying 2 things: - The link between category theory and supervised learning algorithm and neural networks through the concepts described in this amazing paper Backprop as Functor: A compositional perspective on supervised learning which tries to unify (partly at least) category theory & supervised learning concepts & simple neural networks/

  • how to represent matrices (and thus neural networks) which dimensions are checked at compile-time using the new feature singleton-types described in SIP-23 which allows to manipulate the integer
    as the value
    but also the type
    . My matrix experimentations use the little & really cool library singleton-ops by Frank S. Thomas the author of the great
    library too. Shapeless
    is a nice idea but not good for big naturals because this is a recursive structure (down to 0) checked at compile-time so

Very superficially, the idea of the paper is quite simple (minus a few details): - A supervised learning algorithm can be seen as a structure able to approximate a function

A -> B
relying on parameters
which are updated through an optimization/training process using a set of training samples. - This paper shows that the set of supervised learning algorithms equipped with 3 functions (implement, update-params, request-input) forms a symmetric monoidal category
and then demonstrates that supervised learning algorithms can be composed - It also shows that there exists a Functor from the category
of parametrised functions
P -> A -> B

ParaFn -> Learn
  • Then it shows that a (trained) neural network can be seen as an approximation of a function
    InputLayer -> OutputLayer
    parametrised by the weights
  • Thus it demonstrates there is also a Functor from the category of neural network (W, InputLayer, OutputLayer) to the category of parametrised functions (W -> InputLayer -> OutputLayer)

I: NNet -> ParaFn
  • By simple functor composition, you have then a Functor from neural networks to supervised learning algorithms:

NNet -> Learn : (ParaFn -> Learn) ∘ (NNet -> ParaFn)

I'll stop there for now but my work has just started and there are more concepts about the bimonoidal aspects of neural networks under euclidean space constraints and pending studies about recurrent networks and more.

Discovering that formulation, I just said: "Whoaaa that's cool, exactly what I had in mind without being able to put words on it".

Why? Because everything I've seen about neural networks looks like programming from the 70s, not like I program nowadays with Functional Programming, types & categories.

This starts unifying concepts and is exactly the reason of being of category theory in maths. I think programming learning algorithms will change a lot in the future exactly as programming backends changed a lot those last 10 years.

I'm just scratching the surface of all of those concepts. I'm not a NeuralNetwork expert at all neither a good mathematician so I just want to open this field of study in a language which now has singleton-types allowing really cool new ways of manipulating data structures

So first, have a look at this sample:

  • Basic Compile-time Matrix calculus:
  • Neural network layers transformed into Learn instances and then composed and trained:
  • Neural network + Huge Matrices that compiles in human time:

For info, to manipulate matrices, I used ND4J to have an array abstraction to test both in CPU or GPU mode but any library doing this could be used naturally.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.