metakernel

by Calysto

Calysto / metakernel

Jupyter/IPython Kernel Tools

219 Stars 69 Forks Last release: Not found BSD 3-Clause "New" or "Revised" License 1.0K Commits 66 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

A Jupyter kernel base class in Python which includes core magic functions (including help, command and file path completion, parallel and distributed processing, downloads, and much more).

.. image:: https://badge.fury.io/py/metakernel.png/ :target: http://badge.fury.io/py/metakernel

.. image:: https://coveralls.io/repos/Calysto/metakernel/badge.png?branch=master :target: https://coveralls.io/r/Calysto/metakernel

.. image:: https://travis-ci.org/Calysto/metakernel.svg :target: https://travis-ci.org/Calysto/metakernel

.. image:: https://anaconda.org/conda-forge/metakernel/badges/version.svg :target: https://anaconda.org/conda-forge/metakernel

.. image:: https://anaconda.org/conda-forge/metakernel/badges/downloads.svg :target: https://anaconda.org/conda-forge/metakernel

See Jupyter's docs on

wrapper kernels
_.

Additional magics can be installed within the new kernel package under a

magics
subpackage.

Features

  • Basic set of line and cell magics for all kernels.
    • Python magic for accessing python interpreter.
    • Run kernels in parallel.
    • Shell magics.
    • Classroom management magics.
  • Tab completion for magics and file paths.
  • Help for magics using ? or Shift+Tab.
  • Plot magic for setting default plot behavior.

Kernels based on Metakernel

  • matlabkernel, https://github.com/Calysto/matlabkernel
  • octavekernel, https://github.com/Calysto/octavekernel
  • calystoscheme, https://github.com/Calysto/calystoscheme
  • calystoprocessing, https://github.com/Calysto/calystoprocessing
  • java9kernel, https://github.com/Bachmann1234/java9kernel
  • xonshkernel, https://github.com/Calysto/xonshkernel
  • calystohy, https://github.com/Calysto/calystohy
  • gnuplotkernel, https://github.com/has2k1/gnuplotkernel
  • spylon_kernel, https://github.com/mariusvniekerk/spylon-kernel
  • wolfram_kernel, https://github.com/mmatera/iwolfram
  • saskernel, https://github.com/palmer0914/saskernel
  • pysyshkernel, https://github.com/Jaesin/psyshkernel
  • calystobash, https://github.com/Calysto/calystobash

... and many others.

Installation

You can install Metakernel through

pip
:

.. code::bash

pip install metakernel --upgrade

Installing

metakernel
from the
conda-forge
channel can be achieved by adding
conda-forge
to your channels with:

.. code::bash

conda config --add channels conda-forge

Once the

conda-forge
channel has been enabled,
metakernel
can be installed with:

.. code::bash

conda install metakernel

It is possible to list all of the versions of

metakernel
available on your platform with:

.. code::bash

conda search metakernel --channel conda-forge

Use MetaKernel Magics in IPython

Although MetaKernel is a system for building new kernels, you can use a subset of the magics in the IPython kernel.

.. code:: python

from metakernel import registeripythonmagics registeripythonmagics()

Put the following in your (or a system-wide)

ipython_config.py
file:

.. code:: python

# /etc/ipython/ipythonconfig.py c = getconfig() startup = [ 'from metakernel import registeripythonmagics', 'registeripythonmagics()', ] c.InteractiveShellApp.exec_lines = startup

Use MetaKernel Languages in Parallel

To use a MetaKernel language in parallel, do the following:

  1. Make sure that the Python module
    ipyparallel
    is installed. In the shell, type:

.. code:: bash

pip install ipyparallel

  1. To enable the extension in the notebook, in the shell, type:

.. code:: bash

ipcluster nbextension enable

  1. To start up a cluster, with 10 nodes, on a local IP address, in the shell, type:

.. code:: bash

ipcluster start --n=10 --ip=192.168.1.108

  1. Initialize the code to use the 10 nodes, inside the notebook from a host kernel
    MODULE
    and
    CLASSNAME
    (can be any metakernel kernel):

.. code:: bash

%parallel MODULE CLASSNAME

For example:

.. code:: bash

%parallel calysto_scheme CalystoScheme

  1. Run code in parallel, inside the notebook, type:

Execute a single line, in parallel:

.. code:: bash

%px (+ 1 1)

Or execute the entire cell, in parallel:

.. code:: bash

%%px (* clusterrank clusterrank)

Results come back in a Python list (Scheme vector), in

cluster_rank
order. (This will be a JSON representation in the future).

Therefore, the above would produce the result:

.. code:: bash

#10(0 1 4 9 16 25 36 49 64 81)

You can get the results back in any of the parallel magics (

%px
,
%%px
, or
%pmap
) in the host kernel by accessing the variable
_
(single underscore), or by using the
--set_variable VARIABLE
flag, like so:

.. code:: bash

%%px --setvariable results (* clusterrank cluster_rank)

Then, in the next cell, you can access

results
.

Notice that you can use the variable

cluster_rank
to partition parts of a problem so that each node is working on something different.

In the examples above, use

-e
to evaluate the code in the host kernel as well. Note that
cluster_rank
is not defined on the host machine, and that this assumes the host kernel is the same as the parallel machines.

Configuration

Metakernel
subclasses can be configured by the user. The configuration file name is determined by the
app_name
property of the subclass. For example, in the
Octave
kernel, it is
octave_kernel
. The user of the kernel can add an
octave_kernel_config.py
file to their
jupyter
config path. The base
MetaKernel
class offers
plot_settings
as a configurable trait. Subclasses can define other traits that they wish to make configurable.

As an example:

.. code:: bash

cat ~/.jupyter/octave_kernel_config.py
# use Qt as the default backend for plots
c.OctaveKernel.plot_settings = dict(backend='qt')

Documentation

Example notebooks can be viewed here_.

Documentation is available online. Magics have interactive help (and online).

For version information, see the Revision History_.

.. _here: http://nbviewer.ipython.org/github/Calysto/metakernel/tree/master/examples/

.. _help: https://github.com/Calysto/metakernel/blob/master/metakernel/magics/README.md

.. _online: http://Calysto.github.io/metakernel/

.. _History: https://github.com/Calysto/metakernel/blob/master/HISTORY.rst

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.