Using Project Jupyter for data science.
A few (hopefully) useful tips and tricks to using the Jupyter Notebook with an eye to pragmatic usage. This is not, in any way, an exhaustive demonstration of the features of the Jupyter notebook. Further, you can go through these notebooks on your own, but I usually demonstrate using them and give lots of information verbally.
If you have any suggestions, edits, or corrections, please open an issue or let me know some other way. Best of luck!
cd ~/Downloads wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh bash Miniconda3-latest-MacOSX-x86_64.sh # go through the licensing and accept the defaults source ~/.bashrc conda update conda
The following commands are how I set up both my conda config and the enviroments that I use.
Add conda-forge to your automatic channels, and the second line makes it so that you don't have to confirm that you want to install when you do things like
conda install numpy.
conda config --add channels conda-forge conda config --set always_yes yes
This following block is bash -- I recommend pasting in the commands one at a time to see what's happening.
# set the environment name here
packages=' altair anaconda-client black bqplot ipyvolume ipywebrtc ipywidgets jupyter jupyter_contrib_nbextensions jupyterlab matplotlib mkl mpld3 notebook numpy pandas pip pivottablejs pyparsing pyscaffold qgrid scikit-learn scipy seaborn sphinx statsmodels vaex vega vega_datasets xlrd yapf '
conda create -n $envname python=3.8 $packages conda activate $envname
Pause here, double check that this pip is the correct one
the correct one will say something like...
$ type pip
pip is /Users/jonathan/miniconda3/envs/dspy3/bin/pip
python -m pip install pyhive sql_magic SQLAlchemy nbdime papermill nbdev
lets the notebook extension (like ToC2) be enabled.
Might not be needed!
jupyter nbextension enable --py --sys-prefix widgetsnbextension
This sets the name of the kernel that you want to select from the Kernel menu
python -m ipykernel install --user --name $envname --display-name "$envname"
conda install -c conda-forge nodejs
jupyter labextension install @jupyter-widgets/jupyterlab-manager jupyter labextension install ipyvolume jupyter-threejs @jupyterlab/toc jupyter labextension install jupyter-threejs bqplot nbgather qgrid
If you see an error message that says something about the iopubdatarate_limit when you're trying to plot, try starting the notebook/lab with the following modified commands:
# Run to get a notebook jupyter notebook --NotebookApp.iopub_data_rate_limit=10000000
Run to get lab
jupyter lab --NotebookApp.iopub_data_rate_limit=10000000
# create environment conda env export -n dspy3 -f environment.lock.yaml # load conda env update --file environment.yaml
to create new python package I did this from root directory of this repo
putup insight cd insight/ python setup.py develop