Manage conda environments and Jupyterlab easily.
Managing environments in Python can be tricky. I used to have a default script to create a conda environment, in that script I would install all the packages I consider basic, plus Jupyterlab and some plugins. Yes a new fresh Jupyterlab+plugins for every environment.
A few days ago I read this tweet by Vicki Boykis and one of the answers from Peter Baumgartner and decided to change my workflow a little bit. Now I use the 2 scripts I linked above, and I will go through explaining what they do. Hopefully you can also learn a bit about conda too.
condacreate_dev.sh: this first one is used to create a base environment that will hold jupyterlab, and the package nb_conda_kernels which enables a Jupyter Notebook or JupyterLab application in one conda environment to access kernels for Python, R, and other languages found in other environments.
conda create --name "$NAME" python="$VERSION"
With that we create a new conda environment with the name we want and a Python version, in my case I execute the script like:
./condacreate_dev.sh dev 3.7 and it creates an environment called “dev” using Python 3.7.
conda install -y --name "$NAME" -c conda-forge ipython jupyterlab nb_conda_kernels black
Now we install the python packages I mentioned before. Using the flag
-y we accept the installation by default, so we don’t need to interact with our terminal the flag
--name lets you execute the command in the environment with that name (in this case it’s “dev”).
Then I install the extensions I use:
conda run -n "$NAME" jupyter labextension install @jupyterlab/toc --no-build conda run -n "$NAME" jupyter labextension install @jupyterlab/celltags --no-build conda run -n "$NAME" jupyter labextension install @jupyter-widgets/jupyterlab-manager --no-build conda run -n "$NAME" jupyter labextension install @ryantam626/jupyterlab_code_formatter --no-build
conda run executes a command,
-n is the same as
--name, the tag
--no-build tells Jupyterlab to not rebuild itself after installing every extension. Before knowing that tag existed I spent quite a lot of time creating new environments.
conda install -y --name "$NAME" -c conda-forge jupyterlab-git conda install -y --name "$NAME" -c conda-forge jupyterlab_code_formatter conda run -n "$NAME" jupyter serverextension enable --py jupyterlab_code_formatter conda run -n "$NAME" jupyter lab build
Finally, I install some more extensions and I rebuild Jupyterlab with the last command.
The last lines:
if [[ $3 == 'alias' ]]; then echo "alias $NAME='conda activate $NAME'" >> ~/dotfiles/.condalias fi
With those I can create an alias with the same name of the environment to activate it, so if my initial command was
./condacreate_dev.sh dev 3.7 alias it would have created an alias so that by running the command
dev in my terminal would activate that environment.
condacreate.sh: the second one creates an environment with the base packages I use. At the end we run:
conda run -n "$NAME" ipython kernel install --user --name="$NAME"
And with that we register the environment to be used every time I run Jupyter.
Apart from that, I use a couple of bash aliases that simplify the workflow a lot:
alias createpy='bash ~/Projects/scripts/condacreate.sh' alias createdev='bash ~/Projects/scripts/condacreate_dev.sh' alias lab='conda run -n dev jupyter lab' alias blab='nohup conda run -n dev jupyter lab --browser firefox &>/dev/null &'
The first two are for creating the environments, the other are for running Jupyterlab from the base environment (called “dev”). The most special one is the last one, which runs Jupyterlab in the background so that it does not die if you mistakenly close the terminal.
My final workflow would be:
# create base env createdev dev 3.7 # create env for nlp and make an alias called "nlp" to activate it createpy nlp 3.7 alias # activate that env nlp # and install whatever I need conda install -c conda-forge spacy gensim # deactivate the environment conda deactivate
And now from anywhere I can run
blab to run Jupyterlab and create a new notebook using the environment called “nlp”.
Lastly, you can run
jupyter kernelspec list with the “dev” environment activated to list the registered kernels and
jupyter kernelspec uninstall <name> to remove the kernel called