# How I Set up Multiple Python Versions with pyenv on Linux

**TL;DR:** In this tutorial, I’ll teach you what I do to have multiple Python versions and tools installed without conflicts.

I’ve been working with Python for the last 6 years. In the beginning academically and as a hobby, then professionally. One thing that has always been very common is having multiple projects in different Python versions. 

Regardless of the language you use, having projects targeting at distinct versions can be a nightmare. When it comes to Python, it’s no different. As a matter of fact, it’s worse. 

For instance, there are libraries that only work with Python 2, despite not being supported anymore. Fortunately, there are a couple of tools that can help us have seamless experience handling different Python versions.

This guide will focus on Linux, but you can easily adapt to macOS. In fact, I have a MacBook at work and follow the same steps to configure my workspace with little modifications. Also, I use `zsh` as my default shell, but this guide applies to `bash` as well.

So, without further ado, here's how I setup my Python workspace:

##  Step 1. Define your requirements

Before getting into the actual configuration, you must take some time to define what your optimal setup looks like. For me, it looks like this:

1. It must have Python 2.7+, Python 3.6, Python 3.7, and Python 3.8 installed.
2. Python 3.8 must be the default version.
3. I must be able to switch between versions easily.
4. I must be able to fire up a [`ipython`](https://ipython.org/) using the default python version I specified.
5. Dependency tools I use - `pipenv` and `poetry` - must work with all Python 3+ versions.

## Step 2. Installing and configuring multiple Python versions

To install multiple versions and being able to switch between them, I use [`pyenv`](https://github.com/pyenv/pyenv).
`pyenv` has several benefits such as:

- Lets you change the global Python version on a per-user basis.
- Provides support for per-project Python versions.
- Everything is installed in the `$HOME` directory. This means no risk of messing up the default Python installation.
- Supports `pypy`, `anaconda`, `CPython`, `Stackless-Python` and others!

In addition to `pyenv`, I also use the [`pyenv-virtualenv`](https://github.com/pyenv/pyenv-virtualenv) plugin to manage my virtual environments.

### Installing `pyenv` and its plugins

Since last year, I've been using [Homebrew](https://brew.sh/) both on macOS and Linux. So, the following steps are OS agnostic for me.

```console
brew install pyenv
brew install pyenv-virtualenv
```
#### Creating a directory for the *virtualenvs*

Each project should have each own virtual environment associated with. This way we can work on more than one project at a time without introducing conflicts in their dependencies. I keep all my *virtualenvs* under the `$HOME/.ve` directory.

```bash
mkdir -p $HOME/.ve
```

Once the folder is created I add the path to my `~/.zshrc` (for bash users, add it to `~/.bashrc`):

```bash
cat <<"EOT" >> ~/.zshrc
# pyenv config
# Set virtualenv dir
export WORKON_HOME=~/.ve
# Initialize pyenv
if command -v pyenv 1>/dev/null 2>&1; then
  eval "$(pyenv init -)"
fi
# Initialize pyenv-virtualenv
eval "$(pyenv virtualenv-init -)"
EOT
```
The variable `WORKON_HOME` tells `pipenv` [where to place your virtual environments](https://pipenv-fork.readthedocs.io/en/latest/advanced.html#custom-virtual-environment-location). 

Once that's done, you can restart your shell by either closing and opening a new window or running: 
```bash
exec $SHELL
```

### Installing all Python versions I need.

```bash
PY_DEFAULT=3.8.5
PY_VERSIONS=( $PY_DEFAULT 3.7.8 3.6.11 2.7.18 )

for py_version in "${PY_VERSIONS[@]}"
do
    echo -e "Installing Python $py_version...\n\n"
    # Install specific Python version
    pyenv install $py_version
done
```

## Step 3. Installing the tools

For dependency management I use two different tools, [`pipenv`](https://pipenv.pypa.io/en/latest/), and more recently [`poetry`](https://python-poetry.org/). Eventually I will probably settle on `poetry` by at the moment I need both. 

Also, I rely a lot on *jupyter notebooks*, for quick data analysis tasks; and [`ipython`](https://ipython.org/) as a fancy Python interpreter.

Again, I don't want to pollute the global installation. To avoid that we can use `pyenv-virtualenv` to create *virtualenvs* for the tools.

Let's install *jupyter*, *pipenv*, and *poetry*.

```console
# Creating tools3 venv
pyenv virtualenv $PY_DEFAULT tools3

# Activating
pyenv activate tools3

# Upgrade pip
pip install --upgrade pip

# Install Jupyter
pip install jupyter

# Install Jupyter extensions
pip install jupyter_nbextensions_configurator rise
jupyter nbextensions_configurator enable --user

# pipenv
pip install pipenv

# poetry (using preview version to fix a bug in the 1.0.10 version)
pip install --pre poetry -U
```

If you use `pipenv`, make sure to install its completion script. You can add it to your shell like so:

```bash
echo 'eval "$(pipenv --completion)"' >> ~/.zshrc
```

And for `poetry`:
```bash
# Oh-My-Zsh
mkdir -p $ZSH/plugins/poetry
poetry completions zsh > $ZSH/plugins/poetry/_poetry
```

Since the completion is a [`oh-my-zsh`](https://ohmyz.sh/) plugin, I need to add poetry to the plugins list in my ~/.zshrc file.

```bash
plugins(
    poetry
    ...
)
```

This can be accomplished using `sed`: 
```console
sed -i.bak 's/^plugins=(\(.*\)/plugins=(poetry\n        \1/' ~/.zshrc
```

Now, we need to configure `poetry` to create *virtualenvs* inside `~/.ve`, just like `pipenv`.

```bash
echo 'export POETRY_VIRTUALENVS_PATH=$WORKON_HOME' >> ~/.zshrc
```

### More poetry configurations

One more thing, `pipenv`'s default behaviour is to load any environment variables defined in the `.env` in the root of the project. It does that on two occasions, when running `pipenv shell` and `pipenv run`. `poetry`, on the other hand, does not support that. As a workaround, I need to load it manually. 

To automate the process, I created a shell function that loads it whenever I run `poetry shell` or `poetry run`. If I want to disable it, I can set the `POETRY_DONT_LOAD_ENV` variable. This is, again, similar to how `PIPENV_DONT_LOAD_ENV` works.

```bash
cat <<"EOT" >> ~/.zshrc
# Get the poetry's full path
POETRY_CMD=$(which poetry)
# Allow poetry to load .env files
function poetry() {
    # Define the full command. i.e. poetry [run|shell|version]
    POETRY_FULL_CMD=($POETRY_CMD "$@")

    # if POETRY_DONT_LOAD_ENV is *not* set, then load .env if it exists
    # also, only loads when for "run" and "shell" commands.
    if [[ -z "$POETRY_DONT_LOAD_ENV" && -f .env && ("$1" = "run" || "$1" = "shell") ]]; then
        echo 'Loading .env environment variables…'
        env $(grep -v '^#' .env | tr -d ' ' | xargs) $POETRY_FULL_CMD
    else
        $POETRY_FULL_CMD
    fi
}
EOT
```

Now deactivate the *virtualenv* `tools3`.

```bash
# Deactivating the venv
pyenv deactivate
```

## Step 4. Setting interpreters priority

Now that we have all the versions we need installed we must establish some sort of priority. Basically, I want to use `poetry`, or any other tool, without activating the *virtualenv* where I installed them. We can do that using `pyenv global` command.

```console
PY_DEFAULT=3.8.5
PY_VERSIONS=( $PY_DEFAULT 3.7.8 3.6.11 2.7.18 )
pyenv global $PY_VERSIONS tools3 system
```

Let's check:

```console
$ pyenv versions
* system (set by /home/miguel/.pyenv/version)
* 2.7.18 (set by /home/miguel/.pyenv/version)
  2.7.18/envs/tools2
* 3.6.11 (set by /home/miguel/.pyenv/version)
* 3.7.8 (set by /home/miguel/.pyenv/version)
* 3.8.5 (set by /home/miguel/.pyenv/version)
  3.8.5/envs/tools3
* tools3 (set by /home/miguel/.pyenv/version)
```

Everything looks good, it's time to restart the shell and that's it.

### Preventing accidental installation of packages

One thing that happened a lot to me was installing packages inside one of the global interpreters using `pip`. As I mentioned earlier, it's a good idea to keep each Python interpreter intact. If we need to install anything, I prefer to create a new virtualenv, like I did for *tools3*. 

Now, how can we lock each Python installation?

That's actually pretty straightforward. We can set `pip` to only install packages if there's a *virtualenv* active.

```bash
echo 'export PIP_REQUIRE_VIRTUALENV=true' >> ~/.zshrc
```

### What about pipenv and poetry?

Now, if I have a `poetry` project that requires Python 3.6, I tell `poetry` to use the 3.6 version and it will automatically create a *virtualenv* using the Python 3.6.11 I installed. 

Example:

```toml
# pyproject.toml 
...
[tool.poetry.dependencies]
python = "~3.6"
...
```

Output:
```console
$ poetry env use 3.6   
Creating virtualenv sandbox-aBhl6cgV-py3.6 in /home/miguel/.ve
Using virtualenv: /home/miguel/.ve/sandbox-aBhl6cgV-py3.6
$ poetry shell        
Loading .env environment variables…
Spawning shell within /home/miguel/.ve/sandbox-aBhl6cgV-py3.6
$ . /home/miguel/.ve/sandbox-aBhl6cgV-py3.6/bin/activate
(sandbox-aBhl6cgV-py3.6) $
```

## Conclusion

That’s pretty much it! I hope this tutorial is useful for you, just as it’s for me. Whenever I need to reconfigure my workspace, I follow those steps. Also, I’ll probably create a shell script to automate it instead of copying and pasting from this guide. This tutorial was initially based on https://medium.com/@henriquebastos/the-definitive-guide-to-setup-my-python-workspace-628d68552e14, which served as inspiration on how to setup my own workspace.