Managing Python Environments with direnv and pyenv - Stack Abuse

# Managing Python Environments with direnv and pyenv

### Introduction

As Python developers, most of us are familiar with Virtual Environments. One of the first things we do when working on a new project is to create an environment. We commonly use virtualenv or venv exactly for that purpose.

Each project we work on uses different packages and may even be compatible with only one Python version.

Doing something repeatedly warrants automation. In this article, we'll see how direnv and pyenv can help us do that.

As a side note, some modern IDEs already automated these steps. For example, PyCharm will create the Virtual environment when initializing a project:

Although automating all these steps is a great win if we use IDEs that support such functionalities, a more generic solution should be IDE-agnostic.

### The Problems of virtualenv

Imagine we found a project on GitHub and we would like to play around with it. Pyweather is a simple script that requests the extended weather forecast for our location and prints it on the terminal.

These are the steps we take in order to try the script on our machine:

$git clone https://github.com/lcofre/pyweather.git$ cd pyweather


Then we create the virtual environment and install the packages the script uses:

$virtualenv --python=python3 env$ source env/bin/activate
(env) $pip install requirements.txt  And only then, we can execute the script: (env)$ ./pyweather.py


We created a virtual environment and saved it in the root folder of our project. While being on that folder we had to activate the environment with the source command.

When we finish working, we need to leave the Virtual Environment by executing deactivate:

(env) $deactivate  All those steps are our responsibility. How many times we may have forgotten to activate an environment and installed a package globally! Let's see how direnv helps us automate this. ### direnv direnv was made mainly to load environment variables, depending on the current directory and has an extension for many shells. In this example, we'll be using using bash, but direnv supports many other shells as well. And what's more important for us, it allows us to manage Python Virtual Environments. To install it we'll run the bash installer they provide. We could use the package manager of our distribution, but the bash installer will ensure we install the latest version available: $ curl -sfL https://direnv.net/install.sh | bash


Now we need to hook direnv to bash. We'll edit ~/.bashrc and then reload it:

$echo 'eval "$(direnv hook bash)"' >> ~/.bashrc
$source ~/.bashrc  This way direnv will link itself to the shell and will be executed always before each prompt. We will never notice it's working on the background. direnv will check if something needs to be loaded on the current folder. It checks the existence of a file named .envrc, with instructions on what should be loaded. To load Python Virtual Environments we run the layout command, followed by the Python version: $ echo 'layout python' > .envrc


Or if we want to use Python 3:

$echo 'layout python3' > .envrc  ## Better understand your data with visualizations. • 30-day no-questions money-back guarantee • Beginner to Advanced • Updated regularly (latest update June 2021) • Updated with bonus resources and guides Running these will tell direnv to look for a python or python3 executable on the path. As soon as we create the .envrc file we'll be warned that we need to allow direnv to access that folder. Let's do that right now: $ direnv allow

direnv: loading .envrc
...
New python executable in /home/myuser/untitled/.direnv/python-3.6.9/bin/python3
...
Installing setuptools, pkg_resources, pip, wheel...direnv:
done.
direnv: export +VIRTUAL_ENV ~PATH


As we can see in the output, the Virtual Environment was immediately created. The prompt is not modified though, so we won't see the name of the environment written at the beginning.

Now we can install the packages we need as we did on the environment we created in the previous section:

$pip install -r requirements.txt  direnv will silently activate the environment in the background. Whenever we move out of the directory, the environment will be deactivated: $ cd ..

direnv: unloading


If we can use any Python version that is installed on the system, direnv is all we need.

Let's suppose now that our pyweather script requires a very specific version though.

### pyenv

pyenv is a version management utility for Python. It allows, among other things, to change Python versions on a per-project basis. direnv provides support for it since version 2.21.0, so together they can give us a higher level of control on the version we use in our environment.

Let's start by installing pyenv:

$curl -L https://pyenv.run | bash  And then ensuring it will always be accessible to our terminal: $ echo 'export PATH="~/.pyenv/bin:$PATH"' >> ~/.bashrc$ echo 'eval "$(pyenv init -)"' >> ~/.bashrc$ echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc$ source ~/.bashrc


Now let's suppose our pyweather script requires a very specific Python version, 3.6.2.

First, we need to install that version of Python:

$pyenv install 3.6.2  And now we can configure our project to use the specific version: $ echo 'layout pyenv 3.6.2' > .envrc
$direnv allow  We can confirm all works as expected by checking the Python version in the environment: $ python --version

Python 3.6.2


If we ever need to change the Python version, it will be enough for us to change the layout in the .envrc file.

Thanks to both utilities we can change the layout to any Python version, and our virtual environment will be updated right away.

Another advantage of using both direnv and pyenv is that we can version our .envrc file in our project repository.

That way all contributors will be able to configure their environment as intended by the project, as long as they install the utilities and Python version needed.

### Conclusion

Virtual Environments are in a way detached from the Python development workflow. We need to remember to configure and activate it before working with our project. Thanks to direnv and pyenv we can automate all this, and entering the project folder will do all the work for us in the background.

Installation of both utilities is not straightforward, but after being done once we will save ourselves a lot of time. We will also always have the certainty that we are working with the right virtual environment and Python version.

Last Updated: July 7th, 2020

Get tutorials, guides, and dev jobs in your inbox.