Comprehensive Guide to Dependency Management in Python

When learning Python, many beginners focus solely on the language and its libraries while completely ignoring virtual environments. As a result, managing Python projects can become a mess: dependencies installed for different projects may have conflicting versions, leading to compatibility issues.

Even when I studied Python, nobody emphasized the importance of virtual environments, which I now find very strange. They are an extremely useful tool for isolating different projects from each other.

In this article, I will explain how virtual environments work, provide several examples, and share useful commands for managing them.

Problem

Imagine you have two Python projects on your laptop, each located in a different directory. You realize that you need to install the latest version of library A for the first project. Later, you switch to the second project and attempt to install library B.

Here’s the problem: library B depends on library A, but it requires a different version than the one you installed earlier.

Since you haven’t used any tool for Dependency Management, all dependencies are installed globally on your computer. Due to the incompatible versions of library A, you encounter an error when trying to install library B.

Solution

To prevent such issues, virtual environments are used. The idea is to allocate a separate storage space for each Python project. Each storage will contain all the externally downloaded dependencies for a specific project in an isolated manner.

More specifically, if we download the same library A for two projects within their own virtual environments, library A will be downloaded twice — once for each environment. Moreover, the versions of the library can differ between the environments because each environment is completely isolated and does not interact with the others.

Now that the motivation behind using virtual environments is clear, let’s explore how to create them in Python.

Virtual environments in Python

It is recommended to create a virtual environment in the root directory of a project. An environment is created using the following command in the terminal:

python -m venv <environment_name>

By convention, <environment_name> is usually named venv, so the command becomes:

python -m venv venv

As a result, this command creates a directory called venv, which contains the virtual environment itself. It is even possible to go inside that directory, but in most cases, it is not very useful, as the venv directory primarily contains system scripts that are not intended to be used directly.

To activate the virtual environment, use the following command:

source venv/bin/activate

Once the environment is activated, we can install dependencies for the project. As long as the venv is activated, any installed dependency will only belong to that environment.

To deactivate the virtual environment, type:

deactivate

Once the environment is deactivated, the terminal returns to its normal state. For example, you can switch to another project and activate its environment there.

Dependency management

Installing libraries

Before installing any dependencies, it is recommended to activate a virtual environment to ensure that installed libraries belong to a single project. This helps avoid global version conflicts.

The most frequently used command for dependency management is pip. Compared to other alternatives, pip is intuitive and simple to use.

To install a library, type:

pip install <library_name>

In the examples below instead of the <library_name>, I will write pandas (the most commonly used data analysis library).

So, for instance, if we wanted to download the latest version of pandas, we should have typed:

pip install pandas

In some scenarios, we might need to install a specific version of a library. pip provides a simple syntax to do that:

pip install pandas==2.1.4 # install pandas of version 2.1.4
pip install pandas>=2.1.4 # install pandas of version 2.1.4 or higher
pip install pandas<2.1.4 # install pandas of version less than 2.1.4
pip install pandas>=2.1.2,<2.2.4 # installs the latest version available between 2.1.2 and 2.2.4 

Viewing dependency details

If you are interested in a particular dependency that you have installed, a simple way to get more information about it is to use the pip show command:

pip show pandas

For example, the command in the example will output the following information:

Deleting dependency

To remove a dependency from a virtual environment, use the following command:

pip uninstall pandas

After executing this command, all files related to the specified library will be deleted, thus freeing up disk space. However, if you run a Python program that imports this library again, you will encounter an ImportError.

File with requirements

A common practice when managing dependencies is to create a requirements.txt file that contains a list of all downloaded dependencies in the project along with their versions. Here is an example of what it might look like:

fastapi==0.115.5
pydantic==2.10.1
PyYAML==6.0.2
requests==2.32.3
scikit-learn==1.5.2
scipy==1.14.1
seaborn==0.13.2
streamlit==1.40.2
torch==2.5.1
torchvision==0.20.1
tornado==6.4.2
tqdm==4.67.1
urllib3==2.2.3
uvicorn==0.32.1
yolo==0.3.2

Ideally, every time you use the pip install command, you should add a corresponding line to the requirements.txt file to keep track of all the libraries used in the project.

However, if you forget to do that, there is still an alternative: the pip freeze command outputs all of the installed dependencies in the project. Nevertheless, pip freeze can be quite verbose, often including many other library names that are dependencies of the libraries you are using in the project.

pip freeze > requirements.txt

Given this, it’s a good habit to add installed requirements with their versions to the requirements.txt file.

Whenever you clone a Python project, it is expected that a requirements.txt file is already present in the Git repository. To install all the dependencies listed in this file, you use the pip install command along with the -r flag followed by the requirements filename.

pip install -r requirements.txt

Conversely, whenever you work on a Python project, you should create a requirements.txt file so that other collaborators can easily install the necessary dependencies.

.gitignore

When working with version control systems, virtual environments should never be pushed to Git! Instead, they must be mentioned in a .gitignore file.

Virtual environments tend to be very large, and if there is an existing requirements.txt file, there should be no problem downloading all necessary dependencies.

Conclusion

In this article, we have looked at the very important concept of virtual environments. By isolating downloaded dependencies for different projects, they allow for easier management of multiple Python Projects.

All images are by the author unless noted otherwise.