Choosing a tool

Use conda for most science work. Use venv for small projects. Use pip only to add packages to an environment you already have.

ToolPrimary purposeKey benefitsSample command
Conda (recommended)Scientific work with substantial dependenciesPrecompiled binaries, cross-language, environment exportconda create -n myenv python=3.10
venvSmaller projects, personal utilitiesUses system Python, built into Python 3python3 -m venv myenv
pipAdding packages to an existing environmentStraightforward, broadly compatiblepip install requests
Avoid: don't use pip install --user. It puts packages straight into your home folder. Use a conda or venv environment instead, so each project stays separate.

Conda workflow

module load anaconda/3
conda create -n analysis python=3.10
conda activate analysis
conda install numpy scipy matplotlib

venv workflow

python3 -m venv myenv
source myenv/bin/activate
pip install numpy scipy matplotlib

Recommendations

  • Load anaconda before you work with any environment.
  • Keep environments in your $HOME or project folder.
  • Export your environment so others can rebuild it: conda env export > environment.yml
  • Try not to mix pip and conda in one environment.
Applies to All clusters