How I Work

How I Work: pipx

How I use pipx

The Problem

You want to install command line Python tools and make them available. You want to type ipython or vd and have it run a version you installed.

You could pip install into the system Python but that leads to all sorts of fun (what happens when you upgrade your Python or when two CLI tools have conflicting dependencies, or won't even work with your system Python?).

You could pip install --local and shunt it into your ~/.local/bin. This mostly works but has some of the same problems as above. At least it's not messing with your system.

You could use Homebrew but I've found two problems with this: the first is you are beholden to the Homebrew upgrade schedule and, this affects me anyway, the second is every time you upgrade the Homebrew Python it seems to mess up all your brew installed python apps.

If you're on Linux you could use whatever comes with your package manager but 99% of the time it's a very old version of the tool.

I also experimented with creating a ~/.virtualenv/tool virtualenv per tool and symlinking into my ~/.local/bin, this mostly works. Can be a pain to manage though.

A Solution: pipx

pipx install sometool creates a virtualenv, pip installs the tool and makes all the bin/* commands available in ~/.local/bin. As a bonus you can easily upgrade versions and have different pythons for each tool. As a further bonus each tool can have wildly incompatible library dependencies without any problems as each lives in a separate virtual environment.

Installation

I usually opt for the non-homebrew installation instructions:

$ python3 -m pip install --user pipx
$ python3 -m pipx ensurepath

# Optional: shell completions
$ pipx completions

This plays well with asdf to have a completely self contained setup.

Usage

Usage is pretty straight forward: where you would use pip use pipx.

$ pipx install jupyterlab
  installed package jupyterlab 2.2.9, Python 3.9.0
  These apps are now globally available
    - jlpm
    - jupyter-lab
    - jupyter-labextension
    - jupyter-labhub
done! ✨ 🌟 ✨

$ which jupyter-lab
/Users/mick/.local/bin/jupyter-lab

Upgrading a Package

You can upgrade a package using the upgrade command:

$ pipx upgrade ipython
ipython is already at latest version 7.19.0 (location: /Users/mick/.local/pipx/venvs/ipython)

More interesting is the upgrade-all which upgrades everything:

$ pipx upgrade-all
Versions did not change after running 'pip upgrade' for each package 😴

Injecting More Packages Into a Tool

This is really handy for somehing like ipython which usually wants a bunch of other libraries like pandas or numpy.

You can use the inject command to do this:

$ pipx inject ipython pytz iso8601 tzdata
  injected package pytz into venv ipython
done! ✨ 🌟 ✨
  injected package iso8601 into venv ipython
done! ✨ 🌟 ✨
  injected package tzdata into venv ipython
done! ✨ 🌟 ✨

You can see what's injected with the list --include-injected command:

$ pipx list --include-injected
venvs are in /Users/mick/.local/pipx/venvs
apps are exposed on your $PATH at /Users/mick/.local/bin
...
   package ipython 7.19.0, Python 3.9.0
    - iptest
    - iptest3
    - ipython
    - ipython3
    Injected Packages:
      - iso8601 0.1.13
      - pytz 2020.1
      - tzdata 2020.2
...

Daily Actions

Every day I usually run this:

$ pipx upgrade-all

This keeps my pipx managed CLI tools fresh.

A Note on Fiddly C Dependencies

Some packages don't have pre-made wheels for your particular platform and Python combination, so you might need to help them and point to libraries for C extensions.

This is typically done by setting environment variables to point at libraries for the C builds.

For example: on my setup pgcli couldn't easily build psycopg2 which depended on OpenSSL. Since I'm using Homebrew I need to give a hand locating dependencies:

# sh/bash/zsh
PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig" \
CPPFLAGS="-I/usr/local/opt/openssl@1.1/include" \
LDFLAGS="-L/usr/local/opt/openssl@1.1/lib" \
pipx install pgcli

# fish
set -lx PKG_CONFIG_PATH "/usr/local/opt/openssl@1.1/lib/pkgconfig"
set -lx CPPFLAGS "-I/usr/local/opt/openssl@1.1/include"
set -lx LDFLAGS "-L/usr/local/opt/openssl@1.1/lib"
pipx install pgcli

Upgrading Your Python Version

If you upgrade your Python version you can either use it in pipx with pipx install --python pythonNew.Version some-package or

Read next