Peter Bengtsson: Pip-Outdated.py - a script to compare requirements.in with the output of pip list --outdated

Simply by posting this, there's a big chance you'll say "Hey! Didn't you know there's already a well-known script that does this? Better." Or you'll say "Hey! That'll save me hundreds of seconds per year!"

The problem

Suppose you have a requirements.in file that is used, by pip-compile to generate the requirements.txt that you actually install in your Dockerfile or whatever server deployment. The requirements.in is meant to be the human-readable file and the requirements.txt is for the computers. You manually edit the version numbers in the requirements.in and then run pip-compile --generate-hashes requirements.in to generate a new requirements.txt. But the "first-class" packages in the requirements.in aren't the only packages that get installed. For example:

▶ cat requirements.in | rg '==' | wc -l
      54

▶ cat requirements.txt | rg '==' | wc -l
     102

In other words, in this particular example, there are 76 "second-class" packages that get installed. There might actually be more stuff installed that you didn't describe. That's why pip list | wc -l can be even higher. For example, you might have locally and manually done pip install ipython for a nicer interactive prompt.

The solution

The command pip list --outdated will list packages based on the requirements.txt not the requirements.in. To mitigate that, I wrote a quick Python CLI script that combines the output of pip list --outdated with the packages mentioned in requirements.in:

#!/usr/bin/env pythonimportsubprocessdefmain(*args):ifnotargs:requirements_in="requirements.in"else:requirements_in=args[0]required={}withopen(requirements_in)asf:forlineinf:if"=="inline:package,version=line.strip().split("==")package=package.split("[")[0]required[package]=versionres=subprocess.run(["pip","list","--outdated"],capture_output=True)ifres.returncode:raiseException(res.stderr)lines=res.stdout.decode("utf-8").splitlines()relevant=[lineforlineinlinesifline.split()[0]inrequired]longest_package_name=max([len(x.split()[0])forxinrelevant])ifrelevantelse0forlineinrelevant:p,installed,possible,*_=line.split()ifpinrequired:print(p.ljust(longest_package_name+2),"INSTALLED:",installed.ljust(9),"POSSIBLE:",possible,)if__name__=="__main__":importsyssys.exit(main(*sys.argv[1:]))

Installation

To install this, you can just download the script and run it in any directory that contains a requirements.in file.

Or you can install it like this:

curl -L https://gist.github.com/peterbe/099ad364657b70a04b1d65aa29087df7/raw/23fb1963b35a2559a8b24058a0a014893c4e7199/Pip-Outdated.py > ~/bin/Pip-Outdated.py
chmod +x ~/bin/Pip-Outdated.py

Pip-Outdated.py