Tuesday, January 28, 2014

setuptools detritus

Pre pip-1.5.1, you used to have to install setuptools prior to Pip. Especially after the distribute/setuptools merger, this potentially left you with a mishmash of setuptools and distribute folders and egg files, as easy-install and pip install packages differently.

If you used the recommended ez-setup.py provided by setuptools, this would always download a tarball or zipfile in whatever directory ez-setup.py was in, probably downloads, but in the case of packages bundled with it, it could be anywhere, never to be deleted, completely undetected by you, and if you never installed pip, and never found the easy way to update your packages, then everytime you used easy-install or distutils (and probably pip too, since ez-setup.py specifies the version of setuptools required before setup.py runs) to install a package bundled with setuptools' ez-setup.py it would leave another piece of detritus.

Then mix in the pip way of doing things, which is indifferent to egg-files. If you use pip to update setuptools, (pip install -U setuptools) which is a great way, because once you have cleaned up all of the random pth and egg leftovers, pip always takes care of cleaning up its own mess. However as mentioned previously, unless that mess is an egg file. Pip doesn't touch these files, which leads to two installs of setuptools, an egg file, which for those wondering is like a wheel, or a zip file, essentially a binary, platform specific installation, and the pip way, which is an uncompressed folder with python source code, byte compiled code and any extensions, similar to what distutils would do, if it weren't monkeypatched.

Well no longer. Pip now takes care of everything. So start from scratch, delete all instances of easy-install, setuptools(, distribute if you still have it) and sure what the heck pip, everywhere, in scripts(/bin [1]) and lib. Then use get-pip.py to install both at the same time. Then periodically update setuptools using pip install -U setuptools. Ah. All better.

[1] Note: that this is really only a Windows or Mac issue, not Linux, because Python packages are included in each distro's repository. It can be an issue for virtualenvs anywhere, even Linux, so it's good to understand in principle. On a Linux share, without root access, you might have this issue with packages installed in .local. On Mac if you are using an official binary installation of Python, then you will find your scripts in /Library/Frameworks/Python.framework/Versions/2.7/bin. In a virtualenv you should find scripts in .virtualenvs/name-of-venv/bin and .virtualenvs\name-of-venv\scripts for Mac/Linux and Windows systems respectively. For non-root local installs on Linux shares the scripts folder is .local/bin.
Fork me on GitHub