Saturday, August 11, 2012

Building numpy, scipy, matplotlib and PIL in virtualev on both windows and linux

This really should be split onto several posts. And I want to send a patch to matplotlib to make it easier to build in windows.
  1. You need several dev files, most of which come standard on Linux and mingw-users. Still for PIL you new Tk.h and tcl.h, zlib, libjpg and ... Note on windows, you need the same version of Tck/Tk as in Python, which is probably 8.5.2. Look at `init.tcl` in `C:\Python27\tcl\tcl8.5` and it will tell you exactly which package is required.
  2. On Linux, look for the corresponding dev packages, which may not be installed. On Ubuntu they are in the Ubuntu Software Center. In particular you will want the ATLAS, BLAS and LAPACK dev packages for Numpy and Scipy.
  3. On Windows, in some cases, I untarred/unzipped the downloads to edit the setup.py or other relevant file, then zipped it back up and used pip. A few packages took command line options which I passed through pip using --install-options=" --my-option='blah blah blah' " as an example.
  4. I added a pydistutils.cfg file to my home directory with [build] compiler=mingw32 which worked well for pip except with pyzmq, which I had to edit, because it makes its own compiler. So I edited it to make a mingw32 compiler, instead of the default which on Windows is msvcr90. In general I had to edit cygwinccompiler in distutils to remove the "-mno-cygwin" gcc option for the Mingw32Ccompiler class, which is no longer accepted for most gcc versions.
  5. On Linux this was all very easy, as long as I had the correct dev packages, but be prepared on Windows, this took a lot of work.
[POSTSCRIPT 2014-08-01]
This post is seriously out of date, and it was a bit vague to begin with. I don't remember what steps I took to get these tools running on windows anymore, and a lot has changed, most significantly I have switched to 64-bit, I have found that more python packages can be built with microsoft compilers and openBLAS has been released which greatly simplifies building these packages as BLAS and LAPACK are already compiled and optimized.
  • Consider using Windows SDK-7 for Python-2.7 after fixing vcvarsall.bat instead of mingw32. Windows SDK-7 replaces VC90 (aka V90) and comes with both 32-bit ad 64-bit compilers.
  • MinGW only works on 32-bit Windows systems so an alternative I started using recently is Win-Builds which uses mingw-w64 compilers which can cross compile both 32-bit and 64-bit applications. Win-Builds works in MSYS, Cygwin or the native CMD shells, although I don't know how you would use autotools in a Windows CMD shell.
  • Unfortunately win-builds doesn't have gfortran, so either stick with mingw-w64 or build it, ugh!
  • Try using one of the new openBLAS binaries, maybe everything can be done with VC90 instead of using gcc.
Fork me on GitHub