Thursday, March 26, 2015

Building Python x64 on Windows 7 with SDK 7.0

[UPDATE 2015-07-02] Check out Python Bootstrap a continuously integrated build of Python-2.7 for Windows that can be installed without admin rights.

I know I was just ranting about the inability to distribute Python27 w/o admin rights, but surprise! This is a piece of cake, thanks to the whoever the amazing Python developers are who maintain the PCbuild and external buildbot tools for Windows. There are also a few sites out there that have similar information on building Python for windows, but honestly everything you need is in PCbuild/readme.txt. Read it, then read it again. Seriously. Also check out the Python docs developer's guide. Hmm, thinking of setting up an AppVeyor buildbot for this.

OK, let's do this:
  1. Get Python and install it on your system. You may need a working binary to bootstrap the amd64 build.
  2. Get a working version of Microsoft SDK for Windows 7 (7.0). AFAIK Visual Studio 2013 Express Desktop or Community editions include both SDK 7.0 and 7.1, so alternately install that. Make sure that you include the redistributables in when installing the SDK because you will need them to distribute your Python build. See upgrade to vs2013 for fixes to some issues you may encounter especially if you have some other VC components already installed.
  3. Get a working svn binary for windows and put it on your path. I use CollabNet SubVersion commandline binaries. The easiest way to get and build all of the external libraries (bzip2, sqlite3, tk/tcl, etc.) is to use the Tools/buildbot batch scripts which call svn.exe.
  4. Either download the gzipped source tarball from python.org or clone the tag v2.7.10 of the cpython mercurial repository. Archives of the Hg repo are also conveniently available from the repo viewer.
  5. Read the PCbuild/readme.txt again
  6. Open the SDK command shell from the Startmenu. Change the target to Release x86, the default is Debug x64, by typing the following:
  7. C:\Program Files\Microsoft SDKs\Windows\v7.0>setenv /Release /x86
  8. Change to the directory where the source tarball is extracted.
  9. Patch the Tools/buildbot/externals batch script exactly as described in the PCbuild readme. I added the release build immediately after the debug fields.
  10. if not exist tcltk\bin\tcl85.dll (
        @rem all and install need to be separate invocations, otherwise nmakehlp is not found on install
        cd tcl-8.5.15.0\win
        nmake -f makefile.vc INSTALLDIR=..\..\tcltk clean all
        nmake -f makefile.vc INSTALLDIR=..\..\tcltk install
        cd ..\..
    )
    
    if not exist tcltk\bin\tk85.dll (
        cd tk-8.5.15.0\win
        nmake -f makefile.vc INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.15.0 clean
        nmake -f makefile.vc INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.15.0 all
        nmake -f makefile.vc INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.15.0 install
        cd ..\..
    )
    
    if not exist tcltk\lib\tix8.4.3\tix84.dll (
        cd tix-8.4.3.5\win
        nmake -f python.mak DEBUG=0 MACHINE=IX86 TCL_DIR=..\..\tcl-8.5.15.0 TK_DIR=..\..\tk-8.5.15.0 INSTALL_DIR=..\..\tcltk clean
        nmake -f python.mak DEBUG=0 MACHINE=IX86 TCL_DIR=..\..\tcl-8.5.15.0 TK_DIR=..\..\tk-8.5.15.0 INSTALL_DIR=..\..\tcltk all
        nmake -f python.mak DEBUG=0 MACHINE=IX86 TCL_DIR=..\..\tcl-8.5.15.0 TK_DIR=..\..\tk-8.5.15.0 INSTALL_DIR=..\..\tcltk install
        cd ..\..
    )
    
  11. From the archive root (Python-2.7.9) call the externals batch script. It will copy and build all of the externals from svn.python.org in a folder called externals/.
  12. Now cd to PCbuild and call build.bat. Voila, python.exe for x86.
  13. Almost there. go back up to the root of the extracted tarball and rename externals to externals-x86.
  14. Change the target to Release x64 by typing the following:
  15. C:\Users\myname\downloads\Python-2.7.9>setenv /Release /x64
  16. Set an environment variable HOST_PYTHON=C:\Python27\python.exe. You may not need this at all or you might be able to use the 32-bit version just built.
  17. Patch the buildbot externals-amd64 batch script just like the x86 script.
  18. if not exist tcltk64\bin\tcl85.dll (
        cd tcl-8.5.15.0\win
        nmake -f makefile.vc MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 clean all
        nmake -f makefile.vc MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 install
        cd ..\..
    )
    
    if not exist tcltk64\bin\tk85.dll (
        cd tk-8.5.15.0\win
        nmake -f makefile.vc MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.15.0 clean
        nmake -f makefile.vc MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.15.0 all
        nmake -f makefile.vc MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.15.0 install
        cd ..\..
    )
    
    if not exist tcltk64\lib\tix8.4.3\tix84.dll (
        cd tix-8.4.3.5\win
        nmake -f python.mak DEBUG=0 MACHINE=AMD64 TCL_DIR=..\..\tcl-8.5.15.0 TK_DIR=..\..\tk-8.5.15.0 INSTALL_DIR=..\..\tcltk64 clean
        nmake -f python.mak DEBUG=0 MACHINE=AMD64 TCL_DIR=..\..\tcl-8.5.15.0 TK_DIR=..\..\tk-8.5.15.0 INSTALL_DIR=..\..\tcltk64 all
        nmake -f python.mak DEBUG=0 MACHINE=AMD64 TCL_DIR=..\..\tcl-8.5.15.0 TK_DIR=..\..\tk-8.5.15.0 INSTALL_DIR=..\..\tcltk64 install
        cd ..\..
    )
    
  19. From the archive root (Python-2.7.9) call the externals-amd64 batch script
  20. Finally cd back to PCbuild and call build.bat -p x64. Voila, python.exe for x64.
  21. Add externals/tcltk to your path and run the tests
  22. C:\Users\myname\downloads\Python-2.7.9\PCbuild>set PATH=C:\Users\myname\downloads\Python-2.7.9\externals;%PATH%
    C:\Users\myname\downloads\Python-2.7.9\PCbuild>rt
    
  23. Copy the VC runtime from Program Files (x86)\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT\to PCbuild folder, and Program Files (x86)\Microsoft Visual Studio 9.0\VC\redist\amd64\Microsoft.VC90.CRT\to PCbuild\amd64 folder.
  24. To distribute create a similar file structure for both archtypes and copy the files into the folders
  25. Python27
    |
    +-python.exe
    |
    +-pythonw.exe
    |
    +-python27.dll
    |
    +-msvcr90.dll <- from VC/redist/MICROSOFT.VC90.CRT
    |
    +-msvcp90.dll <- from VC/redist/MICROSOFT.VC90.CRT
    |
    +-msvcm90.dll <- from VC/redist/MICROSOFT.VC90.CRT
    |
    +-MICROSOFT.VC90.CRT.manifest <- from VC/redist/MICROSOFT.VC90.CRT
    |
    +-DLLs <- all externals/tcltk/bin, PCbuild/*.dll & PCbuild/*.pyd files
    |         & PC/py.ico, PC/pyc.ico & PC/pycon.ico.
    |
    +-Lib <- same as source archive except all *.pyc files, all plat-* folders
    |        & ensurepip folder
    |
    +-libs <- all PCbuild/*.lib files
    |
    +-include <- same as source archive + PC/pyconfig.h
    |
    +-tcl <- everything in tcltk/lib
    |
    +-Scripts <- PCbuild/idle.bat
    |
    +-Doc <- set %PYTHON%=python.exe and build Sphinx docs with Sphinx if you have it
    

Note: wish85.exe and tclsh85.exe won't work with this Python installed file structure although it will work in the externals bin folder because they look for the tcl85.dll in ../lib. Also note that idle.bat needs some fixin. And also it's very important to know that most executables in Scripts created by installer packages have the path to python.exe hardwired, IE: they are initially not portable, however check out this blog for a few quick tricks to fix them.

You can download my x64 build and x86 build from dropbox. Congratulations!

Fork me on GitHub