Showing posts with label mingw mangle. Show all posts
Showing posts with label mingw mangle. Show all posts

Sunday, April 7, 2013

The -mno-cygwin option has been deprecated for years

[UPDATE: 2015-03-19] Yay! Issue #12641 was closed in September-2013, and the latest releases of Python>=2.7.6 no longer have it.

So true. And there's even a bug filed in Python tracker. I also talked about it in this post. Here's how they deal with it in Mercurial (a.k.a. Hg).

# the -mno-cygwin option has been deprecated for years
Mingw32CCompiler = cygwinccompiler.Mingw32CCompiler
class HackedMingw32CCompiler(cygwinccompiler.Mingw32CCompiler):
    def __init__(self, *args, **kwargs):
        Mingw32CCompiler.__init__(self, *args, **kwargs)
        for i in 'compiler compiler_so linker_exe linker_so'.split():
            try:
                getattr(self, i).remove('-mno-cygwin')
            except ValueError:
                passcygwinccompiler.Mingw32CCompiler = HackedMingw32CCompiler

Thursday, January 24, 2013

Console 2 by bozho

[UPDATE 2015-04-27]: After trying ConEmu by Maximus5, there is no turning back. ConEmu is so obviously superior, it has full 256-color xterm support, properly resizes windows like normal windows when maximized and as if that wasn't enough, comes with all of the most common shells like git bash and various windows compiler environments preconfigured, including admin versions. Seriously! And I'm not alone; other Console2 users have professed their conversion to ConEmu as well.

I just went to Maximus5 GitHub release page for ConEmu, downloaded the 7z archive of the most current release and unzipped it into C:\ConEmu and awesome console emulator installed. Make a short cut of ConEmu (either 32 or 64 bit versions executable to your desktop then start - first time it asks for some simple setup questions, like where to store config file, what shells to use (select from preconfigured) and then starts. It just works. Better than Windows CMD, mintty, rxvt, powershell and yest even the revered Console2. Thanks Bozho, but it's time to move on.



The console terminal for MS Windows kicks ass compared to mintty, rxvtcmd or powershell.


Why? Let me enumerate:

  1. Resizable windows
  2. Tabs
  3. Customizable shells.
  4. Transparent background setting.
  5. Customizable copy/paste commands.
I could go on and on. This is like 7zip, just get it and start being more productive right now!

Thursday, August 16, 2012

Cygwin, MinGW, MSYS, GnuWin32, GNU Utilities for Win32 and now MinGW-w64

Introduction

GNU on Windows has a sordid past. Not really, but sordid sounds so much more interesting than complicated. So here's how I understand it, without any emotion, this is really just a navigational exercise.

Disclaimer

What was just said and what I am about to say is entirely opinion and not based on fact. I hope that I do not offend anyone, but I am sure that I inevitably will, as much of my opinion is based on hearsay. In that case I apologize, and please remember that the purpose of this blog is purely selfish; it is a note to myself to remind me of what took me so long to finally understand. So instead of getting angry with me, perhaps post a comment and disabuse me of my still as yet incomplete knowledge.

Cygwin

In the beginning there was Cygwin. Their tagline is "Get that Linux feeling - on Windows!" This does a lot to explain exactly what Cygwin is, Linux emulated on Windows. What this means in practice is that Cygwin applications use the cygwin.dll, a common runtime that must be linked to any applications that run on Cygwin, and consequently any applicaton compiled with Cygwin gcc. So essentially, when you use Cygwin you are more or less stuck in Cygwin. True, you could distribute the cygwin.dll with your application, so it would appear to be native, but it wouldn't be truly native. By native here I mean that it runs on one of the windows runtimes like the win32api or msvcrt. The Windows binary of GNU nano is an example of an application compiled using Cygwin that requires cygwin.dll to run. The probable downside of using this layer between the Windows API and your application may be potentially slower speed and some limits in its features.

MinGW

From Cygwin was born MinGW. I don't know if this is actually true, but that is what I've heard. Either way it doesn't really matter. The essential thing to realize here is this.
MinGW creates native applications for Windows using native libraries.
This means exactly what it sounds like, applications compiled using MinGW's compilers (gcc, g++, gfortran, etc.) will run on any Windows machine natively without any runtime other than the Windows API or mscvrt. Now it might be fun to speculate about some internal strife or a passionate drive of a group of individuals with some bold ideal, but that's irrelevant to this fundamental difference between the Cygwin and MinGW.

MSYS

MSYS may seem like a variation on Cygwin, except it is really meant to be used as an environment for running shell scripts, similar to bash (the Bourne Again SHell). It's true that if you compile an application using the MSYS version of gcc, then you will need to link it to the msys-1.0.dll, so in a sense it is the same as Cygwin. The difference is really in a state of mind. MSYS is presumably part of MinGW, which stands for Minimalist GNU for Windows. What that translates to is that MinGW and MSYS contain only the most essential tools required for developing native applications for Windows using the GNU open source suite of compilers, whereas Cygwin aims to provide Windows users every application available to Linux. In Cygwin you will find Python, GTK, nano, Ruby, Git, even some games I think, whereas in MinGW you will only find libraries most often used in common code. However there is a huge chunk of open source code that is compiled using autotools and depending on shell scripting, which I think is where MSYS gets involved. MSYS makes it easier to do this. True autotools, make and shell script functions have been ported as native win32 applications, but since they are only used as temporary tools to get to the finished product (a natively compiled win32 application) and the results don't depend on any of those temporary tools, why go through the effort of duplicating what Cygwin has already done so nicely?

GnuWin32 and GNU Utilities for Windows

If you were just reading the section above, then continuing on, GnuWin32 and the now defunct GNU Utilities for Windows seem to do precisely that. Both of these generally utilize MinGW to port GNU applications that were native to Linux or Cygwin to run natively on Windows. And in fact many open source projects now try to configure options for some Windows compiler, be it MinGW, MSVC or BCC (really, I haven't seen this too often although apparently it is still out there, wow they even have Delphi!). Some projects are optionally opting for CMake and alternative to Autotools. OK, so the key distinction here is that these are both compiled using MinGW and are basically an alternate to MSYS, if you prefer to continue to use the native Windows CMD console or something like Powershell.

MinGW-w64

MinGW-w64 is a fork of MinGW that allows you to cross compile code from one machine to another. For example, say you want to make 64-bit code, but you are on a 32-bit machine or a Linux box, or using Cygwin, then you would use MinGW-w64. The "w64" comes from it's ability to compile 64-bit code, which is not currently available (AFAIK) with MinGW (sometimes called mingw32). There are alternate distributions of both MinGW and MinGW-w64 provided by TDM. Also there are other cross-compilers, such as mxe.cc, but it is only for *nix (Linux/Unix). Coincidentally MinGW are MinGW-w64 both available as cross-compilers in most Linux distro repositories. Ubuntu, Fedora and Suse all have it. MacPorts, Fink and Homebrew may have them too. Same probably goes for mxe.cc. And who knows there are probably more cross-compilers out there as well.

Epilogue

Ah. So glad to get that all down, finally. So the moral of the story is this.
If you want to use native Windows apps compiled using GNU gcc use MinGW, but do not polute your toolchain with any MSYS or Cygwin libraries or you will be sorry.

Saturday, August 11, 2012

Building numpy, scipy, matplotlib and PIL in virtualev on both windows and linux

This really should be split onto several posts. And I want to send a patch to matplotlib to make it easier to build in windows.
  1. You need several dev files, most of which come standard on Linux and mingw-users. Still for PIL you new Tk.h and tcl.h, zlib, libjpg and ... Note on windows, you need the same version of Tck/Tk as in Python, which is probably 8.5.2. Look at `init.tcl` in `C:\Python27\tcl\tcl8.5` and it will tell you exactly which package is required.
  2. On Linux, look for the corresponding dev packages, which may not be installed. On Ubuntu they are in the Ubuntu Software Center. In particular you will want the ATLAS, BLAS and LAPACK dev packages for Numpy and Scipy.
  3. On Windows, in some cases, I untarred/unzipped the downloads to edit the setup.py or other relevant file, then zipped it back up and used pip. A few packages took command line options which I passed through pip using --install-options=" --my-option='blah blah blah' " as an example.
  4. I added a pydistutils.cfg file to my home directory with [build] compiler=mingw32 which worked well for pip except with pyzmq, which I had to edit, because it makes its own compiler. So I edited it to make a mingw32 compiler, instead of the default which on Windows is msvcr90. In general I had to edit cygwinccompiler in distutils to remove the "-mno-cygwin" gcc option for the Mingw32Ccompiler class, which is no longer accepted for most gcc versions.
  5. On Linux this was all very easy, as long as I had the correct dev packages, but be prepared on Windows, this took a lot of work.
[POSTSCRIPT 2014-08-01]
This post is seriously out of date, and it was a bit vague to begin with. I don't remember what steps I took to get these tools running on windows anymore, and a lot has changed, most significantly I have switched to 64-bit, I have found that more python packages can be built with microsoft compilers and openBLAS has been released which greatly simplifies building these packages as BLAS and LAPACK are already compiled and optimized.
  • Consider using Windows SDK-7 for Python-2.7 after fixing vcvarsall.bat instead of mingw32. Windows SDK-7 replaces VC90 (aka V90) and comes with both 32-bit ad 64-bit compilers.
  • MinGW only works on 32-bit Windows systems so an alternative I started using recently is Win-Builds which uses mingw-w64 compilers which can cross compile both 32-bit and 64-bit applications. Win-Builds works in MSYS, Cygwin or the native CMD shells, although I don't know how you would use autotools in a Windows CMD shell.
  • Unfortunately win-builds doesn't have gfortran, so either stick with mingw-w64 or build it, ugh!
  • Try using one of the new openBLAS binaries, maybe everything can be done with VC90 instead of using gcc.

Monday, July 30, 2012

Nano editor for windows

There are 2 binary versions of nano (gnu Pico), a replacement for vim. The one on the gnu nano site is built with Cygwin, and comes with several Cygwin dlls. The other is built with MinGW, and has many syntax files, which is truly cool.
Both have great color, and are nearly identical. But have the same issues, when called from a shell, (1) window can't be resized, and (2) after closing nano it erases whatever was in the shell previously, and leaves behind some garbage. Vim doesn't do this, and nano doesn't either on Linux.
I tried to compile it myself with mingw32, but there are several patches which you can find online, so I gave up and tried a different approach. I made q small bash script that is s windows START command to open a new shell with a preset size. It works for editing, but doesn't work with Git.

Wednesday, July 18, 2012

building dbus-python on windows with mingw

dbus-python Build Instructions for Windows (x86) with MinGW

Revision history

  1. r0.0.20120706 - First draft
  2. r0.1.20120718 - Clean up markdown and post to poquitopicante.blogspot.com
  3. r1.0.20120720 - Use autolauch
    • Need --with-dbus-session-bus-default-address=autolaunch: for dbus to work out-of-the-box and to keep dbus-python from throwing exception.
    • Also If using git repo, use ./autogen.sh instead of ./configure. Make sure you have autotools installed in your MinGW toolchain.
  4. r1.1.20120723 - Correct GTK path
    • Remove dbus-glib examples Makefile patch; it's not needed if path to GTK runtime bin is correct, i.e.: no quotes in MS-Windows paths!
    • This was not an issue with dbus-binding-tool.exe, #52322 in bugzilla. There are no problems for dbus-glib to create binding "glue" when in glib-server mode if path to GTK bin folder is correctly.
    • setenv.cmd has an issue in which the quotation marks are in the wrong place, causing the quotes to be copied into the path, which is a big no-no in MS-Windows, because it causes inconsistent results. E.g. dbus-binding-tool.exe could be built, but not run.

Overview

These are instructions for building dbus-python, a binding to dbus on a MS-Windows x86 system using MinGW.

Set up MinGW build environment

  1. Download and install most recent version of mingw-get-inst from sourceforge, current version is 20120426. This only needs to be done once because afterwards you can just run ...
    $ mingw-get update
    $ mingw-get install mingw-get
    
    ... to get the latest version. During the install, which is an Inno-Setup GUI, select the option to install the "C compiler" and "MSYS Basic System" but leave other default install options. It should install MinGW in C:\mingw and MSYS in C:\mingw\msys\1.0. It doesn't change your path or add any environmental variables, but it does add a shortcut to the MSYS bash shell on your desktop and start menu, and register mingw-get-inst so you can un-install it later. There is lots of rich material on MinGW and MSYS on the Internet. Also take a look at Configuring and Using mingw-get.
  2. Start the MSYS shell, from the start menu, and install (or update) the developers toolkit. If you need to upgrade a component, use mingw-get upgrade <component>.
    $ mingw-get update
    $ mingw-get install gcc
    $ mingw-get install g++
    $ mingw-get install msys-make
    $ mingw-get install mingw32-make
    $ mingw-get install mingw-developer-toolkit
    
  3. While still in the MSYS shell, install expat.
    $ mingw-get install expat
    $ mingw-get install libexpat
    $ mingw-get install msys-expat
    $ mingw-get install msys-libexpat
    
  4. You also need libz and zlib.
    $ mingw-get install libz
    $ mingw-get install zlib
    $ mingw-get install msys-zlib
    
  5. You may also need or want the following:
    • GNU patch, wget, tar, libxml2 and possibly others (libxslt and libffi). If possible install them from MinGW/MSYS with mingw-get. Use mingw-get list <package> to search for packages.
      $ mingw-get install msys-patch
      $ mingw-get install msys-wget
      $ mingw-get install msys-tar
      $ mingw-get install msys-libxml2
      
    • lib2xml and libxslt are also available from xmlsoft as windows binaries. If use them, copy the files into the matching folders in /local, e.g. bin files go in /local/bin.
    • zlib is also available from zlib, there are links to binaries, but it is easy to build by following the directions on Fragrant Memories, however, the generated library zlib1.dll may conflict with your Intel Wifi controller or something else. The MinGW/MSYS libraries are libz-1.dll and msys-z.dll. The files zlib1.dll, libexpat-1.dll and libxml2-2.dll may actually already be in your GTK runtime bin folder.
      $ which zlib1.dll /c/Program Files/Intel/WiFi/bin/zlib1.dll
    • libffi is available here and on Github and installation instructions are here.

Python

Download and install Python (2.7.3) with the Python Windows Installer.

PyGTK or GTK+

  1. Download and install either the PyGTK all-in-one installer for your Python version (e.g. 2.7) or the GTK-2.0 all-in-one installer. If you're determined to build GLib from source see this post on Bootstrapping GLIB with MinGW.
  2. Fix the setenv.bat by either removing the quotes entirely from the SET PATH statements, or correcting them as in the patch below.
    --- a/python27/lib/site-packages/gtk-2.0/runtime/bin/setenv.cmd
    +++ b/python27/lib/site-packages/gtk-2.0/runtime/bin/setenv.cmd
    @@ -23,14 +23,14 @@
     call:abspath PYTHON_PKGCONFIG "%PYTHON_ROOT%\Lib\pkgconfig\"
    
     if defined PATH (
    -    set PATH="%PYTHON_ROOT%;%PYTHON_SCRIPTS%;%RUNTIME_BIN%;%PATH%"
    +    set "PATH=%PYTHON_ROOT%;%PYTHON_SCRIPTS%;%RUNTIME_BIN%;%PATH%"
     ) else (
    -    set PATH="%PYTHON_ROOT%;%PYTHON_SCRIPTS%;%RUNTIME_BIN%"
    +    set "PATH=%PYTHON_ROOT%;%PYTHON_SCRIPTS%;%RUNTIME_BIN%"
     )
     if defined PKG_CONFIG_PATH (
    -    set PKG_CONFIG_PATH="%PYTHON_PKGCONFIG%;%RUNTIME_PKGCONFIG%;%PKGCONFIG_PATH%"
    +    set "PKG_CONFIG_PATH=%PYTHON_PKGCONFIG%;%RUNTIME_PKGCONFIG%;%PKGCONFIG_PATH%"
     ) else (
    -    set PKG_CONFIG_PATH="%PYTHON_PKGCONFIG%;%RUNTIME_PKGCONFIG%"
    +    set "PKG_CONFIG_PATH=%PYTHON_PKGCONFIG%;%RUNTIME_PKGCONFIG%"
     )
    
     cls
    
  3. After correcting the setenv.cmd script as above, make a GTK bash shell environment by copying the MSYS/MinGW shell icon on the desktop (or making a shortcut to c:\mingw\msys\1.0\msys.bat) , right-clicking, selecting Properties, editing the Target: to be C:\WINDOWS\system32\cmd.exe /C ""C:\Python27\Lib\site-packages\gtk-2.0\runtime\bin\setenv.cmd"&&"C:\MinGW\msys\1.0\msys.bat" and renaming it to GTK MinGW shell. If the icon gets screwed up, select Properties again, Change Icon and point it to the same location as the original MSYS/MinGW shell shortcut's icon path. Note you will still have to point PKG_CONFIG to the GTK runtime pkgconfig.exe and add both /usr/local/lib/pkgconfig, /mingw/lib/pkgconfig and /usr/lib/pkgconfig to your PKG_CONFIG_PATH in your ./configure commands, but this should add all of the other GTK and PKG_CONFIG paths and environmental variables you'll need, especially for dbus-glib. An alternative is to simply add the GTK runtime to the path of your bash shell by editing (or creating) your ~/.bashrc file.
    $ export GTK_BASEPATH="/c/Python27/Lib/site-packages/gtk-2.0/runtime/"
    $ export PATH=$GTK_BASEPATH/bin:$PATH
    
    Another alternative is to start the PyGTK/GTK Command Prompt which executes setenv.cmd, creating several environmental variables in that shell such as PYTHON_ROOT, PYTHON_PKGCONFIG, RUNTIME_PKGCONFIG, PKG_CONFIG_PATH and RUNTIME_BIN, thereby making sure that your GTK environment is correctly configured. Then type c:\mingw\msys\1.0\msys.bat to start a MSYS/MinGW shell which will also set up your MinGW/MSYS environment. Note: you may have already added GTK to your path during installation, depending on what options you selected.
    The library libgio-2.0-0.dll and other binaries are used by dbus-binding-tool.exe and others, so you will need to use this shell, or make the GTK runtime bin folder available, whenever you wan to use dbus-glib.

Build DBus

  1. Download the tarball or clone the git repository a folder in C:\, e.g. C:\DBus. Note: If you clone the repo, then instead of ./configure you will need to use ./autogen.sh which requires autotools. Use mingw-get to install autoconf, automake, m4 and libtool for both MSYS and MinGW. Also you may want to checkout the latest stable tag into your own private branch git checkout -b <myBranch> dbus-1.6.4.
    $ mingw-get install autoconf
    $ mingw-get install automake
    $ mingw-get install libtool
    $ mingw-get install msys-autoconf
    $ mingw-get install msys-automake
    $ mingw-get install msys-m4
    $ mingw-get install msys-libtool
    
  2. Apply the MemoryBarrier macro patch. The location of this patch is debatable, and there have been some mailing list posts, tickets, forks and suggeted patches [1, 2, 3, 4, 5, 6, 7, 8]. I applied it to dbus\dbus-sysdeps-win.c which is where the macro is used. I also added a prototype in the header so that gcc won't warn me about it. I have no idea whether this works as intended, or even what the intent was, so use it at your own risk. It is identical to the macro used by mingw-w64 in winnt.h. The other options are to get rid of the MemoryBarrier() call or use a different compiler such as MSVC or MinGW-w64.

    MemoryBarrier.patch for dbus-sysdeps-win.c

    --- a/dbus/dbus-sysdeps-win.c
    +++ b/dbus/dbus-sysdeps-win.c
    @@ -54,6 +54,13 @@
     #include <windows.h>
     #include <ws2tcpip.h>
     #include <wincrypt.h>
    +
    +__CRT_INLINE VOID MemoryBarrier(VOID)
    +{
    +  LONG Barrier = 0;
    +  __asm__ __volatile__("xchgl %%eax,%0 "
    +    :"=r" (Barrier));
    +}
    
     /* Declarations missing in mingw's headers */
     extern BOOL WINAPI ConvertStringSidToSidA (LPCSTR  StringSid, PSID *Sid);
    

    MemoryBarrierHeader.patch for dbus-sysdeps-win.h

    --- a/dbus/dbus-sysdeps-win.h
    +++ b/dbus/dbus-sysdeps-win.h
    @@ -38,6 +38,8 @@
    
     #define DBUS_CONSOLE_DIR "/var/run/console/"
    
    +
    +void MemoryBarrier(void);
    
     void _dbus_win_set_errno (int err);
     const char* _dbus_win_error_from_last_error (void);
    
    You can use git apply --ignore-space-change --ignore-whitespace <path/to/MemoryBarrier.patch> followed by commit -am "<your message>" or patch <path/to/dbus-sysdeps-win.c> <path/to/MemoryBarrier.patch> if you are using the tarball.
  3. configure, make and install
    $ ./configure --with-dbus-session-bus-default-address=autolaunch: PYTHON=/c/Python27/python PYTHON_INCLUDES=-I/c/Python27/include/ PYTHON_LIBS='-L/c/Python27/Lib -lpython27' PKG_CONFIG=/c/Python27/Lib/site-packages/gtk-2.0/runtime/bin/pkg-config.exe PKG_CONFIG_PATH=/c/Python27/Lib/pkgconfig:/c/Python27/Lib/site-packages/gtk-2.0/runtime/lib/pkgconfig:/c/mingw/msys/1.0/local/lib/pkgconfig:/c/mingw/msys/1.0/lib/pkgconfig:/c/mingw/lib/pkgconfig am_cv_python_pythondir='${prefix}/Lib/site-packages' am_cv_python_pyexecdir='${exec_prefix}/Lib/site-packages'
    $ make
    $ make install
    
  4. Check that it works by opening a daemon, and monitoring it using two MSYS shells.

    MSYS shell with daemon

    $ dbus-daemon --session
    

    MSYS shell with monitor

    $ dbus-monitor
    signal sender=org.freedesktop.DBus -> dest=:1.0 serial=2 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=NameAcquired   string ":1.0"
    method call sender=:1.0 -> dest=org.freedesktop.DBus serial=3 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='method_call'"
    method call sender=:1.0 -> dest=org.freedesktop.DBus serial=4 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='method_return'"
    method call sender=:1.0 -> dest=org.freedesktop.DBus serial=5 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='error'"
    
    You should see dbus-daemon and dbus-monitor as processes in ms-windows task manager. Use ctrl-c to kill the processes in each MSYS window.

Buld dbus-glib

  1. Download the tarball or clone the git repository on your hardrive, e.g. c:\dbus-glib. If you are using the git repo, checkout the latest tag into your own branch, and then use ./autogen.sh instead of ./configure. Also you may need to add --disable-gtk-doc as a configure option. I did not, but I used the tarball.
  2. confgure, make and install
    $ ./configure PYTHON=/c/Python27/python PYTHON_INCLUDES=-I/c/Python27/include/ PYTHON_LIBS='-L/c/Python27/Lib -lpython27' PKG_CONFIG=/c/Python27/Lib/site-packages/gtk-2.0/runtime/bin/pkg-config.exe PKG_CONFIG_PATH=/c/Python27/Lib/pkgconfig:/c/Python27/Lib/site-packages/gtk-2.0/runtime/lib/pkgconfig:/c/mingw/msys/1.0/local/lib/pkgconfig:/c/mingw/msys/1.0/lib/pkgconfig:/c/mingw/lib/pkgconfig am_cv_python_pythondir='${prefix}/Lib/site-packages' am_cv_python_pyexecdir='${exec_prefix}/Lib/site-packages'
    $ make
    $ make install
    

Build dbus-python, finally!

  1. Download the wip-windows tarball or clone the git repository and checkout 2012-07-04 branch on your hardrive, e.g. c:\dbus-python. Again, if using Git, you should use ./autogen.sh instead of ./configure, with the same arguments.
  2. configure, make and install
    $ ./configure PYTHON=/c/Python27/python PYTHON_INCLUDES=-I/c/Python27/include/ PYTHON_LIBS='-L/c/Python27/Lib -lpython27' PKG_CONFIG=/c/Python27/Lib/site-packages/gtk-2.0/runtime/bin/pkg-config.exe PKG_CONFIG_PATH=/c/Python27/Lib/pkgconfig:/c/Python27/Lib/site-packages/gtk-2.0/runtime/lib/pkgconfig:/c/mingw/msys/1.0/local/lib/pkgconfig:/c/mingw/msys/1.0/lib/pkgconfig:/c/mingw/lib/pkgconfig am_cv_python_pythondir='${prefix}/Lib/site-packages' am_cv_python_pyexecdir='${exec_prefix}/Lib/site-packages'
    
  3. Check that it works by starting Python by adding your MSYS environment site-packages folder to your Python path, starting Python and import dbus.
    $ export PYTHONPATH="c:\\mingw\\msys\\1.0\\local\\lib\\site-packages"
    $ python
    Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import dbus
    >>> session_bus = dbus.SessionBus()
    
  4. You now have a working dbus-python module. Yay!

Links

  1. https://bitbucket.org/bradpitcher/mingw-cross-env/src/5f91fb4e0199/src/dbus-1-fixes.patch
  2. https://sourceforge.net/tracker/?func=detail&atid=102435&aid=3420424&group_id=2435
  3. http://msdn.microsoft.com/en-us/library/ms684208%28VS.85%29.aspx
  4. https://bugs.freedesktop.org/show_bug.cgi?id=41423
  5. https://sourceforge.net/mailarchive/message.php?msg_id=24260705
  6. http://lists.gnu.org/archive/html/mingw-cross-env-list/2011-09/msg00033.html
  7. https://github.com/mxe/mxe/commit/d51ad53b664859b38c3cee61f18c4e8e8810fcef
  8. https://bitbucket.org/vog/mingw-cross-env/changeset/8bfc6d601bb1
This zip file contains an install script, patches, generated binaries, libraries and doc files.

Do not use it for evil.

Generated using markdown_py 2.2.0

Sunday, July 1, 2012

Building DBus for Windows XP with MinGW

Background

I was trying to contribute to Meld, a great diff/merge tool. On Linux, Meld has great version control (VC) integration, which is convenient, because if you run git difftool from the terminal, git locks the terminal until you are done. Great for merges.

So I cloned the meld repository, but the master branch no uses dbus, so that if you do a diff and Meld is already open, it will just add a new tab to the current instance of Meld, instead of starting a new one. Unfortunately there is no nicely packaged windows installer for dbus or dbus-python, so I started on the murky compile and build quest. Obviously without the module, import dbus fails.

Building DBus using MinGW32 (aka MinGW)

This was nearly an exercise in futility. Unfortunately (or fortunately) I didn't read the fine print on the DBus [link] page on Windows porting, that says that it has been successfully ported using MinGW-w64 which, who knew?, is different from MinGW. How is it different? Who knows? I haven't tried MinGW-w64 yet, so I'll let you know when I do. Familiar always seems better, but I will reserve judgement until some later date. One thing I like about MinGW is `mingw-get` which is kind of like `apt-get` or `yum` and makes is super easy to add packages from either MSYS or MinGW.

Step 1: MinGW

Download and install mingw-get-inst. Then install the basic packages: make, msys-make, gcc, g++. Then make sure you download  expat, msys-expat, libexpat, msys-libexpat, which are used during the build process. You'll probably also want the mingw-developers-toolkit which has autoconf/make, libtools, etc.

Step 1.5: GTK

download and install either pyGTK, GTK-2.0 all-in-one installers for windows, or get the glib tarball and see this to bootstrap glib.
http://www.mingw.org/wiki/Bootstrapping_GLIB_with_MinGW

Step 1.75: Doxygen

For kicks install doxygen for really nice help documents.

Step 2: DBus

Download the latest dbus tarball and extract it to your c:\ folder. This is important because in general spaces cause issues, and if you need to add it to your path, it will have the shortest possible length, which is nice.

Step 2.5: add MemoryBarrier macro

Patch the "dbus/dbus-sysdeps-win.c" file at the top right after the "#includes" and "#defines" with the following snippet:

__CRT_INLINE VOID MemoryBarrier(VOID)
{
  LONG Barrier = 0;
  __asm__ __volatile__("xchgl %%eax,%0 "
    :"=r" (Barrier));
}

Step 3: configure, make and install

Read the INSTALL text file in the extracted folder. The build, make & installation process is the same for almost all Linux/Unix packages packaged using GNU autotools. There is a copy of pkg-config.exe in your GTK distribution. Otherwise you can try to build it, but it depends on glib, which depends on it - sort of a catch-22.

    $ ./configure PKG_CONFIG=/c/Python27/Lib/site-packages/gtk-2.0/runtime/bin/pkg-config.exe PYTHON=/c/Python27/python
    $ make
    $ make install

Step 4: check that it works

This test is the same as for building dbus with cmake, except that you don't need to copy the libexpat.dll to dbus bin folder, because mingw/msys/autotools does a better job of helping dbus find this file.
Open two msys shells, and start a dbus-daemon in the first shell using the default session configuration, but output the address to a file. Unfortunately, unless you enabled verbose output during the build with the flag -DDBUS_VERBOSE, you can't add it now. You could try make uninstall, make clean, make distclean and start over again, if you really need to debug dbus.

msys shell #1

$ dbus-daemon --session --print-address > /home/dbus.txt

msys shell #2

$ export DBUS_SESSION_BUS_ADDRESS="<paste from dbus.txt, don't forget quotes>"
$ dbus-monitor --address $DBUS_SESSION_BUS_ADDRESS

signal sender=org.freedesktop.DBus -> dest=:1.0 serial=2 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=NameAcquired   string ":1.0"
method call sender=:1.0 -> dest=org.freedesktop.DBus serial=3 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='method_call'"
method call sender=:1.0 -> dest=org.freedesktop.DBus serial=4 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='method_return'"
method call sender=:1.0 -> dest=org.freedesktop.DBus serial=5 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='error'"
If you see this, "yay!" it works, I think...

build and install dbus with cmake, msvc and nmake

Here are the directions for building dbus using the cmake folder and Microsoft Visual Studio C++ (MSVC).
  1. Install MSVC of your choice, e.g.: 2008 Express (MSVC90) or  2010 Express (MSVC100), then get all updates and service packs.
  2. Download and install CMake (2.8.8). I'll list the software versions I used in parentheses.
  3. Download and install Expat (2.1.0)
  4. Download and install doxygen (1.8.1.1)
  5. Download and install NullSoft (2.46)
  6. Download and extract dbus (1.6.2) to folder with no spaces, e.g.: c:\dbus-1.6.2-msvc90
  7. Start CMake gui from start menu
  8. Enter the path to your source, e.g.: "C:\dbus-1.6.2-msvc90\cmake"
  9. Make a folder called, "build" in the same folder and enter it as the build directory, e.g.:  "C:\dbus-1.6.2-msvc90\cmake\build"
  10. add the following variables using the "Add Entry" button. The format I use here is NAME:TYPE=value, which is the how cmake expects it at the command line, but in the gui you just fill in the boxes.
    • LIBEXPAT_LIBRARIES:FILEPATH="C:\Program Files\Expat 2.1.0\Bin\libexpat.lib"
    • LIBEXPAT_INCLUDE_DIR:PATH="C:\Program Files\Expat 2.1.0\Source\lib"
  11. (UPDATED - 2012-07-17)  Optional: Edit CMakeLists.txt so that the default session address is autolaunch: instead of the nonce-tcp:, see the "D-Bus Windows help" thread in the mailing list.
    • DBUS_SYSTEM_BUS_DEFAULT_ADDRESS="autolaunch:"
  12. Click Configure, select either MSVC90 (Visual Studio 2008) or MSVC100 (Visual Studio 2010), then generate.
  13. close CMake and start MSVC from the start menu.
  14. Load the dbus.sln from the build folder you created in step 8.
  15. Select configuration manager from build pull down in menu bar, and select doc, INSTALL and PACKAGE.
  16. Build the solution.
  17. Make sure that all projects built successfully.
  18. If there are any errors clean the the solution, quit and go back to step 1.
  19. Use the configuration manager to change from Debug to Release, and build the solution again.
  20. (UPDATED - 2012-07-03) Copy the libexpat.dll to the dbus bin folder ("C:\Program Files\dbus\bin") and the libexpat.lib file to the dbus lib folder, otherwise you will get an error saying that dbus-daemon can't find expat application and that re-installing expat might fix the problem. Setting the PATH to include expat does not fix this issue, sorry. Not sure if dbus also needs library but it can't hurt.
  21. (UPDATED - 2012-07-03) Test the daemon using a windows command shell, navigate to the dbus bin folder and try dbus-daemon --session --print-address > dbus_address.txt (or whatever), and then open dbus_address select and copy the address. Note for added excitement, set DBUS_VERBOSE=1 first, and the dbus-daemon will display everything that it's thinking in the terminal, which gives you the sense that it's actually working!
  22. $ cd "C:\Program Files\dbus\bin"
    $ set DBUS_VERBOSE=1
    $ dbus-daemon --session --print-address > dbus_address.txt

  23. (UPDATED - 2012-07-03) Open another windows shell, navigate to the dbus bin folder again and set DBUS_SESSION_BUS_ADDRESS="<paste the address from the dbus_address.txt file here>". Don't forget the quotes, or you;'ll get a string of errors. (Note: if you edited the CMakeLists.txt file in the optional step 11, then you can just use dbus-monitor, and it should work out of the box.) Now type dbus-monitor --address %DBUS_SESSION_BUS_ADDRESS% and you should see the following text in the monitor shell (and a ton of stuff if you enabled verbose in the daemon shell):
  24. $ cd "C:\Program Files\dbus\bin"
    $ set DBUS_SESSION_BUS_ADDRESS="<paste from dbus_address.text, don't forget quotes>
    $ dbus-monitor --address %DBUS_SESSION_BUS_ADDRESS%
    signal sender=org.freedesktop.DBus -> dest=:1.0 serial=2 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=NameAcquired   string ":1.0"
    method call sender=:1.0 -> dest=org.freedesktop.DBus serial=3 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='method_call'"
    method call sender=:1.0 -> dest=org.freedesktop.DBus serial=4 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='method_return'"
    method call sender=:1.0 -> dest=org.freedesktop.DBus serial=5 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch   string "eavesdrop=true,type='error'"

Done!

Now try using nmake.

from the start menu start msvc command prompt
add cmake to the path by typing the following:
    $ set PATH="C:\Program Files\Cmake-2.8.0\bin";%PATH%
change the folder where you extracted dbus, e.g. C:\dbus-1.6.0 and make a new folder called build
    $ cd C:\dbus-1.6.0
    $ mkdir build
    $ cd build
Run cmake with the following options
    $ cmake -G "NMake Makefiles" -D CMAKE_BUILD_TYPE=Release -D LIBEXPAT_LIBRARIES:FILEPATH="C:\Program Files\Expat 2.1.0\Bin\libexpat.lib" -D LIBEXPAT_INCLUDE_DIR:PATH="C:\Program Files\Expat 2.1.0\Source\lib" "C:\dbus-1.6.0\cmake"
Now build the solution
    $ nmake /A
then finally ...
    $ nmake install
I think you can build the package this way too ...
    $ nmake package
Fork me on GitHub