Thursday, July 24, 2014

Django from development to production: Apache and PostgreSQL

Perhaps like many, I start my Django projects with the default settings. That means that my database backend is SQLite and I use the simple HTTP server provided by Django for debugging. (See this post for a hack to get the debug server to run as a Windows service.) Now it's time for production, and that means using a real HTTP server like Apache and a perhaps a more flexible database like PostgreSQL. Here are some notes on the steps I took, and a couple of missteps as well. There are many other blogs with similar notes, eg Salty Crane. By the way I am using Django-1.6.5 and South-1.0. The Django How-To Deployment and Installation guides both recommend using Apache with mod-wsgi. The Install FAQs recommend PostgreSQL with psycopg2.

Apache HTTP Server

  1. Download Apache from ApacheLounge. I chose the 64-bit Windows binary built with VC10 because my system is 64-bit and I have VC10. Also it matches the mod-wsgi binary available for Windows-x64. I also chose Apache-2.4 version instead of the older 2.2.
  2. The ApacheLounge zip file has instructions, but it's simple, just extract it to c:\Apahce24. I also made a shortcut to ApacheMonitor.exe in my Startup folder. This nifty app runs in the system tray giving you the server status, and lets you restart, stop or start the server or open services. Finally I changed ownership of the folder recursively to SYSTEM.
  3. Download mod_wsgi from Graham Dumpleton's GitHub releases page - it has all of the windows binaries you need. Unfortunately Christoph Gohlke's version on his website - Python Extension Packages for Windows - doesn't have a 64-bit version. Extract the library, and copy it to your Apache/modules folder.
  4. edit httpd.conf to load the mod_wsgi module. See the Django documentation on how to Use Apache with mod_wsgi and the mod_wsgi quick installation guide for configuration. Specifically add the line LoadModule wsgi_module modules/ - I added it to the end of the list of modules. Note comments are preceded by # (aka: the hash symbol).
  5. Provide the mod_wsgi parameters that allow the server to serve the Django folder:
    WSGIScriptAlias / /path/to/
    WSGIPythonPath /path/to/
    <Directory /path/to/>
    Require all granted
  6. Alias and allow the static and media folders:
    Alias /static/ /path/to/
    Alias /media/ /path/to/
    <Directory /path/to/>
    Require all granted
    <Directory /path/to/>
    Require all granted
    I put these and the preceding lines in the section of httpd.conf where it says to specify which folders to allow access.
  7. Use collectstatic to make all admin files and other css/js files available to server. Make sure STATIC_ROOT is set to the same folder that is aliased in the httpd.conf file, and that it is empty because it will be overwritten. I keep all of my bootstrap and tablesorter files as well as images and icons in an assets folder that is on my STATIC_DIRS list.
  8. Turn DEBUG and TEMPLATE_DEBUG off.
  9. You must specify ALLOWED_HOSTS, e.g., or you will get a 500 server error.
  10. Install Apache as service, from admin COM window, navigate to C:\Apache24\bin and type httpd.exe -k install, now go to your site and see if it works? You may need to start the service. You can use the ApacheMonitor to start it or open services. If you are not an admin, it will prompt you for admin creds.

Now for Postgre

  1. first make a copy of your db.sqlite3 file.
  2. Make a mental check list of all of the apps and models you have and use dumpdata --natural myapp1 myapp2 myapp3.mymodel auth.user etc. > myfixtures.json to save them all to a JSON file. Note: only select your models, or you may get integrity or other errors when loading fixtures, and don't forget auth.user or auth.groups if you are using them. Specify models of apps using dot notation. You may need the --natural option for auth objects, not sure, but it doesn't hurt.
  3. Download PostgreSQL from EnterpriseDB and install it. Note: If you are on a domain network and are using Vista or Server 2003/2008 you may experience a well known issue with the message, "database cluster initialisation failed". To resolve the issue, (1) create a local user with admin rights, (2) uninstall Postgres and delete C:\Program Files\PostgreSQL, (3) log out and log on as the local user, (4) turn off antivirus and reinstall PostgreSQL into C:\PostgreSQL not C:\Program Files, (5) recursively add full-control permissions of C:\PostgreSQL to new user, (6) without uninstalling, rerun installer which will state that it detects your previous installation and is updating it and make sure that you use the same data folder.
  4. I also made a file in my profile's bin folder with the following code
    C:\PROGRA~1\PostgreSQL\9.3\bin\psql.exe %*
    so that I can use dbshell in my Git Bash shell which has %USERPROFILE%\bin (aka ~/bin) on my path by default, and I made another script for Bash
    #! /bin/sh
    /c/Program\ Files/PostgreSQL/9.3/bin/psql.exe "$@"
    so that I can use psql -U -h localhost -p 5432 in my Git Bash shell.
  5. Download the Python PostgreSQL binding psycopg (aka psycopg2) from Stickpeople Project another extraordinary good Samaritan service like Christoph Gohlke's Python Extension Packages for Windows, who also has another version of psycopg2 build from PostgreSQL-9.3 available for download.
  6. Use the pgAdmin3 panel to connect to the server and create a new user and set the password, e.g. django.
    Just right click on Postgre-9.3 (localhost:5432), select Connect and enter your password. Then right click on Databases and select New Database. For new users right click on Login Roles and select New Login Role.
  7. Create a new database for you Django project and set the new user as the owner. See the PostgreSQL notes in the Django Database documentation.
  8. Update your settings for the new database. A PostgreSQL example is given in the Settings documentation for DATABASE. Set the ENGINE key to postgresql_psycopg2, NAME to the name you gave your Django project's database, USER to the owner you set for the new database, and PASSWORD to the database owner's password. HOST is probably 'localhost' and PORT is probably 5432.
  9. Check one last time that you've backed up the old database and used dumpdata to save fixtures of your apps and models, including auth.users and auth.groups in a JSON file. Then use syncdb to install your Django project in the new database. Say yes or no when it asks to create a superuser, because when you load the fixtures it will overwrite any rows in your tables.
  10. Use South to migrate the databases, eg migrate <app>, etc. for all apps in your project.
  11. Now use loaddata myfixtures.json to load your app and model data into the new database. You should be able to load them with one file containing all of the fixtures, because Django will reference the fixture for tables or rows not yet created. Do not include any extra Django fixtures, or you will raise exceptions. Specifically do not use dumpdata without specifying any of your apps or models or Django saves extra duplicate info such as content_types, which can not be loaded into the new database because they already exist.
Finally, does your app work? Test the data. Hopefully everything is hunky dory.

Wednesday, July 16, 2014

WinMerge with Git

WinMerge is a fine diff tool for Windows platforms.
I downloaded a portable version and extracted into my root folder as C:\WinMerge\. Then I used it as a Git difftool by executing these lines in my Git Bash shell.
$ git config --global difftool.winmerge.cmd '/c/winmerge/winmergeu.exe -e -u -x -wl -wr -dl LOCAL -dr REMOTE "${LOCAL}" "${REMOTE}"'
$ git config --global mergetool.winmerge.cmd '/c/winmerge/winmergeu.exe -e -u -x -dl LOCAL -dr REMOTE "${LOCAL}" "${REMOTE}" "${MERGED}"'
The command line options are explained in the WinMerge Documentation. Now you can call it as your diff or merge tool.
$ git difftool -t winmerge
Fork me on GitHub