Wednesday, April 15, 2015

Recommended Python Project Layout

[UPDATE 2016-07-19] Lately I've preferred using core instead of lib for the main package modules.

[UPDATE 2015-06-04] Create top level package to bundle all sub-packages and package-data together for install.

Been looking for a good, comprehensive, credible guide:

  1. Pretty good links in this SO Q&A:
    1. What is the best project structure for a Python application?
    2. Especially this one:
      1. Open Sourcing a Python Project the Right Way
  2. And maybe, maybe theses ones:
    1. Learn Python The Hard Way Exercise 46: A Project Skeleton
    2. The Hitchhiker’s Guide to Python! Structuring Your Project by Kenneth Reitz
      1. Repository Structure and Python also by Kenneth Reitz
    3. How to Package Your Python Code: Minimal Structure by Scott Torborg
    4. Interesting Things, Largely Python and Twisted Related: Filesystem structure of a Python project by Jean-Paul Calderone
  3. Of course understanding Python Modules and Packages
    1. The Python Tutorial: 6. Modules
  4. An understanding of how to install packages, and roughly I guess how pip and setuptools interact with distutils is good
    1. Python Documentation: Installing Python Modules
  5. Way later down the line it helps to understand distutils and setuptools for deploying packages
    1. Python Packaging User Guide
    2. Setuptools
    3. Python Documentation: Distributing Python Modules
    4. How to Package Your Python Code by Scott Torborg
    5. The Hitchhiker’s Guide to Packaging
  6. There is also a package that will create a boiler plate project layout for you but I don’t recommend it
    1. PyPI: Python Boilerplate Template

It's hard to pin a standard style down. Here’s mine:

MyProject/ <- git repository
|
+- .gitignore <- *.pyc, IDE files, venv/, build/, dist/, doc/_build, etc.
|
+- requirements.txt <- to install into a virtualenv
|
+- setup.py <- use setuptools, include packages, extensions, scripts and data 
|
+- MANIFEST.in <- files to include in or exclude from sdist
|
+- readme.rst <- incorporate into setup.py and docs
|
+- changes.rst <- release notes, incorporate into setup.py and docs
|
+- myproject_script.py <- script to run myproject from command line, use Python
|                         argparse for command line arguments put shebang
|                         `#! /usr/bin/env python` on 1st line and end with a
|                         `if __name__ == "__main__":` section, include in
|                         setup.py scripts section for install
|
+- any_other_scripts.py <- scripts for configuration, documentation generation
|                          or downloading assets, etc., include in setup.py
|
+- venv/ <- virtual environment to run tests, validate setup.py, development
|
+- myproject/ <- top level package keeps sub-packages and package-data together
   |             for install
   |
   +- __init__.py <- contains __version__, an API by importing key modules,
   |                 classes, functions and constants, __all__ for easy import
   |
   +- docs/ <- use Sphinx to auto-generate documentation
   |
   +- tests/ <- use nose to perform unit tests
   |
   +- other_package_data/ <- images, data files, include in setup.py
   |
   +- core/ <- main source code for myproject, sometimes called `lib`
   |  |
   |  +- __init__.py <- necessary to make mypoject_lib a sub-package
   |  |
   |  +- … <- the rest of the folders and files in myproject
   |
   +- related_project/ <- a GUI library that uses myproject_lib or tools that
      |                   myproject_lib depends on that's bundled together, etc.
      |
      +- __init__.py <- necessary to make related_project a sub-package
      |
      +- … <- the rest of the folders and files in your the related project
Fork me on GitHub