Wednesday, April 15, 2015

Recommended Python Project Layout

[UPDATE 2018-09-04] Links to Cookiecutter and Bootstrap a Scientific Python Library from the National Synchrotron Light Source II (NLSL-II).

[UPDATE 2016-07-19] Lately I've preferred using core instead of lib for the main package modules.

[UPDATE 2015-06-04] Create top level package to bundle all sub-packages and package-data together for install.

Been looking for a good, comprehensive, credible guide:

  1. Pretty good links in this SO Q&A:
    1. What is the best project structure for a Python application?
    2. Especially this one:
      1. Open Sourcing a Python Project the Right Way
  2. And maybe, maybe theses ones:
    1. Learn Python The Hard Way Exercise 46: A Project Skeleton
    2. The Hitchhiker’s Guide to Python! Structuring Your Project by Kenneth Reitz
      1. Repository Structure and Python also by Kenneth Reitz
    3. How to Package Your Python Code: Minimal Structure by Scott Torborg
    4. Interesting Things, Largely Python and Twisted Related: Filesystem structure of a Python project by Jean-Paul Calderone
  3. Of course understanding Python Modules and Packages
    1. The Python Tutorial: 6. Modules
  4. An understanding of how to install packages, and roughly I guess how pip and setuptools interact with distutils is good
    1. Python Documentation: Installing Python Modules
  5. Way later down the line it helps to understand distutils and setuptools for deploying packages
    1. Python Packaging User Guide
    2. Setuptools
    3. Python Documentation: Distributing Python Modules
    4. How to Package Your Python Code by Scott Torborg
    5. The Hitchhiker’s Guide to Packaging
  6. There are also a packages that will create a boiler plate project layout for you but I wouldn't recommend them except as reference guides - the tutorial by NSLS-2 being the notable exception, PTAL!
    1. Bootstrap a Scientific Python Library: This is a tutorial with a template for packaging, testing, documenting, and publishing scientific Python code.
    2. Cookiecutter: A command-line utility that creates projects from cookiecutters (project templates), e.g. creating a Python package project from a Python package project template.
    3. PyPI: Python Boilerplate Template

It's hard to pin a standard style down. Here’s mine:

MyProject/ <- git repository
+- .gitignore <- *.pyc, IDE files, venv/, build/, dist/, doc/_build, etc.
+- requirements.txt <- to install into a virtualenv
+- <- use setuptools, include packages, extensions, scripts and data 
+- <- files to include in or exclude from sdist
+- readme.rst <- incorporate into and docs
+- changes.rst <- release notes, incorporate into and docs
+- <- script to run myproject from command line, use Python
|                         argparse for command line arguments put shebang
|                         `#! /usr/bin/env python` on 1st line and end with a
|                         `if __name__ == "__main__":` section, include in
|                scripts section for install
+- <- scripts for configuration, documentation generation
|                          or downloading assets, etc., include in
+- venv/ <- virtual environment to run tests, validate, development
+- myproject/ <- top level package keeps sub-packages and package-data together
   |             for install
   +- <- contains __version__, an API by importing key modules,
   |                 classes, functions and constants, __all__ for easy import
   +- docs/ <- use Sphinx to auto-generate documentation
   +- tests/ <- use nose to perform unit tests
   +- other_package_data/ <- images, data files, include in
   +- core/ <- main source code for myproject, sometimes called `lib`
   |  |
   |  +- <- necessary to make mypoject_lib a sub-package
   |  |
   |  +- … <- the rest of the folders and files in myproject
   +- related_project/ <- a GUI library that uses myproject_lib or tools that
      |                   myproject_lib depends on that's bundled together, etc.
      +- <- necessary to make related_project a sub-package
      +- … <- the rest of the folders and files in your the related project

1 comment:

Fork me on GitHub