Python Package Scaffolding with Web Interface

Introduction

This post is about creating a distributable package for a application written in Python. Beside file based configuration options and a CLI, it is always a user friendly approach to include a web based interface as well. The sample project in this post uses flask and sqlalchemy for the data and user interface and a very basic configuration system that works well with python package management.

Complete code can be found here

Cluedo : Sample Application

The sample application includes two inbuilt applications, one to launch the business logic which is tied up with the cluedo command and another to start the web server for data manipulation cluedo-webserver.

Most of the focus is given to the configuration management and web server integration.

MVPs of python package

• setup.py : the file that contains the package information
• Manifest.in : to include non-python files into distributable
• config : config files and how to create local folder for user specific configuration and settings
• www : the submodule that contains html templates for the UI and flask setup

Manifest

This file decides what goes in the python package distributable. By default all python files are part of it and to make sure other essentials go in as well you will have to specifically mention them here.

include cluedo/cluedo.cfg
graft cluedo/www
graft cluedo/www/static
graft cluedo/www/templates
global-exclude __pycache__  *.pyc

include adds individual files, graft is for including directories. Also there is an exclude statement to avoid including unnecessary files.

Configuration

Installing and using the application configures it automatically based on the logic present in configuration.py. It creates a app folder in user’s home directory along with coniguration file, log files and sqlite DB file.

Persisting Data

Database is an essential part of maintaining a stateful application. A production system should be flexible enough to handle migration as well to maintain compatibility. This example includes the migration mechanism for the DB models using alembic. There is a script to make sure that the DB is initialized properly but it is the maintainers responsibility to make sure that subsequent updates in tables is managed as smoothly as possible.

./scripts/setupdb.sh

make sure to provide necessary permissions to script files.

cd scripts
chmod +x *

This script will create the sample model defined in cluedo/models

Flask

Flask is probably the most lightweight and simple way to include a web server into an application. Since this is supposed to be used locally no additional setup is required and the server included for debugging is more than enough. But if the intention is to deploy the package as part of a production stack then further configuration and consideration is highly recommended, which is clearly mentioned in flask documentation itself.

Flask specific configuration are specified in cluedo/flask_configuration.py.

Other relevant configurations are present in cluedo.cfg under webserver section.

Flask application is defined in www with dependency setup and admin panel configuration.

Internal apps

Python packaging provides a way to define command line applications to be shipped with the distributable in the form of console scripts.

    entry_points = {
        'console_scripts': [
            'cluedo=cluedo.apps.main:run',
            'cluedo-webserver=cluedo.apps.webserver:run'
            ],
    },

Two of them are defined here, the main CLI and to start the included webserver. Both of them make use of click for argument parsing.

Dependency

Last thing to consider is dependency, no application can truly be stand alone and requires a lot of supporting libraries. Python provides two ways to include them in a package, as part of setup.py

    install_requires=[
    ],

or in form of requirements.txt, which can be generated and installed as such.

pip freeze > requirements.txt
pip install -r requirements.txt

Running the system

• Clone the code and create a virtual environment
• run ./scripts/build.sh to build cluedo
• install pip install dist/cluedo-0.0.1.tar.gz
• run pip install -r requirements.txt to install all dependencies
• setup db ./scripts/setupdb.sh

  This will create the Initial Migration and apply them to the DB located as per the configuration file

• run CLI cluedo --help
• run webserver cluedo-webserver
• http://localhost:5000 should be running the admin panel

Conclusion

In short, Python packages are easy to prototype and distribute and if you are planning to distribute an non-library standalone application then do consider including a configuration mechanism and web UI for the user.