Software Management

This guide asks a range of questions to consider when planning your software management. Each question has resources for more information.

Examples are not an exhaustive list of possible options and many are Python-specific.

Checklist

• Use version control for all of your code (e.g., GitHub).

• Create one environment for each project (e.g., conda).

• Choose the most appropriate method for capturing your computational environment.

• Capture your computational environment.

• Share your captured computational environment (along with your results/analysis) with a citable DOI and license.

• Ensure (at least) the key functionality is correct, by writing, and regularly running, tests.

• Ensure (at least) the key functionality is documented.

Questions

What is the software environment setup?

Integrated Development Environment (IDE)

• e.g., Jupyter Lab, Visual Studio Code, PyCharm (Python)

Version control

• Code

  • e.g., Git (GitHub, GitLab, Bitbucket), Subversion, Mercurial

• Data

  • e.g., Dolt (DoltHub), Git Large File Storage (LFS), Datalad, git-annex

• Resources

  • Version control (The Alan Turing Institute)

Reproducibility

Package management

• User-level package management systems

  • e.g., conda, spack

• Virtual environments

  • e.g., Python: pip and venv, virtualenv, pipenv, poetry

  • binder (guide), conda-store

Entire computational environment

• Containers (guide)

  • e.g., Singularity (guide), Docker, repo2docker

  • Manage and deploy containers e.g., Kubernetes

• Virtual machines

  • e.g., Vagrant (guide), VirtualBox

Workflow management

• e.g., SnakeMake, Luigi, Nextflow (with Singularity), Make, Research Compendium, protocols.io, Ploomber

Resources

• Reproducible environments (The Turing Way)

• Reusable Code (The Turing Way)

• Tools for Reproducible Research

• Reproducible research (CodeRefinery)

• An Imperfect Guide to Imperfect Reproducibility, Gabriel Becker

How will software correctness be verified?

Testing

• e.g., pytest (Python), unittest (Python), NumPy testing (Python)

• Various levels (when), types (what), and methods (how)

• Debugging

  • pdb (Python)

• Mocking

• Refactoring

  • Design (objects), Classes

• Resources

  • Software testing fundamentals

  • Testing (The Alan Turing Institute)

Continuous integration (CI)

• e.g., GitHub Actions (Guide), Travis CI, GitLab CI, Jenkins, Azure Pipelines, Circle CI

Code quality

• Formatting

  • e.g., Black (Python)

• Style

  • e.g., PEP 8 (Python), Flake8 (Python), NumPy (Python), Google Style Guides

• Static code analysis (linting)

  • e.g., Pylint (Python), autopep8 (Python), Mypy (Python)

Code review

• e.g., Codacy, CodeFactor

• Code coverage

  • e.g., codecov, Coveralls

How to manage the software?

• Project structure

  • e.g., cookiecutter (Data Science, Jupyter Book, EasyData)

• Share, publish, and release

  • Citable releases (with DOI, Digital Object Identifier)

    • e.g., Zenodo (guide), Open Science Framework, FigShare, Dryad

  • Executable environment

    • e.g., Binder, BinderHub

• Documentation

  • e.g., Jupyter Book, Read the Docs, Sphinx, nbdev, fastdoc

  • Publish online using GitHub Pages (for a Jupyter Book)

• Licensing

• Issue tracking

• Security

  • e.g, GitHub Workflows and Essentials

Training

• SWD3: Software development practices for Research, Research Computing, University of Leeds.

• SWD7: Introduction to reproducible workflows in Python, Research Computing, University of Leeds.

• Intermediate Research Software Development in Python, The Carpentries.

Other general resources

• Open Research (Library)

• Software Carpentry

• CodeRefinery

• UK Reproducibility Network

• Versioning (UK Data Service)

Examples

• SHAP (SHapley Additive exPlanations)

  • Version control with GitHub

  • Continuous integration with GitHub Actions, Travis CI

  • Capture and share project environment via conda or pip

  • MIT license

  • Documentation with examples

  • Tests with pytest

  • Reproducible via Binder

• TPOT (Tree-based Pipeline Optimization Tool)

  • Version control with GitHub

  • Continuous integration with GitHub Actions, Travis CI, AppVeyor

  • Capture and share project environment via conda or pip

  • LGPL-3.0 license

  • Documentation with examples

  • Tests with pytest

  • Citable DOI through Zenodo

  • Code coverage with Coveralls

• PyHealth (A Python Library for Health Predictive Models)

  • Version control with GitHub

  • Continuous integration with Circle CI, Travis CI, AppVeyor

  • Capture and share project environment via pip

  • BSD-2-Clause license

  • Documentation with examples

  • Tests with pytest

  • Reproducible via Binder

• VEROS (versatile ocean simulator Python / JAX)

  • Version control with GitHub

  • Continuous integration with GitHub Actions

  • Capture and share project environment via conda or pip

  • MIT license

  • Documentation with examples

  • Tests with pytest

  • Code coverage with codecov