Developer Guidelines

Everyone contributing code to TOAST should aim to follow these guidelines in order to keep the code consistent. Whenever you make changes and before opening a pull request, run the script src/format_source.sh to apply the standard formatting rules to the Python and C++ code. If you use an editor with some other automatic code formatting, you should disable it unless you can configure it identically to the action of this script.

Python Code

We aim to follow PEP8 style guidelines whenever possible. There are some reasonable exceptions to this. In particular, import statements might not always be placed at the top of the code if:

  • The import is an optional feature that has large performance impacts and which is only used infrequently. For example matplotlib, astropy, etc.
  • Some initialization code is needed prior to the import statement. For example, setting the matplotlib backend.

Other style choices:

  • Double quotes for strings unless the string contains double quotes, resulting in excessive backslash escaping. This should be handled automatically by the code formatter.

We use the “black” command line tool to format our source. This needs to be installed on your system before running the format_source.sh script.

Compiled Code

For consistency with python, class names follow python CamelCase convention. Function names follow python_underscore_convention. Formatting is set by uncrustify with a custom config file and this is run by the format_source.sh script.

All code that is exposed through pybind11 is in a single toast namespace. Nested namespaces may be used for code that is internal to the C++ code.

The “using” statement is allowed for aliasing a specific class or type:

using ShapeContainer = py::detail::any_container<ssize_t>;

But should not be used to import an entire namespace:

using std;

Header files included with “#include” should use angle brackets (“<>”) for the header file name. If this fails for some reason then that indicates a problem with the build system and its header file search paths. Using double quotes for “#include” statements is OK when including raw source files in the same directory (for example, when including raw *.cpp contents generated by external scripts).

When including C standard header files in C++, use the form:

#include <cstdio>

Rather than:

#include <stdio.h>

Pointer / reference declarations: this allows reading from right to left as “a pointer to a constant double” or “a reference to a constant double”:

double const * data
double const & data

Not:

const double * data
const double & data

When indexing the size of an STL container, the index variable should be either of the size type declared in the container class or size_t.

When describing time domain sample indices or intervals, we using int64_t everywhere for consistency. This allows passing, e.g. “-1” to communicate unspecified intervals or sample indices.

Single line conditional statements:

if (x > 0) y = x;

Are permitted if they fit onto a single line. Otherwise, insert braces.

Internal toast source files should not include the main “toast.hpp”. Instead they should include the specific headers they need. For example:

#include <toast/sys_utils.hpp>
#include <toast/math_lapack.hpp>
#include <toast/math_qarray.hpp>

If attempting to vectorize code with OpenMP simd constructs, be sure to check that any data array used in the simd region are aligned (see toast::is_aligned). Otherwise this can result in silent data corruption.

Documentation: sphinx is used. All python code should have docstrings. All C++ code exposed through pybind11 should also have docstrings defined in the bindings. C++ code that is not exposed to python is considered internal, expert-level code that does not require formal documentation. However, such code should have sufficient comments to describe the algorithm and design choices.

Testing Workflow

We are using a github workflow to pull docker containers with our dependencies and run our unit tests. Those docker containers are re-generated whenever a new tag is made on the cmbenv git repository. So if there are dependencies that need to be updated, open a PR against cmbenv which updates the version or build in the package file. After merging and tagging a new cmbenv release the updated docker images will be available in an hour or two and be used automatically.

Release Process

There are some github workflows that only run when a new tag is created. Unless you are sure everything works, create a “release candidate” tag first. Before making a tag, ensure that the docs/changes.rst file contains all pull requests that have been merged since the last tag. Also edit the src/toast/RELEASE file and set the version to a PEP-440 compatible string. Next go onto the github releases page and create a new release from the master branch. Briefly, the format for a stable release, a release candidate or alpha version is:

2.6.9 2.6.7rc2 2.6.8a1

After tagging the release, verify that the github workflow to deploy pip wheels runs and uploads these to PyPI. The conda-forge bots will automatically detect the new tag and open a PR to update the feedstock. Also, after the release is complete, update the RELEASE file to be at the “alpha” of the next release. For example, after tagging version 3.4.5, set the version in the RELEASE file to 3.4.6a1. This version is only used when building pip wheels. Anyone installing locally with setup.py or with pip running on the local source tree will get a version constructed from the number of commits since the last git tag.