Processing Model#
A TOAST workflow usually consists of loading or simulating different
types of data into one or more [Observation]{.title-ref} objects (See
section observations{.interpreted-text role=”ref”}) inside an overall
[Data]{.title-ref} instance. The workflow classes that populate and
manipulate these Data objects are called “Operators”.
Operators#
An operator class inherits from the [toast.ops.Operator]{.title-ref} class, and has several key characteristics:
Each Operator is configured using the traitlets package. This allows easy configuration of an operator at construction time or afterwards, and allows modular documentation of parameters, parameter checking, and construction from parameter files.
Operators can be called repeatedly on subsets of data (both observations and detectors) with the [exec()]{.title-ref} method.
Operators have a [finalize()]{.title-ref} method that performs any final calculations or other steps after all timestream data has been processed.
- class toast.ops.Operator(**kwargs: Any)[source]#
Base class for Operators.
An operator has methods which work with a toast.dist.Data object. This base class defines some interfaces and also some common helper methods.
- exec(data, detectors=None, use_accel=None, **kwargs)[source]#
Perform operations on a Data object.
If a list of detectors is specified, only process these detectors. Any extra kwargs are passed to the derived class internal method.
Accelerator use: If the derived class supports OpenMP target offload and all the required data objects exist on the device, then the _exec() method will be called with the “use_accel=True” option. Any operator that returns “True” from its _supports_accel() method should also accept the “use_accel” keyword argument.
- Parameters
data (toast.Data) – The distributed data.
detectors (list) – A list of detector names or indices. If None, this indicates a list of all detectors.
- Returns
None
- finalize(data, use_accel=None, **kwargs)[source]#
Perform any final operations / communication.
A call to this function indicates that all calls to the ‘exec()’ method are complete, and the operator should perform any final actions. Any extra kwargs are passed to the derived class internal method.
- Parameters
data (toast.Data) – The distributed data.
- Returns
None or an Operator-dependent result.
- Return type
(value)
- apply(data, detectors=None, use_accel=None, **kwargs)[source]#
Run exec() and finalize().
This is a convenience wrapper that calls exec() exactly once with an optional detector list and then immediately calls finalize(). This is really only useful when working interactively to save a bit of typing. When a Pipeline is calling other operators it will always use exec() and finalize() explicitly.
After calling this, any future calls to exec() may produce unexpected results, since finalize() has already been called.
- Parameters
data (toast.Data) – The distributed data.
detectors (list) – A list of detector names or indices. If None, this indicates a list of all detectors.
- Returns
None or an Operator-dependent result.
- Return type
(value)
- requires()[source]#
Dictionary of Observation keys directly used by this Operator. Including optional keys that will be created by the operator if they do not exist.
This dictionary should have 5 keys, each containing a list of “global”, “metadata”, “detdata”, “shared”, and “intervals” fields. Global keys are contained in the top-level data object. Metadata keys are those contained in the primary observation dictionary. Detdata, shared, and intervals keys are those contained in the “detdata”, “shared”, and “intervals” observation attributes.
- Returns
The keys in the Observation dictionary required by the operator.
- Return type
(dict)
- provides()[source]#
Dictionary of Observation keys created or modified by this Operator.
This dictionary should have 5 keys, each containing a list of “global”, “metadata”, “detdata”, “shared”, and “intervals” fields. Global keys are contained in the top-level data object. Metadata keys are those contained in the primary observation dictionary. Detdata, shared, and intervals keys are those contained in the “detdata”, “shared”, and “intervals” observation attributes.
- Returns
- The keys in the Observation dictionary that will be created
or modified.
- Return type
(dict)
- classmethod get_class_config(input=None)[source]#
Return a dictionary of the default traits of an Operator class.
This returns a new or appended dictionary. The class instance properties are contained in a dictionary found in result[“operators”][cls.name].
If the specified named location in the input config already exists then an exception is raised.
- Parameters
input (dict) – The optional input dictionary to update.
- Returns
The created or updated dictionary.
- Return type
(dict)
- get_config(input=None)[source]#
Return a dictionary of the current traits of an Operator instance.
This returns a new or appended dictionary. The operator instance properties are contained in a dictionary found in result[“operators”][self.name].
If the specified named location in the input config already exists then an exception is raised.
- Parameters
input (dict) – The optional input dictionary to update.
- Returns
The created or updated dictionary.
- Return type
(dict)
For details about specific operators see the relevant sections
(simulation-operators{.interpreted-text role=”ref”},
reduction-operators{.interpreted-text role=”ref”},
utility-operators{.interpreted-text role=”ref”}).
Pipeline Operator#
Although one can run a single operator on the whole dataset before running the next operator, a common processing paradigm is to run a sequence of operations on one detector at a time or on sets of detectors.