Utility Operators#

class toast.ops.MemoryCounter(*, API=0, enabled=True, kernel_implementation=ImplementationType.DEFAULT, name=None, prefix='', silent=False, **kwargs)[source]#

Compute total memory used by Observations in a Data object.

Every process group iterates over their observations and sums the total memory used by detector and shared data. Metadata and interval lists are assumed to be negligible and are not counted.

Note

The following traits can be set at construction or afterwards

Variables

  • API (int) – Internal interface version for this operator (default = 0)

  • enabled (bool) – If True, this class instance is marked as enabled (default = True)

  • kernel_implementation (enum) – Which kernel implementation to use (DEFAULT, COMPILED, NUMPY, JAX). (default = 0)

  • name (str) – The ‘name’ of this class instance (default = None)

  • prefix (str) – Prefix for log messages (default = )

  • silent (bool) – If True, return the memory used but do not log the result (default = False)

API#

Internal interface version for this operator

silent#

If True, return the memory used but do not log the result

prefix#

Prefix for log messages

class toast.ops.Copy(*, API=0, detdata, enabled=True, intervals, kernel_implementation=ImplementationType.DEFAULT, meta, name=None, shared, **kwargs)[source]#

Class to copy data.

This operator takes lists of shared, detdata, and meta keys to copy to a new location in each observation.

Each list contains tuples specifying the input and output key names.

Note

The following traits can be set at construction or afterwards

Variables

  • API (int) – Internal interface version for this operator (default = 0)

  • detdata (list) – List of tuples of Observation detdata keys to copy (default = traitlets.Undefined)

  • enabled (bool) – If True, this class instance is marked as enabled (default = True)

  • intervals (list) – List of tuples of Observation intervals keys to copy (default = traitlets.Undefined)

  • kernel_implementation (enum) – Which kernel implementation to use (DEFAULT, COMPILED, NUMPY, JAX). (default = 0)

  • meta (list) – List of tuples of Observation meta keys to copy (default = traitlets.Undefined)

  • name (str) – The ‘name’ of this class instance (default = None)

  • shared (list) – List of tuples of Observation shared keys to copy (default = traitlets.Undefined)

API#

Internal interface version for this operator

meta#

List of tuples of Observation meta keys to copy

detdata#

List of tuples of Observation detdata keys to copy

shared#

List of tuples of Observation shared keys to copy

intervals#

List of tuples of Observation intervals keys to copy

class toast.ops.Delete(*, API=0, detdata, enabled=True, intervals, kernel_implementation=ImplementationType.DEFAULT, meta, name=None, shared, **kwargs)[source]#

Class to purge data from observations.

This operator takes lists of shared, detdata, intervals and meta keys to delete from observations.

Note

The following traits can be set at construction or afterwards

Variables

  • API (int) – Internal interface version for this operator (default = 0)

  • detdata (list) – List of Observation detdata keys to delete (default = traitlets.Undefined)

  • enabled (bool) – If True, this class instance is marked as enabled (default = True)

  • intervals (list) – List of tuples of Observation intervals keys to delete (default = traitlets.Undefined)

  • kernel_implementation (enum) – Which kernel implementation to use (DEFAULT, COMPILED, NUMPY, JAX). (default = 0)

  • meta (list) – List of Observation dictionary keys to delete (default = traitlets.Undefined)

  • name (str) – The ‘name’ of this class instance (default = None)

  • shared (list) – List of Observation shared keys to delete (default = traitlets.Undefined)

API#

Internal interface version for this operator

meta#

List of Observation dictionary keys to delete

detdata#

List of Observation detdata keys to delete

shared#

List of Observation shared keys to delete

intervals#

List of tuples of Observation intervals keys to delete

class toast.ops.Reset(*, API=0, detdata, enabled=True, intervals, kernel_implementation=ImplementationType.DEFAULT, meta, name=None, shared, **kwargs)[source]#

Class to reset data from observations.

This operator takes lists of shared, detdata, intervals, and meta keys to reset. Numerical data objects and arrays are set to zero. String objects are set to an empty string. Any object that defines a clear() method will have that called. Any object not matching those criteria will be set to None. Since an IntervalList is not mutable, any specified intervals will simply be deleted.

Note

The following traits can be set at construction or afterwards

Variables

  • API (int) – Internal interface version for this operator (default = 0)

  • detdata (list) – List of Observation detdata keys to reset (default = traitlets.Undefined)

  • enabled (bool) – If True, this class instance is marked as enabled (default = True)

  • intervals (list) – List of tuples of Observation intervals keys to reset (default = traitlets.Undefined)

  • kernel_implementation (enum) – Which kernel implementation to use (DEFAULT, COMPILED, NUMPY, JAX). (default = 0)

  • meta (list) – List of Observation dictionary keys to reset (default = traitlets.Undefined)

  • name (str) – The ‘name’ of this class instance (default = None)

  • shared (list) – List of Observation shared keys to reset (default = traitlets.Undefined)

Members

HDF5 I/O#

class toast.ops.SaveHDF5(*, API=0, config, detdata, detdata_float32=False, enabled=True, force_serial=False, intervals, kernel_implementation=ImplementationType.DEFAULT, meta, name=None, shared, times='times', verify=False, volume='toast_out_hdf5', **kwargs)[source]#

Operator which saves observations to HDF5.

This creates a file for each observation.

Note

The following traits can be set at construction or afterwards

Variables

  • API (int) – Internal interface version for this operator (default = 0)

  • config (dict) – Write this job config to the file (default = traitlets.Undefined)

  • detdata (list) – Only save this list of detdata objects (default = traitlets.Undefined)

  • detdata_float32 (bool) – If True, convert any float64 detector data to float32 on write. (default = False)

  • enabled (bool) – If True, this class instance is marked as enabled (default = True)

  • force_serial (bool) – Use serial HDF5 operations, even if parallel support available (default = False)

  • intervals (list) – Only save this list of intervals objects (default = traitlets.Undefined)

  • kernel_implementation (enum) – Which kernel implementation to use (DEFAULT, COMPILED, NUMPY, JAX). (default = 0)

  • meta (list) – Only save this list of meta objects (default = traitlets.Undefined)

  • name (str) – The ‘name’ of this class instance (default = None)

  • shared (list) – Only save this list of shared objects (default = traitlets.Undefined)

  • times (str) – Observation shared key for timestamps (default = times)

  • verify (bool) – If True, immediately load data back in and verify (default = False)

  • volume (str) – Top-level directory for the data volume (default = toast_out_hdf5)

API#

Internal interface version for this operator

volume#

Top-level directory for the data volume

meta#

Only save this list of meta objects

detdata#

Only save this list of detdata objects

shared#

Only save this list of shared objects

intervals#

Only save this list of intervals objects

times#

Observation shared key for timestamps

config#

Write this job config to the file

force_serial#

Use serial HDF5 operations, even if parallel support available

detdata_float32#

If True, convert any float64 detector data to float32 on write.

verify#

If True, immediately load data back in and verify

class toast.ops.LoadHDF5(*, API=0, detdata, enabled=True, files, force_serial=False, intervals, kernel_implementation=ImplementationType.DEFAULT, meta, name=None, pattern='.*h5', process_rows=None, shared, sort_by_size=False, volume=None, **kwargs)[source]#

Operator which loads HDF5 data files into observations.

This operator expects a top-level volume directory.

Note

The following traits can be set at construction or afterwards

Variables

  • API (int) – Internal interface version for this operator (default = 0)

  • detdata (list) – Only load this list of detdata objects (default = traitlets.Undefined)

  • enabled (bool) – If True, this class instance is marked as enabled (default = True)

  • files (list) – Override volume and load a list of files (default = traitlets.Undefined)

  • force_serial (bool) – Use serial HDF5 operations, even if parallel support available (default = False)

  • intervals (list) – Only load this list of intervals objects (default = traitlets.Undefined)

  • kernel_implementation (enum) – Which kernel implementation to use (DEFAULT, COMPILED, NUMPY, JAX). (default = 0)

  • meta (list) – Only load this list of meta objects (default = traitlets.Undefined)

  • name (str) – The ‘name’ of this class instance (default = None)

  • pattern (str) – Regexp pattern to match files against (default = .*h5)

  • process_rows (int) – The size of the rectangular process grid in the detector direction. (default = None)

  • shared (list) – Only load this list of shared objects (default = traitlets.Undefined)

  • sort_by_size (bool) – If True, sort observations by size before load balancing (default = False)

  • volume (str) – Top-level directory containing the data volume (default = None)

API#

Internal interface version for this operator

volume#

Top-level directory containing the data volume

pattern#

Regexp pattern to match files against

files#

Override volume and load a list of files

meta#

Only load this list of meta objects

detdata#

Only load this list of detdata objects

shared#

Only load this list of shared objects

intervals#

Only load this list of intervals objects

sort_by_size#

If True, sort observations by size before load balancing

process_rows#

The size of the rectangular process grid in the detector direction.

force_serial#

Use serial HDF5 operations, even if parallel support available

SPT3G Interfaces#