hydro.exchange

Submodules

Attributes

CCHDO_VERSION

log

SENTINEL_PREFIX

SENTINEL_PARAM

FLAG_SCHEME

WHPNameIndex

WHPParamUnit

ExchangeIO

Classes

_ExchangeData

Dataclass containing exchange data which has been parsed into ndarrays

_ExchangeInfo

Low level dataclass containing the parts of an exchange file

CheckOptions

Flags and config that controll how strict the file checks are

Functions

_has_no_nones(val)

_transform_whp_to_csv(params, units)

_get_params(params_units[, ignore_params])

_ctd_get_header(line[, dtype])

finalize_ancillary_variables(dataset)

Turn the ancillary variable attr into a space seperated string

combine_bottle_time(dataset)

Combine the bottle dates and times if present

check_is_subset_shape(a1, a2[, strict])

Ensure that the shape of the data in a2 is a subset (or strict subset) of the data shape of a1

_get_fill_locs(arr[, fill_values])

_is_valid_exchange_numeric(data)

_load_raw_exchange(filename_or_obj, *[, ...])

read_csv(filename_or_obj, *[, fill_values, ftype, ...])

read_exchange(filename_or_obj, *[, fill_values, ...])

Loads the data from filename_or_obj and returns a xr.Dataset with the CCHDO

_from_exchange_data(exchange_data, *[, ftype, checks])

Package Contents

hydro.exchange.CCHDO_VERSION = ''[source]
hydro.exchange.log[source]
hydro.exchange.SENTINEL_PREFIX = '__SENTINEL'[source]
hydro.exchange.SENTINEL_PARAM[source]
hydro.exchange.FLAG_SCHEME: dict[str, type[cchdo.hydro.flags.ExchangeFlag]][source]
hydro.exchange.WHPNameIndex[source]
hydro.exchange.WHPParamUnit[source]
hydro.exchange._has_no_nones(val)[source]
Parameters:

val (list[str | None])

Return type:

TypeGuard[list[str]]

hydro.exchange._transform_whp_to_csv(params, units)[source]
Parameters:
Return type:

list[str]

hydro.exchange._get_params(params_units, ignore_params=None)[source]
Parameters:
Return type:

tuple[WHPNameIndex, WHPNameIndex, WHPNameIndex]

hydro.exchange._ctd_get_header(line, dtype=str)[source]
hydro.exchange.finalize_ancillary_variables(dataset)[source]

Turn the ancillary variable attr into a space seperated string

It is nice to have the ancillary variable be a list while things are being read into it

Parameters:

dataset (xarray.Dataset)

hydro.exchange.combine_bottle_time(dataset)[source]

Combine the bottle dates and times if present

Raises if only one is present

Parameters:

dataset (xarray.Dataset)

hydro.exchange.check_is_subset_shape(a1, a2, strict='disallowed')[source]

Ensure that the shape of the data in a2 is a subset (or strict subset) of the data shape of a1

For a given set of param, flag, and error arrays you would want to ensure that:

  • errors are a subset of params (strict is allowed)

  • params are a subset of flags (strict is allowed)

For string vars, the empty string is considered the “nothing” value. For woce flags, flag 9s should be converted to nans (depending on scheme flag 5 and 1 may not have param values)

Return a boolean array of invalid locations

Parameters:

  • a1 (numpy.typing.NDArray)

  • a2 (numpy.typing.NDArray)

Return type:

numpy.typing.NDArray[numpy.bool_]

class hydro.exchange._ExchangeData[source]

Dataclass containing exchange data which has been parsed into ndarrays

single_profile: bool[source]
param_cols: dict[cchdo.params.WHPName, numpy.ndarray][source]
flag_cols: dict[cchdo.params.WHPName, numpy.ndarray][source]
error_cols: dict[cchdo.params.WHPName, numpy.ndarray][source]
param_precisions: dict[cchdo.params.WHPName, numpy.typing.NDArray[numpy.int_]][source]
error_precisions: dict[cchdo.params.WHPName, numpy.typing.NDArray[numpy.int_]][source]
comments: str[source]
__post_init__()[source]
set_expected(params, flags, errors)[source]

Puts fill columns for expected params which are missing

This can occur when there are disjoint columns in CTD files

Parameters:

  • params (set[cchdo.params.WHPName])

  • flags (set[cchdo.params.WHPName])

  • errors (set[cchdo.params.WHPName])

split_profiles()[source]

Split into single profile containing _ExchangeData instances

Done by looking at the expocode+station+cast composate keys

property str_lens: dict[cchdo.params.WHPName, int][source]

Figure out the length of all the string params

The char size can vary by platform.

Return type:

dict[cchdo.params.WHPName, int]

hydro.exchange._get_fill_locs(arr, fill_values=('-999',))[source]
Parameters:

fill_values (tuple[str, Ellipsis])

class hydro.exchange._ExchangeInfo[source]

Low level dataclass containing the parts of an exchange file

stamp_slice: slice[source]
comments_slice: slice[source]
ctd_headers_slice: slice[source]
params_idx: int[source]
units_idx: int[source]
data_slice: slice[source]
post_data_slice: slice[source]
_raw_lines: tuple[str, Ellipsis][source]
_ctd_override: bool = False[source]
_ignore_columns: collections.abc.Iterable[str] = ()[source]
property stamp[source]

Returns the filestamp of the exchange file

e.g. “BOTTLE,20210301CCHSIOAMB”

property comments[source]

Returns the comments of the exchange file with leading # stripped

property ctd_headers[source]

Returns a dict of the CTD headers and their value

property params[source]

Returns a list of all parameters in the file (including CTD “headers”)

property units[source]

Returns a list of all the units in the file (including CTD “headers”)

Will have the same shape as params

property data[source]

Returns the data block of an exchange file as a tuple of strs. One line per entry.

property post_data[source]

Returns any post data content as a tuple of strs

property _whp_param_info[source]

Parses the params and units for base parameters

Returns a dict with a WHPName to column index mapping

property whp_params[source]
property whp_flags[source]

Parses the params and units for flag values

returns a dict with a WHPName to column index of flags mapping

property whp_errors[source]

Parses the params and units for uncertanty values

returns a dict with a WHPName to column index of errors mapping

property _np_data_block[source]
finalize(fill_values=('-999',), precision_source=PrecisionSource.FILE)[source]

Parse all the data into ndarrays of the correct dtype and shape

Returns an ExchangeData dataclass

Parameters:

precision_source (cchdo.hydro.types.PrecisionSourceType)

Return type:

_ExchangeData

classmethod from_lines(lines, ftype, ignore_columns)[source]

Figure out the line numbers/indicies of the parts of the exchange file

Parameters:

  • lines (tuple[str, Ellipsis])

  • ftype (cchdo.hydro.types.FileTypeType)

hydro.exchange._is_valid_exchange_numeric(data)[source]
Parameters:

data (numpy.typing.NDArray[numpy.str_])

Return type:

numpy.bool_

hydro.exchange.ExchangeIO[source]
hydro.exchange._load_raw_exchange(filename_or_obj, *, file_seperator=None, keep_seperator=True, encoding='utf8')[source]
Parameters:

  • filename_or_obj (ExchangeIO)

  • file_seperator (str | None)

Return type:

list[str]

class hydro.exchange.CheckOptions[source]

Bases: TypedDict

Flags and config that controll how strict the file checks are

flags: bool[source]
hydro.exchange.read_csv(filename_or_obj, *, fill_values=('-999',), ftype=FileType.BOTTLE, checks=None, precision_source=PrecisionSource.FILE, encoding='utf8', ignore_columns=None)[source]
Parameters:

  • filename_or_obj (ExchangeIO)

  • ftype (cchdo.hydro.types.FileTypeType)

  • checks (CheckOptions | None)

  • precision_source (cchdo.hydro.types.PrecisionSourceType)

  • ignore_columns (collections.abc.Iterable[str] | None)

Return type:

xarray.Dataset

hydro.exchange.read_exchange(filename_or_obj, *, fill_values=('-999',), checks=None, precision_source=PrecisionSource.FILE, file_seperator=None, keep_seperator=True, encoding='utf8', ignore_columns=None)[source]

Loads the data from filename_or_obj and returns a xr.Dataset with the CCHDO CF/netCDF structure

Parameters:
Return type:

xarray.Dataset

hydro.exchange._from_exchange_data(exchange_data, *, ftype=FileType.BOTTLE, checks=None)[source]
Parameters:
Return type:

xarray.Dataset