util
Reasonably helpful functions that can be often used.
- ncas_amof_netcdf_template.util.add_metadata_from_dict(ncfile: Dataset, metadata_dict: dict[str, str | int | float]) None[source]
Take metadata from dictionary and add to global attributes in netCDF file. Can also include latitude and longitude variables if they are single values (e.g. point deployment), using update_variable function.
- Parameters:
ncfile (netCDF Dataset) – Dataset object of netCDF file.
metadata_dict (dict) – Dictionary of attribute name/value pairs.
- ncas_amof_netcdf_template.util.add_metadata_to_netcdf(ncfile: Dataset, metadata_file: Optional[str] = None, file_format: Optional[str] = None) None[source]
Reads metadata from csv file using get_metadata, adds values to global attributes in netCDF file. Numbers in metadata file are converted to integers or floats unless they are strings in the format ‘number’ (e.g. ‘123’). Can also include latitude and longitude variables if they are single values (e.g. point deployment), using update_variable function.
- Parameters:
ncfile (netCDF Dataset) – Dataset object of netCDF file.
metadata_file (str) – file with metadata, one attribute per line.
file_format (str) – format of metadata file. One of ‘CSV’, ‘JSON’, ‘YAML’, or ‘XML’ if given. Default is None, which means the function will attempt to detect file type.
- ncas_amof_netcdf_template.util.change_qc_flags(ncfile: Dataset, ncfile_varname: str, flag_meanings: list[str] = [], flag_values: Optional[list[int]] = None) None[source]
Change the flag meanings and flag values in a quality control variable from the default options. The first two flag meanings must be “not_used” and “good_data”, and all spaces in flag meanings will be replaced with underscores. If given, the first two flag values must be 0 and 1. If not given, flag values will be worked out based on the number of flag meanings and will be sequential values.
- Parameters:
ncfile (netCDF Dataset) – Dataset object of netCDF file.
ncfile_varname (str) – Name of variable in netCDF file.
flag_meanings (list) – List of flag meanings to be used in variable.
flag_values (list) – List of integer flag values to be used in variable. Will be automatically worked out if not provided.
- ncas_amof_netcdf_template.util.check_float(value: Any) bool[source]
Returns True if value can be converted to float, otherwise returns False.
- Parameters:
value (str) – string to test
- Returns:
True if value is a float
- Return type:
bool
- ncas_amof_netcdf_template.util.check_int(value: Any) bool[source]
Returns True if value can be converted to integer, otherwise returns False.
- Parameters:
value (str) – string to test
- Returns:
True if value is an integer
- Return type:
bool
- ncas_amof_netcdf_template.util.check_type_convert(value: Any, dtype: type) bool[source]
Returns True if value can be converted to type dtype, otherwise returns False.
- Parameters:
value (str) – string to test
dtype (type) – type to test
- Returns:
True if value can be of type dtype
- Return type:
bool
- ncas_amof_netcdf_template.util.get_metadata(metafile: str, file_format: Optional[str] = None) dict[str, dict[str, Union[str, type]]][source]
Returns a dict from of metadata from file. Metadata can be in a CSV, JSON, YAML, or XML file. Can also include latitude and longitude variables if they are single values (e.g. point deployment).
- Parameters:
metafile (file) – file with metadata
file_format (str) – format of metadata file. One of ‘CSV’, ‘JSON’, ‘YAML’, or ‘XML’ if given. Default is None, which means the function will attempt to detect file type.
- Returns:
metadata as dictionary
- Return type:
dict
- ncas_amof_netcdf_template.util.get_times(dt_times: list[datetime.datetime]) tuple[list[float], list[float], list[int], list[int], list[int], list[int], list[int], list[float], float, float, str][source]
Returns all time units for AMOF netCDF files from series of datetime objects.
- Parameters:
dt_times (list-like object) – object with datetime objects for times
- Returns:
unix_times, day-of-year, years, months, days, hours, minutes, seconds floats: unix time of first and last times (time_coverage_start and time_coverage_end) str: date in YYYYmmdd format of first time, (file_date)
- Return type:
lists
- ncas_amof_netcdf_template.util.read_csv_metadata(metafile: str) dict[str, dict[str, Union[str, type]]][source]
Returns a dict from a csv with metadata. Can also include latitude and longitude variables if they are single values (e.g. point deployment).
- Parameters:
metafile (file) – csv file with metadata, one attribute per line
- Returns:
metadata from csv as dictionary
- Return type:
dict
- ncas_amof_netcdf_template.util.read_json_metadata(metafile: str) dict[str, dict[str, Union[str, type]]][source]
Returns a dict from a JSON with metadata. Can also include latitude and longitude variables if they are single values (e.g. point deployment).
- Parameters:
metafile (file) – JSON file with metadata
- Returns:
metadata from JSON as dictionary
- Return type:
dict
- ncas_amof_netcdf_template.util.read_xml_metadata(metafile: str) dict[str, dict[str, Union[str, type]]][source]
Returns a dict from a XML with metadata. Can also include latitude and longitude variables if they are single values (e.g. point deployment).
- Parameters:
metafile (file) – XML file with metadata
- Returns:
metadata from XML as dictionary
- Return type:
dict
- ncas_amof_netcdf_template.util.read_yaml_metadata(metafile: str) dict[str, dict[str, Union[str, type]]][source]
Returns a dict from a YAML with metadata. Can also include latitude and longitude variables if they are single values (e.g. point deployment).
- Parameters:
metafile (file) – YAML file with metadata
- Returns:
metadata from YAML as dictionary
- Return type:
dict
- ncas_amof_netcdf_template.util.update_variable(ncfile: Dataset, ncfile_varname: str, data: Union[ndarray[Any, Any], list[Any]], qc_data_error: bool = True) None[source]
- Adds data to variable, and updates valid_min and valid_max
variable attrs if they exist.
- Parameters:
ncfile (netCDF Dataset) – Dataset object of netCDF file.
ncfile_varname (str) – Name of variable in netCDF file.
data (array or list) – Data to be added to netCDF variable.
qc_data_error (bool) – Raise error if trying to add values to QC flag variables that are not in the flag_values attribute. Otherwise, just a warning is printed. Default True.