create_netcdf

Create netCDF files for NCAS AMOF instruments that adhere to the NCAS-AMF-2.0.0 standard.

ncas_amof_netcdf_template.create_netcdf.add_attributes(ncfile: Dataset, instrument_dict: Optional[dict[str, dict[str, Union[str, list[str], dict[str, dict[str, Union[str, float]]]]]]] = None, product: Optional[str] = None, created_time: Optional[str] = None, location: Optional[str] = None, loc: str = 'land', use_local_files: Optional[str] = None, tag: str = 'latest', instrument_file_info: Optional[FileInfo] = None) → None[source]

Adds all global attributes for a given product to the netCDF file.

Parameters:

  • ncfile (obj) – netCDF file object

  • instrument_dict (dict) – -DEPRECATED- information about the instrument from tsv2dict.isntrument_dict. Use instrument_file_info instead. Will be removed in version 2.7.0.

  • product (str) – -DEPRECATED- name of data product. Value will be obtained from instrument_file_info. Option will be removed in version 2.7.0.

  • created_time (str or None) – time of file creation. If ‘None’, now will be used.

  • location (str or None) – -DEPRECATED- value for the ‘platform’ global attribute. Value will be obtained from instrument_file_info. Option will be removed in version 2.7.0.

  • loc (str) – -DEPRECATED- value for the ‘deployment_mode’ global attribute, should be one of ‘land’, ‘sea’, ‘air’, or ‘trajectory’. Value will be obtained from instrument_file_info. Option will be removed in version 2.7.0.

  • instrument_file_info (FileInfo) – information about instrument for making netCDF file, from ncas_amof_netcdf_template.file_info.FileInfo.

  • tag (str) – -DEPRECATED- tagged release version of AMF_CVs, or “latest” to get most recent release. Ignored if use_local_files is not None. Value will be obtained from instrument_file_info. Option will be removed in version 2.7.0.

  • use_local_files (str or None) – path to local directory where tsv files are stored. If “None”, read from online. Default None.

ncas_amof_netcdf_template.create_netcdf.add_dimensions(ncfile: Dataset, instrument_dict: Optional[dict[str, dict[str, Union[str, list[str], dict[str, dict[str, Union[str, float]]]]]]] = None, product: Optional[str] = None, dimension_lengths: Optional[dict[str, int]] = None, instrument_file_info: Optional[FileInfo] = None) → None[source]

Adds all dimensions for a given product to the netCDF file.

Parameters:

  • ncfile (obj) – netCDF file object

  • instrument_dict (dict) – -DEPRECATED- information about the instrument from tsv2dict.isntrument_dict. Use instrument_file_info instead. Will be removed in version 2.7.0.

  • product (str) – -DEPRECATED- name of data product. Value will now be obtained from instrument_file_info. Option will be removed from version 2.7.0.

  • dimension_lengths (dict) – -DEPRECATED- length of each dimension. Values will now be obtained from instrument_file_info. Option will be removed from version 2.7.0.

  • instrument_file_info (FileInfo) – information about instrument for making netCDF file, from ncas_amof_netcdf_template.file_info.FileInfo.

ncas_amof_netcdf_template.create_netcdf.add_variables(ncfile: Dataset, instrument_dict: Optional[dict[str, dict[str, Union[str, list[str], dict[str, dict[str, Union[str, float]]]]]]] = None, product: Optional[str] = None, instrument_file_info: Optional[FileInfo] = None, verbose: int = 0) → None[source]

Adds all variables and their attributes for a given product to the netCDF file.

Parameters:

  • ncfile (obj) – netCDF file object

  • instrument_dict (dict) – -DEPRECATED- information about the instrument from tsv2dict.isntrument_dict. Use instrument_file_info instead. Will be removed in version 2.7.0.

  • product (str) – -DEPRECATED- name of data product. Value will be obtained from instrument_file_info. Option will be removed in version 2.7.0.

  • instrument_file_info (FileInfo) – information about instrument for making netCDF file, from ncas_amof_netcdf_template.file_info.FileInfo.

  • verbose (int) – level of additional info to print. At the moment, there is only 1 additional level. Default 0.

ncas_amof_netcdf_template.create_netcdf.list_products(instrument: str = 'all', use_local_files: Optional[str] = None, tag: str = 'latest') → list[str][source]

Lists available products, either for a specific instrument or all data products.

Parameters:

  • instrument (str) – ncas instrument name, or “all” for all data products. Default “all”.

  • use_local_files (str or None) – path to local directory where tsv files are stored. If “None”, read from online. Default None.

  • tag (str) – tagged release of definitions, or ‘latest’ to get most recent release. Ignored if use_local_files is not None. Default “latest”.

Returns:

list of products available for the given instrument

ncas_amof_netcdf_template.create_netcdf.main(instrument: str, date: Optional[str] = None, dimension_lengths: dict[str, int] = {}, platform: Optional[str] = None, loc: str = 'land', products: Optional[Union[str, list[str]]] = None, verbose: int = 0, options: str = '', product_version: str = '1.0', file_location: str = '.', use_local_files: Optional[str] = None, tag: str = 'latest', chunk_by_dimension: Optional[dict[str, int]] = None, compression: Optional[Union[str, dict[str, str]]] = None, complevel: Union[int, dict[str, int]] = 4, shuffle: Union[bool, dict[str, bool]] = True) → Union[Dataset, list[netCDF4._netCDF4.Dataset]][source]

Create ‘just-add-data’ AMOF-compliant netCDF file

Parameters:

  • instrument (str) – ncas instrument name

  • date (str) – date for file, format YYYYmmdd. If not given, finds today’s date

  • dimension_lengths (dict) – dictionary of dimension:length. If length not given for needed dimension, user will be asked to type in dimension length

  • platform (str) – observatory or location of the instrument. If not given or is None, will use default platform for instrument from instrument vocabularies. Default None.

  • loc (str) – one of ‘land’, ‘sea’, ‘air’, ‘trajectory’

  • products (str or list) – string of one product or list of multiple products to make netCDF file for this instrument. If None, then all available products for the defined instrument are made. -DEPRECATION WARNING- option to specify either a list of ‘None’ is being deprecated and will be removed in version 2.7.0. Use single data product.

  • verbose (int) – level of info to print out. Note that at the moment there is only one additional layer, this may increase in future.

  • options (str) – options to be included in file name. All options should be in one string and separated by an underscore (‘_’), with up to three options permitted. Default ‘’.

  • product_version (str) – version of the data file. Default ‘1.0’.

  • file_location (str) – where to write the netCDF file. Default ‘.’.

  • use_local_files (str or None) – path to local directory where tsv files are stored. If “None”, read from online. Default None.

  • tag (str) – tagged release of definitions, or ‘latest’ to get most recent release. Ignored if use_local_files is not None. Default “latest”.

  • chunk_by_dimension (dict) – chunk sizes to use in each dimension Default None (no chunking).

  • compression (str or dict) – compression algorithm to be used to store data. If string, then compression option passed to all variables. If dictionary, should be variable:compression pairs, and compression option passed to just the specified variables. Should be one of the options available in

  • complevel (int or dict) – level of compression to be used, between 0 (no compression) and 9 (most compression). Either integer value or dictionary with variable:integer pairs. Default is 4. Ignored if no compression used.

  • shuffle (bool or dict) –

    whether to use the HDF5 shuffle filter before compressing with zlib, significantly improving compression. Default is True. Ignored if compression is not zlib.

    the netCDF4 python module. Default is None (no compression).

Returns:

netCDF file object or nothing

ncas_amof_netcdf_template.create_netcdf.make_netcdf(instrument: Optional[str] = None, product: Optional[str] = None, time: str = '20260506', instrument_dict: Optional[dict[str, dict[str, Union[str, list[str], dict[str, dict[str, Union[str, float]]]]]]] = None, loc: str = 'land', dimension_lengths: dict[str, int] = {}, verbose: int = 0, options: str = '', product_version: str = '1.0', file_location: str = '.', use_local_files: Optional[str] = None, tag: str = 'latest', chunk_by_dimension: Optional[dict[str, int]] = None, compression: Optional[Union[str, dict[str, str]]] = None, complevel: Union[int, dict[str, int]] = 4, shuffle: Union[bool, dict[str, bool]] = True, instrument_file_info: Optional[FileInfo] = None) → Dataset[source]

Makes netCDF file for given instrument and arguments.

Parameters:

  • instrument (str or None) – -DEPRECATED- ncas instrument name. Value will be retrived from instrument_file_info. Option will be removed in version 2.7.0. Default None.

  • product (str or None) – -DEPRECATED- name of data product. Value will be retrieved from instrument_file_info. Option will be removed in version 2.7.0. Default None.

  • time (str) – time that the data represents, in YYYYmmdd-HHMMSS format or as much of as required. Default is now in YYYYmmdd format.

  • instrument_dict (dict or None) – -DEPRECATED- information about the instrument from tsv2dict.instrument_dict. Use instrument_file_info argument instead. Will be removed in version 2.7.0.

  • instrument_file_info (FileInfo or None) – information about the instrument, from file_info.FileInfo.

  • loc (str) – -DEPRECATED- location of instrument, one of ‘land’, ‘sea’, ‘air’ or ‘trajectory’. Value will be retrieved from instrument_file_info. Option will be removed in version 2.7.0. Default ‘land’.

  • dimension_lengths (dict) – -DEPRECATED- lengths of dimensions in file. If not given, python will prompt the user to enter lengths for each dimension. Values will be retrieved from instrument_file_info. Option will be removed in version 2.7.0. Default {}.

  • verbose (int) – level of additional info to print. At the moment, there is only 1 additional level. Default 0.

  • options (str) – options to be included in file name. All options should be in one string and separated by an underscore (‘_’), with up to three options permitted. Default ‘’.

  • product_version (str) – version of the data file. Default ‘1.0’.

  • file_location (str) – where to write the netCDF file. Default ‘.’.

  • use_local_files (str or None) – path to local directory where tsv files are stored. If “None”, read from online. Default None.

  • tag (str) – -DEPRECATED- tagged release version of AMF_CVs, or ‘latest’ to get most recent release. Ignored if use_local_files is not None. Value will be retrieved from instrument_file_info. Option will be removed in verison 2.7.0. Default “latest”.

  • chunk_by_dimension (dict) – chunk sizes to use in each dimension. Default None (no chunking).

  • compression (str or dict) – compression algorithm to be used to store data. If string, then compression option passed to all variables. If dictionary, should be variable:compression pairs, and compression option passed to just the specified variables. Should be one of the options available in the netCDF4 python module. Default is None (no compression).

  • complevel (int or dict) – level of compression to be used, between 0 (no compression) and 9 (most compression). Either integer value or dictionary with variable:integer pairs. Default is 4. Ignored if no compression used.

  • shuffle (bool or dict) – whether to use the HDF5 shuffle filter before compressing with zlib, significantly improving compression. Default is True. Ignored if compression is not zlib.

Returns:

netCDF file object or nothing.

ncas_amof_netcdf_template.create_netcdf.make_product_netcdf(product: str, instrument_name: str, date: Optional[str] = None, dimension_lengths: dict[str, int] = {}, platform: str = '', deployment_loc: str = 'land', verbose: int = 0, options: str = '', product_version: str = '1.0', file_location: str = '.', use_local_files: Optional[str] = None, tag: str = 'latest', chunk_by_dimension: Optional[dict[str, int]] = None, compression: Optional[Union[str, dict[str, str]]] = None, complevel: Union[int, dict[str, int]] = 4, shuffle: Union[bool, dict[str, bool]] = True) → Dataset[source]

Create an AMOF-like netCDF file for a given data product. This means files can be made to the NCAS-GENERAL standard for instruments that aren’t part of the AMOF instrument suite.

Parameters:

  • product (str) – name of data product

  • instrument_name (str) – instrument name for use in file name

  • date (str) – date for file, format YYYYmmdd. If not given, finds today’s date

  • dimension_lengths (dict) – dictionary of dimension:length. If length not given for needed dimension, user will be asked to type in dimension length

  • platform (str) – observatory or location of the instrument. Default “”.

  • deployment_loc (str) – one of ‘land’, ‘sea’, ‘air’, ‘trajectory’. Default “land”.

  • verbose (int) – level of info to print out. Note that at the moment there is only one additional layer, this may increase in future.

  • options (str) – options to be included in file name. All options should be in one string and separated by an underscore (“_”), with up to three options permitted. Default “”.

  • product_version (str) – version of the data file. Default “1.0”.

  • file_location (str) – where to write the netCDF file. Default “.”.

  • use_local_files (str or None) – path to local directory where tsv files are stored. If “None”, read from online. Default None.

  • tag (str) – tagged release of definitions, or ‘latest’ to get most recent release. Ignored if use_local_files is not None. Default “latest”.

  • chunk_by_dimension (dict) – chunk sizes to use in each dimension Default None (no chunking).

  • compression (str or dict) – compression algorithm to be used to store data. If string, then compression option passed to all variables. If dictionary, should be variable:compression pairs, and compression option passed to just the specified variables. Should be one of the options available in the netCDF4 python module. Default is None (no compression).

  • complevel (int or dict) – level of compression to be used, between 0 (no compression) and 9 (most compression). Either integer value or dictionary with variable:integer pairs. Default is 4. Ignored if no compression used.

  • shuffle (bool or dict) – whether to use the HDF5 shuffle filter before compressing with zlib, significantly improving compression. Default is True. Ignored if compression is not zlib.

Returns:

netCDF file object or nothing.