Loading...
Searching...
No Matches
Appendix F. Specifying Paths for NetCDF-C QuickStart

A key concept in netcdf-c is the notion of a "path". A path specifies some dataset that is of interest to a user. It is the primary argument to the nc_open and nc_create functions in the NetCDF-C API, as defined by the file netcdf.h. It is also the primary option for the NetCDF-C utilities: ncdump, nccopy, and ncgen. Hence understanding what kind of paths are acceptable is important for using the NetCDF-C library.

# Classification of Paths

Basically, there are two kinds of paths:

1. File system paths, and
2. Uniform Resource Locator (URL) paths.

## File System Paths

The most common form of path accepted by the NetCDF-C library is a file system path. Every user of some computer operating system is familiar with the idea of a file system path.

Each operating system has some special quirks when specifying file system paths. Here are some example paths for various version of Linux/Unix.

- /
- /a/b/c/d

As a rule, Linux/Unix has a single root path typically indicated by "/", and "/" is also used as the file separator.

For Windows, some example paths would be as follows:

- c:\
- d:\a\b\c

Windows has a notion of a drive ("d:") and each drive serves as the root of its own file system. Windows uses "\\" as its file separator, although many programs also accept "/". @section qpaths_url Uniform Resource Locator (URL) Paths The NetCDF-C library can access datasets that reside on remote computers, Hence NetCDF-C now also accepts URLs to specify those remote datasets. The general form of a URL is as follows: @icode <protocol>://<user>:<pwd>@<host>:<port>/<path>?<query>#<fragment> @endicode * \<protocol\> - specifies the format of the messages between the local and remote computers. Some examples used by NetCDF-C are <em>http</em>, <em>https</em>, <em>s3</em>, <em>dap</em>, <em>dap4</em>, or <em>file</em>. * \<user:pwd\> - Pass authorization credentials to the remote computer. * \<host\>:\<port\> - The specific remote computer to access, such as <em>thredds.ucar.edu:8080</em>. * \<path\> - A specification of file-path-like string that indicates some specific resource on the remote computer. * \<query\> - A sequence of (key=value) pairs, separated by "&", and providing special information to the remote computer. * \<fragment\> - A sequence of (key=value) pairs, separated by "&", and providing special information to the local computer. Most parts of a URL are optional. As a rule, the protocol is always required, and either the host is required or, if the protocol is "file", then the path is required. The query and fragment are optional. @subsection autotoc_md418 Examples of URL Paths for NetCDF-C * https://thredds.ucar.edu/catalog @section nc_paths_s3_urls Addendum A. Amazon S3 Specific URLS A URL path is required for accessing datasets on the Amazon S3 storage cloud. Unfortunately S3 URLs are complicated. It has the following features: * Protocol: <em>https</em> or <em>s3</em>. The <em>s3</em> scheme is equivalent to "https" plus setting various tags in the query and/or fragment part of the URL. * Host: Amazon S3 defines three forms: <em>Virtual</em>, <em>Path</em>, and <em>S3</em> + <em>Virtual</em>: the host includes the bucket name as in <strong>bucket.s3.&lt;region&gt;.amazonaws.com</strong> or <strong>bucket.s3.amazonaws.com</strong> + <em>Path</em>: the host does not include the bucket name, but rather the bucket name is the first segment of the path. For example <strong>s3.&lt;region&gt;.amazonaws.com/bucket</strong> or <strong>s3.amazonaws.com/bucket</strong> + <em>S3</em>: the protocol is "s3:" and if the host is a single name, then it is interpreted as the bucket. The region is determined using an algorithm defined in the nczarr documentation. + <em>Other</em>: It is possible to use other non-Amazon cloud storage, but that is cloud library dependent. * Query: currently not used. * Fragment: the fragment is of the form <em>key=value&key=value&...</em>. Depending on the key, the <em>value</em> part may be left out and some default value will be used. The exact set of possible keys is defined in the nczarr documentation. @section nc_paths_frag_keys Addendum B. Known Fragment Keys The fragment part of a URL is used to pass information deep into the netcdf-c library to control its actions. This appendix list known keys, although it may be somewhat out-of-date. The current set of keys used in the netcdf-c library is as follows. * <em>mode</em> &ndash; A special key that is used to provide single values for controlling the netcdf-c library. It consists of a comma separated sequence of values primarily used to control the file format. The <em>mode</em> key supports the following values - <em>dap2</em> &ndash; Specifies that the URL accesses a resource using the DAP2 protocol - <em>dap4</em> &ndash; Specifies that the URL accesses a resource using the DAP4 protocol - <em>netcdf-3</em> &ndash; Specifies that a file is a netcdf-classic file - <em>classic</em> &ndash; Alias for <em>netcdf-3</em> - <em>netcdf-4</em> &ndash; Specifies that a file is a netcdf-enhanced file - <em>enhanced</em> &ndash; Alias for <em>netcdf-4</em> - <em>udf0</em> &ndash; Specifies that the file format is defined by a User Defined format - <em>udf1</em> &ndash; Specifies that the file format is defined by a User Defined format - <em>nczarr</em> &ndash; Specifies that the file is in NCZarr format - <em>zarr</em> &ndash; Specifies that the file is in Zarr format - <em>xarray</em> &ndash;Specifies that the file is in Zarr format and uses the XARRAY convention - <em>noxarray</em> &ndash;Specifies that the file is in Zarr format and does not use the XARRAY convention - <em>s3</em> &ndash;Specifies that the file is remote and is stored on the Amazon S3 cloud - <em>file</em> &ndash;Specifies that the file is an NCZarr/Zarr file stored as a file tree - <em>zip</em> &ndash;Specifies that the file is an NCZarr/Zarr file stored as a zip file - <em>bytes</em> &ndash; Specifies that the file is remote and is to be read using byte-range support in NCZarr format * <em>dap2</em> &ndash; equivalent to "mode=dap2" * <em>dap4</em> &ndash; equivalent to "mode=dap4" * <em>bytes</em> &ndash; equivalent to "mode=bytes"

- log – turn on logging for the duration of the data request
- show=fetch – log curl fetch commands