$Id: libtcd.html 1803 2007-01-22 15:29:56Z flaterco $

# Tide Constituent Database

Comments on this documentation should be returned to David Flater.

Contents

1. Introduction
2. Compiling libtcd
3. Tide record (version 2.2)
4. Public database header (version 2.2)
5. libtcd 2.2 API
6. V1 to V2 migration guide
7. Changelog
8. References

## 1. Introduction

libtcd provides a software API for reading and writing Tide Constituent Database (TCD) files.

The TCD file format and schema are used by XTide to retrieve constituent definitions (speeds, equilibrium arguments, and node factors), harmonic constants, subordinate station offsets and associated metadata for use in generating tide predictions.

The TCD file format and schema were originally designed by Jan Depner to improve the performance of XTide and to meet additional requirements of the U.S. Naval Oceanographic Office (NAVO). They are now maintained primarily by David Flater.

The design goals for TCD included:

- Available without installing heavy external packages.
- Efficient for indexing and loading of stations.
- Compact file format.
- Translatable without loss from legacy XTide .txt and .xml files.
- Exportable to legacy .txt and .xml formats.
- Modifiable by end user using tideEditor.
- Able to detect tampering (usually, misguided attempts to edit the database in a word processor).

The TCD file format and schema and libtcd are in the public domain. They are distributed in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

For additional background, see Jan Depner, "Format for the Oceanographic and Atmospheric Master Library (OAML) Tide Constituent Database," rev. 2003-06-06.

## 2. Compiling and installing libtcd

The normal route:

>     bash-3.1$ ./configure
>     bash-3.1$ make
>     bash-3.1$ su
>     bash-3.1# make install

libtcd is packaged with the popular and portable GNU automake, so all usual GNU tricks should work. Help on configuration options can be found in the INSTALL file or obtained by entering `./configure --help`.

## 3. Tide record (version 2.2)

This section documents the tide record schema of the Tide Constituent Database as it stands in libtcd version 2.2.

Fields designated as "header" fields are retrieved at indexing time and stored in memory for quick access. The rest are retrieved only when individual tide stations are loaded.

Type 1 records are for reference stations; type 2 records are for subordinate stations. Some fields are only encoded for one record type while others are encoded for both.

intX means signed X-bit integer. uintX means unsigned X-bit integer. TCD files are encoded with a bit-packing function that allows odd-sized fields to be stored efficiently. int varying and uint varying indicate fields whose lengths in bits are dynamically configured for each TCD file.

Floating-point numbers are encoded in the TCD file using scaled integers. An integer with a scale of X represents the real number obtained by dividing the integer by X.

Please note: All character strings stored in a TCD file use the character set ISO-8859-1.

Record type | TIDE_RECORD field | TIDE_RECORD data type | File encoding | Encodable range | Semantics
--- | --- | --- | --- | --- | ---
all | `header.record_number` | int32 | none | N/A | Records in TCD file are implicitly numbered [0, N-1].
all | `header.record_size` | uint32 | uint16 | [0, 65535] | Length of encoded record in bytes.
all | `header.record_type` | uint8 | uint4 | [0, 15] | See `enum TIDE_RECORD_TYPE` in Section 5. Type 1 is reference station. Type 2 is subordinate station. Others are as yet undefined.
all | `header.latitude` | float64 | int25, scale 100000 | [-167.77216, 167.77215] | Latitude in degrees north; use [-90.00000, 90.00000]. Lat,lng of 0,0 is interpreted as NULL.
all | `header.longitude` | float64 | int26, scale 100000 | [-335.54432, 335.54431] | Longitude in degrees east; use [-180.00000, 180.00000]. Lat,lng of 0,0 is interpreted as NULL.
all | `header.reference_station` | int32 | int18 | [-131072, 131071] | Index, references record_number of reference station for type 2 records, or -1 for none.

all

`header.tzfile`

int16

uint10

[0, 1023]

Index, references table of time zone values like ":America/New_York" that control which time zone output should be rendered into. These time zone values are defined in the zoneinfo time zone database that is included with most flavors of Unix. Not to be confused with `zone_offset`, which actually affects the calibration of the results relative to real time. See comments for `zone_offset` below.

all | `header.name` | char[90] | 0-terminated string | Length constrained by TIDE_RECORD. | Station name.
all | `country` | int16 | uint9 | [0, 511] | Index, references table of country names.
all | `source` | char[90] | 0-terminated string | Length constrained by TIDE_RECORD. | Where you got the data, or "" for NULL.
all | `restriction` | uint8 | uint4 | [0, 15] | Index, references table of restrictions like "Public domain," "Non-commercial use only," "DoD/DoD contractors only."
all | `comments` | char[10000] | 0-terminated string | Length constrained by TIDE_RECORD. | Human-readable text that is not provided to the user of a data set at the time of use but may be retrieved on demand. Use "" for NULL.
all | `notes` | char[10000] | 0-terminated string | Length constrained by TIDE_RECORD. | Human-readable text that MAY be delivered to the user of a data set at the time of use. XTide will print the notes in modes where this is convenient but ignore them when it is not.
all | `legalese` | uint8 | uint4 | [0, 15] | Index, references table of human-readable legal notices that MUST be delivered to the user of a data set at the time of use. Use 0 for NULL.
all | `station_id_context` | char[90] | 0-terminated string | Length constrained by TIDE_RECORD. | Name of the authority that has defined the Station ID, or "" for NULL.
all | `station_id` | char[90] | 0-terminated string | Length constrained by TIDE_RECORD. | An identifier for a tide station, defined by the authority specified in station_id_context, or "" for NULL.
all | `date_imported` | uint32 | uint27 | [0, 134217727] | The date on which the data set was imported into the database, encoded Year * 10000 + Month [1, 12] * 100 + Day [1, 31], or 0 for NULL. N.B., this is not the date of creation of the TCD file, but the date on record in a tide data management package such as Harmbase.

all

`xfields`

char[10000]

0-terminated string

Length constrained by TIDE_RECORD.

Space for backward-compatible addition of expansion fields as text. Encoding:

    xfields: xfield*
    xfield: field-name ":" field-value "\n"
    field-name: [^:\n ]+
    field-value: [^\n]* continuation*
    continuation: "\n " field-value

When decoding field-values, the string "\n " is replaced by "\n" to unmangle multi-line values. (Yes, multi-line values remain multi-line, unlike RFC 822 headers.)

all | `direction_units` | uint8 | uint varying | [0, 255] | Index, references table of units like "degrees true." This field is used to indicate the units of the direction fields.
all | `min_direction` | int32 | uint9 | [0, 511] | Direction of ebb current. Use [0, 359], or 361 for NULL.
all | `max_direction` | int32 | uint9 | [0, 511] | Direction of flood current. Use [0, 359], or 361 for NULL.
all | `level_units` | uint8 | uint varying | [0, 255] | Index, references table of units like "feet," "meters," "knots," "knots^2." This field indicates the units of the datum and the amplitudes in type 1 records and the units of the level_add fields in type 2 records.*
1 | `datum_offset` | float32 | int28, scale 10000 | [-13421.7728, 13421.7727] | The datum (Z0).
1 | `datum` | int16 | uint7 | [0, 127] | Index, references table of datum kinds like "Mean Lower Low Water." For currents, set to 0 (value should be ignored).

1

`zone_offset`

int32

int13

[-4096, 4095]

The standard time to which epochs are adjusted, a.k.a. the meridian, in hours and minutes east of UTC, encoded Hours * 100 + Minutes. `zone_offset` affects the calibration of the predictions relative to real time. Not to be confused with `tzfile`, which only affects which time zone the results are rendered into. For example, adjusting `zone_offset` will change high tide from 5:00 EST to 4:00 EST, while adjusting `tzfile` will change it from 5:00 EST to 4:00 CST (actually the same "real time" in two different time zones). Technically, `zone_offset` could be done away with by specifying that all data should be calibrated with UTC; however, this would make it more difficult to identify data sets with source data by visual inspection (all of the epochs would be different).

1 | `expiration_date` | uint32 | uint27 | [0, 134217727] | Use-until date for data set, encoded Year * 10000 + Month [1, 12] * 100 + Day [1, 31], or 0 for NULL.
1 | `months_on_station` | uint16 | uint10 | [0, 1023] | Number of months in time series used to derive harmonic constants, or 0 for NULL.
1 | `last_date_on_station` | uint32 | uint27 | [0, 134217727] | Last date in time series used to derive harmonic constants, encoded Year * 10000 + Month [1, 12] * 100 + Day [1, 31], or 0 for NULL.
1 | `confidence` | uint8 | uint4 | [0, 15] | A meaningless indicator of data quality, generally initialized to 10. (Jan wants to keep this.)
1 | `amplitude` | float32[255] | uint19, scale 10000 (each) | [0.0000, 52.4287] | Amplitudes of constituents. Constituents with amplitude less than `AMPLITUDE_EPSILON` are not encoded.
1 | `epoch` | float32[255] | uint16, scale 100 (each) | [0.00, 655.35] | Epochs (phases) of constituents, in degrees. Use [0.00, 359.99]. Constituents with amplitude less than `AMPLITUDE_EPSILON` are not encoded.
2 | `min_time_add` | int32 | int13 | [-4096, 4095] | The time adjustment for Low Tide or Max Ebb events, encoded Hours * 100 + Minutes. 0 and NULL are equivalent (no adjustment).** ***
2 | `min_level_add` | float32 | int17, scale 1000 | [-65.536, 65.535] | Additive adjustment for Low Tide or Max Ebb events.*** 0 and NULL are equivalent (no adjustment).
2 | `min_level_multiply` | float32 | uint16, scale 1000 | [0.000, 65.535] | Ratio for Low Tide or Max Ebb events, or 0 for NULL (equivalent to 1, no adjustment).***
2 | `max_time_add` | int32 | int13 | [-4096, 4095] | The time adjustment for High Tide or Max Flood events, encoded Hours * 100 + Minutes. 0 and NULL are equivalent (no adjustment).** ***
2 | `max_level_add` | float32 | int17, scale 1000 | [-65.536, 65.535] | Additive adjustment for High Tide or Max Flood events.*** 0 and NULL are equivalent (no adjustment).
2 | `max_level_multiply` | float32 | uint16, scale 1000 | [0.000, 65.535] | Ratio for High Tide or Max Flood events, or 0 for NULL (equivalent to 1, no adjustment).***
2 | `flood_begins` | int32 | int13 | [-4096, 4095] | The time adjustment for Slack Water or Min Flood before Max Flood, encoded Hours * 100 + Minutes, or 2560 for NULL.**
2 | `ebb_begins` | int32 | int13 | [-4096, 4095] | The time adjustment for Slack Water or Min Ebb before Max Ebb, encoded Hours * 100 + Minutes, or 2560 for NULL.**

* In case of units = knots^2 (hydraulic currents), it should be understood that only the amplitudes of the constants are such as to give results in knots squared. The datum and any level_add corrections are in plain knots; i.e., the square root of the amplitude is taken before the datum and any corrections are added in.

** Time corrections DO NOT incorporate adjustments to Local Standard Time when the reference and sub station are in different time zones. Time corrections are specified as if all calculations and predictions were done in UTC. Time zone differences are handled by adjusting the `tzfile` field. This differs from current NOS practice, in which LST adjustments are incorporated. When this "time warp" is undone for NOS sub stations that use reference stations across the International Date Line, time corrections in excess of 24 hours can result.

*** See http://www.flaterco.com/xtide/mincurrents.html regarding usage for Min Flood and Min Ebb events.

## 4. Public database header (version 2.2)

This section describes the "public portion" of the database header structure, which is declared in `tcd.h` and made available to applications via the `get_tide_db_header` operation.

    char version[90];

libtcd version string.

    uint32 major_rev;

libtcd major revision number.

    uint32 minor_rev;

libtcd minor revision number.

    char last_modified[90];

Last modification of TCD file.

    uint32 number_of_records;

Number of records in TCD file.

    int32 start_year;

Year corresponding to 0 index in speed, equilibrium argument, and node factor arrays.

    uint32 number_of_years;

Number of years in speed, equilibrium argument, and node factor arrays.

    uint32 constituents;

Number of constituents.

    uint32 level_unit_types;

Number of entries in table used by `get_level_units`.

    uint32 dir_unit_types;

Number of entries in table used by `get_dir_units`.

    uint32 restriction_types;

Number of entries in table used by `get_restriction`.

    uint32 datum_types;

Number of entries in table used by `get_datum`.

    uint32 countries;

Number of entries in table used by `get_country`.

    uint32 tzfiles;

Number of entries in table used by `get_tzfile`.

    uint32 legaleses;

Number of entries in table used by `get_legalese`.

    uint32 pedigree_types;

For backward compatibility. Ignore.

The encodings of constituent speeds, equilibrium arguments, and node factors are as follows.

Name | Program data type | File encoding | Encodable range | Semantics
--- | --- | --- | --- | ---
Speed | float64 | uint varying (max 31), scale 10000000, with offset | [0.0000000, 214.7483647]* | Speed of constituent in degrees per hour.
Equilibrium argument | float32 | uint varying (max 31), scale 100, with offset | [0.00, bignum] | Equilibrium argument in degrees. Use [0.00, 359.99].
Node factor | float32 | uint varying (max 31), scale 10000, with offset | [0.0000, bignum] | Node factor.

The offsets are adjustments so that the minimum value in the data set is represented with a zero and the maximum by max−offset. For node factors, this can save bits. However, for speeds and equilibrium arguments they are a needless complication. Speeds in excess of the limit quoted above will overflow the integers before the offset is taken into account, and the minimum equilibrium argument is always zero anyway.

* The highest speed in the IHO list of constituents as of 2003-07-31 is 203.904625 degrees/hour for 6MS14.

## 5. libtcd 2.2 API

The API is provided in the C language and is made available by including the header file `tcd.h`.

    #define NV_BYTE    int8_t
    #define NV_INT16   int16_t
    #define NV_INT32   int32_t
    #define NV_INT64   int64_t
    #define NV_U_BYTE  uint8_t
    #define NV_U_INT16 uint16_t
    #define NV_U_INT32 uint32_t
    #define NV_U_INT64 uint64_t

    #define NV_BOOL    unsigned char
    #define NV_CHAR    char
    #define NV_U_CHAR  unsigned char
    #define NV_FLOAT32 float
    #define NV_FLOAT64 double

This section may appear slightly different from one platform to the next. The build process for libtcd generates #includes and data type definitions as needed to provide integer types of specific sizes.

    #define NVFalse         0
    #define NVTrue          1
    #define NV_U_INT32_MAX  4294967295
    #define NV_INT32_MAX    2147483647
    #define NV_U_INT16_MAX  65535
    #define NV_INT16_MAX    32767

Defined values.

    #define LIBTCD_VERSION   "PFM Software - libtcd v2.2 - 2006-11-22"
    #define LIBTCD_MAJOR_REV 2
    #define LIBTCD_MINOR_REV 2

These defines describe the version of libtcd to which the header file belongs.

    /* One-line character strings */
    #define ONELINER_LENGTH      90
    /* Verbose character strings */
    #define MONOLOGUE_LENGTH  10000
    #define MAX_CONSTITUENTS    255

These defines describe the sizes of the fixed-size arrays in the tide record.

    typedef struct ... DB_HEADER_PUBLIC;
    typedef struct ... TIDE_STATION_HEADER;
    typedef struct ... TIDE_RECORD;

The structure for the database header, and the header and main part of the tide record.

    enum TIDE_RECORD_TYPE {REFERENCE_STATION=1, SUBORDINATE_STATION=2};

Syntactic sugar for values of `header.record_type`.

    #define NULLSLACKOFFSET 0xA00

Magic constant used to indicate NULL in the `flood_begins` and `ebb_begins` fields of the tide record.

    #define AMPLITUDE_EPSILON 0.00005

This is the level below which an amplitude rounds to zero.

    void dump_tide_record (TIDE_RECORD *rec);

Prints a low-level dump of the tide record to stderr.

    NV_CHAR *get_country (NV_INT32 num);
    NV_CHAR *get_tzfile (NV_INT32 num);
    NV_CHAR *get_level_units (NV_INT32 num);
    NV_CHAR *get_dir_units (NV_INT32 num);
    NV_CHAR *get_restriction (NV_INT32 num);
    NV_CHAR *get_datum (NV_INT32 num);
    NV_CHAR *get_legalese (NV_INT32 num);

For fields in the tide record that are indices into tables of character string values, these functions are used to retrieve the character string value corresponding to a particular index. The value "Unknown" is returned when no translation exists. The return value is a pointer into static memory.

    NV_CHAR *get_constituent (NV_INT32 num);

Get the name of the constituent corresponding to index `num` [0,`constituents`-1]. The return value is a pointer into static memory.

    NV_CHAR *get_station (NV_INT32 num);

Get the name of the station whose `record_number` is `num` [0,`number_of_records`-1]. The return value is a pointer into static memory.

    NV_FLOAT64 get_speed (NV_INT32 num);

Returns the speed of the constituent indicated by `num` [0,`constituents`-1].

    NV_FLOAT32 get_equilibrium (NV_INT32 num, NV_INT32 year);
    NV_FLOAT32 get_node_factor (NV_INT32 num, NV_INT32 year);

Get the equilibrium argument and node factor for the constituent indicated by `num` [0,`constituents`-1], for the year `start_year`+`year`.

    NV_FLOAT32 *get_equilibriums (NV_INT32 num);
    NV_FLOAT32 *get_node_factors (NV_INT32 num);

Get all available equilibrium arguments and node factors for the constituent indicated by `num` [0,`constituents`-1]. The return value is a pointer into static memory which is an array of `number_of_years` floats, corresponding to the years `start_year` through `start_year`+`number_of_years`-1.

    NV_INT32 get_time (NV_CHAR *string);
    NV_CHAR *ret_time (NV_INT32 time);
    NV_CHAR *ret_time_neat (NV_INT32 time);

Convert between character strings of the form "[+-]HH:MM" and the encoding Hours * 100 + Minutes. `ret_time` pads the hours with a leading zero when less than 10; `ret_time_neat` omits the leading zero and omits the sign when the value is 0:00. Returned pointers point into static memory.

    NV_CHAR *ret_date (NV_U_INT32 date);

Convert the encoding Year * 10000 + Month [1, 12] * 100 + Day [1, 31] to a character string of the form "YYYY-MM-DD", or "NULL" if the value is zero. The returned pointer points into static memory. (The compact form, without hyphens, is obtainable just by printing the integer.)

    NV_INT32 search_station (NV_CHAR *string);

When invoked multiple times with the same `string`, returns record numbers of all stations that have that string anywhere in the station name. This search is case insensitive. When no more records are found it returns -1.

    NV_INT32 find_station (NV_CHAR *name);
    NV_INT32 find_tzfile (NV_CHAR *name);
    NV_INT32 find_country (NV_CHAR *name);
    NV_INT32 find_level_units (NV_CHAR *name);
    NV_INT32 find_dir_units (NV_CHAR *name);
    NV_INT32 find_restriction (NV_CHAR *name);
    NV_INT32 find_datum (NV_CHAR *name);
    NV_INT32 find_constituent (NV_CHAR *name);
    NV_INT32 find_legalese (NV_CHAR *name);

Inverses of the corresponding `get_` operations. Return -1 for not found.

    NV_INT32 add_restriction (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 add_tzfile (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 add_country (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 add_datum (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 add_legalese (NV_CHAR *name, DB_HEADER_PUBLIC *db);

Add the value of `name` to the corresponding lookup table and return the index of the new value. If `db` is not NULL, the database header struct pointed to will be updated to reflect the changes. The maximum length of `name` is restricted by the corresponding size definition in `tide_db_default.h` (restriction 30, tzfile 30, country 50, datum 70, legalese 70, including the terminating null).

    NV_INT32 find_or_add_restriction (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 find_or_add_tzfile (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 find_or_add_country (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 find_or_add_datum (NV_CHAR *name, DB_HEADER_PUBLIC *db);
    NV_INT32 find_or_add_legalese (NV_CHAR *name, DB_HEADER_PUBLIC *db);

Add the value of `name` to the corresponding lookup table if and only if it is not already present. Return the index of the value. If `db` is not NULL, the database header struct pointed to will be updated to reflect the changes. The maximum length of `name` is restricted by the corresponding size definition in `tide_db_default.h` (restriction 30, tzfile 30, country 50, datum 70, legalese 70, including the terminating null).

    void set_speed (NV_INT32 num, NV_FLOAT64 value);

Set the speed for the constituent corresponding to index `num` [0,`constituents`-1].

    void set_equilibrium (NV_INT32 num, NV_INT32 year, NV_FLOAT32 value);
    void set_node_factor (NV_INT32 num, NV_INT32 year, NV_FLOAT32 value);

Set the equilibrium argument and node factor for the constituent corresponding to index `num` [0,`constituents`-1], for the year `start_year`+`year`.

    NV_BOOL open_tide_db (NV_CHAR *file);

Opens the specified TCD file. If a different database is already open, it will be closed. libtcd maintains considerable internal state and can only handle one open database at a time. Returns false if the open failed.

    void close_tide_db ();

Closes the open database.

    NV_BOOL create_tide_db (NV_CHAR *file, NV_U_INT32 constituents,
        NV_CHAR *constituent[], NV_FLOAT64 *speed, NV_INT32 start_year,
        NV_U_INT32 num_years, NV_FLOAT32 *equilibrium[],
        NV_FLOAT32 *node_factor[]);

Creates a TCD file with the supplied constituents and no tide stations. Returns false if creation failed. The database is left in an open state.

    DB_HEADER_PUBLIC get_tide_db_header ();

Returns a copy of the database header for the open database.

    NV_BOOL get_partial_tide_record (NV_INT32 num, TIDE_STATION_HEADER *rec);

Gets "header" portion of tide record for the station whose `record_number` is `num` [0,`number_of_records`-1] and writes it into `rec`. Returns false if `num` is out of range. `num` is preserved in the static variable `current_index`.

    NV_INT32 get_next_partial_tide_record (TIDE_STATION_HEADER *rec);

Invokes `get_partial_tide_record` for `current_index`+1. Returns the record number or -1 for failure.

    NV_INT32 get_nearest_partial_tide_record (NV_FLOAT64 lat, NV_FLOAT64 lon,
                                              TIDE_STATION_HEADER *rec);

Invokes `get_partial_tide_record` for a station that appears closest to the specified `lat` and `lon` in the Cylindrical Equidistant projection. Returns the record number or -1 for failure.

    NV_INT32 read_tide_record (NV_INT32 num, TIDE_RECORD *rec);

Gets tide record for the station whose `record_number` is `num` [0,`number_of_records`-1] and writes it into `rec`. `num` is preserved in the static variable `current_record`. Returns `num`, or -1 if `num` is out of range.

    NV_INT32 read_next_tide_record (TIDE_RECORD *rec);

Invokes `read_tide_record` for `current_record`+1. Returns the record number or -1 for failure.

    NV_BOOL add_tide_record (TIDE_RECORD *rec, DB_HEADER_PUBLIC *db);
    #ifdef COMPAT114
    /* Omission of db parameter was a bug. */
    NV_BOOL update_tide_record (NV_INT32 num, TIDE_RECORD *rec);
    #else
    NV_BOOL update_tide_record (NV_INT32 num, TIDE_RECORD *rec, DB_HEADER_PUBLIC *db);
    #endif
    NV_BOOL delete_tide_record (NV_INT32 num, DB_HEADER_PUBLIC *db);

Add a new record, update an existing record, or delete an existing record. If the deleted record is a reference station, all dependent subordinate stations will also be deleted. Add and update return false if the new record is invalid; delete and update return false if the specified `num` is invalid or there is insufficient memory available. If `db` is not NULL, the database header struct pointed to will be updated to reflect the changes.

    NV_BOOL infer_constituents (TIDE_RECORD *rec);

Computes inferred constituents when M2, S2, K1, and O1 are given and fills in the remaining unfilled constituents. The inferred constituents are developed or decided based on Article 230 of "Manual of Harmonic Analysis and Prediction of Tides," Paul Schureman, C&GS Special Publication No. 98, October 1971. Returns false if M2, S2, K1, or O1 is missing.

For more information, see Jan Depner, "Format for the Oceanographic and Atmospheric Master Library (OAML) Tide Constituent Database," rev. 2003-06-06.

## 6. V1 to V2 migration guide

### libtcd vs. TCD file version compatibilities

libtcd 1 can only read and modify files written by v1. Versions of libtcd prior to 1.99 will fail inelegantly when attempting to read v2 files and will corrupt the database if a modification is attempted. Versions 1.99 and 1.100 will detect and report the version mismatch.

libtcd 2 can read files written by v1 or v2, but it can only create or modify v2 files. If it is necessary to create a v1 file, you must compile your application using libtcd 1. Version 1.100 is recommended for this purpose. It is available at ftp://ftp.flaterco.com/xtide/libtcd-1.100.tar.gz.

To convert a v1 TCD file to a v2 TCD file, use the application rewrite_tide_db in the package tcd-utils, available from http://www.flaterco.com/xtide/files.html.

### Compiling old applications without changes

This option is for emergencies only, or in case of such extreme laziness that maintaining the version-integrity of the API is less important than avoiding minor changes to your applications.

Applications written against the TCD version 1 API should compile and run without changes using libtcd 2 if libtcd 2 is built with the `COMPAT114` option, as follows:

>     bash-3.1$ ./configure --enable-COMPAT114

This will allow old applications to read and write version 2 TCD files. However, the resulting library is NOT API-compatible with libraries built in the normal way, which will break applications that are properly written for the v2 API. Therefore, any library built with this option is for local use only. Only a static library will be built.

Even with `COMPAT114` enabled, some fields do not behave as they did in libtcd 1. Specifically, `COMPAT114` does the following.

- The preprocessor defines `NAME_LENGTH`, `SOURCE_LENGTH` and `COMMENTS_LENGTH` are provided.
- The tide record field `units` is provided and is copied to and from `level_units` as needed to support reference stations.
- The tide record fields `pedigree`, `avg_level_units`, `min_avg_level`, and `max_avg_level` are provided, but they are never used.
- The deprecated functions `get_pedigree`, `find_pedigree` and `add_pedigree` are provided, but their behavior is to change all pedigrees to "Unknown."
- The function `update_tide_record` accepts only two parameters `(NV_INT32 num, TIDE_RECORD *rec)`.
- The function `check_simple` is provided.
- Reading of pre-version-1.02 TCD files with old style header checksums is supported.

The following subsections detail the application changes that are required and recommended to migrate to the version 2 API without `COMPAT114`.

### Checklist of required changes when migrating applications from v1 API to v2 API

1. Replace all uses of the `units` field with `level_units`.
2. Replace all uses of `NAME_LENGTH` and `SOURCE_LENGTH` with `ONELINER_LENGTH`.
3. Replace all uses of `COMMENTS_LENGTH` with `MONOLOGUE_LENGTH`.
4. Provide a third parameter, `DB_HEADER_PUBLIC *db`, on each invocation of `update_tide_record`. It can be NULL if the calling function has no need of the db header.
5. Stop using the tide record fields `pedigree`, `units`, `avg_level_units`, `min_avg_level`, and `max_avg_level`.
6. Stop using the functions `get_pedigree`, `find_pedigree`, `add_pedigree`, and `check_simple`.

### Checklist of optional changes when migrating applications from v1 API to v2 API

1. Ensure that the `max_direction` and `min_direction` fields are initialized to 361 (the value indicating null) and/or set to their correct values when writing reference stations. Ensure that `direction_units` is set properly.
2. Change variable data types and casts to match the new type of `months_on_station`. This was changed from unsigned char (uint8) to unsigned short (uint16) in libtcd 1.100.
3. Change variable data types and casts to match the new types of the `constituents` and `num_years` parameters to `create_tide_db` and any fields in `DB_HEADER_PUBLIC` that were changed from signed to unsigned in libtcd 1.99.
4. Make use of the new fields `notes`, `legalese`, `xfields`, `station_id_context`, `station_id`, and `date_imported` as appropriate.
5. Make use of the new functions `find_or_add...`, `...legalese`, `get_equilibriums`, `get_node_factors`, `ret_time_neat`, and `ret_date` as appropriate.
6. When invoking functions that accept a `DB_HEADER_PUBLIC *db` parameter, if the db header has no use in the calling function except to fill in this parameter, a NULL can be passed instead.

## 7. Changelog

### libtcd 2.2.2 (libtcd.so.0.0.2)

David Flater
2007-01-22

(Bugs, tide_db.c) Corrected more off-by-one errors in the use of calculate_bits by the function create_tide_db. The function write_tide_db_header was rounding values when it encoded them but create_tide_db was truncating them when it was calculating the number of bits. So if the maximum speed, equilibrium argument, or node factor value as encoded in integer form happened to round up to an exact power of 2, an invalid TCD file could have been produced.

(Nit, bit_pack.c) Eliminated the special case in calculate_bits to return 1 on the input 0. A field whose only possible value is 0 consumes no space in the TCD file but is correctly decoded as 0.

### libtcd 2.2.1 (libtcd.so.0.0.1)

David Flater
2007-01-21

(Bug, bit_pack.c) Rewrote calculate_bits to eliminate ambiguous interpretation of "range" argument (now called "value"), eliminate the possibility of roundoff error, return the correct result in the case where the input value is 1, and, as a special case, return 1 if the input value is 0.

(Bugs, tide_db.c) Corrected off-by-one errors in the use of calculate_bits by the function create_tide_db.

The preceding issues could have resulted in the production of an invalid TCD file if the number of constituents, the number of distinct speed values, the number of distinct equilibrium argument values, or the number of distinct node factor values that were required by a particular database happened to be exactly a power of 2 (including 1). These conditions are unlikely to have occurred in normal use. I discovered the problems while attempting to generate a test case for XTide using exactly one constituent.

### libtcd 2.2 (libtcd.so.0.0.0) release 2

David Flater
2006-11-26

Added warranty disclaimer to COPYING and libtcd.html. No source code changes.

### libtcd 2.2 (libtcd.so.0.0.0)

David Flater
2006-11-22

Split out libtcd into a separate tarball (was previously bundled with XTide).

Started using automake and libtool; started building shared lib.

Resolved preprocessor define clash between automake and libtcd on VERSION.

Merged nvtypes.h and tide_db_version.h into tcd.h to reduce pollution of public include directories.

Removed extraneous #includes from tcd.h.

Changed COMPAT114 into a configure option (--enable-COMPAT114).

Caused tcd.h to be generated at configure time with the following substitutions.

- COMPAT114 is defined or undefined per --enable-COMPAT114.
- Version string reflects enabling of COMPAT114.
- NV_ data type definitions and any pertinent #includes are synchronized with what configure found.

### libtcd 2.1.3

David Flater
2006-10-05

Changed asserts with side-effects to use require macro instead.

Minor clean-ups.

### libtcd 2.1.2

David Flater
2006-08-03

Improved configure script.

### libtcd 2.1.1

David Flater
2006-08-02

Added configure script.

Cleaned up nvtypes.h.

Corrected date on version 2.1 (said 2005, was 2006).

### libtcd 2.1

David Flater
2006-05-28

Fixed bug in delete_tide_record: linkage from subordinate stations to reference stations other than the one being deleted was getting broken. Bug report credit: August Hahn.

Fixed a memory management error in update_tide_record. Bug report credit: August Hahn.

Added enum for TIDE_RECORD_TYPE.

In update or delete_tide_record, if cannot allocate memory, return failure to the caller instead of exiting.

Added more error traps for invalid input.

Miscellaneous cleanups and nit fixes.

### libtcd 2.0

David Flater
2004-10-15

Lots of changes to implement the revised database schema.

`COMPAT114` is no longer defined by default in tcd.h (i.e. the v2 API is now effective).

Removed the unused define `DEFAULT_NUMBER_OF_OBSOLETE_RECORDS` from tide_db_default.h.

Fixed repeated reporting of amplitude/epoch errors.

Fixed wasted byte bug on speeds, equilibrium arguments, and node factors.

### libtcd 1.100 "enhanced edition"

David Flater
2004-10-13

With luck, this will be the final release in the version 1 series and the last release that will write files that can be read by libtcd 1.02 through 1.14. It includes an additional round of cleanups that will enhance upward compatibilty but should not break backward compatibility.

`check_simple` was placed inside `#ifdef COMPAT114` (which is still defined in tcd.h). AFAIK, this function is now of interest only in restore_tide_db, which uses it to determine which XML format to output.

Copied the libtcd.html descriptions of operations into the header file.

Made the `db` parameters to `add_` and `find_or_add_` operations optional (can be null).

Gave `delete_tide_record` a return value. Gave `update_tide_record` a `db` parameter, conditional on `COMPAT114` not being defined.

In `delete_tide_record`, assert that the record is found.

Deleted unused `systemtime` var in `write_tide_db_header`.

Deleted unnecessary `file` parameter to `read_tide_db_header`.

Added static function `check_tide_record` to do sanity checks on records before committing them to the database and called it from `add_tide_record` and `update_tide_record`.

Initialized an uninitialized var in `search_station`.

Added `clip_string` calls in `find_tzfile` and `find_constituent`.

Replaced some weird code in `find_constituent` with more normal code.

Got rid of unnecessary copy operations in `add_` functions.

Renamed `tname` parms to `name`.

Added checks to ensure that database is open when expected to be open and closed when expected to be closed.

Added `AMPLITUDE_EPSILON` and used it where applicable.

Moved duplicate record-size-figuring code into a new function, `figure_size`. Fixed wasted byte when record size is even multiple of 8 bits. (The same bug exists for calculating the size of the speeds, equilibrium arguments, and node factors, but fixing that would break compatibility, and the wastage is at most 3 bytes.)

Changed `months_on_station` to uint16.

Fixed bounds checking on input to `add_` functions -- they all have their own static limits. Added similar bounds checking in `write_tide_db_header`.

Lots of documentation nits fixed.

### libtcd 1.99 "collector's edition"

David Flater
2004-10-05

With luck, this will be the final release in the version 1 series and the last release that will write files that can be read by libtcd 1.02 through 1.14. It includes a gigantic round of cleanups to libtcd code that were made in anticipation of the upcoming major revision. Except for the bug fixes, the changes should be invisible to all wholesome client applications. Unwholesome client code or the use of unwholesome databases built by old, buggy versions of libtcd may now trigger errors or warnings that did not appear with libtcd 1.14.

In the process of testing the cleanups, I discovered some unrelated, really horrible bugs.

(Bug) Constituents with zero phases were not being encoded by `write_tide_record`. The affected reference stations in harmonics-dwf-2004-09-14 are Delfzijl, Netherlands; The Battery, New York Harbor, NY; and South Pass, LA. `write_tide_record` now encodes any constituent with a positive amplitude. Constituents with zero amplitude but nonzero phase are dropped, and negative amplitudes yield assertion failures. Made analogous changes to `update_tide_record` (the counts were supposed to match).

(Bug) `update_tide_record` was counting constituents for subordinate stations.

(Bug) Backed out the change to the control flow in `delete_tide_record` made in 1.14, which broke the deletion of reference stations with sub stations.

(Bug) Fixed logic error in `delete_tide_record` that corrupted the database when reference stations with sub stations were deleted.

(Bug) Made `read_next_tide_record` return -1 if at end of file, instead of going off the deep end.

Following are details of the gigantic cleanup.

The defines `NAME_LENGTH`, `SOURCE_LENGTH` and `COMMENTS_LENGTH` were generalized to `ONELINER_LENGTH` to set the length of one-line character string buffers and `MONOLOGUE_LENGTH` to set the length of verbose character string buffers.

`COMPAT114` is defined in tcd.h. The effects are:

- `NAME_LENGTH`, `SOURCE_LENGTH` and `COMMENTS_LENGTH` get defined.
- Pre-version-1.02 TCD files using the old checksum code are accepted.

Added commentary on the commentary about OS-specific defines in nvtypes.h and tide_db_header.h, noting that they are not actually used.

Cleaned up all warnings and errors compiling under g++ -Wall:

- (Errors) Corrected casts in union initializers in `tide_db_header.h`.
- (Warnings) Added casts or changed data types, as appropriate, to resolve flagged signed-unsigned conversions.
- (Errors) Added casts where `NV_U_BYTE` buffers were passed to functions from `string.h` (invalid conversion from `unsigned char*` to `const char*`).

Added asserts everywhere that I came across potentially dangerous assumptions.

Changed `-O2` to `-O` in Makefile (Forte doesn't like `O2`).

Moved changelogs from source files to libtcd.html.

Made `write_tide_record` static; no applications should be calling this directly.

In `clip_string`, expanded static buffer to `MONOLOGUE_LENGTH`, fixed a bug that was previously masked, simplified code and added bounds checking.

Eliminated repeated calls to `clip_string` in find functions.

Eliminated `clip_string` duplication in `write_tide_record`.

Changed 256-char static buffers in `add_pedigree`, `add_tzfile`, `add_country`, `add_datum`, and `add_restriction` to `ONELINER_LENGTH` and added bounds checking.

Made filename `MONOLOGUE_LENGTH`.

Made version and last_modified `ONELINER_LENGTH`.

Handled overlong strings in TCD file by complaining and truncating instead of returning garbarge.

Eliminated `get_string`.

Eliminated repeated parsing of input in `read_tide_db_header`. Changed input buffer size to `ONELINER_LENGTH`.

Made support for pre-version-1.02 header checksums conditional on `COMPAT114`.

Changed many int32 fields in the DB header that had no reason ever to be negative to uint32. Exceptions include `start_year` (in case some crazy goes B.C.) and the offsets for speeds, equilibrium arguments, and node factors (required to read pre-version-1.11 TCD files). Also changed `record_size` in the tide record header, but left alone the rest of the tide record because the risk of breaking client code is too great: build_tide_db temporarily stores negative results from find functions in these fields, and tideEditor puts -1 in the `record_number` field to flag a new record. -1 in `reference_station` means no reference, though normally it's only -1 for reference stations.

In signature of `create_tide_db`, `constituents` and `num_years` became unsigned.

Changed many looper, index, size, etc. variables from signed to unsigned.

Corrected unsigned scan formats in `read_tide_db_header`.

Removed unused fields, unused data types and overcomplicated infrastructure in `tide_db_header.h` and `read_tide_db_header`.

Corrected signature of `bit_unpack` in `tide_db.c`.

Made `get_speed`, `get_equilibrium` and `get_node_factor` give assertion failures instead of returning -1.0 when invoker attempts to retrieve nonsense values. Added a missing bounds check.

Made `set_speed`, `set_equilibrium` and `set_node_factor` give assertion failures instead of doing nothing when invoker attempts to set nonsense values. Added a missing bounds check.

Made `record_size` in `TIDE_INDEX` struct 32 bits.

Added a warning about TCD files with negative offsets for speeds, eq args or node factors (caused by the bug fixed in 1.11).

Changed format of date string in [LAST MODIFIED] to XTide style. Changed identifer in [VERSION] to libtcd.

Added these functions:

- `NV_FLOAT32 *get_equilibriums (NV_INT32 num);`
- `NV_FLOAT32 *get_node_factors (NV_INT32 num);`
- `NV_INT32 find_or_add_restriction (NV_CHAR *tname, DB_HEADER_PUBLIC *db);`
- `NV_INT32 find_or_add_tzfile (NV_CHAR *tname, DB_HEADER_PUBLIC *db);`
- `NV_INT32 find_or_add_country (NV_CHAR *tname, DB_HEADER_PUBLIC *db);`
- `NV_INT32 find_or_add_datum (NV_CHAR *tname, DB_HEADER_PUBLIC *db);`
- `NV_CHAR *ret_time_neat (NV_INT32 time);`
- `NV_CHAR *ret_date (NV_U_INT32 date);`

Added header fields for major and minor revision number. Added check for attempt to read higher major revision than supported.

### libtcd 1.14

David Flater
2004-09-03

(tide_db.c) Certain database update operations that ended with closing and re-opening the database were broken by nulling out the filename in close_tide_db. Incorporated patch from Jan Depner that rectifies this and tweaks the control flow in delete_tide_record.

### libtcd 1.13

David Flater
2004-08-15

(create_tide_db) Write a valid end_of_file record even if someone creates a database with no tide records.

(tide_db_default) Removed a superfluous space character in table of default datum types (after Lowest Astronomical Tide).

### libtcd 1.12

David Flater
2003-12-04

(tcd.h) Deleted #include <malloc.h>.

### libtcd 1.11

David Flater
2003-11-16

(create_tide_db) Fixed horrible bug: offsets for speeds, equilibrium args, and node factors were sign-reversed with respect to their usage in read_tide_db_header and write_tide_db_header, resulting in possible overflows.

(read_tide_db_header) Added handling for zero tide records, which happens on new database create.

(open_tide_db) Added check of modified flag to 2003-10-14 code.

(close_tide_db) Deleted repeat free of tindex introduced 2003-10-14. Cleared modified flag on close.

### libtcd 1.10

David Flater
2003-10-14

Incorporated patch from Phil Thornton that closes a memory leak and improves performance on repeat calls to open_tide_db. See http://www.flaterco.com/xtide/tcd_notes.html

### libtcd 1.09

Jan C. Depner
2003-09-04

Bug fix - modifying last record in file and changing size of record caused a big problem. Duh, I should have seen that one coming.

### libtcd 1.08

Jan C. Depner
2003-07-23

Bug fix - deleting last record in file caused a big problem.

### libtcd 1.07

David Flater
2003-03-27

(This version was also released briefly without an updated version string.)

Check_simple altered per resolution to abolish "simplified" type 2 records. See http://www.flaterco.com/xtide/tcd_notes.html

### libtcd 1.06

David Flater
2003-03-18

(This version failed to report an updated version string.)

Added NULLSLACKOFFSET to tcd.h per 2003-03-18 change to slack offsets. See http://www.flaterco.com/xtide/tcd_notes.html

### libtcd 1.05

David Flater
2003-03-11

Incorporated nit fixes from August Hahn: added clip_string(tname) to take care of cases when the country name had leading/trailing spaces, and removed a couple of unused variables.

### libtcd 1.04

David Flater
2002-12-13

While integrating 1.03, deleted an unused variable (whoop de do).

### libtcd 1.03

Wade Ladner/Jan C. Depner
2002-12-09

Small fix for Micro$oft Windoze. We don't need to add CR's to strings (apparently).

### libtcd 1.02

August Hahn
2002-11-15

Replaced the simplistic checksum with a real CRC. If the first checksum fails it will check for an old-style checksum.

### libtcd 1.01

David Flater
2002-10-01

Many changes since 2002-08-01. Bumped version upon integration with XTide distribution. Renamed resulting library to libtcd.

### tidelib 1.00

Jan C. Depner
2002-08-01

First tested release.

### tidelib 0.99

Jan C. Depner
2002-07-15

Prototype for testing.

## 8. References

Jan Depner, "Format for the Oceanographic and Atmospheric Master Library (OAML) Tide Constituent Database," rev. 2003-06-06.

libtcd version 1.100.

XTide.

ISO-8859-1.

Manual of Harmonic Analysis and Prediction of Tides. Special Publication No. 98, Revised (1940) Edition (reprinted 1958 with corrections; reprinted again 1994). United States Government Printing Office, 1994.