| Appendix B. HyperSQL Database Files and Recovery |
Appendix A. Lists of Keywords |  | Appendix C. Building HSQLDB Jars

# HyperSQL Database Files and Recovery

$Revision: 6692 $

2024-10-25

## Database Files

Database catalogs opened with the file: protocol are stored as a set of files. This document describes the contents of these files and how they are stored.

A database named 'test' is used in this description. The database files will be as follows.

Database Files

test.properties | Contains the entry 'modified'. If the entry 'modified' is set to 'yes' then the database is either running or was not closed correctly. When the database is properly shutdown, 'modified' is set to 'no'. You can add the optional entry readonly=true to make the database files read only.
test.script | This file contains the SQL statements that makes up the database up to the last checkpoint. All the data for MEMORY tables is also stored in this file.
test.data | This file contains the binary data records for CACHED tables only.
test.backup | This file contains a backup of the parts of the `test.data` that have been modified since the last checkpoint. The file is deleted at each checkpoint or shutdown. The size of this file can be as large as the `test.data` file.
test.log | This file contains the extra SQL statements that have modified the database since the last checkpoint. It is used as a redo log.
test.lobs | This file contains the lobs. If a database has no BLOB or CLOB object, this file will not be present. This file contains all the lobs that are currently in the database, as well as those that belong to rows that have been deleted since the last checkpoint. The space for deleted lobs is always reused after a CHECKPOINT.

A CHECKPOINT is an operations that saves all the changed data and removes the `test.log` followed by the creation of an empty log. It also deletes the `test.backup` file. A SHUTDOWN is equivalent to a CHECKPOINT followed by closing the database.

The disk space needed for the database depends on the amount of data. After the database is opened more space, up to slightly greater than the size of the `test.script` and `test.data` files is needed during operation.

## States

Database is closed correctly

State after running the `SHUTDOWN` statement

- The `test.data` file is fully updated.

- There is no `test.backup` file.

- The `test.script` contains all the metadata and CREATE TABLE and other DDL statements. It also contains the data for MEMORY tables.

- The `test.properties` contains the entry 'modified' set to 'no'.

- There is no `test.log` file.

Database is closed correctly with SHUTDOWN SCRIPT

State after running the `SHUTDOWN SCRIPT` statement

- The `test.data` file does not exist; all CACHED table data is now in the `test.script` file

- There is no `test.backup` file.

- The `test.script` contains all the metadata and DDL statements, followed by the data for MEMORY, CACHED and TEXT tables.

- The `test.properties` contains the entry 'modified' set to 'no'.

- There is no `test.log` file.

Database is aborted

If the database process was terminated without a SHUTDOWN, or the SHUTDOWN IMMEDIATELY command was used, the database is in aborted state.

Aborted database state

- The `test.properties` contains 'modified=yes'.

- The `test.script` contains a snapshot of the database at the last checkpoint.

- The `test.data` file is not necessarily consistent.

- The `test.backup` file contains just sections of the original `test.data` file that correspond to `test.script` at the time of the last checkpoint.

- The `test.log` file contain all data change statements executed since the last checkpoint. As a result of abnormal termination, the end of file may be incomplete.

## Procedures

The database engine performs the following procedures internally in different circumstances.

### Clean Shutdown

Procedure B.1. Clean HyperSQL database shutdown

1. The `test.data` file is written completely (all the modified cached table rows are written out) and closed.

2. The file `test.script.new` is created using the current state of the database.

3. The entry 'modified' in the properties file is set to 'yes-new-files' (Note: after this step, the `test.data` and `test.script.new` files constitute the database)

4. The file `test.log` is deleted

5. The file `test.script` is deleted

6. The file `test.script.new` is renamed to `test.script`

7. The file `test.backup` is deleted

8. The entry 'modified' in the properties file is set to 'no'

### Startup

Procedure B.2. Opening the Database

1. Check if the database files are in use by checking a special `test.lck` file.

2. See if the `test.properties` file exists, otherwise create it.

3. If the `test.script` did not exist, then this is a new database.

4. If it is an existing database, check in the `test.properties` file if 'modified=yes'. In this case the RESTORE operation is performed before the database is opened normally.

5. Otherwise, if in the `test.properties` file 'modified=yes-new-files', then the (old) `test.backup` and `test.script` files are deleted if they exist and the new `test.script.new` file is renamed to `test.script`.

6. Open the `test.script` file and create the database objects.

7. Create the empty `test.log` to append any data change statements.

### Restore

The current `test.data` file is not necessarily consistent. The database engine takes these steps:

Procedure B.3. Restore a Database

1. Restore the old `test.data` file from the backup by copying the original sections from the `test.backup` file.

2. Execute all the statements in the `test.script` file.

3. Execute all statements in the `test.log` file. If due to incomplete statements in this file an exception is thrown, the rest of the lines in the `test.log` file are ignored. This can be overridden with the database connection property `hsqldb.full_log_replay=true` which results in the startup process to fail and allows the user to examine and edit the `test.log` file.

4. Close the database files.

5. Open the restored database.

$Revision: 6787 $

Appendix A. Lists of Keywords |  | Appendix C. Building HSQLDB Jars