# LDBS disk image format v0.3

## John Elliott, 14 March 2017

## Introduction

LDBS is a disk image format originally designed for internal use in LibDsk, with the intended use case being for archiving FM / MFM floppy disks. This specification is provided for comment and suggestions, in case the format proves useful in other situations.

The suggested file extension is .ldbs.

The reference implementation for this version of LDBS is ldbs.c / ldbs.h, supplied with LibDsk v1.5.4.

The file structure is defined at two levels: a generic block store, and the specific block types used to specify a disk image.

Rationale: Why a new disk image format? As mentioned above, it was designed for use as an intermediate format in LibDsk. The aim is to have feature parity with CPCEMU extended .DSK, but in a more extensible format with greater support for in-place rewrites. It is not, for example, possible to reformat a track from 8 to 9 512-byte sectors in an EDSK file, without rewriting all subsequent tracks.

## Change history

0.3 (8 March 2017)
  - Altered the definition of the track and sector headers to support some CPC copy-protection methods and add further extensibility. Since this is a breaking change to the binary file format, I have increased the version number in the header.
0.2 (3 March 2017)
  - Added a warning against storing file offsets in custom blocks.
  - Implemented a few more helper methods. The disk image format itself remains unchanged.
0.1 (16 March 2016)
  Original version.

## Data storage: Block Store

All 16-bit and 32-bit values are stored in Intel format (low byte first).

File offsets and block lengths are 32 bit little-endian doublewords. To avoid signed / unsigned issues, they should not exceed 2^31.

### Header

All LDBS files begin with a 20-byte header:

Offset | Type | Meaning
--- | --- | ---
0x0000 | 4 bytes | LDBS magic number: `0x4C 0x42 0x53 0x01` ('`LBS\01`')

0x0004

4 bytes

File type. For a disk image this is `0x44 0x53 0x4B 0x02` ('`DSK\02`'). If it is `0x44 0x53 0x4B 0x01` ('`DSK\01`') then it is in LDBS 0.2 or earlier format. Such disc images are not likely to be in wide circulation and I don't think you need to bother implementing support for them. The supplied `ldbs_v2` utility can convert disc images in the earlier format to the current version.

0x0008 | offset | Offset of first block in 'used' linked list. Zero means there are no used data blocks.
0x000C | offset | Offset of first block in 'free' linked list. Zero means there are no free data blocks.
0x0010 | offset | Offset of track directory. Zero means there is no track directory.

If the offsets at 0x0008 / 0x000C / 0x0010 are non-zero, they must be file offsets of valid data blocks. It is an error for them to point to any other location in the file.

Within the block store layer the track directory is optional, but a valid LDBS disk image must have a track directory.

Although the block store layer is currently only defined as containing a disk image, it would be usable as a container for other file formats; this would be indicated by a different value at offset 0x0004. The use of the block store for other file formats is not covered further by this document.

### Data blocks

The header is followed by one or more data blocks (a file with no data blocks is structurally valid, but would not contain a track directory). There is no constraint on the order of blocks in a file; a track header does not have to be stored together with its associated sectors, for example.

For an LDBS disc image, is recommended that a given data block should not exceed 64k in size.

There are no alignment requirements; blocks can begin at any file offset.

Each data block has its own 20-byte header:

Offset | Type | Meaning
--- | --- | ---
0x0000 | 4 bytes | Block signature: `0x4C 0x44 0x42 0x01` ('`LDB\01`')
0x0004 | 4 bytes | Block type. 4-byte value as defined below. `0x00 0x00 0x00 0x00` indicates a free block, other values indicate a used block.
0x0008 | length | Length of block on disk (not including this 20-byte header)
0x000C | length | Length of block contents, less than or equal to length of block on disk. Should be set to zero for a free block.
0x0010 | offset | Offset of next block in used / free linked list. Zero if this is the last block in the list.

The header is followed immediately by the block data.

If a block is recorded in the track directory (see below) its block type is used to identify it, and so must be unique within the file. The block type is not required to be unique for blocks not recorded in the directory.

If the offset at 0x0010 is non-zero, it must point to another valid data block.

If the length of the block on disk is greater than the length of the block contents, an implementation is encouraged to fill the remainder with zero bytes, but is not obliged to. Similarly a free block should be filled with zeroes, but is not required to be.

Data blocks can be enumerated either by stepping through the file in linear order, or by following the used / free linked lists.

It is an error for there to be a 'free' block on the 'used' linked list, or a 'used' block on the 'free' linked list.

### Operations

Typical operations performed at block level are:

- Get the length of a block. Given an offset, load the block header at that offset and return its length.
- Load a block. Given an offset, load the block header and data at that offset.
- Add a block. Search the free list for an entry greater than or equal to the required size. If one is found, remove the entry from the free list, add it to the used list, and populate with the data. If not found, append a new block of the required size to the end of the file, and add it to the used list.
- Delete a block. Remove the block from the used list, add it to the free list, and zero its block type.
- Update a block. If the new size is less than or equal to the block's current size on disk, write the data into the block at its current location. Otherwise, treat as delete followed by add.

Note that there is no requirement to coalesce adjacent free blocks, or to split a block if the size requested is less than its current size on disk.

## Data storage: Disc image

### Track directory

A disk image must contain a track directory block. The intention is that an implementation will use the track directory to locate data rather than performing linear searches of the file or walking its linked lists.

The format of the track directory is:

Offset | Type | Meaning
--- | --- | ---
0x0000 | 20 bytes | LDBS block header. Type field contains `0x44 0x49 0x52 0x01` ('`DIR\01`')
0x0014 | 16-bit word | Number of directory entries
0x0016 | 8 bytes per entry | Directory entries

The format of a directory entry is one of `Txxx`, `INFO`, `CREA`, `DPB` or `GEOM`, or a custom block type (beginning with a lowercase 'a'-'z'). In each case, this is the block type of the corresponding block in the store.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 1 byte | 0x54 ('T')
0x0001 | 2 bytes | Cylinder number
0x0003 | 1 byte | Head number
0x0004 | offset | Offset of track header block

Reference to a track header. There is one of these for each track in the disk image.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 4 bytes | 'INFO'
0x0004 | offset | Offset of comment block

Reference to a comment block. There is at most one of these in the disk image file.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 4 bytes | 'CREA'
0x0004 | offset | Offset of creator block

Reference to a creator block. There is at most one of these in the disk image file.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 4 bytes | 'GEOM'
0x0004 | offset | Offset of geometry block

Reference to a geometry block. There is at most one of these in the disk image file.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 4 bytes | 'DPB '
0x0004 | offset | Offset of DPB block

Reference to a CP/M Disk Parameter Block block. There is at most one of these in the disk image file. It will only be present on disks containing a CP/M filesystem.

### Custom block types

Other block types than the ones listed here may be defined in later versions of this spec; implementations should ignore unknown block types. It may, for example, be useful at some point to create a 'BOOT' block type containing the keystrokes necessary to boot a disk, or a 'PICT' block containing a picture of the disk.

All future block types defined by this spec will start with a capital letter, 'A'-'Z'. Block types beginning with a lowercase letter 'a'-'z' can be used for private purposes — there is, of course, no guarantee that someone else won't pick the same block type for their own use.

You should avoid storing file offsets in your custom blocks. There is nothing stopping an LDBS implementation from rearranging blocks in a file so they end up at different offsets (for example, to remove unused space). However, such an implementation won't be aware of offsets held in custom blocks, and so won't update them. This isn't just a theoretical danger: `ldbs_clone()` reverses the order of blocks when duplicating an LDBS file.

The LibDsk TeleDisk, CopyQM and QRST drivers use custom blocks to hold details from the original file headers which aren't used by other disk image formats. These are, respectively, 'utd0', 'ucqm', and 'uqrs'.

### Track header

Offset | Type | Meaning
--- | --- | ---
0x0000 | 20 bytes | LDBS block header. Type field contains `0x54 cyl cyl head` , corresponding to directory entry.
0x0014 | 2 bytes | [LDBS 0.3] Length of fixed track header in bytes (= offset of first sector header less 0x0014). Currently 0x000C, but you should not rely on this; future extensions may require extra bytes to be added to the track header.
0x0016 | 2 bytes | [LDBS 0.3] Length of each sector entry in bytes. Currently 0x0010, but you should not rely on this; future extensions may require extra bytes to be added to the sector header.
0x0018 | 2 bytes | Number of sector entries in the track header.

0x001A

1 byte

Data rate when track was recorded:

    0 => unknown
    1 => Single density (125 kbit/s or 150 kbit/s FM) or double density (250 kbit/s or 300 kbit/s MFM)
    2 => High density (500 kbit/s MFM, or [unlikely] 250 kbit/s FM)
    3 => Extended density (1000 kbit/s MFM, or [unlikely] 500 kbit/s FM)

Note: The 250 kbit/s and 300 kbit/s rates (and their FM equivalents) are combined because the same disc may appear to use either rate, depending whether it is read in a 360k drive or a 1.2Mb drive.

0x001B

1 byte

Recording mode used for the track:

    0 => unknown
    1 => FM
    2 => MFM

0x001C | 1 byte | Format gap length
0x001D | 1 byte | Default filler byte
0x001E | 2 bytes | [LDBS 0.3] Approximate length of track (including address marks and gaps) in bytes. For some timing-based copy protection schemes. Zero if this value was not captured when the disc was imaged.
0x0020 |  | [LDBS 0.3] Future versions may add extra bytes here. See offset 0x0014
See offset 0x0014 | See offset 0x0016 | [LDBS 0.3] Sector headers

Each sector descriptor is formed:

Offset | Type | Meaning
--- | --- | ---
0x0000 | 1 byte | Sector ID: Cylinder
0x0001 | 1 byte | Sector ID: Head
0x0002 | 1 byte | Sector ID: Sector
0x0003 | 1 byte | Sector ID: Size (0 => 128, 1 => 256, 2 => 512 ... 7 => 8192)

0x0004

1 byte

8272 status 1. The following bits may be set if the archiver encountered errors reading this sector:

    Bit 7: End of cylinder
    Bit 5: Data error in ID or data field
    Bit 2: No data
    Bit 0: Missing address mark in ID or data field

0x0005

1 byte

8272 status 2. The following bits may be set if the archiver encountered errors reading this sector, or if the sector has a deleted data control mark:

    Bit 6: Control mark (sector marked as deleted data)
    Bit 5: Data error in data field
    Bit 0: Missing address mark in data field

0x0006 | 1 byte | Number of copies held. Some copy-protection systems use a 'weak' sector which returns different values each time it is read. This is simulated by holding multiple copies of the sector, and returning a random one each time it is read. If this is zero, the sector is treated as blank (ie, entirely filled with the byte at 0x0007).
0x0007 | 1 byte | Filler byte if sector is blank
0x0008 | offset | Offset of sector data block for this sector. If number of copies is zero, this must be zero too.
0x000C | 2 bytes | Number of trailing bytes. If nonzero, then the specified number of CRC and gap bytes follow each copy of the sector itself.
0x000E | 2 bytes | Approximate offset of sector within track. This is used for some timing-based copy protection schemes. If this value was not captured it will be zero for all sectors on the track.
0x0010 |  | [LDBS 0.3] Future versions may add extra bytes here. See offset 0x0016 of track header.

### Sector data

Offset | Type | Meaning
--- | --- | ---
0x0000 | 20 bytes | LDBS block header. Type field contains `0x53 cylinder head sector`. Note that cylinder and head are the sector's actual physical location on the disk, which may not be the same as the values in its ID header.

0x0014

As specified in descriptor

The sector data. In normal circumstances the length should match the descriptor in the track header: ((128 << id_psh) + trailing bytes) * number of copies. However implementations should be prepared to handle the data being either shorter or longer than the expected size; this may arise if the LDBS file was converted from a disc image file with the same anomaly.

Since the track directory does not record the locations of individual sectors, the block type field is not required to be unique. If an LDBS file contained more than 256 cylinders, for example, sectors on cylinder 256 would have the same block type as those on cylinder 0. Similarly if a track contained several copies of the same sector, all these copies would have the same block type.

### Comment block

The comment block is optional, and contains a human-readable comment describing the disk image.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 20 bytes | LDBS block header. Type field contains `0x49 0x4E 0x46 0x4F` ('INFO')
0x0014 | As specified in header | Disk comment. Lines should be separated with DOS-style newlines ('\r\n').

On character sets: If possible, stick to ASCII. Failing that, UTF-8.

### Creator block

The creator block is optional, and contains human-readable text naming the utility that created the disk image.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 20 bytes | LDBS block header. Type field contains `0x43 0x52 0x45 0x41` ('CREA')
0x0014 | As specified in header | Creator, ASCII, no newlines.

### DPB block

The DPB block is optional, and contains a CP/M Plus Disk Parameter Block describing the filesystem.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 20 bytes | LDBS block header. Type field contains `0x44 0x50 0x42 0x20` ('DPB ')
0x0014 | 17 bytes | CP/M DPB

Rationale: CP/M filesystems are not self-describing. If the disk image was generated by an emulator or utility that is aware of the CP/M filesystem, it may be helpful to record the filesystem parameters used. This is also necessary for round-trip compatibility with the YAZE YDSK format.

### Geometry block

The geometry block is optional. It contains the drive geometry used by LibDsk when it last formatted a track in the disk image. On a 'straightforward' disk image where all tracks have the same format, this will give the geometry of the entire file.

Offset | Type | Meaning
--- | --- | ---
0x0000 | 20 bytes | LDBS block header. Type field contains `0x47 0x45 0x4F 0x4D` ('GEOM')

0x0014

Byte

Sidedness: the order in which to process tracks.

    0 => Single sided, or alternating sides. Track n is cylinder
                                 (n / heads) head (n % heads).

    1 => Out and back. Tracks go (head 0) 0,1,2,3,...,37,38,39 then
                                 (head 1) 39,38,37,...2,1,0

    2 => Out and out.  Tracks go (head 0) 0,1,2,3,...,37,38,39 then
                                 (head 1) 0,1,2,3,...,37,38,39

    3 => Extended surface.  As SIDES_ALT, but sectors on head 1 identify
                               as head 0, with numbers in sequence
                               eg: Head 0 has sectors 1-9, head 1 has 10-18

0x0015 | Word | Number of cylinders
0x0017 | Byte | Number of heads
0x0018 | Byte | Number of sectors per track
0x0019 | Byte | First physical sector number
0x001A | Word | Bytes per sector
0x001C | Byte | Data rate: 0 => 500 kbit/s, 1 => 300 kbit/s, 2 => 250 kbit/s, 3 => 1 mbit/s
0x001D | Byte | Read/write gap
0x001E | Byte | Format gap
0x001F | Byte | Recording mode: 0 => MFM, 1=> FM
0x0020 | Byte | Complement flag, 1 if bytes are stored complemented
0x0021 | Byte | Disable multitrack read/writes
0x0022 | Byte | Do not skip deleted data

Note that the representation of the data rate and recording mode isn't the same as in a track header.

Rationale: If the imaged disk contains no superblock or other means to specify its own format, this can be used to suggest a geometry for accessing it.

## Compression

The only form of compression is omitting sectors that are blank or unreadable (by setting the 'copies' field in the sector header to 0). Should it prove necessary to compress LDBS disk images further, this should be done using an external utility such as gzip (LibDsk can transparently decompress gzipped disk images).

## Reference Implementation

ldbs.h / ldbs.c in LibDsk 1.5.4 contain an X11-licensed reference implementation of LDBS. A few sample utilities are also supplied:

- dsk2ldbs: Converts DSK / EDSK disk images to LDBS.
- ldbs2dsk: Converts LDBS disk images to DSK / EDSK.
- ldbstest: Sanity test of block layer functions.
- ldbsdump: Displays the contents of an LDBS file.
- ldbscopy: Copies one LDBS file to another using the ldbs_clone() function. As currently written this happens to reverse the order of blocks in the file, so all block offsets will change.
- ldbs_v2: Upgrades a file in LDBS v0.1 or v0.2 format to the current specification.