# NiBabel

### Access a cacophony of neuro-imaging file formats

# BIAP7 - Loading multiple images

Author:

  Matthew Brett

Status:

  Draft

Type:

  Standards

Created:

  2015-07-18

## Background

### Some formats store images with different shapes in the same file

The ECAT file format can contain more than one type of image in a single image file.

ECAT can store many frames in a single image file. Each frame has its own subheader. The subheader specifies the 3D image size; each frame can therefore have a different image size.

We currently raise an error if you try and load an ECAT file where the frames do not have the same 3D dimensions.

It would be better if we could allow loading multiple images with different image dimensions, from a single ECAT file.

Vista data format and Lipsia format are other formats that allow saving multiple images with different image dimensions in the same file. We don’t currently support Lipsia or Vista formats and it is not clear how we would do that with the current `load` API.

We have had some discussion about saving multiple images into a single HDF5 file - see https://github.com/nipy/nibabel/pull/215#issuecomment-122357444

### It can be useful to load 4D images as multiple 3D images

We sometimes want to load a 4D image as multiple 3D images.

When we are doing motion correction, we often want to split up a 4D image into separate 3D images.

Motion estimation results in different affines for each volume in the 4D time series. At the moment we have no API for returning these affines with a 4D image. One way of doing that is to load the 4D image and affines as a sequence of 3D images, each with their own affine.

We currently have a proposal open for a JSON header extension that can store these 4D affines for a 4D NIfTI file.

SPM saves the affines in an associated `.mat` file, with one affine per volume in the 4D image.

## Options

## Return an image sequence from `load` for some file formats

We don’t currently load ECAT files from the top-level `nibabel.load` function.

We do have `nibabel.ecat.load`, which raises an error for an ECAT file having frames with different image dimensions.

We could therefore choose to return a sequence of images from `nibabel.load` on an ECAT file, with one element per frame in the ECAT file.

Most ECAT images are 4D images, in the sense that the frames in the file do all have the same image dimensions and data type, so this might be cumbersome as a default.

We would have to work out how to deal with `nibabel.ecat.load`.

The same principles apply to the Lipsia / Vista formats, except we have no backward-compatibility problems, and it seems to be more common for these formats to mix image types in a single file.

## Add a `load_multi` top-level function

`nibabel.load_multi` always returns an image sequence.

`nibabel.load` always returns a single image.

`nibabel.load` on ECAT (etc) files could first do `load_multi`, then check the resulting image dimensions, raising an error if incompatible, concatenating otherwise.

`load_multi` on current formats like NIfTI could return one image per volume, where each volume might have its own affine, as loaded from the JSON header extension or the SPM `.mat` file.

## Next steps:

- Make sure there are use-cases where you could wish to call load vs. load_multi on the same image (perhaps a Nifti image with different affines for each volume)

- Investigate AFNI file formats as a use-case for this.

- Check the nilearn codebase, see if iter_img and slice_img functions might offer a post-load alternative. Also check if those functions could be deprecated in favor of slicing / iterating on dataobj

- Create a new issue to implement getting an iterator on dataobj?