# NiBabel

### Access a cacophony of neuro-imaging file formats

# BIAP1 - Towards immutable images

Author:

  Matthew Brett

Status:

  Rejected

Type:

  Standards

Created:

  2011-03-23

## Resolution

Retired as of nibabel 2.0 in favor of exposed dataobj property. See:

- http://nipy.org/nibabel/nibabel_images.html#the-image-data-array

- http://nipy.org/nibabel/images_and_memory.html

See image in_memory attribute and uncache method.

We haven’t implemented an is_as_loaded attribute yet.

## Background

Nibabel implicitly has two types of images

- array images

- proxy images

### Array images

Array images are the images you get from a typical constructor call:

    import numpy as np
    import nibabel as nib
    arr = np.arange(24).reshape((2,3,4))
    img = nib.Nifti1Image(arr, np.eye(4))

`img` here is an array image, that is to say that, internally, the private `img._data` attribute is reference to `arr` above. `img.get_data()` just returns `img._data`. If you modify `arr`, you will modify the result of `img.get_data()`.

### Proxy images

Proxy images are what you get from a call to `load`:

    px_img = nib.load('test.nii')

It’s a proxy image in the sense that, internally, `px_arr._data` is a proxy object that does not yet contain an array, but can get an array by the application of:

    actual_arr = np.asarray(px_img._data)

This is in fact what `px_img.get_data()` does. Actually, `px_img.get_data()` also stores the read array in `px_img._data`, so that:

    px_img = nib.load('test.nii')
    assert not isinstance(px_img._data, np.ndarray) # it's a proxy
    actual_arr = px_img.get_data()
    assert isinstance(px_img._data, np.ndarray) # it's an array now

So, at this point, if you change `actual_arr` you’ll also be changing `px_img._data` and therefore the result of `px_img.get_data()`.

In other words, `actual_arr = px_img.get_data()` turns the proxy image into an array image.

### Issues for design

The code at the moment is a little bit confusing because:

- there isn’t an explicit API to check if you have an array image or a proxy image and

- there isn’t anywhere in the docs that you can go and see this distinction.

### Use cases

#### Loading images, minimizing memory

I want to load lots of images, or several large images. I’m going to do something with the image data. I want to minimize memory use. This tempts me to do something like this:

    large_img1 = nib.load('large1.nii')
    large_img2 = nib.load('large2.nii')
    li1_mean = large_img1.get_data().mean()
    li2_mean = large_img2.get_data().mean()

The problem with the current design is that, after the `li1_mean =` line, `large_img1` got unproxied, and there’s a huge array inside it.

#### Loading images, maximizing speed

On the other hand, I might want to do the same thing, but each call to unproxy the data (loading off disk, applying scalefactors) will be expensive. So, when I do `li1_mean = large_img1.get_data().mean()` I want any subsequent call to to `large_img1.get_data()` to be much faster. This is the case at the moment, at the expense of the memory hit above.

#### Loading images, assert not modified

In pipelines in particular, we frequently want to load images, maybe have a look at some parameters, and then pass that image filename to some other program such as SPM or FSL. At the moment we’ve got a problem:

    img = nib.load('test.nii')
    # do stuff
    run_spm_thing_on(img) # is 'img' the same as test.nii?

The problem is that when the routine `run_spn_thing` receives `img`, it can know that `img` has a filename, `test.nii`, but it can’t currently know if `img` is the same object that it was when it was loaded. That is, it can’t know whether `test.img` still corresponds to `img` or not. In practice that means that `run_spm_thing` will need to save every `img` to another file before passing that filename to the SPM routine, just in case `img` has been modified. So, we would like a dirty bit for the image, something like:

    # Not implemented yet
    if not img.is_as_loaded():
        save(img, 'some_filename.nii')

The last line, like it or not, modifies `img` in-place.

### Array images, proxy images, copy, view

With thanks to Roberto Viviani for some clarifying thoughts on the nipy mailing list.

At the moment, `img.get_data()` always returns a reference to an array. That is, whenever you call:

    data = img.get_data()

Then, if you modify `data` you will modify the next result of `img.get_data()`.

In particular, the interface currently intends that there should be no functional difference between proxied images and non-proxied images. The proposal below exposes a functional difference between them.

#### When do you want a copy and when do you want a view?

This is a discussion of this proposal:

    img.get_data(copy=True|False)

compared to:

    img.get_data(unproxy=True|False)

Summary:

- array images - you nearly always want a view

- proxy images - you may want a copy, but you want a copy only because you want to leave the image as a proxy. You might want to leave the image as a proxy because you want to be sure the image corresponds to the file, or save memory.

For array images, it doesn’t make sense to return a copy from `img.get_data()`, because it buys you nothing that you would not get from `data = img.get_data().copy()`. This is because you can’t save memory (the image already contains the whole array), and it won’t help you be sure that the image has not been modified compared to the original array, because there may be references to the array that existed before the image was made, that can be used to modify the data. So, for array images, you always want a reference, or you want to do a manual copy, as above.

For proxied images, it does make sense to get a copy, because a) you want to preserve memory by not unproxying the image, and / or b) you want to be able to be sure that the file associated with the image still corresponds to the data.

For the `img.get_data(copy=False)` proposal, on a proxied image, the `copy=False` call, in order to return a view, must implicitly unproxy the image.

Similarly, `img.get_data(unproxy=False)` must implicitly copy the image.

It seems to me (MB) that an implicit copy is familiar to a numpy user, but the implicit unproxying may be less obvious.

My (MBs) reasons then for preferring ‘unproxy’ to ‘copy=True’ or ‘copy=False’ or get_data_copy() is that ‘unproxy’ is closer to how I think the user would think about deciding what they wanted to do.

The `unproxy=False` case covers the situation where you want to preserve memory. It doesn’t fully cover the cases where we want to keep track of when the image data has been modified.

Here there are three cases:

- array image, instantiated with an array; the image data can be modified using the array reference passed into the image - we can’t know whether the data has been modified without doing hashing or similar.

- proxy image; the array data is still in the file, so we know it corresponds to the file.

- proxy images that have been converted to array images, but have not passed out a reference to the data. Let’s call these shy unproxied images. For example, with an API like this:

      img = load('test.nii')
      data = img.get_data(copy=True)

  the `img` is now an array image, but there’s no public reference to the internal array object. Someone could get one by cheating with `ref = img._data`, but, we don’t need to worry about that - following Python’s “mess around if you like but take the consequences” philosophy.

### Proposal

An `is_proxy` property:

    img.is_proxy

This is just for clarity.

Allow the user to specify what unproxying they want with a kwarg to `get_data()`:

    arr = large_img1.get_data(unproxy=False)

- for proxied images, `unproxy=False` would leave the underlying array data as a pointer to the file. The returned `arr` would be therefore a copy of the data as loaded from file, and `arr[0] = 99` would have no effect on the data in the image. `unproxy=True` would convert the proxy image into an array image (load the data into memory, return reference). Here `arr[0] = 99` would affect the data in the image

- for array images, `unproxy` would always be ignored.

Thus `unproxy=True` in fact means, `unproxy_if_this_is_a_proxy_do_nothing_otherwise`.

The default would continue to be `unproxy=True` so that the proxied image would continue, by default, to behave the same way as an unproxied image (`get_data` returns a view).

If `img.is_proxy` is True, then we know that the array data has not changed. We then need to be sure that the `header` and `affine` data haven’t changed. We might be able to do this with default `copy` kwargs to the `get_header` and `get_affine` methods:

    hdr = img.get_header(copy=True) # will be default
    aff = img.get_affine(copy=True) # will be default

We could also do that by caching the original header and affine, but the header in particular can be rather large.

For the next version of nibabel, for backwards compatibility, we’ll set `copy=False` to be the default, but warn about the upcoming change. After that we’ll set `copy=True` as the default.

Now we can know whether the image has been modified, because if `get_header` and `get_affine` have only been called with `copy=True` and `img.is_proxy == True` - then it must be the same as when loaded.

This leads to an `is_as_loaded` property:

    if img.is_as_loaded:
        fname = img.get_filename()
    else:
        fname = 'tempname.nii'
        save(img, 'tempname.nii')

### Questions

Should there also be a `set_header` and `set_affine` method?

The header may conflict with the affine. So, would we need something like:

    img.set_header(hdr, hdr_affine_from='affine')

or some other nasty syntax. Or can we avoid this and just do:

    img2 = nib.Nifti1Image(img.get_data(), new_affine, new_header)

?

How about the names in the proposal? `is_proxy`; `unproxy=True`?