## VIPS from Python

Using VIPS — How to use the VIPS library from Python

#### Introduction

VIPS comes with a convenient, high-level Python API built on on `gobject-introspection`. As long as you can get GOI for your platform, you should be able to use libvips.

To test the binding, start up Python and at the console enter:

    >>> from gi.repository import Vips
    >>> x = Vips.Image.new_from_file("/path/to/some/image/file.jpg")
    >>> x.width
    1450
    >>>

1. If import fails, check you have the Python gobject-introspection packages installed, that you have the libvips typelib installed, and that the typelib is either in the system area or on your `GI_TYPELIB_PATH`.

2. If `.new_from_file()` fails, the vips overrides have not been found. Make sure `Vips.py` is in your system overrides area.

#### Example program

Here's a complete example program:

    #!/usr/bin/python

    import sys

    from gi.repository import Vips

    im = Vips.Image.new_from_file(sys.argv[1])

    im = im.extract_area(100, 100, im.width - 200, im.height - 200)
    im = im.similarity(scale = 0.9)
    mask = Vips.Image.new_from_array([[-1, -1,  -1],
                                      [-1, 16,  -1],
                                      [-1, -1,  -1]], scale = 8)
    im = im.conv(mask)

    im.write_to_file(sys.argv[2])

Reading this code, the first interesting line is:

    from gi.repository import Vips

When Python executes the import line it performs the following steps:

1. It searches for a file called `Vips-x.y.typelib`. This is a binary file generated automatically during libvips build by introspection of the libvips shared library plus scanning of the C headers. It lists all the API entry points, all the types the library uses, and has an extra set of hints for object ownership and reference counting. The typelib is searched for in `/usr/lib/gi-repository-1.0` and along the path in the environment variable `GI_TYPELIB_PATH`.

2. It uses the typelib to make a basic binding for libvips. It's just the C API with a little very light mangling, so for example the enum member `VIPS_FORMAT_UCHAR` of the enum `VipsBandFormat` becomes `Vips.BandFormat.UCHAR`.

3. The binding you get can be rather unfriendly, so it also loads a set of overrides from `Vips.py` in `/usr/lib/python2.7/dist-packages/gi/overrides` (on my system at least). If you're using python3, it's `/usr/lib/python3/dist-packages/gi/overrides`. Unfortunately, as far as I know, there is no way to extend this search using environment variables. You MUST have `Vips.py` in exactly this directory. If you install vips via a package manager this will happen automatically, since vips and pygobject will have been built to the same prefix, but if you are installing vips from source and the prefix does not match, it will not be installed for you, you will need to copy it.

4. Finally, `Vips.py` makes the rest of the binding. In fact, `Vips.py` makes almost all the binding: it defines `__getattr__` on `Vips.Image` and binds at runtime by searching libvips for an operation of that name.

The next line is:

    im = Vips.Image.new_from_file(sys.argv[1])

This loads the input image. You can append load options to the argument list as keyword arguments, for example:

    im = Vips.Image.new_from_file(sys.argv[1], access = Vips.Access.SEQUENTIAL)

See the various loaders for a list of the available options for each file format. The C equivalent to this function, `vips_image_new_from_file()`, has more extensive documentation. Try `help(Vips.Image)` to see a list of all the image constructors --- you can load from memory, or create from an array, for example.

The next line is:

    im = im.extract_area(100, 100, im.width - 200, im.height - 200)

The arguments are left, top, width, height, so this crops 100 pixels off every edge. Try `help(im.extract_area)` and the C API docs for `vips_extract_area()` for details. You can use `.crop()` as a synonym, if you like.

`im.width` gets the image width in pixels, see `help(Vips.Image)` and `vips_image_get_width()` and friends for a list of the other getters.

The next line:

    im = im.similarity(scale = 0.9)

shrinks by 10%. By default it uses bilinear interpolation, use `interpolate` to pick another interpolator, for example:

    im = im.similarity(scale = 0.9, interpolate = Vips.Interpolate.new("bicubic"))

see `vips_similarity()` for full documentation. The similarity operator will not give good results for large resizes (more than a factor of two). See `vips_resize()` if you need to make a large change.

Next:

    mask = Vips.Image.new_from_array([[-1, -1,  -1],
                                      [-1, 16,  -1],
                                      [-1, -1,  -1]], scale = 8)
    im = im.conv(mask)

makes an image from a 2D array, then convolves with that. The `scale` keyword argument lets you set a divisor for convolution, handy for integer convolutions. You can set `offset` as well. See `vips_conv()` for details on the vips convolution operator.

Finally,

    im.write_to_file(sys.argv[2])

sends the image back to the filesystem. There's also `.write_to_buffer()` to make a string containing the formatted image, and `.write()` to write to another image.

As with `.new_from_file()` you can append save options as keyword arguments. For example:

    im.write_to_file("test.jpg", Q = 90)

will write a JPEG image with quality set to 90. See the various save operations for a list of all the save options, for example `vips_jpegsave()`.

#### Getting help

Try `help(Vips)` for everything, `help(Vips.Image)` for something slightly more digestible, or something like `help(Vips.Image.black)` for help on a specific class member.

You can't get help on dynamically bound member functions like `.add()` this way. Instead, make an image and get help from that, for example:

    image = Vips.Image.black(1, 1)
    help(image.add)

And you'll get a summary of the operator's behaviour and how the arguments are represented in Python.

The API docs have a handy table of all vips operations, if you want to find out how to do something, try searching that.

The vips command can be useful too. For example, in a terminal you can type vips jpegsave to get a summary of an operation:

    $ vips jpegsave
    save image to jpeg file
    usage:
       jpegsave in filename
    where:
       in           - Image to save, input VipsImage
       filename     - Filename to save to, input gchararray
    optional arguments:
       Q            - Q factor, input gint
                          default: 75
                          min: 1, max: 100
       profile      - ICC profile to embed, input gchararray
       optimize-coding - Compute optimal Huffman coding tables, input gboolean
                          default: false
       interlace    - Generate an interlaced (progressive) jpeg, input gboolean
                          default: false
       no-subsample - Disable chroma subsample, input gboolean
                          default: false
       trellis-quant - Apply trellis quantisation to each 8x8 block, input gboolean
                          default: false
       overshoot-deringing - Apply overshooting to samples with extreme values, input gboolean
                          default: false
       optimize-scans - Split the spectrum of DCT coefficients into separate scans, input gboolean
                          default: false
       strip        - Strip all metadata from image, input gboolean
                          default: false
       background   - Background value, input VipsArrayDouble
    operation flags: sequential-unbuffered nocache

#### `pyvips8` basics

As noted above, the Python interface comes in two main parts, an automatically generated binding based on the vips typelib, plus a set of extra features provided by overrides. The rest of this chapter runs through the features provided by the overrides.

#### Automatic wrapping

The overrides intercept member lookup on the `Vips.Image` class and look for vips operations with that name. So the vips operation "add", which appears in the C API as `vips_add()`, appears in Python as `image.add()`.

The first input image argument becomes the `self` argument. If there are no input image arguments, the operation appears as a class member. Optional input arguments become keyword arguments. The result is a list of all the output arguments, or a single output if there is only one.

Optional output arguments are enabled with a boolean keyword argument of that name. For example, "min" (the operation which appears in the C API as `vips_min()`), can be called like this:

    min_value = im.min()

and `min_value` will be a floating point value giving the minimum value in the image. "min" can also find the position of the minimum value with the `x` and `y` optional output arguments. Call it like this:

    min_value, opts = im.min(x = True, y = True)
    x = opts['x']
    y = opts['y']

In other words, if optional output args are requested, an extra dictionary is returned containing those objects. Of course in this case, the `.minpos()` convenience function would be simpler, see below.

Because operations are member functions and return the result image, you can chain them. For example, you can write:

    result_image = image.sin().pow(2)

to calculate the square of the sine for each pixel. There is also a full set of arithmetic operator overloads, see below.

VIPS types are also automatically wrapped. The override looks at the type of argument required by the operation and converts the value you supply, when it can. For example, "linear" takes a VipsArrayDouble as an argument for the set of constants to use for multiplication. You can supply this value as an integer, a float, or some kind of compound object and it will be converted for you. You can write:

    result_image = image.linear(1, 3)
    result_image = image.linear(12.4, 13.9)
    result_image = image.linear([1, 2, 3], [4, 5, 6])
    result_image = image.linear(1, [4, 5, 6])

And so on. A set of overloads are defined for `.linear()`, see below.

It does a couple of more ambitious conversions. It will automatically convert to and from the various vips types, like VipsBlob and VipsArrayImage. For example, you can read the ICC profile out of an image like this:

    profile = im.get_value("icc-profile-data")

and `profile` will be a string.

You can use array constants instead of images. A 2D array is simply changed into a one-band double image. This is handy for things like `.erode()`, for example:

    im = im.erode([[128, 255, 128],
                   [255, 255, 255],
                   [128, 255, 128]])

will erode an image with a 4-connected structuring element.

If an operation takes several input images, you can use a 1D array constant or a number constant for all but one of them and the wrapper will expand it to an image for you. For example, `.ifthenelse()` uses a condition image to pick pixels between a then and an else image:

    result_image = condition_image.ifthenelse(then_image, else_image)

You can use a constant instead of either the then or the else parts, and it will be expanded to an image for you. If you use a constant for both then and else, it will be expanded to match the condition image. For example:

    result_image = condition_image.ifthenelse([0, 255, 0], [255, 0, 0])

Will make an image where true pixels are green and false pixels are red.

This is also useful for `.bandjoin()`, the thing to join two or more images up bandwise. You can write:

    rgba = rgb.bandjoin(255)

to add a constant 255 band to an image, perhaps to add an alpha channel. Of course you can also write:

    result_image = image1.bandjoin(image2)
    result_image = image1.bandjoin([image2, image3])
    result_image = image1.bandjoin([image2, 255])

and so on.

#### Exceptions

The wrapper spots errors from vips operations and raises the `Vips.Error` exception. You can catch it in the usual way. The `.detail` member gives the detailed error message.

#### Reading and writing areas of memory

You can use the C API functions `vips_image_new_from_memory()` and `vips_image_write_to_memory()` directly from Python to read and write areas of memory. This can be useful if you need to get images to and from other other image processing libraries, like PIL or numpy.

Use them from Python like this:

    image = Vips.Image.new_from_file("/path/to/some/image/file.jpg")
    memory_area = image.write_to_memory()

`memory_area` is now a string containing uncompressed binary image data. For an RGB image, it will have bytes `RGBRGBRGB...`, being the first three pixels of the first scanline of the image. You can pass this string to the numpy or PIL constructors and make an image there.

Note that `.write_to_memory()` will make a copy of the image. It would be better to use a Python buffer to pass the data, but sadly this isn't possible with gobject-introspection, as far as I know.

Going the other way, you can construct a vips image from a string of binary data. For example:

    image = Vips.Image.new_from_file("/path/to/some/image/file.jpg")
    memory_area = image.write_to_memory()
    image2 = Vips.Image.new_from_memory(memory_area,
                                        image.width, image.height, image.bands,
                                        Vips.BandFormat.UCHAR)

Now `image2` should be an identical copy of `image`.

Be careful: in this direction, vips does not make a copy of the memory area, so if `memory_area` is freed by the Python garbage collector and you later try to use `image2`, you'll get a crash. Make sure you keep a reference to `memory_area` around for as long as you need it.

#### Draw operations

Paint operations like `draw_circle` and `draw_line` modify their input image. This makes them hard to use with the rest of libvips: you need to be very careful about the order in which operations execute or you can get nasty crashes.

The wrapper spots operations of this type and makes a private copy of the image in memory before calling the operation. This stops crashes, but it does make it inefficient. If you draw 100 lines on an image, for example, you'll copy the image 100 times. The wrapper does make sure that memory is recycled where possible, so you won't have 100 copies in memory. At least you can execute these operations.

If you want to avoid the copies, you'll need to call drawing operations yourself.

#### Overloads

The wrapper defines the usual set of arithmetic, boolean and relational overloads on `image`. You can mix images, constants and lists of constants (almost) freely. For example, you can write:

    result_image = ((image * [1, 2, 3]).abs() < 128) | 4

The wrapper overloads `[]` to be `vips_extract_band()`. You can write:

    result_image = image[2]

to extract the third band of the image. It implements the usual slicing and negative indexes, so you can write:

    result_image = image[1:]
    result_image = image[:3]
    result_image = image[-2:]
    result_image = [x.avg() for x in image]

and so on.

The wrapper overloads `()` to be `vips_getpoint()`. You can write:

    r, g, b = image(10, 10)

to read out the value of the pixel at coordinates (10, 10) from an RGB image.

#### Expansions

Some vips operators take an enum to select an action, for example `.math()` can be used to calculate sine of every pixel like this:

    result_image = image.math(Vips.OperationMath.SIN)

This is annoying, so the wrapper expands all these enums into separate members named after the enum. So you can write:

    result_image = image.sin()

See `help(Vips.Image)` for a list.

#### Convenience functions

The wrapper defines a few extra useful utility functions: `.get_value()`, `.set_value()`, `.bandsplit()`, `.maxpos()`, `.minpos()`, `.median()`. Again, see `help(Vips.Image)` for a list.

#### Command-line option parsing

GLib includes a command-line option parser, and Vips defines a set of standard flags you can use with it. For example:

    import sys
    from gi.repository import GLib, Vips

    context = GLib.OptionContext(" - test stuff")
    main_group = GLib.OptionGroup("main",
                                  "Main options", "Main options for this program",
                                  None)
    context.set_main_group(main_group)
    Vips.add_option_entries(main_group)
    context.parse(sys.argv)