osmium::memory::Buffer Class Reference

`#include <buffer.hpp>`

## Public Types

enum | auto_grow : bool { auto_grow::yes = true, auto_grow::no = false }
using | value_type = Item
template<typename T >
using | t_iterator = osmium::memory::ItemIterator< T >
template<typename T >
using | t_const_iterator = osmium::memory::ItemIterator< const T >
using | iterator = t_iterator< osmium::OSMEntity >
using | const_iterator = t_const_iterator< osmium::OSMEntity >

## Public Member Functions

| Buffer () noexcept
| Buffer (unsigned char *data, size_t size)
| Buffer (unsigned char *data, size_t capacity, size_t committed)
| Buffer (size_t capacity, auto_grow auto_grow=auto_grow::yes)
| Buffer (const Buffer &)=delete
Buffer & | operator= (const Buffer &)=delete
| Buffer (Buffer &&)=default
Buffer & | operator= (Buffer &&)=default
| ~Buffer ()=default
unsigned char * | data () const noexcept
size_t | capacity () const noexcept
size_t | committed () const noexcept
size_t | written () const noexcept
bool | is_aligned () const noexcept
OSMIUM_DEPRECATED void | set_full_callback (std::function< void(Buffer &)> full)
void | grow (size_t size)
size_t | commit ()
void | rollback ()
size_t | clear ()
template<typename T >
T & | get (const size_t offset) const
unsigned char * | reserve_space (const size_t size)
template<typename T >
T & | add_item (const T &item)
void | add_buffer (const Buffer &buffer)
void | push_back (const osmium::memory::Item &item)
template<typename T >
t_iterator< T > | begin ()
iterator | begin ()
template<typename T >
t_iterator< T > | get_iterator (size_t offset)
iterator | get_iterator (size_t offset)
template<typename T >
t_iterator< T > | end ()
iterator | end ()
template<typename T >
t_const_iterator< T > | cbegin () const
const_iterator | cbegin () const
template<typename T >
t_const_iterator< T > | get_iterator (size_t offset) const
const_iterator | get_iterator (size_t offset) const
template<typename T >
t_const_iterator< T > | cend () const
const_iterator | cend () const
template<typename T >
t_const_iterator< T > | begin () const
const_iterator | begin () const
template<typename T >
t_const_iterator< T > | end () const
const_iterator | end () const
| operator bool () const noexcept
void | swap (Buffer &other)
template<typename TCallbackClass >
void | purge_removed (TCallbackClass *callback)

## Private Attributes

std::unique_ptr< unsigned char[]> | m_memory
unsigned char * | m_data
size_t | m_capacity
size_t | m_written
size_t | m_committed
auto_grow | m_auto_grow {auto_grow::no}
std::function< void(Buffer &)> | m_full

## Detailed Description

A memory area for storing OSM objects and other items. Each item stored has a type and a length. See the Item class for details.

Data can be added to a buffer piece by piece using reserve_space() and add_item(). After all data that together forms an item is added, it must be committed using the commit() call. Usually this is done through the Builder class and its derived classes.

You can iterate over all items in a buffer using the iterators returned by begin(), end(), cbegin(), and cend().

Buffers exist in two flavours, those with external memory management and those with internal memory management. If you already have some memory with data in it (for instance read from disk), you create a Buffer with external memory management. It is your job then to free the memory once the buffer isn't used any more. If you don't have memory already, you can create a Buffer object and have it manage the memory internally. It will dynamically allocate memory and free it again after use.

By default, if a buffer gets full it will throw a buffer_is_full exception. You can use the set_full_callback() method to set a callback functor which will be called instead of throwing an exception. The full callback functionality is deprecated and will be removed in the future. See the documentation for set_full_callback() for alternatives.

## Member Typedef Documentation

using osmium::memory::Buffer::const_iterator = t_const_iterator<osmium::OSMEntity>

A const iterator that can be used to iterate over all OSMEntity objects in a buffer.

using osmium::memory::Buffer::iterator = t_iterator<osmium::OSMEntity>

An iterator that can be used to iterate over all OSMEntity objects in a buffer.

template<typename T >

using osmium::memory::Buffer::t_const_iterator = osmium::memory::ItemIterator<const T>

A const iterator that can be used to iterate over all items of type T in a buffer.

template<typename T >

using osmium::memory::Buffer::t_iterator = osmium::memory::ItemIterator<T>

An iterator that can be used to iterate over all items of type T in a buffer.

using osmium::memory::Buffer::value_type = Item

## Member Enumeration Documentation

enum osmium::memory::Buffer::auto_grow : bool

strong

Enumerator
yes |
no |

## Constructor & Destructor Documentation

osmium::memory::Buffer::Buffer | ( |  | ) |

inline noexcept

The constructor without any parameters creates an invalid, buffer, ie an empty hull of a buffer that has no actual memory associated with it. It can be used to signify end-of-data.

Most methods of the Buffer class will not work with an invalid buffer.

osmium::memory::Buffer::Buffer | ( | unsigned char * | data,
|  | size_t | size
| ) |  |

inline explicit

Constructs a valid externally memory-managed buffer using the given memory and size.

Parameters

  data | A pointer to some already initialized data.
  size | The size of the initialized data.

Exceptions

  std::invalid_argument | if the size isn't a multiple of the alignment.

osmium::memory::Buffer::Buffer | ( | unsigned char * | data,
|  | size_t | capacity,
|  | size_t | committed
| ) |  |

inline explicit

Constructs a valid externally memory-managed buffer with the given capacity that already contains 'committed' bytes of data.

Parameters

  data | A pointer to some (possibly initialized) data.
  capacity | The size of the memory for this buffer.
  committed | The size of the initialized data. If this is 0, the buffer startes out empty.

Exceptions

  std::invalid_argument | if the capacity or committed isn't a multiple of the alignment.

osmium::memory::Buffer::Buffer | ( | size_t | capacity,
|  | auto_grow | auto_grow = `auto_grow::yes`
| ) |  |

inline explicit

Constructs a valid internally memory-managed buffer with the given capacity. Will internally get dynamic memory of the required size. The dynamic memory will be automatically freed when the Buffer is destroyed.

Parameters

  capacity | The (initial) size of the memory for this buffer.
  auto_grow | Should this buffer automatically grow when it becomes to small?

Exceptions

  std::invalid_argument | if the capacity isn't a multiple of the alignment.

osmium::memory::Buffer::Buffer | ( | const Buffer & |  | ) |

delete

osmium::memory::Buffer::Buffer | ( | Buffer && |  | ) |

default

osmium::memory::Buffer::~Buffer | ( |  | ) |

default

## Member Function Documentation

void osmium::memory::Buffer::add_buffer | ( | const Buffer & | buffer | ) |

inline

Add committed contents of the given buffer to this buffer.

Precondition
  The buffer must be valid.

Note that you have to eventually call commit() to actually commit this data.

Parameters

  buffer | The source of the copy. Must be valid.

template<typename T >

T& osmium::memory::Buffer::add_item | ( | const T & | item | ) |

inline

Add an item to the buffer. The size of the item is stored inside the item, so we know how much memory to copy.

Note that you have to eventually call commit() to actually commit this data.

Precondition
  The buffer must be valid.

Template Parameters

  T | Class of the item to be copied.

Parameters

  item | Reference to the item to be copied.

Returns
  Reference to newly copied data in the buffer.

template<typename T >

t_iterator<T> osmium::memory::Buffer::begin | ( |  | ) |

inline

Get iterator for iterating over all items of type T in the buffer.

Precondition
  The buffer must be valid.

Returns
  Iterator to first item of type T in the buffer.

iterator osmium::memory::Buffer::begin | ( |  | ) |

inline

Get iterator for iterating over all objects of class OSMEntity in the buffer.

Precondition
  The buffer must be valid.

Returns
  Iterator to first OSMEntity in the buffer.

template<typename T >

t_const_iterator<T> osmium::memory::Buffer::begin | ( |  | ) | const

inline

const_iterator osmium::memory::Buffer::begin | ( |  | ) | const

inline

size_t osmium::memory::Buffer::capacity | ( |  | ) | const

inline noexcept

Returns the capacity of the buffer, ie how many bytes it can contain. Always returns 0 on invalid buffers.

template<typename T >

t_const_iterator<T> osmium::memory::Buffer::cbegin | ( |  | ) | const

inline

const_iterator osmium::memory::Buffer::cbegin | ( |  | ) | const

inline

template<typename T >

t_const_iterator<T> osmium::memory::Buffer::cend | ( |  | ) | const

inline

const_iterator osmium::memory::Buffer::cend | ( |  | ) | const

inline

size_t osmium::memory::Buffer::clear | ( |  | ) |

inline

Clear the buffer.

No-op on an invalid buffer.

Returns
  Number of bytes in the buffer before it was cleared.

size_t osmium::memory::Buffer::commit | ( |  | ) |

inline

Mark currently written bytes in the buffer as committed.

Precondition
  The buffer must be valid and aligned properly (as indicated by is_aligned().

Returns
  Number of committed bytes before this commit. Can be used as an offset into the buffer to get to the object being committed by this call.

size_t osmium::memory::Buffer::committed | ( |  | ) | const

inline noexcept

Returns the number of bytes already filled in this buffer. Always returns 0 on invalid buffers.

unsigned char* osmium::memory::Buffer::data | ( |  | ) | const

inline noexcept

Return a pointer to data inside the buffer.

Precondition
  The buffer must be valid.

template<typename T >

t_iterator<T> osmium::memory::Buffer::end | ( |  | ) |

inline

Get iterator for iterating over all items of type T in the buffer.

Precondition
  The buffer must be valid.

Returns
  End iterator.

iterator osmium::memory::Buffer::end | ( |  | ) |

inline

Get iterator for iterating over all objects of class OSMEntity in the buffer.

Precondition
  The buffer must be valid.

Returns
  End iterator.

template<typename T >

t_const_iterator<T> osmium::memory::Buffer::end | ( |  | ) | const

inline

const_iterator osmium::memory::Buffer::end | ( |  | ) | const

inline

template<typename T >

T& osmium::memory::Buffer::get | ( | const size_t | offset | ) | const

inline

Get the data in the buffer at the given offset.

Precondition
  The buffer must be valid.

Template Parameters

  T | Type we want to the data to be interpreted as.

Returns
  Reference of given type pointing to the data in the buffer.

template<typename T >

t_iterator<T> osmium::memory::Buffer::get_iterator | ( | size_t | offset | ) |

inline

Get iterator for iterating over all items of type T in the buffer.

Precondition
  The buffer must be valid.

Returns
  Iterator to first item of type T after given offset in the buffer.

iterator osmium::memory::Buffer::get_iterator | ( | size_t | offset | ) |

inline

Get iterator for iterating over all objects of class OSMEntity in the buffer.

Precondition
  The buffer must be valid.

Returns
  Iterator to first OSMEntity after given offset in the buffer.

template<typename T >

t_const_iterator<T> osmium::memory::Buffer::get_iterator | ( | size_t | offset | ) | const

inline

const_iterator osmium::memory::Buffer::get_iterator | ( | size_t | offset | ) | const

inline

void osmium::memory::Buffer::grow | ( | size_t | size | ) |

inline

Grow capacity of this buffer to the given size. This works only with internally memory-managed buffers. If the given size is not larger than the current capacity, nothing is done. Already written but not committed data is discarded.

Precondition
  The buffer must be valid.

Parameters

  size | New capacity.

Exceptions

  std::logic_error | if the buffer doesn't use internal memory management.
  std::invalid_argument | if the size isn't a multiple of the alignment.
  std::bad_alloc | if there isn't enough memory available.

bool osmium::memory::Buffer::is_aligned | ( |  | ) | const

inline noexcept

This tests if the current state of the buffer is aligned properly. Can be used for asserts.

Precondition
  The buffer must be valid.

osmium::memory::Buffer::operator bool | ( |  | ) | const

inline explicit noexcept

In a bool context any valid buffer is true.

Buffer& osmium::memory::Buffer::operator= | ( | const Buffer & |  | ) |

delete

Buffer& osmium::memory::Buffer::operator= | ( | Buffer && |  | ) |

default

template<typename TCallbackClass >

void osmium::memory::Buffer::purge_removed | ( | TCallbackClass * | callback | ) |

inline

Purge removed items from the buffer. This is done by moving all non-removed items forward in the buffer overwriting removed items and then correcting the m_written and m_committed numbers.

Note that calling this function invalidates all iterators on this buffer and all offsets in this buffer.

For every non-removed item that moves its position, the function 'moving_in_buffer' is called on the given callback object with the old and new offsets in the buffer where the object used to be and is now, respectively. This call can be used to update any indexes.

Precondition
  The buffer must be valid.

void osmium::memory::Buffer::push_back | ( | const osmium::memory::Item & | item | ) |

inline

Add an item to the buffer. This function is provided so that you can use std::back_inserter.

Precondition
  The buffer must be valid.

Parameters

  item | The item to be added.

unsigned char* osmium::memory::Buffer::reserve_space | ( | const size_t | size | ) |

inline

Reserve space of given size in buffer and return pointer to it. This is the only way of adding data to the buffer. You reserve the space and then fill it.

Note that you have to eventually call commit() to actually commit this data.

If there isn't enough space in the buffer, one of three things can happen:

- If you have set a callback with set_full_callback(), it is called. After the call returns, you must have either grown the buffer or cleared it by calling buffer.clear(). (Usage of the full callback is deprecated and this functionality will be removed in the future. See the documentation for set_full_callback() for alternatives.
- If no callback is defined and this buffer uses internal memory management, the buffers capacity is grown, so that the new data will fit.
- Else the buffer_is_full exception is thrown.

Precondition
  The buffer must be valid.

Parameters

  size | Number of bytes to reserve.

Returns
  Pointer to reserved space. Note that this pointer is only guaranteed to be valid until the next call to reserve_space().

Exceptions

  osmium::buffer_is_full | if the buffer is full there is no callback defined and the buffer isn't auto-growing.

void osmium::memory::Buffer::rollback | ( |  | ) |

inline

Roll back changes in buffer to last committed state.

Precondition
  The buffer must be valid.

OSMIUM_DEPRECATED void osmium::memory::Buffer::set_full_callback | ( | std::function< void(Buffer &)> | full | ) |

inline

Set functor to be called whenever the buffer is full instead of throwing buffer_is_full.

The behaviour is undefined if you call this on an invalid buffer.

Precondition
  The buffer must be valid.

Deprecated:
  Callback functionality will be removed in the future. Either detect the buffer_is_full exception or use a buffer with auto_grow::yes. If you want to avoid growing buffers, check that the used size of the buffer (committed()) is small enough compared to the capacity (for instance small than 90% of the capacity) before adding anything to the Buffer. If the buffer is initialized with auto_grow::yes, it will still grow in the rare case that a very large object will be added taking more than the difference between committed() and capacity().

void osmium::memory::Buffer::swap | ( | Buffer & | other | ) |

inline

size_t osmium::memory::Buffer::written | ( |  | ) | const

inline noexcept

Returns the number of bytes currently filled in this buffer that are not yet committed. Always returns 0 on invalid buffers.

## Member Data Documentation

auto_grow osmium::memory::Buffer::m_auto_grow {auto_grow::no}

private

size_t osmium::memory::Buffer::m_capacity

private

size_t osmium::memory::Buffer::m_committed

private

unsigned char* osmium::memory::Buffer::m_data

private

std::function<void(Buffer&)> osmium::memory::Buffer::m_full

private

std::unique_ptr<unsigned char[]> osmium::memory::Buffer::m_memory

private

size_t osmium::memory::Buffer::m_written

private

The documentation for this class was generated from the following file:

- osmium/memory/buffer.hpp