Loading...
Searching...
No Matches
casacore::ISMBucket Class Reference
tables package (libcasa_tables) » DataMan_module » DataMan_module_internal_classes

More...

`#include <ISMBucket.h>`

## Public Member Functions

| ISMBucket (ISMBase *parent, const char *bucketStorage)
| Create a bucket with the given parent.
| ~ISMBucket ()
| ISMBucket (const ISMBucket &)=delete
| Forbid copy constructor.
ISMBucket & | operator= (const ISMBucket &)=delete
| Forbid assignment.
uInt | getInterval (uInt colnr, rownr_t rownr, rownr_t bucketNrrow, rownr_t &start, rownr_t &end, uInt &offset) const
| Get the row-interval for given column and row.
Bool | canAddData (uInt leng) const
| Is the bucket large enough to add a value?
void | addData (uInt colnr, rownr_t rownr, uInt index, const char *data, uInt leng)
| Add the data to the data part.
Bool | canReplaceData (uInt newLeng, uInt oldLeng) const
| Is the bucket large enough to replace a value?
void | replaceData (uInt &offset, const char *data, uInt newLeng, uInt fixedLength)
| Replace a data item.
const char * | get (uInt offset) const
| Get a pointer to the data for the given offset.
uInt | getLength (uInt fixedLength, const char *data) const
| Get the length of the data value.
uInt & | getOffset (uInt colnr, rownr_t rownr)
| Get access to the offset of the data for given column and row.
Block< rownr_t > & | rowIndex (uInt colnr)
| Get access to the index information for the given column.
Block< uInt > & | offIndex (uInt colnr)
| Return the offsets of the values stored in the data part.
uInt & | indexUsed (uInt colnr)
| Return the number of values stored.
rownr_t | split (ISMBucket *&left, ISMBucket *&right, Block< Bool > &duplicated, rownr_t bucketStartRow, rownr_t bucketNrrow, uInt colnr, rownr_t rownr, uInt lengToAdd)
| Split the bucket in the middle.
Bool | simpleSplit (ISMBucket *left, ISMBucket *right, Block< Bool > &duplicated, rownr_t &splitRownr, rownr_t rownr)
| Determine whether a simple split is possible.
uInt | getSplit (uInt totLeng, const Block< uInt > &rowLeng, const Block< uInt > &cumLeng)
| Return the index where the bucket should be split to get two parts with almost identical length.
void | shiftLeft (uInt index, uInt nr, Block< rownr_t > &rowIndex, Block< uInt > &offIndex, uInt &nused, uInt leng)
| Remove nr items from data and index part by shifting to the left.
void | copy (const ISMBucket &that)
| Copy the contents of that bucket to this bucket.
void | show (ostream &os) const
| Show the layout of the bucket.
Bool | check (uInt &offendingCol, uInt &offendingIndex, rownr_t &offendingRow, rownr_t &offendingPrevRow) const
| Check that there are no repeated rowIds in the bucket.

## Static Public Member Functions

static char * | readCallBack (void *owner, const char *bucketStorage)
| Callback function when BucketCache reads a bucket.
static void | writeCallBack (void *owner, char *bucketStorage, const char *bucket)
| Callback function when BucketCache writes a bucket.
static char * | initCallBack (void *owner)
| Callback function when BucketCache adds a new bucket to the data file.
static void | deleteCallBack (void *, char *bucket)
| Callback function when BucketCache removes a bucket from the cache.

## Private Member Functions

void | removeData (uInt offset, uInt leng)
| Remove a data item with the given length.
uInt | insertData (const char *data, uInt leng)
| Insert a data value by appending it to the end.
uInt | copyData (ISMBucket &other, uInt colnr, rownr_t toRownr, uInt fromIndex, uInt toIndex) const
| Copy a data item from this bucket to the other bucket.
void | read (const char *bucketStorage)
| Read the data from the storage into this bucket.
void | write (char *bucketStorage) const
| Write the bucket into the storage.

## Private Attributes

ISMBase * | stmanPtr_p
| Pointer to the parent storage manager.
uInt | uIntSize_p
| The size (in bytes) of an uInt and rownr_t (used in index, etc.).
uInt | rownrSize_p
uInt | dataLeng_p
| The size (in bytes) of the data.
uInt | indexLeng_p
| The size (in bytes) of the index.
Block< Block< rownr_t > * > | rowIndex_p
| The row index per column; each index contains the row number of each value stored in the bucket (for that column).
Block< Block< uInt > * > | offIndex_p
| The offset index per column; each index contains the offset (in bytes) of each value stored in the bucket (for that column).
Block< uInt > | indexUsed_p
| Nr of used elements in each index; i.e.
char * | data_p
| The data space (in external (e.g.

## Detailed Description

A bucket in the Incremental Storage Manager

### Intended use:

Internal

### Review Status

Reviewed By:
  UNKNOWN
Date Reviewed:
  before2004/08/25

### Prerequisite

- IncrementalStMan
- BucketCache

### Etymology

ISMBucket represents a bucket in the Incremental Storage Manager.

### Synopsis

The Incremental Storage Manager uses a BucketCache object to read/write/cache the buckets containing the data. An ISMBucket object is the internal representation of the contents of a bucket. ISMBucket contains static callback functions which are called by BucketCache when reading/writing a bucket. These callback functions do the mapping of bucket data to ISMBucket object and vice-versa.

A bucket contains the values of several rows of all columns bound to this Incremental Storage Manager. A bucket is split into a data part and an index part. Each part has an arbitrary length but together they do not exceed the fixed bucket length.

The beginning of the data part contains the values of all columns bound. The remainder of the data part contains the values of the rows/columns with a changed value.
The index part contains an index per column. Each index contains the row number and an offset for a row with a stored value. The row numbers are relative to the beginning of the bucket, so the bucket has no knowledge about the absolute row numbers. In this way deletion of rows is much simpler.

The contents of a bucket looks like:

-------------------------------------------------------------------
| index offset | data part | index part | free |
-------------------------------------------------------------------
0 4 4+length(data part)
<--------------------------bucketsize----------------------------->
free
free(pool)
casacore::offset
int offset(int, int) const
compute a linear offset from array indicies
casacore::length
LatticeExprNode length(const LatticeExprNode &expr, const LatticeExprNode &axis)
2-argument function to get the length of an axis.

The data part contains all data value belonging to the bucket. The index part contains for each column the following data:

-----------------------------------------------------------------------
| \#values stored | row numbers of values | offset in data part of |
| for column i | stored for column i | values stored for column i |
-----------------------------------------------------------------------
0 4 4+4*nrval

Note that the row numbers in the bucket start at 0, thus are relative to the beginning of the bucket. The main index kept in ISMIndex knows the starting row of each bucket. In this way bucket splitting and especially row removal is much easier.

The bucket can be stored in canonical or local (i.e. native) data format. When a bucket is read into memory, its data are read, converted, and stored in the ISMBucket object. When flushed, the contents are written. ISMBucket takes care that the values stored in its object do not exceed the size of the bucket. When full, the user can call a function to split it into a left and right bucket. When the new value has to be written at the end, the split merely consist of creating a new bucket. In any case, care is taken that a row is not split. Thus a row is always entirely contained in one bucket.

Class ISMColumn does the actual writing of data in a bucket and uses the relevant ISMBucket functions.

### Motivation

ISMBucket encapsulates the data of a bucket.

Definition at line 130 of file ISMBucket.h.

## Constructor & Destructor Documentation

## ISMBucket() [1/2]

casacore::ISMBucket::ISMBucket | ( | ISMBase * | parent,
|  | const char * | bucketStorage )

Create a bucket with the given parent.

When bucketStorage is non-zero, reconstruct the object from it. It keeps the pointer to its parent (but does not own it).

Referenced by copy(), copyData(), ISMBucket(), operator=(), simpleSplit(), and split().

## ~ISMBucket()

casacore::ISMBucket::~ISMBucket | ( |  | ) |

## ISMBucket() [2/2]

casacore::ISMBucket::ISMBucket | ( | const ISMBucket & |  | ) |

delete

Forbid copy constructor.

References ISMBucket().

## Member Function Documentation

## addData()

void casacore::ISMBucket::addData | ( | uInt | colnr,
|  | rownr_t | rownr,
|  | uInt | index,
|  | const char * | data,
|  | uInt | leng )

Add the data to the data part.

It updates the bucket index at the given index. An exception is thrown if the bucket is too small.

## canAddData()

Bool casacore::ISMBucket::canAddData | ( | uInt | leng | ) | const

Is the bucket large enough to add a value?

## canReplaceData()

Bool casacore::ISMBucket::canReplaceData | ( | uInt | newLeng,
|  | uInt | oldLeng ) const

Is the bucket large enough to replace a value?

## check()

Bool casacore::ISMBucket::check | ( | uInt & | offendingCol,
|  | uInt & | offendingIndex,
|  | rownr_t & | offendingRow,
|  | rownr_t & | offendingPrevRow ) const

Check that there are no repeated rowIds in the bucket.

## copy()

void casacore::ISMBucket::copy | ( | const ISMBucket & | that | ) |

Copy the contents of that bucket to this bucket.

This is used after a split operation.

References ISMBucket().

## copyData()

uInt casacore::ISMBucket::copyData | ( | ISMBucket & | other,
|  | uInt | colnr,
|  | rownr_t | toRownr,
|  | uInt | fromIndex,
|  | uInt | toIndex ) const

private

Copy a data item from this bucket to the other bucket.

References ISMBucket().

## deleteCallBack()

void casacore::ISMBucket::deleteCallBack | ( | void * | ,
|  | char * | bucket )

static

Callback function when BucketCache removes a bucket from the cache.

This function dletes the ISMBucket bucket object.

## get()

const char * casacore::ISMBucket::get | ( | uInt | offset | ) | const

inline

Get a pointer to the data for the given offset.

Definition at line 313 of file ISMBucket.h.

References data_p, and casacore::offset().

## getInterval()

uInt casacore::ISMBucket::getInterval | ( | uInt | colnr,
|  | rownr_t | rownr,
|  | rownr_t | bucketNrrow,
|  | rownr_t & | start,
|  | rownr_t & | end,
|  | uInt & | offset ) const

Get the row-interval for given column and row.

It sets the start and end of the interval to which the row belongs and the offset of its current value. It returns the index where the row number can be put in the bucket index.

References casacore::end(), and casacore::offset().

## getLength()

uInt casacore::ISMBucket::getLength | ( | uInt | fixedLength,
|  | const char * | data ) const

Get the length of the data value.

It is fixedLength when non-zero, otherwise read it from the data value.

## getOffset()

uInt & casacore::ISMBucket::getOffset | ( | uInt | colnr,
|  | rownr_t | rownr )

Get access to the offset of the data for given column and row.

It allows to change it (used for example by replaceData).

## getSplit()

uInt casacore::ISMBucket::getSplit | ( | uInt | totLeng,
|  | const Block< uInt > & | rowLeng,
|  | const Block< uInt > & | cumLeng )

Return the index where the bucket should be split to get two parts with almost identical length.

## indexUsed()

uInt & casacore::ISMBucket::indexUsed | ( | uInt | colnr | ) |

inline

Return the number of values stored.

Definition at line 325 of file ISMBucket.h.

References indexUsed_p.

## initCallBack()

char * casacore::ISMBucket::initCallBack | ( | void * | owner | ) |

static

Callback function when BucketCache adds a new bucket to the data file.

This function creates an empty ISMBucket object. It returns the pointer to ISMBucket object which gets part of the cache. The object gets deleted by the deleteCallBack function.

## insertData()

uInt casacore::ISMBucket::insertData | ( | const char * | data,
|  | uInt | leng )

private

Insert a data value by appending it to the end.

It returns the offset of the data value.

## offIndex()

Block< uInt > & casacore::ISMBucket::offIndex | ( | uInt | colnr | ) |

inline

Return the offsets of the values stored in the data part.

Definition at line 321 of file ISMBucket.h.

References offIndex_p.

Referenced by shiftLeft().

## operator=()

ISMBucket & casacore::ISMBucket::operator= | ( | const ISMBucket & |  | ) |

delete

Forbid assignment.

References ISMBucket().

## read()

void casacore::ISMBucket::read | ( | const char * | bucketStorage | ) |

private

Read the data from the storage into this bucket.

## readCallBack()

char * casacore::ISMBucket::readCallBack | ( | void * | owner,
|  | const char * | bucketStorage )

static

Callback function when BucketCache reads a bucket.

It creates an ISMBucket object and converts the raw bucketStorage to that object. It returns the pointer to ISMBucket object which gets part of the cache. The object gets deleted by the deleteCallBack function.

## removeData()

void casacore::ISMBucket::removeData | ( | uInt | offset,
|  | uInt | leng )

private

Remove a data item with the given length.

If the length is zero, its variable length is read first.

References casacore::offset().

## replaceData()

void casacore::ISMBucket::replaceData | ( | uInt & | offset,
|  | const char * | data,
|  | uInt | newLeng,
|  | uInt | fixedLength )

Replace a data item.

When its length is variable (indicated by fixedLength=0), the old value will be removed and the new one appended at the end. An exception is thrown if the bucket is too small.

References casacore::offset().

## rowIndex()

Block< rownr_t > & casacore::ISMBucket::rowIndex | ( | uInt | colnr | ) |

inline

Get access to the index information for the given column.

This is used by ISMColumn when putting the data.

Return the row numbers with a stored value.

Definition at line 317 of file ISMBucket.h.

References rowIndex_p.

Referenced by shiftLeft().

## shiftLeft()

void casacore::ISMBucket::shiftLeft | ( | uInt | index,
|  | uInt | nr,
|  | Block< rownr_t > & | rowIndex,
|  | Block< uInt > & | offIndex,
|  | uInt & | nused,
|  | uInt | leng )

Remove nr items from data and index part by shifting to the left.

The rowIndex, offIndex, and nused get updated. The caller is responsible for removing data when needed (e.g. ISMIndColumn removes the indirect arrays from its file).

References offIndex(), and rowIndex().

## show()

void casacore::ISMBucket::show | ( | ostream & | os | ) | const

Show the layout of the bucket.

## simpleSplit()

Bool casacore::ISMBucket::simpleSplit | ( | ISMBucket * | left,
|  | ISMBucket * | right,
|  | Block< Bool > & | duplicated,
|  | rownr_t & | splitRownr,
|  | rownr_t | rownr )

Determine whether a simple split is possible.

If so, do it. This is possible if the new row is at the end of the last bucket, which will often be the case.
A simple split means adding a new bucket for the new row. If the old bucket already contains values for that row, those values are moved to the new bucket.
This fuction is only called by split, which created the left and right bucket.

References ISMBucket().

## split()

rownr_t casacore::ISMBucket::split | ( | ISMBucket *& | left,
|  | ISMBucket *& | right,
|  | Block< Bool > & | duplicated,
|  | rownr_t | bucketStartRow,
|  | rownr_t | bucketNrrow,
|  | uInt | colnr,
|  | rownr_t | rownr,
|  | uInt | lengToAdd )

Split the bucket in the middle.

It returns the row number where the bucket was split and the new left and right bucket. The caller is responsible for deleting the newly created buckets. When possible a simple split is done.
The starting values in the right bucket may be copies of the values in the left bucket. The duplicated Block contains a switch per column indicating if the value is copied.

References ISMBucket().

## write()

void casacore::ISMBucket::write | ( | char * | bucketStorage | ) | const

private

Write the bucket into the storage.

## writeCallBack()

void casacore::ISMBucket::writeCallBack | ( | void * | owner,
|  | char * | bucketStorage,
|  | const char * | bucket )

static

Callback function when BucketCache writes a bucket.

It converts the ISMBucket bucket object to the raw bucketStorage.

## Member Data Documentation

## data_p

char* casacore::ISMBucket::data_p

private

The data space (in external (e.g.

canonical) format).

Definition at line 309 of file ISMBucket.h.

Referenced by get().

## dataLeng_p

uInt casacore::ISMBucket::dataLeng_p

private

The size (in bytes) of the data.

Definition at line 296 of file ISMBucket.h.

## indexLeng_p

uInt casacore::ISMBucket::indexLeng_p

private

The size (in bytes) of the index.

Definition at line 298 of file ISMBucket.h.

## indexUsed_p

Block<uInt> casacore::ISMBucket::indexUsed_p

private

Nr of used elements in each index; i.e.

the number of stored values per column.

Definition at line 307 of file ISMBucket.h.

Referenced by indexUsed().

## offIndex_p

Block<Block<uInt>*> casacore::ISMBucket::offIndex_p

private

The offset index per column; each index contains the offset (in bytes) of each value stored in the bucket (for that column).

Definition at line 304 of file ISMBucket.h.

Referenced by offIndex().

## rowIndex_p

Block<Block<rownr_t>*> casacore::ISMBucket::rowIndex_p

private

The row index per column; each index contains the row number of each value stored in the bucket (for that column).

Definition at line 301 of file ISMBucket.h.

Referenced by rowIndex().

## rownrSize_p

uInt casacore::ISMBucket::rownrSize_p

private

Definition at line 294 of file ISMBucket.h.

## stmanPtr_p

ISMBase* casacore::ISMBucket::stmanPtr_p

private

Pointer to the parent storage manager.

Definition at line 291 of file ISMBucket.h.

## uIntSize_p

uInt casacore::ISMBucket::uIntSize_p

private

The size (in bytes) of an uInt and rownr_t (used in index, etc.).

Definition at line 293 of file ISMBucket.h.

The documentation for this class was generated from the following file:

- tables/DataMan/ISMBucket.h