Loading...
Searching...
No Matches
casacore::TiledDataStMan Class Reference
tables package (libcasa_tables) » DataMan_module

More...

`#include <TiledDataStMan.h>`

## Public Member Functions

| TiledDataStMan (const String &hypercolumnName, uInt64 maximumCacheSize=0)
| Create a TiledDataStMan storage manager for the hypercolumn with the given name.
| TiledDataStMan (const String &hypercolumnName, const Record &spec)
| ~TiledDataStMan ()
| TiledDataStMan (const TiledDataStMan &)=delete
| Forbid copy constructor.
TiledDataStMan & | operator= (const TiledDataStMan &)=delete
| Forbid assignment.
DataManager * | clone () const
| Clone this object.
String | dataManagerType () const
| Get the type name of the data manager (i.e.
Public Member Functions inherited from casacore::TiledStMan
| TiledStMan ()
| Create a TiledStMan.
| TiledStMan (const String &hypercolumnName, uInt maximumCacheSizeMiB)
| Create a TiledStMan storage manager.
virtual | ~TiledStMan ()
| TiledStMan (const TiledStMan &)=delete
| Forbid copy constructor.
TiledStMan & | operator= (const TiledStMan &)=delete
| Forbid assignment.
virtual String | dataManagerName () const
| Get the name given to the storage manager.
void | setDataManagerName (const String &newHypercolumnName)
virtual Record | dataManagerSpec () const
| Return a record containing data manager specifications.
virtual Record | getProperties () const
| Get data manager properties that can be modified.
virtual void | setProperties (const Record &spec)
| Modify data manager properties.
void | setDataChanged ()
| Set the flag to "data has changed since last flush".
virtual void | setMaximumCacheSize (uInt nMiB)
| Set the maximum cache size (in MiB) in a non-persistent way.
uInt | maximumCacheSize () const
| Get the current maximum cache size (in MiB (MibiByte)).
uInt | cacheSize (rownr_t rownr) const
| Get the current cache size (in buckets) for the hypercube in the given row.
const IPosition & | hypercubeShape (rownr_t rownr) const
| Get the hypercube shape of the data in the given row.
const IPosition & | tileShape (rownr_t rownr) const
| Get the tile shape of the data in the given row.
uInt64 | bucketSize (rownr_t rownr) const
| Get the bucket size (in bytes) of the hypercube in the given row.
virtual Bool | canChangeShape () const
| Can the tiled storage manager handle changing array shapes?
virtual Bool | canAccessColumn () const
| Can the tiled storage manager access an entire column.
virtual Bool | hasMultiFileSupport () const
| The data manager supports use of MultiFile.
uInt | calcCacheSize (rownr_t rownr, const IPosition &sliceShape, const IPosition &windowStart, const IPosition &windowLength, const IPosition &axisPath) const
| Calculate the cache size (in buckets) for accessing the hypercube containing the given row.
void | setCacheSize (rownr_t rownr, const IPosition &sliceShape, const IPosition &windowStart, const IPosition &windowLength, const IPosition &axisPath, Bool forceSmaller)
| Set the cache size using the calcCacheSize function mentioned above.
void | setCacheSize (rownr_t rownr, uInt nbuckets, Bool forceSmaller)
| Set the cache size for accessing the hypercube containing the given row.
void | setHypercubeCacheSize (uInt hypercube, uInt nbuckets, Bool forceSmaller)
| Sets the cache size using the hypercube instead of the row number.
Bool | userSetCache (rownr_t rownr) const
| Determine if the user set the cache size (using setCacheSize).
void | emptyCaches ()
| Empty the caches used by the hypercubes in this storage manager.
void | showCacheStatistics (ostream &os) const
| Show the statistics of all caches used.
uInt64 | getLengthOffset (uInt64 nrPixels, Block< uInt > &dataOffset, Block< uInt > &localOffset, uInt &localTileLength) const
| Get the length of the data for the given number of pixels.
uInt | nrCoordVector () const
| Get the number of coordinate vectors.
rownr_t | nrow () const
| Get the nr of rows in this storage manager.
Bool | canAddRow () const
| Does the storage manager allow to add rows?
virtual IPosition | defaultTileShape () const
| Get the default tile shape.
uInt | nhypercubes () const
| Return the number of hypercubes.
virtual TSMCube * | singleHypercube ()
| Test if only one hypercube is used by this storage manager.
const TSMCube * | getTSMCube (uInt hypercube) const
| Get the given hypercube.
TSMCube * | getTSMCube (uInt hypercube)
const TSMCube * | getHypercube (rownr_t rownr) const
| Get the hypercube in which the given row is stored.
TSMCube * | makeTSMCube (TSMFile *file, const IPosition &cubeShape, const IPosition &tileShape, const Record &values, Int64 fileOffset=-1)
| Make the correct TSMCube type (depending on tsmOption()).
void | readTile (char *local, const Block< uInt > &localOffset, const char *external, const Block< uInt > &externalOffset, uInt nrpixels)
| Read a tile and convert the data to local format.
void | writeTile (char *external, const Block< uInt > &externalOffset, const char *local, const Block< uInt > &localOffset, uInt nrpixels)
| Write a tile after converting the data to external format.
TSMFile * | getFile (uInt sequenceNumber)
| Get the TSMFile object with the given sequence number.
virtual rownr_t | open64 (rownr_t nrrow, AipsIO &)
| Open the storage manager for an existing table.
virtual rownr_t | resync64 (rownr_t nrrow)
| Resync the storage manager with the new file contents.
virtual void | reopenRW ()
| Reopen all files used in this storage manager for read/write access.
virtual void | deleteManager ()
| The data manager will be deleted (because all its columns are requested to be deleted).
DataManagerColumn * | makeScalarColumn (const String &name, int dataType, const String &dataTypeID)
| Create a column in the storage manager on behalf of a table column.
DataManagerColumn * | makeDirArrColumn (const String &name, int dataType, const String &dataTypeID)
| Create a direct array column.
DataManagerColumn * | makeIndArrColumn (const String &name, int dataType, const String &dataTypeID)
| Create an indirect array column.
Bool | canReallocateColumns () const
| The TiledStMan wants to do reallocateColumn.
DataManagerColumn * | reallocateColumn (DataManagerColumn *column)
| Reallocate the column object if it is part of this data manager.
virtual void | setShape (rownr_t rownr, TSMCube *hypercube, const IPosition &shape, const IPosition &tileShape)
| Set the shape and tile shape of a hypercube.
void | checkCubeShape (const TSMCube *hypercube, const IPosition &cubeShape) const
| Check the shape to be set for a hypercube.
int | coordinateDataType (const String &columnName) const
| Get the data type of the coordinate column with the given name.
void | initCoordinates (TSMCube *hypercube)
| Initialize the new coordinates for the given cube.
const TSMDataColumn * | getDataColumn (uInt colnr) const
| Get pointer to data column object.
Public Member Functions inherited from casacore::DataManager
| DataManager ()
| Default constructor.
virtual | ~DataManager ()
| DataManager (const DataManager &)=delete
| The copy constructor cannot be used for this base class.
DataManager & | operator= (const DataManager &)=delete
| Assignment cannot be used for this base class.
void | dataManagerInfo (Record &info) const
| Add SEQNR and SPEC (the DataManagerSpec subrecord) to the info.
virtual Bool | isStorageManager () const
| Is the data manager a storage manager?
uInt | sequenceNr () const
| Get the (unique) sequence nr of this data manager.
uInt | ncolumn () const
| Get the nr of columns in this data manager (can be zero).
Bool | asBigEndian () const
| Have the data to be stored in big or little endian canonical format?
const TSMOption & | tsmOption () const
| Get the TSM option.
std::shared_ptr< MultiFileBase > | multiFile ()
| Get the MultiFile pointer (can be 0).
String | keywordName (const String &keyword) const
| Compose a keyword name from the given keyword appended with the sequence number (e.g.
String | fileName () const
| Compose a unique filename from the table name and sequence number.
ByteIO::OpenOption | fileOption () const
| Get the AipsIO option of the underlying file.
virtual Bool | isRegular () const
| Is this a regular storage manager?
Table & | table () const
| Get the table this object is associated with.
virtual Bool | canRemoveRow () const
| Does the data manager allow to delete rows?
virtual Bool | canAddColumn () const
| Does the data manager allow to add columns?
virtual Bool | canRemoveColumn () const
| Does the data manager allow to delete columns?
virtual Bool | canRenameColumn () const
| Does the data manager allow to rename columns?
virtual void | showCacheStatistics (std::ostream &) const
| Show the data manager's IO statistics.
DataManagerColumn * | createScalarColumn (const String &columnName, int dataType, const String &dataTypeId)
| Create a column in the data manager on behalf of a table column.
DataManagerColumn * | createDirArrColumn (const String &columnName, int dataType, const String &dataTypeId)
| Create a direct array column.
DataManagerColumn * | createIndArrColumn (const String &columnName, int dataType, const String &dataTypeId)
| Create an indirect array column.
DataManager * | getClone () const
| Has the object already been cloned?
void | setClone (DataManager *clone) const
| Set the pointer to the clone.

## Static Public Member Functions

static DataManager * | makeObject (const String &dataManagerType, const Record &spec)
| Make the object from the type name string.
Static Public Member Functions inherited from casacore::TiledStMan
static IPosition | makeTileShape (const IPosition &hypercubeShape, Double tolerance=0.5, uInt64 maxNrPixelsPerTile=4 *1024 *1024)
| Derive the tile shape from the hypercube shape for the given number of pixels per tile.
static IPosition | makeTileShape (const IPosition &hypercubeShape, const Vector< double > &weight, const Vector< double > &tolerance, uInt64 maxNrPixelsPerTile=4 *1024 *1024)
Static Public Member Functions inherited from casacore::DataManager
static void | registerCtor (const String &type, DataManagerCtor func)
| Register a mapping of a data manager type to its static construction function.
static DataManagerCtor | getCtor (const String &dataManagerType)
| Get the "constructor" of a data manager (thread-safe).
static Bool | isRegistered (const String &dataManagerType)
| Test if a data manager is registered (thread-safe).
static DataManager * | unknownDataManager (const String &dataManagerType, const Record &spec)
| Serve as default function for theirRegisterMap, which catches all unknown data manager types.

## Private Member Functions

| TiledDataStMan ()
| Create a TiledDataStMan.
void | addRow64 (rownr_t nrrow)
| Add rows to the storage manager.
void | addHypercube (const IPosition &cubeShape, const IPosition &tileShape, const Record &values)
| Add a hypercube.
void | extendHypercube (uInt64 incrInLastDim, const Record &values)
| Extend the hypercube with the given number of elements in the last dimension.
virtual TSMCube * | getHypercube (rownr_t rownr)
| Get the hypercube in which the given row is stored.
virtual TSMCube * | getHypercube (rownr_t rownr, IPosition &position)
| Get the hypercube in which the given row is stored.
virtual Bool | flush (AipsIO &, Bool fsync)
| Flush and optionally fsync the data.
virtual void | create64 (rownr_t nrrow)
| Let the storage manager create files as needed for a new table.
virtual void | readHeader (rownr_t nrrow, Bool firstTime)
| Read the header info.
void | updateRowMap (uInt cubeNr, uInt64 incrInLastDim)
| Update the map of row numbers to cube number plus offset.
void | checkNrrow (const IPosition &cubeShape, uInt64 incrInLastDim) const
| Check if the table is large enough to hold this hypercube extension.

## Private Attributes

std::vector< rownr_t > | rowMap_p
| The map of row number to cube and position in cube.
std::vector< uInt > | cubeMap_p
std::vector< uInt > | posMap_p
rownr_t | nrrowLast_p
| The row number since the last hypercube extension.

## Friends

class | TiledDataStManAccessor

## Additional Inherited Members

Static Public Attributes inherited from casacore::DataManager
static rownr_t | MAXROWNR32
| Define the highest row number that can be represented as signed 32-bit.
Protected Member Functions inherited from casacore::TiledStMan
void | setPersMaxCacheSize (uInt nMiB)
| Set the persistent maximum cache size (in MiB).
uInt | getBindings (const Vector< String > &columnNames, Block< TSMColumn * > &colSet, Bool mustExist) const
| Get the bindings of the columns with the given names.
virtual void | setupCheck (const TableDesc &tableDesc, const Vector< String > &dataNames) const
| Function setup calls this function to allow the derived class to check specific information.
virtual const TableDesc & | getDesc () const
| Get the table description needed for the hypercolumn description.
void | checkValues (const Block< TSMColumn * > &colSet, const Record &values) const
| Check if values are given in the record for all columns in the block.
void | checkCoordinates (const Block< TSMColumn * > &coordColSet, const IPosition &cubeShape, const Record &values) const
| Check if the coordinate values are correct.
void | checkShapeColumn (const IPosition &shape) const
| Check if the shapes of FixedShape data and coordinate columns match.
void | checkCoordinatesShapes (const TSMCube *hypercube, const IPosition &cubeShape) const
| Check if the cube shape matches that of defined coordinates.
void | checkAddHypercube (const IPosition &cubeShape, const Record &values) const
| Check if the hypercube to be added is correctly defined.
TSMCube * | makeHypercube (const IPosition &cubeShape, const IPosition &tileShape, const Record &values)
| Make a new TSMCube object.
Int | getCubeIndex (const Record &idValues) const
| Get the index of the hypercube with the given id-values.
rownr_t | addedNrrow (const IPosition &shape, uInt incrInLastDim) const
| Determine how many rows need to be added for an extension (in the last dimension) of a hypercube with the given shape.
Bool | flushCaches (Bool fsync)
| Flush the caches of all hypercubes.
AipsIO * | headerFileCreate ()
| Create the TSM header file.
AipsIO * | headerFileOpen ()
| Open the TSM header file.
void | headerFilePut (AipsIO &headerFile, uInt64 nrCube)
| Write the data into the header file.
uInt | headerFileGet (AipsIO &headerFile, rownr_t tabNrrow, Bool firstTime, Int extraNdim)
| Read the data from the header file.
void | headerFileClose (AipsIO *headerFile)
| Close the header file.
void | setup (Int extraNdim=-1)
| Set up the TiledStMan variables from the table description.
void | createFile (uInt index)
| Create a TSMFile object and store its pointer at the given index in the block.
int | arrayDataType (int dataType) const
| Convert the scalar data type to an array data type.
Protected Member Functions inherited from casacore::DataManager
void | decrementNcolumn ()
| Decrement number of columns (in case a column is deleted).
void | setEndian (Bool bigEndian)
| Tell the data manager if big or little endian format is needed.
void | setTsmOption (const TSMOption &tsmOption)
| Tell the data manager which TSM option to use.
void | setMultiFile (const std::shared_ptr< MultiFileBase > &mfile)
| Tell the data manager that MultiFile can be used.
void | throwDataTypeOther (const String &columnName, int dataType) const
| Throw an exception in case data type is TpOther, because the storage managers (and maybe other data managers) do not support such columns.
Protected Attributes inherited from casacore::TiledStMan
String | hypercolumnName_p
| The name of the hypercolumn.
rownr_t | nrrow_p
| The number of rows in the columns.
Block< TSMColumn * > | colSet_p
| The assembly of all columns.
Block< TSMDataColumn * > | dataCols_p
| The assembly of all data columns.
Block< TSMColumn * > | dataColSet_p
Block< TSMColumn * > | idColSet_p
| The assembly of all id columns.
Block< TSMColumn * > | coordColSet_p
| The assembly of all coordinate columns.
Block< TSMFile * > | fileSet_p
| The assembly of all TSMFile objects.
Block< TSMCube * > | cubeSet_p
| The assembly of all TSMCube objects.
uInt | persMaxCacheSize_p
| The persistent maximum cache size (in MiB) for a hypercube.
uInt | maxCacheSize_p
| The actual maximum cache size for a hypercube (in MiB).
uInt | nrdim_p
| The dimensionality of the hypercolumn.
uInt | nrCoordVector_p
| The number of vector coordinates.
IPosition | fixedCellShape_p
| The fixed cell shape.
Bool | dataChanged_p
| Has any data changed since the last flush?

## Detailed Description

Tiled Data Storage Manager.

### Intended use:

Public interface

### Review Status

Reviewed By:
  UNKNOWN
Date Reviewed:
  before2004/08/25

### Prerequisite

- TiledStMan
- TSMCube
- ROTiledStManAccessor for a discussion of the maximum cache size
- Record

### Etymology

TiledDataStMan is the Tiled Storage Manager for general data arrays.

### Synopsis

TiledDataStMan is a derivation from TiledStMan, the abstract tiled storage manager class. A description of the basics of tiled storage managers is given in the Tables module description.

TiledDataStMan allows the user explicit control over the definition and extension of hypercubes by means of the accessor class TiledDataStManAccessor. The user can determine which row should be put in which hypercube, so it is possible to put row 0-9 in hypercube A, row 10-29 in B, row 30-39 in A again, etc.. This makes it possible to use a tiled storage manager for a data column containing data with different shapes (e.g. line and continuum data). Actually, this storage manager is developed for irregularly shaped UV-data, but can be used for any purpose.
Each extensible hypercube uses a file of its own. This means that there shouldn't be too many of them, otherwise the number of files may get too high.

The TiledDataStMan has the following (extra) properties:

- When multiple hypercubes are used, one or more id columns have to be used to differentiate between them. The id values must be defined when the hypercube gets added; they cannot be put explicitly.
- A hypercube can be extensible in its last dimension by setting its last dimension to zero. In that case extendHypercube can be used to extend the hypercube when needed. All fixed sized hypercubes are stored in one file, while there is one file per extensible hypercube.
- The table must be large enough to accommodate the addition or extension of a hypercube. This means that a sufficient number of rows must be added to the table before a hypercube can be added or extended. It is the responsibility of the user to "synchronize" addition of rows and hypercubes.
- It is possible to define coordinates for the hypercube axes in several ways:
  - Use the TiledDataStMan storage manager to hold their values and define the coordinates when adding or extending the hypercube. This is the preferred way.
  - As above, but use explicit puts to write their values. This has to be used when coordinates are defined after the hypercube has been added or extended. Note that several rows may share the same value, so overwriting a value may affect multiple rows.
  - Use another storage manager to hold their values. This is useful when their values depend on other axes, because that cannot be handled by TiledDataStMan.
  Note that it is possible to store one coordinate column with TiledDataStMan and another with another storage manager.

### Motivation

This tiled storage manager allows one to create and extend hypercubes as needed. One has complete control over which row is stored in which hypercube.

### Example

The following example shows how to create a TiledDataStMan tiled storage manager using the hypercolumn as defined in the table description. Furthermore it shows how to use TiledDataStManAccessor to add a hypercube, while defining its tile shape, coordinates, and id-value. The example shows that reading the data back does not require any knowledge of the data manager. It's exactly the same if another data manager was used.
The table created contains the equally shaped data columns "Data" and "Weight". Each cell in those columns contains a 2D array with shape [12,20]. The coordinates of those arrays are "Pol" and "Freq". The tiled storage manager superimposes two more axes ("Baseline"and "Time") on the data resulting in a 4D hypercube with shape [12,20,30,42]. The table contains 42*30 rows (which has to be equal to the number of elements in the superimposed axes).
The tile shape of the hypercube is (arbitrarily) set to [4,5,6,7]. Of course, any tile shape could be chosen. This tile shape results in a tile size of 6720 bytes (4*5*6*7 *(4+4) bytes), which is not that large (32768 as tile size is very reasonable). The number of tiles is integral in each dimension, so no space is wasted. Finally it makes access along the various axes about equally efficient.
Although in this example only one hypercube is added, multiple hypercubes are possible, because an id column has been defined.
Caution: The example uses the global Array function indgen to fill the data and coordinate arrays with arbitrary values;
Note that the description of class ROTiledStManAccessor contains a discussion about the effect of setting the maximum cache size.

// Define the table description and the columns in it.
TableDesc td ("", "1", TableDesc::Scratch);
td.addColumn (ScalarColumnDesc<float> ("Time"));
td.addColumn (ScalarColumnDesc<float> ("Baseline"));
td.addColumn (ArrayColumnDesc<float> ("Pol", 1));
td.addColumn (ArrayColumnDesc<float> ("Freq", 1));
td.addColumn (ScalarColumnDesc<String> ("Id"));
td.addColumn (ArrayColumnDesc<float> ("Data", 2));
td.addColumn (ArrayColumnDesc<float> ("Weight", 2));
// Define the 4-dim hypercolumn with its data, coordinate and id columns.
td.defineHypercolumn ("TSMExample",
4,
stringToVector ("Data,Weight"),
stringToVector ("Pol,Freq,Baseline,Time"),
stringToVector ("Id"));
// Now create a new table from the description.
SetupNewTable newtab("tTiledDataStMan_tmp.data", td, Table::New);
// Create a TiledDataStMan storage manager for the hypercolumn
// and bind the columns to it.
TiledDataStMan sm1 ("TSMExample");
newtab.bindAll (sm1);
// Create the table with 42*30 rows.
Table table(newtab, 42*30);
// Create the accessor to be able to add a hypercube to this
// storage manager.
TiledDataStManAccessor accessor(table, "TSMExample");
// Define the values for the coordinates of the hypercube
// and put them into the record.
Vector<float> timeValues(42);
Vector<float> baselineValues(30);
Vector<float> freqValues(20);
Vector<float> polValues(12);
indgen (timeValues);
indgen (baselineValues, float(100));
indgen (freqValues, float(200));
indgen (polValues, float(300));
Record hyperDef;
hyperDef.define ("Time", timeValues);
hyperDef.define ("Baseline", baselineValues);
hyperDef.define ("Freq", freqValues);
hyperDef.define ("Pol", polValues);
// Define the id value as well.
hyperDef.define ("Id", "");
// Now add the hypercube with the given shape, tile shape,
// and coordinate and id values.
accessor.addHypercube (IPosition(4,12,20,30,42),
IPosition(4,4,5,6,7), hyperDef);
ArrayColumn<float> data (table, "Data");
ArrayColumn<float> weight (table, "Weight");
Matrix<float> array(IPosition(2,12,20));
indgen (array);
// Write some data into the data columns.
for (uInt i=0; i<30*42; i++) {
data.put (i, array);
weight.put (i, array+float(100));
array += float(200);
}
// Prepare for reading the data back.
// Note that time and baseline are in fact scalar columns. They are
// superimposed dimensions on the hypercube.
ScalarColumn<float> time (table, "Time");
ScalarColumn<float> baseline (table, "Baseline");
ArrayColumn<float> freq (table, "Freq");
ArrayColumn<float> pol (table, "Pol");
ScalarColumn<String> id (table, "Id");
float fValue;
String sValue;
for (rownr_t i=0; i<table.nrow(); i++) {
data.get (i, array);
weight.get (i, array);
pol.get (i, polValues);
freq.get (i, freqValues);
baseline.get (i, fValue);
time.get (i, fValue);
id.get (i, sValue);
}
stringToVector
Vector< String > stringToVector(const String &string, char delim=',')
casacore::ArrayColumnDesc
Templated class for description of table array columns.
Definition ArrColDesc.h:235
casacore::ArrayColumn
Definition ArrayColumn.h:98
casacore::DataManager::table
Table & table() const
Get the table this object is associated with.
Definition DataManager.h:323
casacore::IPosition
Definition IPosition.h:117
casacore::Matrix
Definition Matrix.h:87
casacore::ScalarColumnDesc
Definition ScaColDesc.h:162
casacore::ScalarColumn
Definition ScalarColumn.h:83
casacore::SetupNewTable
Create a new table - define shapes, data managers, etc.
Definition SetupNewTab.h:337
casacore::String
String: the storage and methods of handling collections of characters.
Definition String.h:223
casacore::TableDesc
Definition TableDesc.h:189
casacore::TableDesc::Scratch
@ Scratch
Definition TableDesc.h:193
casacore::Table
Definition Table.h:157
casacore::Table::New
@ New
create table
Definition Table.h:175
casacore::Table::nrow
rownr_t nrow() const
Get the number of rows.
Definition Table.h:1187
casacore::TiledDataStMan::TiledDataStManAccessor
friend class TiledDataStManAccessor
Definition TiledDataStMan.h:348
casacore::TiledDataStMan::TiledDataStMan
TiledDataStMan(const String &hypercolumnName, uInt64 maximumCacheSize=0)
Create a TiledDataStMan storage manager for the hypercolumn with the given name.
casacore::Vector
Definition Vector.h:82
casacore::Record
Record()
casacore::indgen
void indgen(TableVector< T > &tv, T start, T inc)
Definition TabVecMath.h:398
casacore::time
TableExprNode time(const TableExprNode &node)
Definition ExprNode.h:1584
casacore::array
T * array
The actual storage.
Definition Block.h:689
casacore::uInt
unsigned int uInt
Definition aipstype.h:49
casacore::rownr_t
uInt64 rownr_t
Define the type of a row number in a table.
Definition aipsxtype.h:44

Note that in this example an id column was not necessary, because there is only one hypercube.

The following example is more advanced. Two (extensible) hypercubes are used for line and continuum data. Writing such a data set could be done as shown. Reading it back is the same as above.
In this example the data columns contain line and continuum data. So there are two types of data, each with their own shape and stored in their own (extensible) hypercube. Note that the last dimension of the hypercube shape is set to zero (to make extensible), but the last tile shape dimension has been filled in, because the exact tile shape must be known.
Before each put of the data the appropriate hypercube is extended. Also the time has to be put, which is done (as an example) in two different ways (using an explicit put and using the extendHypercube).

// Defining TableDesc and storage manager is same as in first example.
// Create the table.
Table table(newtab);
// Create the accessor to be able to add the hypercubes to this
// storage manager.
TiledDataStManAccessor accessor(table, "TSMExample");
// Fill the coordinate values.
// Note that the time axis of the hypercube will have length 0 to
// make it extensible. Therefore the time coordinate can only be
// filled in when the hypercube is extended.
Vector<float> baselineValues(30);
Vector<float> freqValuesCont(1);
Vector<float> freqValuesLine(20);
Vector<float> polValues(4);
indgen (baselineValues, float(100));
indgen (freqValuesLine, float(200));
indgen (freqValuesCont, float(200));
indgen (polValues, float(300));
Record hyperDefLine;
hyperDefLine.define ("Baseline", baselineValues);
hyperDefLine.define ("Pol", polValues);
// Make similar record for line data.
// Fill the correct id and frequency values for each type.
// Add the 2 hypercubes.
Record hyperDefCont (hyperDefLine);
hyperDefLine.define ("Id", "L");
hyperDefLine.define ("Freq", freqValuesLine);
hyperDefCont.define ("Id", "C");
hyperDefCont.define ("Freq", freqValuesCont);
// Add the hypercubes.
// Define their last dimension as zero to make them extensible.
accessor.addHypercube (IPosition(4,4,20,30,0),
IPosition(4,4,5,6,7), hyperDefLine);
accessor.addHypercube (IPosition(4,4,1,30,0),
IPosition(4,4,1,6,7), hyperDefCont);
ScalarColumn<float> time (table, "Time");
ScalarColumn<float> baseline (table, "Baseline");
ArrayColumn<float> freq (table, "Freq");
ArrayColumn<float> pol (table, "Pol");
ArrayColumn<float> data (table, "Data");
ArrayColumn<float> weight (table, "Weight");
Matrix<float> arrayLine(IPosition(2,4,20));
Matrix<float> arrayCont(IPosition(2,4,1));
indgen (arrayLine);
indgen (arrayCont);
// Write some data into the data columns.
// Alternately line and continuum is written.
// Each hypercube requires 30 rows to be added (i.e. nr of baselines).
// The last dimension of each hypercube is extended with 1.
rownr_t rownr = 0;
for (uInt i=0; i<42; i++) {
if (i%2 == 0) {
table.addRow (30);
accessor.extendHypercube (1, hyperDefLine);
time.put (rownr, float(i));
for (uInt j=0; j<30; j++) {
data.put (rownr, arrayLine);
weight.put (rownr, arrayLine);
rownr++;
}
}else{
table.addRow (30);
Vector<float> timeValue(1);
timeValue(0) = float(i);
hyperDefCont.define ("Time", timeValue);
accessor.extendHypercube (1, hyperDefCont);
time.put (rownr, float(i));
for (uInt j=0; j<30; j++) {
data.put (rownr, arrayCont);
weight.put (rownr, arrayCont);
rownr++;
}
}
}
casacore::Table::addRow
void addRow(rownr_t nrrow=1, Bool initialize=False)
Add one or more rows at the end of the table.
Definition Table.h:1219

Note that in this example the time is defined in 2 different ways. The first one by an explicit put, the second one as a record in the extendHypercube call. The second way if the preferred one, although it requires a bit more coding.

Definition at line 346 of file TiledDataStMan.h.

## Constructor & Destructor Documentation

## TiledDataStMan() [1/4]

casacore::TiledDataStMan::TiledDataStMan | ( | const String & | hypercolumnName,
|  | uInt64 | maximumCacheSize = 0 )

Create a TiledDataStMan storage manager for the hypercolumn with the given name.

The hypercolumn name is also the name of the storage manager. The given maximum cache size (default is unlimited) is persistent, thus will be reused when the table is read back. Note that the class ROTiledStManAccessor allows one to overwrite the maximum cache size temporarily.
The constructor taking a Record expects fields in the record with the name of the arguments in uppercase. If not defined, their default value is used.

References casacore::TiledStMan::maximumCacheSize().

Referenced by operator=(), and TiledDataStMan().

## TiledDataStMan() [2/4]

casacore::TiledDataStMan::TiledDataStMan | ( | const String & | hypercolumnName,
|  | const Record & | spec )

References casacore::Record().

## ~TiledDataStMan()

casacore::TiledDataStMan::~TiledDataStMan | ( |  | ) |

## TiledDataStMan() [3/4]

casacore::TiledDataStMan::TiledDataStMan | ( | const TiledDataStMan & |  | ) |

delete

Forbid copy constructor.

References TiledDataStMan().

## TiledDataStMan() [4/4]

casacore::TiledDataStMan::TiledDataStMan | ( |  | ) |

private

Create a TiledDataStMan.

This constructor is private, because it should only be used by makeObject.

## Member Function Documentation

## addHypercube()

void casacore::TiledDataStMan::addHypercube | ( | const IPosition & | cubeShape,
|  | const IPosition & | tileShape,
|  | const Record & | values )

private

Add a hypercube.

The number of rows in the table must be large enough to accommodate this hypercube. The possible id values must be given in the record, while coordinate values are optional. The field names in the record should match the coordinate and id column names. The last dimension in the cube shape can be zero, indicating that the hypercube is extensible.

References casacore::Record(), and casacore::TiledStMan::tileShape().

## addRow64()

void casacore::TiledDataStMan::addRow64 | ( | rownr_t | nrrow | ) |

private virtual

Add rows to the storage manager.

This will only increase the number of rows. When a hypercube is added or extended, it will be checked whether the number of rows is sufficient.

Reimplemented from casacore::DataManager.

## checkNrrow()

void casacore::TiledDataStMan::checkNrrow | ( | const IPosition & | cubeShape,
|  | uInt64 | incrInLastDim ) const

private

Check if the table is large enough to hold this hypercube extension.

## clone()

DataManager * casacore::TiledDataStMan::clone | ( |  | ) | const

virtual

Clone this object.

It does not clone TSMColumn objects possibly used.

Implements casacore::DataManager.

## create64()

virtual void casacore::TiledDataStMan::create64 | ( | rownr_t | nrrow | ) |

private virtual

Let the storage manager create files as needed for a new table.

This allows a column with an indirect array to create its file.

Reimplemented from casacore::DataManager.

## dataManagerType()

String casacore::TiledDataStMan::dataManagerType | ( |  | ) | const

virtual

Get the type name of the data manager (i.e.

TiledDataStMan).

Implements casacore::DataManager.

Referenced by makeObject().

## extendHypercube()

void casacore::TiledDataStMan::extendHypercube | ( | uInt64 | incrInLastDim,
|  | const Record & | values )

private

Extend the hypercube with the given number of elements in the last dimension.

The record should contain the id values (to get the correct hypercube) and optionally coordinate values for the elements added.

References casacore::Record().

## flush()

virtual Bool casacore::TiledDataStMan::flush | ( | AipsIO & | ,
|  | Bool | fsync )

private virtual

Flush and optionally fsync the data.

It returns a True status if it had to flush (i.e. if data have changed).

Implements casacore::DataManager.

## getHypercube() [1/2]

virtual TSMCube * casacore::TiledDataStMan::getHypercube | ( | rownr_t | rownr | ) |

private virtual

Get the hypercube in which the given row is stored.

Implements casacore::TiledStMan.

## getHypercube() [2/2]

virtual TSMCube * casacore::TiledDataStMan::getHypercube | ( | rownr_t | rownr,
|  | IPosition & | position )

private virtual

Get the hypercube in which the given row is stored.

It also returns the position of the row in that hypercube.

Implements casacore::TiledStMan.

## makeObject()

DataManager * casacore::TiledDataStMan::makeObject | ( | const String & | dataManagerType,
|  | const Record & | spec )

static

Make the object from the type name string.

This function gets registered in the DataManager "constructor" map.

References dataManagerType(), and casacore::Record().

## operator=()

TiledDataStMan & casacore::TiledDataStMan::operator= | ( | const TiledDataStMan & |  | ) |

delete

Forbid assignment.

References TiledDataStMan().

## readHeader()

virtual void casacore::TiledDataStMan::readHeader | ( | rownr_t | nrrow,
|  | Bool | firstTime )

private virtual

Read the header info.

Implements casacore::TiledStMan.

## updateRowMap()

void casacore::TiledDataStMan::updateRowMap | ( | uInt | cubeNr,
|  | uInt64 | incrInLastDim )

private

Update the map of row numbers to cube number plus offset.

## Friends And Related Symbol Documentation

## TiledDataStManAccessor

friend class TiledDataStManAccessor

friend

Definition at line 348 of file TiledDataStMan.h.

References TiledDataStManAccessor.

Referenced by TiledDataStManAccessor.

## Member Data Documentation

## cubeMap_p

std::vector<uInt> casacore::TiledDataStMan::cubeMap_p

private

Definition at line 448 of file TiledDataStMan.h.

## nrrowLast_p

rownr_t casacore::TiledDataStMan::nrrowLast_p

private

The row number since the last hypercube extension.

Definition at line 451 of file TiledDataStMan.h.

## posMap_p

std::vector<uInt> casacore::TiledDataStMan::posMap_p

private

Definition at line 449 of file TiledDataStMan.h.

## rowMap_p

std::vector<rownr_t> casacore::TiledDataStMan::rowMap_p

private

The map of row number to cube and position in cube.

Definition at line 447 of file TiledDataStMan.h.

The documentation for this class was generated from the following file:

- tables/DataMan/TiledDataStMan.h