Loading...
Searching...
No Matches
dyscostman::DyscoStMan Class Reference

The main class for the Dysco storage manager. More...

`#include <dyscostman.h>`

## Public Member Functions

| DyscoStMan (unsigned dataBitRate, unsigned weightBitRate, const casacore::String &name="DyscoStMan")
| Convenience constructor to create a new storage manager with some settings without having to fill a 'spec' Record.
void | SetGaussianDistribution ()
| Initialize the storage manager to use a Gaussian distribution for the quantization.
void | SetUniformDistribution ()
| Initialize the storage manager to use a Uniform distribution for the quantization (i.e., use a linear quantizer).
void | SetStudentsTDistribution (double nu)
| Initialize the storage manager to use a Student T distribution for the quantization (i.e., use a linear quantizer).
void | SetTruncatedGaussianDistribution (double truncationSigma)
| Initialize the storage manager to use a Uniform distribution for the quantization (i.e., use a linear quantizer).
void | SetNormalization (Normalization normalization)
| Set the type of normalization.
void | SetStaticSeed (bool staticSeed)
| DyscoStMan (const casacore::String &name, const casacore::Record &spec)
| This constructor is called by Casa when it needs to create a DyscoStMan.
| DyscoStMan (const DyscoStMan &source)
| Copy constructor that initializes a storage manager with similar specs.
| ~DyscoStMan ()
| Destructor.
DyscoStMan & | operator= (const DyscoStMan &source)=delete
| Assignment – new dyscostman takes the settings of the source (but not the columns and/or data).
virtual casacore::DataManager * | clone () const final override
| Polymorphical copy constructor, equal to DyscoStMan(const DyscoStMan&).
virtual casacore::String | dataManagerType () const final override
| Type of manager.
virtual casacore::String | dataManagerName () const final override
| Returns the name of this manager as specified during construction.
virtual casacore::Record | dataManagerSpec () const final override
| Get manager specifications.
uint | getNRow () const
| Get the number of rows in the measurement set.
virtual casacore::Bool | canAddRow () const final override
| Whether rows can be added.
virtual casacore::Bool | canRemoveRow () const final override
| Whether rows can be removed.
virtual casacore::Bool | canAddColumn () const final override
| Whether columns can be added.
virtual casacore::Bool | canRemoveColumn () const final override
| Whether columns can be removed.
Public Member Functions inherited from casacore::DataManager
| DataManager ()
| Default constructor.
virtual | ~DataManager ()
| DataManager (const DataManager &)=delete
| The copy constructor cannot be used for this base class.
DataManager & | operator= (const DataManager &)=delete
| Assignment cannot be used for this base class.
void | dataManagerInfo (Record &info) const
| Add SEQNR and SPEC (the DataManagerSpec subrecord) to the info.
virtual Record | getProperties () const
| Get data manager properties that can be modified.
virtual void | setProperties (const Record &spec)
| Modify data manager properties given in record fields.
virtual Bool | isStorageManager () const
| Is the data manager a storage manager?
virtual Bool | canReallocateColumns () const
| Tell if the data manager wants to reallocate the data manager column objects.
virtual DataManagerColumn * | reallocateColumn (DataManagerColumn *column)
| Reallocate the column object if it is part of this data manager.
uInt | sequenceNr () const
| Get the (unique) sequence nr of this data manager.
uInt | ncolumn () const
| Get the nr of columns in this data manager (can be zero).
Bool | asBigEndian () const
| Have the data to be stored in big or little endian canonical format?
const TSMOption & | tsmOption () const
| Get the TSM option.
std::shared_ptr< MultiFileBase > | multiFile ()
| Get the MultiFile pointer (can be 0).
String | keywordName (const String &keyword) const
| Compose a keyword name from the given keyword appended with the sequence number (e.g.
String | fileName () const
| Compose a unique filename from the table name and sequence number.
ByteIO::OpenOption | fileOption () const
| Get the AipsIO option of the underlying file.
virtual Bool | isRegular () const
| Is this a regular storage manager?
Table & | table () const
| Get the table this object is associated with.
virtual Bool | canRenameColumn () const
| Does the data manager allow to rename columns?
virtual void | setMaximumCacheSize (uInt nMiB)
| Set the maximum cache size (in bytes) to be used by a storage manager.
virtual void | showCacheStatistics (std::ostream &) const
| Show the data manager's IO statistics.
DataManagerColumn * | createScalarColumn (const String &columnName, int dataType, const String &dataTypeId)
| Create a column in the data manager on behalf of a table column.
DataManagerColumn * | createDirArrColumn (const String &columnName, int dataType, const String &dataTypeId)
| Create a direct array column.
DataManagerColumn * | createIndArrColumn (const String &columnName, int dataType, const String &dataTypeId)
| Create an indirect array column.
DataManager * | getClone () const
| Has the object already been cloned?
void | setClone (DataManager *clone) const
| Set the pointer to the clone.

## Static Public Member Functions

static casacore::DataManager * | makeObject (const casacore::String &name, const casacore::Record &spec)
| Create an object with given name and spec.
static void | registerClass ()
| This function makes the DyscoStMan known to casacore.
Static Public Member Functions inherited from casacore::DataManager
static void | registerCtor (const String &type, DataManagerCtor func)
| Register a mapping of a data manager type to its static construction function.
static DataManagerCtor | getCtor (const String &dataManagerType)
| Get the "constructor" of a data manager (thread-safe).
static Bool | isRegistered (const String &dataManagerType)
| Test if a data manager is registered (thread-safe).
static DataManager * | unknownDataManager (const String &dataManagerType, const Record &spec)
| Serve as default function for theirRegisterMap, which catches all unknown data manager types.

## Protected Member Functions

uint64_t | nBlocksInFile () const
| The number of rows that are actually stored in the file.
size_t | nRowsInBlock () const
| Number of rows in one "time-block", i.e.
size_t | nAntennae () const
| Number of antennae used in a time block.
size_t | getBlockIndex (uint64_t row) const
| Return index of block that contains the given measurement set row.
size_t | getRowWithinBlock (uint64_t row) const
| Return the offset of the row within the block.
uint64_t | getRowIndex (size_t block) const
| Calculate first measurement set row index of a given block index.
bool | areOffsetsInitialized () const
| This method returns `true` when the number of rows per block and the number of antennae per block are known.
void | initializeRowsPerBlock (size_t rowsPerBlock, size_t antennaCount, bool writeToHeader)
| To be called by a column once it determines rowsPerBlock and antennaCount.
Protected Member Functions inherited from casacore::DataManager
void | decrementNcolumn ()
| Decrement number of columns (in case a column is deleted).
void | setEndian (Bool bigEndian)
| Tell the data manager if big or little endian format is needed.
void | setTsmOption (const TSMOption &tsmOption)
| Tell the data manager which TSM option to use.
void | setMultiFile (const std::shared_ptr< MultiFileBase > &mfile)
| Tell the data manager that MultiFile can be used.
virtual Bool | hasMultiFileSupport () const
| Does the data manager support use of MultiFile?
void | throwDataTypeOther (const String &columnName, int dataType) const
| Throw an exception in case data type is TpOther, because the storage managers (and maybe other data managers) do not support such columns.

## Private Member Functions

void | readCompressedData (size_t blockIndex, const DyscoStManColumn *column, unsigned char *dest, size_t size)
void | writeCompressedData (size_t blockIndex, const DyscoStManColumn *column, const unsigned char *data, size_t size)
void | readHeader ()
void | writeHeader ()
void | makeEmpty ()
void | setFromSpec (const casacore::Record &spec)
size_t | getFileOffset (size_t blockIndex) const
virtual casacore::Bool | flush (casacore::AipsIO &, casacore::Bool doFsync) final override
| Flush and optionally fsync the data.
virtual void | create64 (casacore::rownr_t nRow) final override
| Let the storage manager create files as needed for a new table.
virtual casacore::rownr_t | open64 (casacore::rownr_t nRow, casacore::AipsIO &) final override
| Open the storage manager file for an existing table.
virtual casacore::DataManagerColumn * | makeScalarColumn (const casacore::String &name, int dataType, const casacore::String &dataTypeID) final override
| Create a column in the storage manager on behalf of a table column.
virtual casacore::DataManagerColumn * | makeDirArrColumn (const casacore::String &name, int dataType, const casacore::String &dataTypeID) final override
| Create a direct array column.
virtual casacore::DataManagerColumn * | makeIndArrColumn (const casacore::String &name, int dataType, const casacore::String &dataTypeID) final override
| Create an indirect array column.
virtual casacore::rownr_t | resync64 (casacore::rownr_t nRow) final override
virtual void | deleteManager () final override
| The data manager will be deleted (because all its columns are requested to be deleted).
virtual void | prepare () final override
| Prepare the columns, let the data manager initialize itself further.
virtual void | reopenRW () final override
| Reopen the storage manager files for read/write.
virtual void | addRow64 (casacore::rownr_t nrrow) final override
| Add rows to the storage manager.
virtual void | removeRow64 (casacore::rownr_t rowNr) final override
| Delete a row from all columns.
virtual void | addColumn (casacore::DataManagerColumn *) final override
| Do the final addition of a column.
virtual void | removeColumn (casacore::DataManagerColumn *) final override
| Remove a column from the data file.

## Private Attributes

uint64_t | _nRow
uint64_t | _nBlocksInFile
uint32_t | _rowsPerBlock
uint32_t | _antennaCount
uint32_t | _blockSize
unsigned | _headerSize
std::mutex | _mutex
std::unique_ptr< std::fstream > | _fStream
std::string | _name
unsigned | _dataBitCount
unsigned | _weightBitCount
DyscoDistribution | _distribution
Normalization | _normalization
double | _studentTNu
double | _distributionTruncation
bool | _staticSeed
std::vector< std::unique_ptr< DyscoStManColumn > > | _columns

## Static Private Attributes

static const unsigned short | VERSION_MAJOR
static const unsigned short | VERSION_MINOR

## Friends

class | DyscoStManColumn

## Additional Inherited Members

Static Public Attributes inherited from casacore::DataManager
static rownr_t | MAXROWNR32
| Define the highest row number that can be represented as signed 32-bit.

## Detailed Description

The main class for the Dysco storage manager.

Definition at line 46 of file dyscostman.h.

## Constructor & Destructor Documentation

## DyscoStMan() [1/3]

dyscostman::DyscoStMan::DyscoStMan | ( | unsigned | dataBitRate,
|  | unsigned | weightBitRate,
|  | const casacore::String & | name = "DyscoStMan" )

Convenience constructor to create a new storage manager with some settings without having to fill a 'spec' Record.

The storage manager will be initialized to AF normalization with a truncated Gaussian distribution for the quantization, and a truncation of sigma = 2.5. To change the settings, use one of the Set...Distribution() methods and SetNormalization().

Parameters

  dataBitRate | The number of bits per float used for visibilities.
  weightBitRate | The number of bits per float used for the weight column.
  name | Storage manager name.

Referenced by clone(), DyscoStMan(), makeObject(), and operator=().

## DyscoStMan() [2/3]

dyscostman::DyscoStMan::DyscoStMan | ( | const casacore::String & | name,
|  | const casacore::Record & | spec )

This constructor is called by Casa when it needs to create a DyscoStMan.

Casa will call makeObject() that will call this constructor. When it loads an DyscoStMan for an existing MS, the "spec" parameter will be empty, thus the class should initialize its properties by reading them from the file. The `spec` is used to make a new storage manager with specs similar to another one.

Parameters

  name | Name of this storage manager.
  spec | Specs to initialize this class with.

References casacore::Record().

## DyscoStMan() [3/3]

dyscostman::DyscoStMan::DyscoStMan | ( | const DyscoStMan & | source | ) |

Copy constructor that initializes a storage manager with similar specs.

The columns are not copied: the new manager will be empty.

References DyscoStMan().

## ~DyscoStMan()

dyscostman::DyscoStMan::~DyscoStMan | ( |  | ) |

Destructor.

## Member Function Documentation

## addColumn()

virtual void dyscostman::DyscoStMan::addColumn | ( | casacore::DataManagerColumn * |  | ) |

final override private virtual

Do the final addition of a column.

Reimplemented from casacore::DataManager.

References addColumn().

Referenced by addColumn().

## addRow64()

virtual void dyscostman::DyscoStMan::addRow64 | ( | casacore::rownr_t | nrrow | ) |

final override private virtual

Add rows to the storage manager.

References addRow64().

Referenced by addRow64().

## areOffsetsInitialized()

bool dyscostman::DyscoStMan::areOffsetsInitialized | ( |  | ) | const

inline protected

This method returns `true` when the number of rows per block and the number of antennae per block are known.

This is only the case once the first time- block was written to the file.

Returns
  True when the nr of rows per block and antennae are available.

Definition at line 296 of file dyscostman.h.

References _rowsPerBlock.

## canAddColumn()

virtual casacore::Bool dyscostman::DyscoStMan::canAddColumn | ( |  | ) | const

inline final override virtual

Whether columns can be added.

Returns
  `true` (but restrictions apply; columns can only be added as long as no writes have been performed on the set).

Reimplemented from casacore::DataManager.

Definition at line 202 of file dyscostman.h.

## canAddRow()

virtual casacore::Bool dyscostman::DyscoStMan::canAddRow | ( |  | ) | const

inline final override virtual

Whether rows can be added.

Returns
  `true`

Reimplemented from casacore::DataManager.

Definition at line 189 of file dyscostman.h.

## canRemoveColumn()

virtual casacore::Bool dyscostman::DyscoStMan::canRemoveColumn | ( |  | ) | const

inline final override virtual

Whether columns can be removed.

Returns
  `true` (but restrictions apply – still to be checked)

Todo
  Describe restrictons

Reimplemented from casacore::DataManager.

Definition at line 209 of file dyscostman.h.

## canRemoveRow()

virtual casacore::Bool dyscostman::DyscoStMan::canRemoveRow | ( |  | ) | const

inline final override virtual

Whether rows can be removed.

Returns
  `true` (but only rows at the end can actually be removed)

Reimplemented from casacore::DataManager.

Definition at line 195 of file dyscostman.h.

## clone()

virtual casacore::DataManager * dyscostman::DyscoStMan::clone | ( |  | ) | const

inline final override virtual

Polymorphical copy constructor, equal to DyscoStMan(const DyscoStMan&).

Returns
  Empty manager with specs as the source.

Implements casacore::DataManager.

Definition at line 158 of file dyscostman.h.

References DyscoStMan().

## create64()

virtual void dyscostman::DyscoStMan::create64 | ( | casacore::rownr_t | nRow | ) |

final override private virtual

Let the storage manager create files as needed for a new table.

This allows a column with an indirect array to create its file.

## dataManagerName()

virtual casacore::String dyscostman::DyscoStMan::dataManagerName | ( |  | ) | const

inline final override virtual

Returns the name of this manager as specified during construction.

Reimplemented from casacore::DataManager.

Definition at line 169 of file dyscostman.h.

References _name.

## dataManagerSpec()

virtual casacore::Record dyscostman::DyscoStMan::dataManagerSpec | ( |  | ) | const

final override virtual

Get manager specifications.

Includes method settings, etc. Can be used to make a second storage manager with

Returns
  Record containing data manager specifications.

Reimplemented from casacore::DataManager.

References casacore::Record().

## dataManagerType()

virtual casacore::String dyscostman::DyscoStMan::dataManagerType | ( |  | ) | const

inline final override virtual

Type of manager.

Returns
  "DyscoStMan".

Implements casacore::DataManager.

Definition at line 164 of file dyscostman.h.

## deleteManager()

virtual void dyscostman::DyscoStMan::deleteManager | ( |  | ) |

final override private virtual

The data manager will be deleted (because all its columns are requested to be deleted).

So clean up the things needed (e.g. delete files).

Implements casacore::DataManager.

## flush()

virtual casacore::Bool dyscostman::DyscoStMan::flush | ( | casacore::AipsIO & | ,
|  | casacore::Bool | doFsync )

final override private virtual

Flush and optionally fsync the data.

The AipsIO stream represents the main table file and can be used by virtual column engines to store SMALL amounts of data.

Implements casacore::DataManager.

## getBlockIndex()

size_t dyscostman::DyscoStMan::getBlockIndex | ( | uint64_t | row | ) | const

inline protected

Return index of block that contains the given measurement set row.

This can only be calculated after a first time block was written (see areOffsetsInitialized()).

Parameters

  row | A measurement set row.

Returns
  Block index.

Definition at line 269 of file dyscostman.h.

References _rowsPerBlock.

## getFileOffset()

size_t dyscostman::DyscoStMan::getFileOffset | ( | size_t | blockIndex | ) | const

inline private

Definition at line 326 of file dyscostman.h.

References _blockSize, and _headerSize.

## getNRow()

uint dyscostman::DyscoStMan::getNRow | ( |  | ) | const

inline

Get the number of rows in the measurement set.

Returns
  Number of rows in the measurement set.

Definition at line 183 of file dyscostman.h.

References _nRow, and getNRow().

Referenced by getNRow().

## getRowIndex()

uint64_t dyscostman::DyscoStMan::getRowIndex | ( | size_t | block | ) | const

inline protected

Calculate first measurement set row index of a given block index.

Parameters

  block | A block index

Returns
  First measurement set row index of given block.

Definition at line 286 of file dyscostman.h.

References _rowsPerBlock.

## getRowWithinBlock()

size_t dyscostman::DyscoStMan::getRowWithinBlock | ( | uint64_t | row | ) | const

inline protected

Return the offset of the row within the block.

This can only be calculated after a first time block was written (see areOffsetsInitialized()).

See also
  getBlockIndex().

Parameters

  row | A measurement set row.

Returns
  offset of row within block.

Definition at line 279 of file dyscostman.h.

References _rowsPerBlock.

## initializeRowsPerBlock()

void dyscostman::DyscoStMan::initializeRowsPerBlock | ( | size_t | rowsPerBlock,
|  | size_t | antennaCount,
|  | bool | writeToHeader )

protected

To be called by a column once it determines rowsPerBlock and antennaCount.

Parameters

  rowsPerBlock | Number of measurement set rows in one time block.
  antennaCount | Highest antenna index+1 used in a time block.
  writeToHeader | Write the header?

## makeDirArrColumn()

virtual casacore::DataManagerColumn * dyscostman::DyscoStMan::makeDirArrColumn | ( | const casacore::String & | name,
|  | int | dataType,
|  | const casacore::String & | dataTypeID )

final override private virtual

Create a direct array column.

Implements casacore::DataManager.

## makeEmpty()

void dyscostman::DyscoStMan::makeEmpty | ( |  | ) |

private

## makeIndArrColumn()

virtual casacore::DataManagerColumn * dyscostman::DyscoStMan::makeIndArrColumn | ( | const casacore::String & | name,
|  | int | dataType,
|  | const casacore::String & | dataTypeID )

final override private virtual

Create an indirect array column.

Implements casacore::DataManager.

## makeObject()

casacore::DataManager * dyscostman::DyscoStMan::makeObject | ( | const casacore::String & | name,
|  | const casacore::Record & | spec )

inline static

Create an object with given name and spec.

This methods gets registered in the DataManager "constructor" map. The caller has to delete the object. New class will be initialized via DyscoStMan(const casacore::String& name, const casacore::Record& spec).

Returns
  A DyscoStMan with given specs.

Definition at line 219 of file dyscostman.h.

References DyscoStMan(), and casacore::Record().

## makeScalarColumn()

virtual casacore::DataManagerColumn * dyscostman::DyscoStMan::makeScalarColumn | ( | const casacore::String & | name,
|  | int | dataType,
|  | const casacore::String & | dataTypeID )

final override private virtual

Create a column in the storage manager on behalf of a table column.

The caller will NOT delete the newly created object. Create a scalar column.

Implements casacore::DataManager.

## nAntennae()

size_t dyscostman::DyscoStMan::nAntennae | ( |  | ) | const

inline protected

Number of antennae used in a time block.

This does not have to be equal to the number of antennae stored in the measurement set. This value is only available after a first time block was written (see areOffsetsInitialized()).

Returns
  Number of antennae.

Definition at line 260 of file dyscostman.h.

References _antennaCount.

## nBlocksInFile()

uint64_t dyscostman::DyscoStMan::nBlocksInFile | ( |  | ) | const

inline protected

The number of rows that are actually stored in the file.

This method is synchronized (i.e., thread-safe).

Definition at line 239 of file dyscostman.h.

References _mutex, and _nBlocksInFile.

## nRowsInBlock()

size_t dyscostman::DyscoStMan::nRowsInBlock | ( |  | ) | const

inline protected

Number of rows in one "time-block", i.e.

a sequence of rows that belong to the same timestep, spw and field. This value is only available after a first time block was written (see areOffsetsInitialized()).

Returns
  Number of measurement set rows in one time block.

Definition at line 251 of file dyscostman.h.

References _rowsPerBlock.

## open64()

virtual casacore::rownr_t dyscostman::DyscoStMan::open64 | ( | casacore::rownr_t | nRow,
|  | casacore::AipsIO & | )

final override private virtual

Open the storage manager file for an existing table.

Return the number of rows in the data file.

## operator=()

DyscoStMan & dyscostman::DyscoStMan::operator= | ( | const DyscoStMan & | source | ) |

delete

Assignment – new dyscostman takes the settings of the source (but not the columns and/or data).

Parameters

  source | Source manager.

References DyscoStMan().

## prepare()

virtual void dyscostman::DyscoStMan::prepare | ( |  | ) |

final override private virtual

Prepare the columns, let the data manager initialize itself further.

Prepare is called after create/open has been called for all columns. In this way one can be sure that referenced columns are read back and partly initialized.

Reimplemented from casacore::DataManager.

References prepare().

Referenced by prepare().

## readCompressedData()

void dyscostman::DyscoStMan::readCompressedData | ( | size_t | blockIndex,
|  | const DyscoStManColumn * | column,
|  | unsigned char * | dest,
|  | size_t | size )

private

References DyscoStManColumn.

## readHeader()

void dyscostman::DyscoStMan::readHeader | ( |  | ) |

private

## registerClass()

void dyscostman::DyscoStMan::registerClass | ( |  | ) |

static

This function makes the DyscoStMan known to casacore.

The function is necessary for loading the storage manager from a shared library. It should have this specific name ("register_" + storage manager's name in lowercase) to be able to be automatically called when the library is loaded. That function will forward the call here.

## removeColumn()

virtual void dyscostman::DyscoStMan::removeColumn | ( | casacore::DataManagerColumn * |  | ) |

final override private virtual

Remove a column from the data file.

Reimplemented from casacore::DataManager.

References removeColumn().

Referenced by removeColumn().

## removeRow64()

virtual void dyscostman::DyscoStMan::removeRow64 | ( | casacore::rownr_t | rowNr | ) |

final override private virtual

Delete a row from all columns.

References removeRow64().

Referenced by removeRow64().

## reopenRW()

virtual void dyscostman::DyscoStMan::reopenRW | ( |  | ) |

final override private virtual

Reopen the storage manager files for read/write.

Reimplemented from casacore::DataManager.

References reopenRW().

Referenced by reopenRW().

## resync64()

virtual casacore::rownr_t dyscostman::DyscoStMan::resync64 | ( | casacore::rownr_t | nRow | ) |

final override private virtual

## setFromSpec()

void dyscostman::DyscoStMan::setFromSpec | ( | const casacore::Record & | spec | ) |

private

References casacore::Record().

## SetGaussianDistribution()

void dyscostman::DyscoStMan::SetGaussianDistribution | ( |  | ) |

inline

Initialize the storage manager to use a Gaussian distribution for the quantization.

This method should only be called directly after creating DyscoStMan, before adding columns, and reading/writing data.

In tests with MWA and LOFAR data, the Gaussian distribution showed lesser compression accuracy compared to the truncated Gaussian and uniform distributions.

See also
  SetUniformDistribution(), SetTruncatedGaussianDistribution()

Definition at line 72 of file dyscostman.h.

References _distribution, and dyscostman::GaussianDistribution.

## SetNormalization()

void dyscostman::DyscoStMan::SetNormalization | ( | Normalization | normalization | ) |

inline

Set the type of normalization.

This method should only be called directly after creating DyscoStMan, before adding columns, and reading/writing data.

Definition at line 121 of file dyscostman.h.

References _normalization.

## SetStaticSeed()

void dyscostman::DyscoStMan::SetStaticSeed | ( | bool | staticSeed | ) |

inline

Definition at line 125 of file dyscostman.h.

References _staticSeed.

## SetStudentsTDistribution()

void dyscostman::DyscoStMan::SetStudentsTDistribution | ( | double | nu | ) |

inline

Initialize the storage manager to use a Student T distribution for the quantization (i.e., use a linear quantizer).

This method should only be called directly after creating DyscoStMan, before adding columns, and reading/writing data.

The Student T distribution performed not very well on test sets, and was mainly added for testing.

Definition at line 95 of file dyscostman.h.

References _distribution, _studentTNu, and dyscostman::StudentsTDistribution.

## SetTruncatedGaussianDistribution()

void dyscostman::DyscoStMan::SetTruncatedGaussianDistribution | ( | double | truncationSigma | ) |

inline

Initialize the storage manager to use a Uniform distribution for the quantization (i.e., use a linear quantizer).

This method should only be called directly after creating DyscoStMan, before adding columns, and reading/writing data.

In tests with MWA and LOFAR data, the truncated Gaussian distribution with a sigma of 1.5 to 2.5 is the recommended distribution.

Parameters

  truncationSigma | At which point the distribution is truncated. Good values are 1.5 to 2.5.

Definition at line 111 of file dyscostman.h.

References _distribution, _distributionTruncation, and dyscostman::TruncatedGaussianDistribution.

## SetUniformDistribution()

void dyscostman::DyscoStMan::SetUniformDistribution | ( |  | ) |

inline

Initialize the storage manager to use a Uniform distribution for the quantization (i.e., use a linear quantizer).

This method should only be called directly after creating DyscoStMan, before adding columns, and reading/writing data.

In tests with MWA and LOFAR data, the Uniform distribution showed very good results, only the truncated Gaussian distribution showed better results for some cases.

Definition at line 84 of file dyscostman.h.

References _distribution, and dyscostman::UniformDistribution.

## writeCompressedData()

void dyscostman::DyscoStMan::writeCompressedData | ( | size_t | blockIndex,
|  | const DyscoStManColumn * | column,
|  | const unsigned char * | data,
|  | size_t | size )

private

References DyscoStManColumn.

## writeHeader()

void dyscostman::DyscoStMan::writeHeader | ( |  | ) |

private

## Friends And Related Symbol Documentation

## DyscoStManColumn

friend class DyscoStManColumn

friend

Definition at line 308 of file dyscostman.h.

References DyscoStManColumn.

Referenced by DyscoStManColumn, readCompressedData(), and writeCompressedData().

## Member Data Documentation

## _antennaCount

uint32_t dyscostman::DyscoStMan::_antennaCount

private

Definition at line 389 of file dyscostman.h.

Referenced by nAntennae().

## _blockSize

uint32_t dyscostman::DyscoStMan::_blockSize

private

Definition at line 390 of file dyscostman.h.

Referenced by getFileOffset().

## _columns

std::vector<std::unique_ptr<DyscoStManColumn> > dyscostman::DyscoStMan::_columns

private

Definition at line 404 of file dyscostman.h.

## _dataBitCount

unsigned dyscostman::DyscoStMan::_dataBitCount

private

Definition at line 397 of file dyscostman.h.

## _distribution

DyscoDistribution dyscostman::DyscoStMan::_distribution

private

Definition at line 399 of file dyscostman.h.

Referenced by SetGaussianDistribution(), SetStudentsTDistribution(), SetTruncatedGaussianDistribution(), and SetUniformDistribution().

## _distributionTruncation

double dyscostman::DyscoStMan::_distributionTruncation

private

Definition at line 401 of file dyscostman.h.

Referenced by SetTruncatedGaussianDistribution().

## _fStream

std::unique_ptr<std::fstream> dyscostman::DyscoStMan::_fStream

private

Definition at line 394 of file dyscostman.h.

## _headerSize

unsigned dyscostman::DyscoStMan::_headerSize

private

Definition at line 392 of file dyscostman.h.

Referenced by getFileOffset().

## _mutex

std::mutex dyscostman::DyscoStMan::_mutex

mutable private

Definition at line 393 of file dyscostman.h.

Referenced by nBlocksInFile().

## _name

std::string dyscostman::DyscoStMan::_name

private

Definition at line 396 of file dyscostman.h.

Referenced by dataManagerName().

## _nBlocksInFile

uint64_t dyscostman::DyscoStMan::_nBlocksInFile

private

Definition at line 387 of file dyscostman.h.

Referenced by nBlocksInFile().

## _normalization

Normalization dyscostman::DyscoStMan::_normalization

private

Definition at line 400 of file dyscostman.h.

Referenced by SetNormalization().

## _nRow

uint64_t dyscostman::DyscoStMan::_nRow

private

Definition at line 386 of file dyscostman.h.

Referenced by getNRow().

## _rowsPerBlock

uint32_t dyscostman::DyscoStMan::_rowsPerBlock

private

Definition at line 388 of file dyscostman.h.

Referenced by areOffsetsInitialized(), getBlockIndex(), getRowIndex(), getRowWithinBlock(), and nRowsInBlock().

## _staticSeed

bool dyscostman::DyscoStMan::_staticSeed

private

Definition at line 402 of file dyscostman.h.

Referenced by SetStaticSeed().

## _studentTNu

double dyscostman::DyscoStMan::_studentTNu

private

Definition at line 401 of file dyscostman.h.

Referenced by SetStudentsTDistribution().

## _weightBitCount

unsigned dyscostman::DyscoStMan::_weightBitCount

private

Definition at line 398 of file dyscostman.h.

## VERSION_MAJOR

const unsigned short dyscostman::DyscoStMan::VERSION_MAJOR

static private

Definition at line 310 of file dyscostman.h.

## VERSION_MINOR

const unsigned short dyscostman::DyscoStMan::VERSION_MINOR

static private

Definition at line 310 of file dyscostman.h.

The documentation for this class was generated from the following file:

- tables/Dysco/dyscostman.h