Loading...
Searching...
No Matches
casacore::MultiFileBase Class Referenceabstract
casa package (libcasa_casa) » IO_module

Abstract base class to combine multiple logical files in a single one. More...

`#include <MultiFileBase.h>`

## Public Member Functions

| MultiFileBase (const String &name, Int blockSize, Bool useODirect)
| Create a MultiFileBase object with the given name.
virtual | ~MultiFileBase ()
| The destructor flushes dirty blocks and closes the container file.
| MultiFileBase (const MultiFileBase &)=delete
| Forbid copy constructor.
MultiFileBase & | operator= (const MultiFileBase &)=delete
| Forbid assignment.
virtual std::shared_ptr< MultiFileBase > | makeNested (const std::shared_ptr< MultiFileBase > &parent, const String &name, ByteIO::OpenOption, Int blockSize) const =0
| Make the correct MultiFileBase object for a nested file.
String | fileName () const
| Get the file name of the MultiFileBase container file.
Bool | isWritable () const
| Is the container file writable?
Int | openFile (const String &name)
| Open the given logical file and return its file id.
Int | createFile (const String &name, ByteIO::OpenOption=ByteIO::New)
| Create a new logical file and return its file id.
void | flushFile (Int fileId)
| Flush the possible dirty buffer of the given logical file.
void | closeFile (Int fileId)
| Close a logical file.
void | deleteFile (Int fileId)
| Delete a logical file.
Int64 | fileSize (Int fileId) const
| Get the size of a logical file.
Int64 | read (Int fileId, void *buffer, Int64 size, Int64 offset)
| Read a block at the given offset in the logical file.
Int64 | write (Int fileId, const void *buffer, Int64 size, Int64 offset)
| Write a block at the given offset in the logical file.
void | truncate (Int fileId, Int64 size)
| Truncate the logical file to the given size.
virtual void | reopenRW ()=0
| Reopen the underlying file for read/write access.
void | flush ()
| Flush the file by writing all dirty data and all header info.
Int64 | blockSize () const
| Get the block size used.
uInt | nfile () const
| Get the nr of logical files.
Int64 | nblock () const
| Get the total nr of data blocks used.
const std::vector< MultiFileInfo > & | info () const
| Get the info object (for test purposes mainly).
const std::vector< Int64 > & | freeBlocks () const
| Get the free blocks (for test purposes mainly).
Int | fileId (const String &name, Bool throwExcp=True) const
| Return the file id of a file in the MultiFileBase object.
Bool | useODirect () const
| Is O_DIRECT used?

## Static Public Member Functions

static std::shared_ptr< MultiFileBase > | openMF (const String &fileName)
| Open the correct MultiFileBase (as plain or HDF5).

## Protected Member Functions

void | resync ()
| Resync with another process by clearing the buffers and rereading the header.
virtual void | fsync ()=0
| Fsync the file (i.e., force the data to be physically written).
void | setNewFile ()
| Set the flags and blockSize for a new MultiFile/HDF5.

## Protected Attributes

String | itsName
Int64 | itsBlockSize
Int64 | itsNrBlock
Int64 | itsHdrCounter
std::vector< MultiFileInfo > | itsInfo
std::shared_ptr< MultiFileBuffer > | itsBuffer
Bool | itsUseODirect
Bool | itsWritable
Bool | itsChanged
std::vector< Int64 > | itsFreeBlocks

## Private Member Functions

void | writeDirty (MultiFileInfo &info)
| Write the dirty block and clear dirty flag.
Int | addFile (const String &name)
| Add a file to the MultiFileBase object.
virtual void | doOpenFile (MultiFileInfo &)=0
| Do the class-specific actions on opening a logical file.
virtual void | doCloseFile (MultiFileInfo &)=0
| Do the class-specific actions on closing a logical file.
virtual void | doAddFile (MultiFileInfo &)=0
| Do the class-specific actions on adding a logical file.
virtual void | doDeleteFile (MultiFileInfo &)=0
| Do the class-specific actions on deleting a logical file.
virtual void | doTruncateFile (MultiFileInfo &info, uInt64 nrblk)=0
| Truncate the container file to nrblk blocks.
virtual void | doFlushFile ()=0
| Flush the container file.
virtual void | close ()=0
| Flush and close the container file.
virtual void | writeHeader ()=0
| Write the header info.
virtual void | readHeader (Bool always=True)=0
| Read the header info.
virtual void | extend (MultiFileInfo &info, Int64 lastblk)=0
| Extend a logical file to fit lastblk.
virtual void | writeBlock (MultiFileInfo &info, Int64 blknr, const void *buffer)=0
| Write a data block of a logical file into the container file.
virtual void | readBlock (MultiFileInfo &info, Int64 blknr, void *buffer)=0
| Read a data block of a logical file from the container file.

## Detailed Description

Abstract base class to combine multiple logical files in a single one.

### Intended use:

Public interface

### Review Status

Test programs:
  tMultiFile

### Synopsis

This class is the abstract base class for classes defining a container file holding multiple logical files. These classes are meant as a container files for the storage manager files of a table to reduce the number of files used (especially for Lustre) and to reduce the number of open files (especially when concatenating tables). The derived classes MultiFile and MultiHDF5 implement such container files using a regular file and HDF5, resp.

MultiFileBase implements several functions with common functionality for the derived classes.

A logical file is represented by an MFFileIO object, which is derived from ByteIO and as such part of the casacore IO framework. It makes it possible for applications to access a logical file in the same way as a regular file.

Definition at line 127 of file MultiFileBase.h.

## Constructor & Destructor Documentation

## MultiFileBase() [1/2]

casacore::MultiFileBase::MultiFileBase | ( | const String & | name,
|  | Int | blockSize,
|  | Bool | useODirect )

Create a MultiFileBase object with the given name.

Upon creation of the container file the block size can be given. If <=0, it uses the block size of the file system the file is on, but it will not be less than the absolute value of the given block size.
If useODIrect=True, it means that O_DIRECT is used. If the OS does not support it (as determined at configure time), the flag will always be set to False. If True, the data buffers will have a proper alignment and size (as needed by O_DIRECT).

References blockSize(), casacore::name(), and useODirect().

Referenced by MultiFileBase(), and operator=().

## ~MultiFileBase()

virtual casacore::MultiFileBase::~MultiFileBase | ( |  | ) |

virtual

The destructor flushes dirty blocks and closes the container file.

## MultiFileBase() [2/2]

casacore::MultiFileBase::MultiFileBase | ( | const MultiFileBase & |  | ) |

delete

Forbid copy constructor.

References MultiFileBase().

## Member Function Documentation

## addFile()

Int casacore::MultiFileBase::addFile | ( | const String & | name | ) |

private

Add a file to the MultiFileBase object.

It returns the file id. Only the base name of the given file name is used. In this way the MultiFileBase container file can be moved.

References casacore::name().

## blockSize()

Int64 casacore::MultiFileBase::blockSize | ( |  | ) | const

inline

Get the block size used.

Definition at line 212 of file MultiFileBase.h.

References itsBlockSize.

Referenced by casacore::MultiFile::makeNested(), makeNested(), casacore::MultiHDF5::makeNested(), casacore::MultiFile::MultiFile(), casacore::MultiFile::MultiFile(), MultiFileBase(), casacore::MultiHDF5::MultiHDF5(), and casacore::MultiHDF5::MultiHDF5().

## close()

virtual void casacore::MultiFileBase::close | ( |  | ) |

private pure virtual

Flush and close the container file.

Implemented in casacore::MultiHDF5.

## closeFile()

void casacore::MultiFileBase::closeFile | ( | Int | fileId | ) |

Close a logical file.

It flushes and deallocates its buffer.

References fileId().

## createFile()

Int casacore::MultiFileBase::createFile | ( | const String & | name,
|  | ByteIO::OpenOption | = ByteIO::New )

Create a new logical file and return its file id.

Only the base name of the given file name is used. In this way the MultiFileBase container file can be moved. If the logical file already exists, it is deleted if ByteIO::New is given. Otherwise an exception is thrown. It allocates the internal buffer of the logical file.

References casacore::name(), and casacore::ByteIO::New.

## deleteFile()

void casacore::MultiFileBase::deleteFile | ( | Int | fileId | ) |

Delete a logical file.

It adds its blocks to the free block list.

References fileId().

## doAddFile()

virtual void casacore::MultiFileBase::doAddFile | ( | MultiFileInfo & |  | ) |

private pure virtual

Do the class-specific actions on adding a logical file.

Implemented in casacore::MultiHDF5.

## doCloseFile()

virtual void casacore::MultiFileBase::doCloseFile | ( | MultiFileInfo & |  | ) |

private pure virtual

Do the class-specific actions on closing a logical file.

Implemented in casacore::MultiHDF5.

## doDeleteFile()

virtual void casacore::MultiFileBase::doDeleteFile | ( | MultiFileInfo & |  | ) |

private pure virtual

Do the class-specific actions on deleting a logical file.

Implemented in casacore::MultiHDF5.

## doFlushFile()

virtual void casacore::MultiFileBase::doFlushFile | ( |  | ) |

private pure virtual

Flush the container file.

Implemented in casacore::MultiHDF5.

## doOpenFile()

virtual void casacore::MultiFileBase::doOpenFile | ( | MultiFileInfo & |  | ) |

private pure virtual

Do the class-specific actions on opening a logical file.

Implemented in casacore::MultiHDF5.

## doTruncateFile()

virtual void casacore::MultiFileBase::doTruncateFile | ( | MultiFileInfo & | info,
|  | uInt64 | nrblk )

private pure virtual

Truncate the container file to nrblk blocks.

Implemented in casacore::MultiHDF5.

References info().

## extend()

virtual void casacore::MultiFileBase::extend | ( | MultiFileInfo & | info,
|  | Int64 | lastblk )

private pure virtual

Extend a logical file to fit lastblk.

Implemented in casacore::MultiHDF5.

References info().

## fileId()

Int casacore::MultiFileBase::fileId | ( | const String & | name,
|  | Bool | throwExcp = True ) const

Return the file id of a file in the MultiFileBase object.

If the name is unknown, an exception is thrown if throwExcp is set. Otherwise it returns -1.

References casacore::name(), and casacore::True.

Referenced by closeFile(), deleteFile(), fileSize(), flushFile(), read(), truncate(), and write().

## fileName()

String casacore::MultiFileBase::fileName | ( |  | ) | const

inline

Get the file name of the MultiFileBase container file.

Definition at line 158 of file MultiFileBase.h.

References itsName.

Referenced by openMF().

## fileSize()

Int64 casacore::MultiFileBase::fileSize | ( | Int | fileId | ) | const

Get the size of a logical file.

References fileId().

## flush()

void casacore::MultiFileBase::flush | ( |  | ) |

Flush the file by writing all dirty data and all header info.

## flushFile()

void casacore::MultiFileBase::flushFile | ( | Int | fileId | ) |

Flush the possible dirty buffer of the given logical file.

References fileId().

## freeBlocks()

const std::vector< Int64 > & casacore::MultiFileBase::freeBlocks | ( |  | ) | const

inline

Get the free blocks (for test purposes mainly).

Definition at line 227 of file MultiFileBase.h.

References itsFreeBlocks.

## fsync()

virtual void casacore::MultiFileBase::fsync | ( |  | ) |

protected pure virtual

Fsync the file (i.e., force the data to be physically written).

Implemented in casacore::MultiFile, and casacore::MultiHDF5.

## info()

const std::vector< MultiFileInfo > & casacore::MultiFileBase::info | ( |  | ) | const

inline

Get the info object (for test purposes mainly).

Definition at line 223 of file MultiFileBase.h.

References itsInfo.

Referenced by doTruncateFile(), casacore::MultiHDF5::doTruncateFile(), extend(), casacore::MultiHDF5::extend(), readBlock(), casacore::MultiHDF5::readBlock(), writeBlock(), casacore::MultiHDF5::writeBlock(), and writeDirty().

## isWritable()

Bool casacore::MultiFileBase::isWritable | ( |  | ) | const

inline

Is the container file writable?

Definition at line 162 of file MultiFileBase.h.

References itsWritable.

## makeNested()

virtual std::shared_ptr< MultiFileBase > casacore::MultiFileBase::makeNested | ( | const std::shared_ptr< MultiFileBase > & | parent,
|  | const String & | name,
|  | ByteIO::OpenOption | ,
|  | Int | blockSize ) const

pure virtual

Make the correct MultiFileBase object for a nested file.

Implemented in casacore::MultiFile, and casacore::MultiHDF5.

References blockSize(), and casacore::name().

## nblock()

Int64 casacore::MultiFileBase::nblock | ( |  | ) | const

inline

Get the total nr of data blocks used.

Definition at line 219 of file MultiFileBase.h.

References itsNrBlock.

## nfile()

uInt casacore::MultiFileBase::nfile | ( |  | ) | const

Get the nr of logical files.

## openFile()

Int casacore::MultiFileBase::openFile | ( | const String & | name | ) |

Open the given logical file and return its file id.

If the name is unknown, an exception is thrown. It allocates the internal buffer of the logical file.

References casacore::name().

## openMF()

std::shared_ptr< MultiFileBase > casacore::MultiFileBase::openMF | ( | const String & | fileName | ) |

static

Open the correct MultiFileBase (as plain or HDF5).

References fileName().

## operator=()

MultiFileBase & casacore::MultiFileBase::operator= | ( | const MultiFileBase & |  | ) |

delete

Forbid assignment.

References MultiFileBase().

## read()

Int64 casacore::MultiFileBase::read | ( | Int | fileId,
|  | void * | buffer,
|  | Int64 | size,
|  | Int64 | offset )

Read a block at the given offset in the logical file.

It returns the actual size read.

References fileId(), casacore::offset(), and casacore::size().

## readBlock()

virtual void casacore::MultiFileBase::readBlock | ( | MultiFileInfo & | info,
|  | Int64 | blknr,
|  | void * | buffer )

private pure virtual

Read a data block of a logical file from the container file.

Implemented in casacore::MultiHDF5.

References info().

## readHeader()

virtual void casacore::MultiFileBase::readHeader | ( | Bool | always = True | ) |

private pure virtual

Read the header info.

If always==False, the info is only read if the header counter has changed.

Implemented in casacore::MultiHDF5.

References casacore::True.

## reopenRW()

virtual void casacore::MultiFileBase::reopenRW | ( |  | ) |

pure virtual

Reopen the underlying file for read/write access.

Nothing will be done if the file is writable already. Otherwise it will be reopened and an exception will be thrown if it is not possible to reopen it for read/write access.

Implemented in casacore::MultiFile, and casacore::MultiHDF5.

## resync()

void casacore::MultiFileBase::resync | ( |  | ) |

protected

Resync with another process by clearing the buffers and rereading the header.

The header is only read if its counter has changed.

## setNewFile()

void casacore::MultiFileBase::setNewFile | ( |  | ) |

protected

Set the flags and blockSize for a new MultiFile/HDF5.

## truncate()

void casacore::MultiFileBase::truncate | ( | Int | fileId,
|  | Int64 | size )

Truncate the logical file to the given size.

References fileId(), and casacore::size().

## useODirect()

Bool casacore::MultiFileBase::useODirect | ( |  | ) | const

inline

Is O_DIRECT used?

Definition at line 236 of file MultiFileBase.h.

References itsUseODirect.

Referenced by casacore::MultiFile::MultiFile(), and MultiFileBase().

## write()

Int64 casacore::MultiFileBase::write | ( | Int | fileId,
|  | const void * | buffer,
|  | Int64 | size,
|  | Int64 | offset )

Write a block at the given offset in the logical file.

It returns the actual size written.

References fileId(), casacore::offset(), and casacore::size().

## writeBlock()

virtual void casacore::MultiFileBase::writeBlock | ( | MultiFileInfo & | info,
|  | Int64 | blknr,
|  | const void * | buffer )

private pure virtual

Write a data block of a logical file into the container file.

Implemented in casacore::MultiHDF5.

References info().

Referenced by writeDirty().

## writeDirty()

void casacore::MultiFileBase::writeDirty | ( | MultiFileInfo & | info | ) |

inline private

Write the dirty block and clear dirty flag.

Definition at line 249 of file MultiFileBase.h.

References casacore::False, info(), and writeBlock().

## writeHeader()

virtual void casacore::MultiFileBase::writeHeader | ( |  | ) |

private pure virtual

Write the header info.

Implemented in casacore::MultiHDF5.

## Member Data Documentation

## itsBlockSize

Int64 casacore::MultiFileBase::itsBlockSize

protected

Definition at line 294 of file MultiFileBase.h.

Referenced by blockSize().

## itsBuffer

std::shared_ptr<MultiFileBuffer> casacore::MultiFileBase::itsBuffer

protected

Definition at line 298 of file MultiFileBase.h.

## itsChanged

Bool casacore::MultiFileBase::itsChanged

protected

Definition at line 301 of file MultiFileBase.h.

## itsFreeBlocks

std::vector<Int64> casacore::MultiFileBase::itsFreeBlocks

protected

Definition at line 302 of file MultiFileBase.h.

Referenced by freeBlocks().

## itsHdrCounter

Int64 casacore::MultiFileBase::itsHdrCounter

protected

Definition at line 296 of file MultiFileBase.h.

## itsInfo

std::vector<MultiFileInfo> casacore::MultiFileBase::itsInfo

protected

Definition at line 297 of file MultiFileBase.h.

Referenced by info().

## itsName

String casacore::MultiFileBase::itsName

protected

Definition at line 293 of file MultiFileBase.h.

Referenced by fileName().

## itsNrBlock

Int64 casacore::MultiFileBase::itsNrBlock

protected

Definition at line 295 of file MultiFileBase.h.

Referenced by nblock().

## itsUseODirect

Bool casacore::MultiFileBase::itsUseODirect

protected

Definition at line 299 of file MultiFileBase.h.

Referenced by useODirect().

## itsWritable

Bool casacore::MultiFileBase::itsWritable

protected

Definition at line 300 of file MultiFileBase.h.

Referenced by isWritable().

The documentation for this class was generated from the following file:

- casa/IO/MultiFileBase.h