Loading...
Searching...
No Matches
casacore::BucketCache Class Reference
casa package (libcasa_casa) » IO_module

Cache for buckets in a part of a file. More...

`#include <BucketCache.h>`

## Public Member Functions

| BucketCache (BucketFile *file, Int64 startOffset, uInt bucketSize, uInt nrOfBuckets, uInt cacheSize, void *ownerObject, BucketCacheToLocal readCallBack, BucketCacheFromLocal writeCallBack, BucketCacheAddBuffer addCallBack, BucketCacheDeleteBuffer deleteCallBack)
| Create the cache for (a part of) a file.
| ~BucketCache ()
Bool | flush (uInt fromSlot=0)
| Flush the cache from the given slot on.
void | clear (uInt fromSlot=0, Bool doFlush=True)
| Clear the cache from the given slot on.
void | resize (uInt cacheSize)
| Resize the cache.
void | resync (uInt nrBucket, uInt nrOfFreeBucket, Int firstFreeBucket)
| Resynchronize the object (after another process updated the file).
uInt | nBucket () const
| Get the current nr of buckets in the file.
uInt | cacheSize () const
| Get the current cache size (in buckets).
void | setDirty ()
| Set the dirty bit for the current bucket.
char * | getBucket (uInt bucketNr)
| Make another bucket current.
void | extend (uInt nrBucket)
| Extend the file with the given number of buckets.
uInt | addBucket (char *data)
| Add a bucket to the file and make it the current one.
void | removeBucket ()
| Remove the current bucket; i.e.
void | get (char *buf, uInt length, Int64 offset)
| Get a part from the file outside the cached area.
void | put (const char *buf, uInt length, Int64 offset)
| Put a part from the file outside the cached area.
Int | firstFreeBucket () const
| Get the bucket number of the first free bucket.
uInt | nFreeBucket () const
| Get the number of free buckets.
void | initStatistics ()
| (Re)initialize the cache statistics.
void | showStatistics (ostream &os) const
| Show the statistics.

## Private Member Functions

| BucketCache (const BucketCache &)
| Copy constructor is not possible.
BucketCache & | operator= (const BucketCache &)
| Assignment is not possible.
void | setLRU ()
| Set the LRU information for the current slot.
void | getSlot (uInt bucketNr)
| Get a cache slot for the bucket.
void | writeBucket (uInt slotNr)
| Write a bucket.
void | readBucket (uInt slotNr)
| Read a bucket.
void | initializeBuckets (uInt bucketNr)
| Initialize the bucket buffer.
void | checkOffset (uInt length, Int64 offset) const
| Check if the offset of a non-cached part is correct.

## Private Attributes

BucketFile * | its_file
| The file used.
void * | its_Owner
| The owner object.
BucketCacheToLocal | its_ReadCallBack
| The read callback function.
BucketCacheFromLocal | its_WriteCallBack
| The write callback function.
BucketCacheAddBuffer | its_InitCallBack
| The add bucket callback function.
BucketCacheDeleteBuffer | its_DeleteCallBack
| The delete callback function.
Int64 | its_StartOffset
| The starting offsets of the buckets in the file.
uInt | its_BucketSize
| The bucket size.
uInt | its_CurNrOfBuckets
| The current nr of buckets in the file.
uInt | its_NewNrOfBuckets
| The new nr of buckets in the file (after extension).
uInt | its_CacheSize
| The size of the cache (i.e.
uInt | its_CacheSizeUsed
| The nr of slots used in the cache.
PtrBlock< char * > | its_Cache
| The cache itself.
uInt | its_ActualSlot
| The cache slot actually used.
Block< Int > | its_SlotNr
| The slot numbers of the buckets in the cache (-1 = not in cache).
Block< uInt > | its_BucketNr
| The buckets in the cache.
Block< uInt > | its_Dirty
| Determine if a block is dirty (i.e.
Block< uInt > | its_LRU
| Determine when a block is used for the last time.
uInt | its_LRUCounter
| The Least Recently Used counter.
char * | its_Buffer
| The internal buffer.
uInt | its_NrOfFree
| The number of free buckets.
Int | its_FirstFree
| The first free bucket (-1 = no free buckets).
uInt | naccess_p
| The statistics.
uInt | nread_p
uInt | ninit_p
uInt | nwrite_p

## Detailed Description

Cache for buckets in a part of a file.

### Intended use:

Public interface

### Review Status

Reviewed By:
  UNKNOWN
Date Reviewed:
  before2004/08/25

### Prerequisite

- BucketFile

### Etymology

BucketCache implements a cache for buckets in (a part of) a file.

### Synopsis

A cache may allow more efficient quasi-random IO. It can, for instance, be used when a limited number of blocks in a file have to be accessed again and again.

The class BucketCache provides such a cache. It can be used on a consecutive part of a file as long as that part is not simultaneously accessed in another way (including another BucketCache object).

BucketCache stores the data as given. It uses (see (group=BucketCache_CallBack))callback functions to allocate/delete buffers and to convert the data to/from local format.

When a new bucket is needed and all slots in the cache are used, BucketCache will remove the least recently used bucket from the cache. When the dirty flag is set, it will first be written.

BucketCache maintains a list of free buckets. Initially this list is empty. When a bucket is removed, it is added to the free list. AddBucket will take buckets from the free list before extending the file.

Since it is possible to handle only a part of a file by a BucketCache object, it is also possible to have multiple BucketCache objects on the same file (as long as they access disjoint parts of the file). Each BucketCache object can have its own bucket size. This can, for example, be used to have tiled arrays with different tile shapes in the same file.

Statistics are kept to know how efficient the cache is working. It is possible to initialize and show the statistics.

### Motivation

A cache may reduce IO traffix considerably. Furthermore it is more efficient to keep a cache in local format. In that way conversion to/from local only have to be done when data gets read/written. It also allows for precalculations.

### Example

// Define the callback function for reading a bucket.
char* bToLocal (void*, const char* data)
{
char* ptr = new char[32768];
memcpy (ptr, data, 32768);
return ptr;
}
// Define the callback function for writing a bucket.
void bFromLocal (void*, char* data, const char* local)
{
memcpy (data, local, 32768);
}
// Define the callback function for initializing a new bucket.
char* bAddBuffer (void*)
{
char* ptr = new char[32768];
for (uInt i=0; i++; i<32768) {
ptr[i] = 0;
}
return ptr;
}
// Define the callback function for deleting a bucket.
void bDeleteBuffer (void*, char* buffer)
{
delete [] buffer;
}
void someFunc()
{
// Open the filebuf.
BucketFile file(...);
file.open();
uInt i;
// Create a cache for the part of the file starting at offset 512
// consisting of 1000 buckets. The cache consists of 10 buckets.
// Each bucket is 32768 bytes.
BucketCache cache (&file, 512, 32768, 1000, 10, 0,
bToLocal, bFromLocal, bAddBuffer, bDeleteBuffer);
// Write all buckets into the file.
for (i=0; i<100; i++) {
char* buf = new char[32768];
cache.addBucket (buf);
}
Flush the cache to write all buckets in it.
cache.flush();
// Read all buckets from the file.
for (i=0; i<1000; i++) {
char* buf = cache.getBucket(i);
..\.
}
cout << cache.nBucket() << endl;
}
casacore::BucketCache::BucketCache
BucketCache(BucketFile *file, Int64 startOffset, uInt bucketSize, uInt nrOfBuckets, uInt cacheSize, void *ownerObject, BucketCacheToLocal readCallBack, BucketCacheFromLocal writeCallBack, BucketCacheAddBuffer addCallBack, BucketCacheDeleteBuffer deleteCallBack)
Create the cache for (a part of) a file.
casacore::uInt
unsigned int uInt
Definition aipstype.h:49
casacore::write
virtual int write(FitsOutput &)
casacore::all
LatticeExprNode all(const LatticeExprNode &expr)

### To Do

- When ready, use HashMap for the internal maps.

Definition at line 215 of file BucketCache.h.

## Constructor & Destructor Documentation

## BucketCache() [1/2]

casacore::BucketCache::BucketCache | ( | BucketFile * | file,
|  | Int64 | startOffset,
|  | uInt | bucketSize,
|  | uInt | nrOfBuckets,
|  | uInt | cacheSize,
|  | void * | ownerObject,
|  | BucketCacheToLocal | readCallBack,
|  | BucketCacheFromLocal | writeCallBack,
|  | BucketCacheAddBuffer | addCallBack,
|  | BucketCacheDeleteBuffer | deleteCallBack )

Create the cache for (a part of) a file.

The file part used starts at startOffset. Its length is bucketSize*nrOfBuckets bytes. When the file is smaller, the remainder is indicated as an extension similarly to the behaviour of function extend.

References cacheSize().

Referenced by BucketCache(), and operator=().

## ~BucketCache()

casacore::BucketCache::~BucketCache | ( |  | ) |

## BucketCache() [2/2]

casacore::BucketCache::BucketCache | ( | const BucketCache & |  | ) |

private

Copy constructor is not possible.

References BucketCache().

## Member Function Documentation

## addBucket()

uInt casacore::BucketCache::addBucket | ( | char * | data | ) |

Add a bucket to the file and make it the current one.

When no more cache slots are available, the one least recently used is flushed.
When no free buckets are available, the file will be extended with one bucket. It returns the new bucket number. The buffer must have been allocated on the heap. It will get part of the cache; its contents are not copied. Thus the buffer should hereafter NOT be used for other purposes. It will be deleted later via the DeleteBuffer callback function. The data is copied into the bucket. A pointer to the data in local format is returned.

## cacheSize()

uInt casacore::BucketCache::cacheSize | ( |  | ) | const

inline

Get the current cache size (in buckets).

Definition at line 401 of file BucketCache.h.

References its_CacheSize.

Referenced by BucketCache(), and resize().

## checkOffset()

void casacore::BucketCache::checkOffset | ( | uInt | length,
|  | Int64 | offset ) const

private

Check if the offset of a non-cached part is correct.

References casacore::length(), and casacore::offset().

## clear()

void casacore::BucketCache::clear | ( | uInt | fromSlot = 0,
|  | Bool | doFlush = True )

Clear the cache from the given slot on.

By default the entire cache is cleared. It will remove the buckets in the cleared part. If wanted and needed, the buckets are flushed to the file before removing them. It can be used to enforce rereading buckets from the file.

References casacore::True.

## extend()

void casacore::BucketCache::extend | ( | uInt | nrBucket | ) |

Extend the file with the given number of buckets.

The buckets get initialized when they are acquired (using getBucket) for the first time.

## firstFreeBucket()

Int casacore::BucketCache::firstFreeBucket | ( |  | ) | const

inline

Get the bucket number of the first free bucket.

-1 = no free buckets.

Definition at line 404 of file BucketCache.h.

References its_FirstFree.

Referenced by resync().

## flush()

Bool casacore::BucketCache::flush | ( | uInt | fromSlot = 0 | ) |

Flush the cache from the given slot on.

By default the entire cache is flushed. When the entire cache is flushed, possible remaining uninitialized buckets will be initialized first. A True status is returned when buckets had to be written.

## get()

void casacore::BucketCache::get | ( | char * | buf,
|  | uInt | length,
|  | Int64 | offset )

Get a part from the file outside the cached area.

It is checked if that part is indeed outside the cached file area.

References casacore::length(), and casacore::offset().

## getBucket()

char * casacore::BucketCache::getBucket | ( | uInt | bucketNr | ) |

Make another bucket current.

When no more cache slots are available, the one least recently used is flushed. The data in the bucket is converted using the ToLocal callback function. When the bucket does not exist yet in the file, it gets added and initialized using the AddBuffer callback function. A pointer to the data in converted format is returned.

## getSlot()

void casacore::BucketCache::getSlot | ( | uInt | bucketNr | ) |

private

Get a cache slot for the bucket.

## initializeBuckets()

void casacore::BucketCache::initializeBuckets | ( | uInt | bucketNr | ) |

private

Initialize the bucket buffer.

The uninitialized buckets before this bucket are also initialized. It returns a pointer to the buffer.

## initStatistics()

void casacore::BucketCache::initStatistics | ( |  | ) |

(Re)initialize the cache statistics.

## nBucket()

uInt casacore::BucketCache::nBucket | ( |  | ) | const

Get the current nr of buckets in the file.

## nFreeBucket()

uInt casacore::BucketCache::nFreeBucket | ( |  | ) | const

inline

Get the number of free buckets.

Definition at line 407 of file BucketCache.h.

References its_NrOfFree.

## operator=()

BucketCache & casacore::BucketCache::operator= | ( | const BucketCache & |  | ) |

private

Assignment is not possible.

References BucketCache().

## put()

void casacore::BucketCache::put | ( | const char * | buf,
|  | uInt | length,
|  | Int64 | offset )

Put a part from the file outside the cached area.

It is checked if that part is indeed outside the cached file area.

References casacore::length(), and casacore::offset().

## readBucket()

void casacore::BucketCache::readBucket | ( | uInt | slotNr | ) |

private

Read a bucket.

## removeBucket()

void casacore::BucketCache::removeBucket | ( |  | ) |

Remove the current bucket; i.e.

add it to the beginning of the free bucket list.

## resize()

void casacore::BucketCache::resize | ( | uInt | cacheSize | ) |

Resize the cache.

When the cache gets smaller, the latter buckets are cached out. It does not take "least recently used" into account.

References cacheSize().

## resync()

void casacore::BucketCache::resync | ( | uInt | nrBucket,
|  | uInt | nrOfFreeBucket,
|  | Int | firstFreeBucket )

Resynchronize the object (after another process updated the file).

It clears the cache (so all data will be reread) and sets the new sizes.

References firstFreeBucket().

## setDirty()

void casacore::BucketCache::setDirty | ( |  | ) |

Set the dirty bit for the current bucket.

## setLRU()

void casacore::BucketCache::setLRU | ( |  | ) |

private

Set the LRU information for the current slot.

## showStatistics()

void casacore::BucketCache::showStatistics | ( | ostream & | os | ) | const

Show the statistics.

## writeBucket()

void casacore::BucketCache::writeBucket | ( | uInt | slotNr | ) |

private

Write a bucket.

## Member Data Documentation

## its_ActualSlot

uInt casacore::BucketCache::its_ActualSlot

private

The cache slot actually used.

Definition at line 348 of file BucketCache.h.

## its_BucketNr

Block<uInt> casacore::BucketCache::its_BucketNr

private

The buckets in the cache.

Definition at line 352 of file BucketCache.h.

## its_BucketSize

uInt casacore::BucketCache::its_BucketSize

private

The bucket size.

Definition at line 336 of file BucketCache.h.

## its_Buffer

char* casacore::BucketCache::its_Buffer

private

The internal buffer.

Definition at line 360 of file BucketCache.h.

## its_Cache

PtrBlock<char*> casacore::BucketCache::its_Cache

private

The cache itself.

Definition at line 346 of file BucketCache.h.

## its_CacheSize

uInt casacore::BucketCache::its_CacheSize

private

The size of the cache (i.e.

#buckets fitting in it).

Definition at line 342 of file BucketCache.h.

Referenced by cacheSize().

## its_CacheSizeUsed

uInt casacore::BucketCache::its_CacheSizeUsed

private

The nr of slots used in the cache.

Definition at line 344 of file BucketCache.h.

## its_CurNrOfBuckets

uInt casacore::BucketCache::its_CurNrOfBuckets

private

The current nr of buckets in the file.

Definition at line 338 of file BucketCache.h.

## its_DeleteCallBack

BucketCacheDeleteBuffer casacore::BucketCache::its_DeleteCallBack

private

The delete callback function.

Definition at line 332 of file BucketCache.h.

## its_Dirty

Block<uInt> casacore::BucketCache::its_Dirty

private

Determine if a block is dirty (i.e.

changed) (1=dirty).

Definition at line 354 of file BucketCache.h.

## its_file

BucketFile* casacore::BucketCache::its_file

private

The file used.

Definition at line 322 of file BucketCache.h.

## its_FirstFree

Int casacore::BucketCache::its_FirstFree

private

The first free bucket (-1 = no free buckets).

Definition at line 364 of file BucketCache.h.

Referenced by firstFreeBucket().

## its_InitCallBack

BucketCacheAddBuffer casacore::BucketCache::its_InitCallBack

private

The add bucket callback function.

Definition at line 330 of file BucketCache.h.

## its_LRU

Block<uInt> casacore::BucketCache::its_LRU

private

Determine when a block is used for the last time.

Definition at line 356 of file BucketCache.h.

## its_LRUCounter

uInt casacore::BucketCache::its_LRUCounter

private

The Least Recently Used counter.

Definition at line 358 of file BucketCache.h.

## its_NewNrOfBuckets

uInt casacore::BucketCache::its_NewNrOfBuckets

private

The new nr of buckets in the file (after extension).

Definition at line 340 of file BucketCache.h.

## its_NrOfFree

uInt casacore::BucketCache::its_NrOfFree

private

The number of free buckets.

Definition at line 362 of file BucketCache.h.

Referenced by nFreeBucket().

## its_Owner

void* casacore::BucketCache::its_Owner

private

The owner object.

Definition at line 324 of file BucketCache.h.

## its_ReadCallBack

BucketCacheToLocal casacore::BucketCache::its_ReadCallBack

private

The read callback function.

Definition at line 326 of file BucketCache.h.

## its_SlotNr

Block<Int> casacore::BucketCache::its_SlotNr

private

The slot numbers of the buckets in the cache (-1 = not in cache).

Definition at line 350 of file BucketCache.h.

## its_StartOffset

Int64 casacore::BucketCache::its_StartOffset

private

The starting offsets of the buckets in the file.

Definition at line 334 of file BucketCache.h.

## its_WriteCallBack

BucketCacheFromLocal casacore::BucketCache::its_WriteCallBack

private

The write callback function.

Definition at line 328 of file BucketCache.h.

## naccess_p

uInt casacore::BucketCache::naccess_p

private

The statistics.

Definition at line 366 of file BucketCache.h.

## ninit_p

uInt casacore::BucketCache::ninit_p

private

Definition at line 368 of file BucketCache.h.

## nread_p

uInt casacore::BucketCache::nread_p

private

Definition at line 367 of file BucketCache.h.

## nwrite_p

uInt casacore::BucketCache::nwrite_p

private

Definition at line 369 of file BucketCache.h.

The documentation for this class was generated from the following file:

- casa/IO/BucketCache.h