Figure 1
User's Guide
and
Developer's Guide
John Ambrosiano
October 18, 1999
Overview *
ByteArray *Object Storage Classes *
StorageSet *
The POOMA framework was engineered to support rapid development of numerical scientific and engineering applications. POOMA presents the user with a high-level, C++ language interface for creating numerical applications (including parallel applications) whose implementation has been optimized to achieve good performance computing platforms ranging from desktop computers to supercomputers with thousands of processors. The POOMA data abstractions and programming are very general and flexible. The framework is also user-extensible.
The POOMA I/O classes were designed to provide efficient I/O services in keeping with the design philosophy of POOMA. POOMA I/O directly supports the abstractions that make the POOMA framework powerful and flexible by making the classes that embody them persistent. In keeping with the rest of POOMA, the I/O system is both flexible and extensible, and may be extended by users as well as developers. It presents a high-level object-oriented interface to users based on a set of a low-level I/O services that can be optimized for performance.
The first part of this document describes the overall design of POOMA I/O. It then gives several use cases and examples including examples of how users can extend the system to manage classes that are not part of the core POOMA framework. The second part contains detailed design information useful to developers.
There are broadly speaking two levels of data management that are important in an object-oriented application. The first level is the essential I/O capability. The second is object persistence.
Any application that wishes to store data, whether object-oriented or not, must be able to write bytes of information to a file and reliably get them back. Within this general scope are several kinds of services. Some of these are essential, while others provide increasingly higher levels of functionality and convenience.
The most rudimentary I/O service is based on input-output bytes streams. Data is inserted into streams and extracted in the same order as stored. All the data is opaque as far as the I/O system is concerned. That is, the system does not know what data has been written and has no information about its internal structure including that of the primitive data types such as int or float. Data is stored sequentially, and although a pointer may be placed at any position in the file, the I/O system does not know where one data item ends and another begins.
The next level of sophistication in I/O organizes the file into blocks or records. Although the records themselves may be opaque to the I/O system, it can be instructed to go to the start of any block and read or write an entire record or sequence of records. A convenient abstraction for this type of organization is a record array. In a system of this kind, storage is organized into a collection of arrays, each array itself being a collection of records. In advanced systems of this type [e.g., hdf5], each record type is known and understood. A type may be an instance of a standard atomic type such as int or double, or a compound data type equivalent to a C struct composed of of several atomic types as well as recursive references to other compound types. With type information supported by the I/O system, it becomes possible to automatically convert data representations when moving data from one computing environment to another.
Both simple and record-oriented I/O systems can be enhanced to support parallel I/O using a variety of strategies [e.g., parallel hdf5, MPI-I/O].
Object Persistence and Storage
An I/O system that provides object storage recognizes object types and allows them to be stored and retrieved as whole entities.
The simplest form of object persistence is object serialization [e.g. JAVA]. Like a simple I/O stream, an I/O system of this kind serializes objects and inserts them into a stream. The objects are retrieved in their entirety, but must be retrieved in the order stored. Typically the system does not have any knowledge of a file's contents. Only the producing and consuming applications have this information.
The next level of sophistication in object storage is something like an object database. Objects stored are seen as a collection of discrete entities, each individually retrievable at random from the collection. A basic object database knows at what types it contains and can obtain an instance of a given type by means of a name or some other key. A full-blown object-oriented database also knows enough about the structure of object types to perform sophisticated queries based on object metadata.
Object-oriented applications especially benefit from object-oriented I/O systems. One of the primary reasons an implementer may choose an object-oriented design is so that they can create and exploit new types not supported directly by the programming language. Object storage systems provide the means to store and retrieve these new types as easily as on intrinsic types.
The overall design of POOMA I/O follows the levels of I/O services described above.
The first level is comprised of the storage classes. These classes organize storage into records. In the current implementation, the records are opaque byte records. The I/O system does not know the internal structure of a record, only its length in bytes. Records are elements of byte arrays. Each array is independently accessible within the file and each record or element of a byte array is also independently addressable. Multiple elements within a byte array can be read or written in the same operation. The byte arrays in the present implementation are also automatically extendible. Arrays are created with an initial number of elements. Whenever a write operation writes past the current number of elements, the array is extended in chunks of the initial array size. Arrays are members of a collection called a storage set. A storage set is the logical interface to storage in terms of arrays. The physical storage in this implementation is a disk file, but a storage set is an abstraction barrier and need not be associated with a file. For example, future implementations may support storage sets based on databases or remote application memory. A storage set is opened and closed like a file and has access modes that resemble file I/O.
The second level is made up of the object storage classes. The object storage classes view storage as a collection of objects of different types. A container of objects is called an object set. Any object of a type supported by the I/O system can be stored with an identifying label in one operation. The object set can be queried to reveal the number of objects, contained, the types of objects contained, the number of objects of each type, and the labels of each object. A single operation is sufficient to retrieve an object given either its name, or its position in a list of objects of its type.
The object storage classes use the I/O services provided by the basic storage classes. To support a different storage type or format, or to optimize I/O for performance, one need only modify the basic storage classes. The object storage classes would remain unchanged. By providing several different implementations of the core storage classes, the I/O system can support a range of storage types, formats, and implementations. Users can then choose among them based on the requirements of a given application.
Users may also use the basic storage classes directly to store and retrieve simple byte records and streams without using the object management system. Or they may take advantage of the underlying to extend object management to new classes.
The main design goal of POOMA I/O was to achieve a fairly high level of support for object storage and management, without incurring the overhead of a full object database system. Therefore straightforward storage and retrieval operations are provided based on simple queries. To avoid overly tight coupling between data-producing and data-consuming applications, objects are randomly accessible based on an object container model rather than a serialized stream model. It was also deemed important to expose the basic I/O mechanisms in such a way that developers and could gauge the performance implications of their design choices. This separation of basic I/O from object management also allows I/O to be straightforwardly optimized for performance without requiring modifications in any portion of the object management layer.
Using the storage classes to store and retrieve byte arrays
Using the Object Storage Adapters
Opening, Closing, and Querying the Object Set
The storage classes are organized into three basic categories:
ByteBuffer
ByteBuffer is smart buffer class. On construction a ByteBuffer object contains a char* buffer of a specified size. The object maintains a cursor that is initially placed at the beginning of the buffer, but may be positioned anywhere within it. Member functions read or write bytes of data starting at the current cursor location. Each of these operations advances the cursor by the number of bytes written. A fatal error is generated by any attempt to read or write past the end. The class deliberately breaks encapsulation by providing an accessor to obtain a void pointer to the buffer. This access is needed by collaborating utility classes.
ByteBuffer() - Default constructor.
ByteBuffer(int size) - Main constructor allocates the specified block of space.
ByteBuffer(const ByteBuffer& r2) - Copy constructor.
~ByteBuffer() - Destructor.
ByteBuffer& operator=(const ByteBuffer& r2) - Assignment operator.
void write(int nBytes, void* source) - Write nBytes bytes from the source given into the buffer starting at the present cursor location.
void read(int nBytes, void* target) - Read nBytes bytes from the buffer into the given target starting at the present cursor location.
void blank(int nBytes=-1) - Blank (fill with zeros) nBytes bytes starting at the current cursor position. If nBytes is unspecified, blank out the whole buffer.
int size() const - Return the size of the buffer.
bool pastEnd() const - Test to see if the cursor is past the end. Since one is not allowed to write past the end, this function actually determines if the cursor is at the end.
void begin() - Place the cursor at the beginning of the buffer.
int pos() const - Return the current position of the cursor.
void seek(int pos) - Seek to a particular position in the buffer. The location must be valid.
void* buffer() - Obtain a pointer to the beginning of the buffer.
ByteRecord
The ByteRecord class is a wrapper for ByteBuffer that organizes the buffer into a set of fields distinguished by a name, a type name, a size, and an offset from the beginning of the buffer. ByteRecord uses a utility class called RecAttribute to store these attributes for each field. RecAttribute is described separately. ByteRecord allows a client to choose a particular field and write data to it. If the number of bytes written is smaller than the field, the field is padded with zeros, if it is larger, the data is clipped to fit the field. Like the ByteBuffer class, the ByteRecord class deliberately breaks encapsulation and returns a void pointer to the beginning of the buffer when requested by a collaborating client.
ByteRecord() - Default and main constructor.
ByteRecord(const ByteRecord& r) - Copy constructor performs a deep copy.
~ByteRecord() - Destructor.
ByteRecord& operator=(const ByteRecord& r) - Assignement operator performs a deep copy.
int recSize() const - Return the size of the record buffer.
void write(int index, void* source, int nBytes=-1) - Write into the field indicated by the field index. If nbytes is not specified, the -1 default will indicate that the full field size is to be written. Otherwise nbytes of the field are written and the rest of the field is padded with zeros. The cursor is positioned at the end of the write.
void write(const std::string& fieldName, void* source, int nBytes=-1) - Similar to write(int index,...), but with the field indicated by name.
void read(int index, void* target, int nBytes=-1) - Read from the field indicated by the field index. If nbytes is not specified, the -1 default will indicate that the full field size is to be read. Otherwise only nbytes of the field are read. The cursor is positioned at the end of the read.
void read(const std::string& fieldName, void* target, int nBytes=-1) - Similar to read(int index,...), but with the field indicated by name.
void defBegin() - Begin record member definitions
void addMember(const std::string& name, const std::string& typeName, int size) - Add a member given its attribute values. Offsets are automatically computed.
int numMembers() const - Return the number of member fields.
RecAttribute attribute(int i) const - Return a copy of the i-th member attribute where i=0...numMembers-1.
void defEnd() - End record member definitions.
void* buffer() - Obtain a pointer to the beginning of the byte buffer.
RecAttribute
RecAttribute is essentially a structure containing the attributes needed to describe a ByteRecord field.
RecAttribute() - Default constructor.
RecAttribute(const std::string& name, const std::string& typeName, int size, int offset) - Main constructor.
RecAttribute(const RecAttribute& r) - Copy constructor
~RecAttribute() - Destructor.
RecAttribute& operator=(const RecAttribute& r) - Assignment operator.
const std::string& name() const - Return the attribute name.
void name(const std::string& name) - Set the attribute name.
int size() const - Return the attribute size.
void size(int s) - Set the attribute size.
const std::string& typeName() const - Return the type name.
void typeName(const std::string& typeName) - Set the type name.
int offset() const - Return the offset.
void offset(int offset) - Set the offset.
These classes represent the generic portion of all storage resource implementations. The classes in this module are:
StorageResource
StorageResource is a base class. Derived classes must implement the methods to perform the actual storage tasks based on the particular type of storage. Any class derived from StorageResource that implements the virtual functions below, and whose associated descendent of ByteArray implements the virtual functions of ByteArray, qualifies as a storage resource.
StorageResource() - Default constructor.
StorageResource(const std::string& name, StorageAccessMode mode) - Main constructor creates and opens the storage resource in the given mode.
~StorageResource() - Destructor.
virtual ByteArray* newArray(int elemSize, int size)=0 - Virtual function to create a new array with the given properties elemSize is the record size in bytes, size is the initial size of the array which will self extend in chunks of the same size whenever an attempt is made to write beyond the current extent. Returns NULL if unsuccessful.
virtual ByteArray* findArray(long id)=0 - Virtual function to return an array pointer given its ID. Returns NULL if unsuccessful.
virtual ByteArray* findArray(const std::string& aName)=0 - Virtual function to find an array given its name. Returns NULL if unsuccessful.
virtual void flush()=0 - Virtual function to flush the storage resource. All metadata is made persistent during this step.
virtual void close()=0 - Virtual function to close the storage resource. Calls flush() first.
virtual int open(const std::string& name, StorageAccessMode mode)=0 - Virtual function opens the resource named in the given access mode. Only valid if the resource has been previously closed. The resource named can be a different one that the one previously closed.
virtual void update(long id)=0 - Used by the ByteArray client to instruct the resource to update its metadata on the array associated with the given ID.
int numArrays() const - Return the number of arrays in the storage resource.
StorageAccessMode mode() const - Return the current access mode.
bool isOpen() const - Return true if the storage resource is open.
bool isClosed() const - Return true if the storage resource is closed.
const std::string& name() const - Return the name of the storage resource.
ByteArray
ByteArray is the class associated with StorageResource that provides the I/O interface to storage. To create a functional storage resource, a descendent of StorageResource and a descendent of ByteArray must be implemented that collaborate in the I/O process. The virtual functions of StorageResource and ByteArray taken together constitute the required interface for a storage resource implementation.
ByteArray() - Default constructor.
~ByteArray() - Destructor.
void init(long id, int elemSize, int size) - Initialize the array using the values given.
void restore(long id) - Restore an array given its ID.
const std::string& name() const - Retrieve the name of the array.
void name(const std::string& name) - Define the name of the array.
long id() const - Return the ID.
int elemSize() const - Return the element (record) size.
int size() const - Return the size of the array; i.e. the number of records available to be written to without extending the array.
int top() const - Return the largest record number actually written.
void top(int t) - Set the value of the largest record number attribute.
virtual void write(int first, int nElems, void* buffer)=0 - Virtual function to write the contents of buffer to nElems elements (records) to the array starting at record first.
virtual void read(int first, int nElems, void* buffer) const=0 - Virtual function to read into the buffer nElems elements (records) from the array starting at record first.
void clear() - clear the state of the array leaving it uninitialized.
ByteArrayRecord
ByteArrayRecord is a base class that provides services for encoding selected member variables of any descendent class into a ByteRecord and reading or writing that record into the element of ByteArray whose elements are compatible in size with the ByteRecord. ByteArrayRecord has member function that allow these selected member variables to be defined and pointers to them given. In this implementation any fixed-length string variable and any C++ atomic data type can be included in a ByteArrayRecord member definition and automatically written to a byte array record or instantiated by reading from it. Any opaque data type whose storage is contiguous and whose size can be obtained from sizeof() can also be included.
ByteArrayRecord() - Default as well as main constructor.
~ByteArrayRecord() - Destructor.
int numMembers() const - Return the number of member fields.
void defBegin() - Begin member definitions.
void addMember(const std::string& name, const std::string& typeName,
int size, void* loc) - Add a member given attribute values.
void defEnd() - End record member definitions.
RecAttribute attribute(int i) const - Return a copy of the i-th ByteRecord attribute record where i=0...numMembers-1.
void* memberLoc(int i) - Return the i-th location pointer where i=0...numMembers-1.
int recSize() const - Return the size of the record in bytes.
int read(ByteArray* array, int recno) - Read a byte array record and set member variables.
int write(ByteArray* array, int recno) - Write member variables into a ByteRecord object and write the record to an element of a byte array.
virtual void defineMembers()=0 - Virtual function for descendent to define member attributes (accessible only to derived classes).
StorageSet is an interface to a particular implemented storage resource. StorageSet serves as a factory for clients who construct a storage set and indicate a type of storage resource. Thereafter it simply passes client requests directly to the particular storage resource it has created.
StorageSet() - Default constructor.
StorageSet(const std::string& name, StorageResourceType type, StorageAccessMode mode) - Main constructor takes a name, resource type, and access mode. There is only one allowed resource type for this implementation: HDF5Storage. The allowed access modes are:
int open(const std::string& name, StorageResourceType type, StorageAccessMode mode) - Open a completely uninitialized storage set specifying all the initialization parameters. This method can also reinitialize a previously initialized, but closed storage set including the respecification of a (new) resource type.
int open(const std::string& name, StorageAccessMode mode) - Open a previously initialized, but closed storage set by specifying only the name and mode. i.e., this function assumes the type is already specified and does not change.
void flush() - Flush the storage resource associated with the set. Preserves all existing records.
void close() - Close the storage resource and set. Flushes before closing.
StorageResourceType type() const - Return the storage resource type.
const std::string& name() const - Return the name of the storage set.
int numArrays() const - Return the number of arrays.
StorageAccessMode mode() const - Return the storage access mode.
ByteArray* newArray(int elemSize, int size) - Return a pointer to a new byte array with the specified characteristics. Return NULL if unsuccessful.
ByteArray* findArray(long id) - Return a pointer to a pre-existing array given its ID. Return NULL if unsuccessful.
ByteArray* findArray(const std::string& aName) - Return a pointer to a pre-existing array given its name. Return NULL if unsuccessful.
bool isOpen() const - Query the open state Return true if the set is open.
bool isClosed() const - Return true if the set is close.
The object storage classes are designed to present a logical view of storage as a container of objects. At the lower levels of the I/O system, data is stored as byte records in byte record arrays using the services of the storage classes. The task of the object storage layer is to take an object of a given type and convert it to byte records. In the process, the object is registered so that the object set is aware of its presence and can retrieve the object in response to queries.
The basic design of the object storage classes uses a collaborating class called StorageAdapter. Storage Adapter is a templated class that is only realized in specializations. Each specialization is written to handle a specific object or category of objects. The ObjectSet class has templated member functions to store and retrieive objects. Only when user code requesting storage or retrieval of specific types is compiled are the actual specializations of storage adapters matched to the objects. Using the services of the storage layer, developers or end users can write storage adapters using simple boiler-plate and a generic storage strategy.
An ObjectSet contains a StorageSet object that performs the actual I/O. It also maintains tables of objects by type. These tables associate an integer ID for each object with an optional user-supplied name. The ObjectSet also manages the boot records for all instances of a given type. The boot record contains the information would be needed by an adapter in order to begin the process of retrieving data and re-instantiating and instance of a particular type. ObjectSet tables are identified by a type name supplied by the adapter for the given type. The storage adapter also supplies the size requirement for the boot record array. The ObjectSet knows nothing else about particular object types.
Object names and IDs are stored sequentially in the object table for each type. Each entry in the table corresponds with a boot record in the boot record array allocated for that type that contains all the boot records. The position of the table entry for a given instance and position of the boot record in the boot record array match. That is, the object table and the boot record array are logically joined. The boot record array for each type, being a byte array, has a unique integer ID. The ordered pair comprised of the boot array ID and the boot record position for a particular instance is the equivalent of a physical address for each object stored in the file. A class called ObjectLoc is contain this pair of integers and is used to convey the physical address of an object from object set to storage adapter. The object primary key for retrieving an object is the pair comprised of a type name and an instance number. In this implementation, an object's boot array record number and its instance number are the same by convention. One of the main tasks of the object set is to maintain the association between the logical object ID (type name, instance number) and its persistent physical address (boot array ID, record number).
Figure 1 illustrates the logical relationship between the boot record
and the object location. It also shows how additional data is linked with
the boot array by using byte array IDs as logical pointers to other data
in the file.
Figure 1
Example: an adapter for a very simple array class might pack the array dimension and element type into the boot record along with the ID of another byte arraycontaining the remaining data, i.e., the interval sizes followed by the actual array data.
For a given type the sequence of events in the storage process is as follows:
2. The adapter then requests a new location from the object set identifying its type by a type name that the implementer of the adapter chooses.
3. If the location returned is invalid, the object set is evidently unaware of any type by that name. The adapter then responds by registering its type with the object set by sending it a type name and a boot record size. It then repeats its request for a valid record location.
4. Using the address of the boot record the adapter writes the boot information for that instance into the record and may write other data into other byte arrays using the services of the storage set classes.
2. In response, the object set invokes the retrieve() function of the adapter for that type passing along a reference to itself and to the object to be instantiated as well as the ID given by the client or found in the type table using the client-supplied name.
3. The adapter requests the byte record location of the boot record for the instance giving the objects set its type name the instance ID.
4. Having obtained the boot record, the adapter obtains the remaining data and instantiates the object.
The object set creates an instance of a StorageSet to perform I/O using operations exactly analogous to those for creating a StorageSet and that in turn resemble the creation of a file. ObjectSet maintains several tables. The main table is a list of all known types for that set. For each type there is a table listing each instance. ObjectSet also contains a reference to a master record in the file that allows it to retrieve these tables. Each of these entities is supported by a structure class derived from ByteRecordArray so that the information they contain can be easily stored and retrieved from byte arrays in the file. The master record is managed by the class ObjectSetBootRec. The type table is a collection of instances of the class TypeRec. Each instance of TypeRec contains a collection of instances of the class ObjectRec. An object location is contained in an instance of the class ObjectLoc.
ObjectSet
ObjectSet() -Default constructor.
ObjectSet(const std::string& name, StorageResourceType type, StorageAccessMode mode) - Main constructor specifies the name, the resource type, and the access mode, and opens the ObjectSet for transactions.
~ObjectSet() - Destructor.
int open(const std::string& name, StorageResourceType type, StorageAccessMode mode) - Opens an object set that has been default constructed, but not opened, or has been closed.
int open(const std::string& name, StorageAccessMode mode) - Opens an object set that has been constructed and subsequently closed.
Queries:
StorageSet* storageSet() const - Returns a pointer to the underlying storage set.
const std::string& name() const - Returns the name of the object set.
StorageAccessMode mode() const - Returns the current access mode.
int numTypes() const - Returns the number of types in the set.
int numInstances(const std::string& typeName) - Returns the number of instances of a given type referred to by name.
int numInstances(long typeID) const - Returns the number of instances of a given type referred to by typeID.
const std::string& typeName(long typeID) const - Returns the type name given a type ID.
long typeID(const std::string& typeName) - Returns the typeID given the type name.
const std::string& objectName(const std::string& typeName, long instanceID) - Returns the object name given type name and instance ID.
const std::string& objectName(long typeID, long instanceID) - Returns the object name given the type ID and the instance ID.
bool isOpen() const - Boolean operation checks whether the set is open.
bool isClosed() const - Checks whether the set is closed.
Operations on the transaction state:
void flush() - Update all current records and flush to storage.
void close() - Flush then close the set.
Object management functions in collaboration with storage adapters:
void addType(const std::string& typeName, int bootRecSize) - Register a new type given its name and boot record size.
ObjectLoc nextObject(const std::string& typeName, const std::string& objectName) - Obtain the next physical location (in the boot record array for objects of this type) and register the name of this instance.
ObjectLoc findObject(const std::string& typeName, long instanceID) - Obtain a physical location (in the boot record array for objects of this type) corresponding to the given instance ID.
ObjectLoc findObject(const std::string& typeName, const std::string& ObjectName) - Obtain a physical location (in the boot record array for objects of this type) corresponding to the given object name.
Generic functions for all adapters:
template <class T>
long store(T& t, const std::string& objectName) - Store an instance of type T with an optional name. Return the instance ID.
template <class T>
int retrieve(T& t, long id) - Retrieve an instance of type T given its instance ID. Return 0 if successful, -1 otherwise.
template <class T>
int retrieve(T& t, const std::string& objectName) - Find an instance of type T given the instance name. Return 0 if successful, -1 otherwise.
This class is used to convey the persistent physical location of an object, i.e. its boot record array ID and its boot record number. It also stores a pointer to the array so that once the array is bound to the ID it can be used repeatedly by the corresponding storage adapter.
ObjectLoc
ObjectLoc() - Default constructor.
ObjectLoc(long arrayID, int recordNum, ByteArray* array) - Main constructor stores the member attributes.
ObjectLoc(const ObjectLoc& r) - Copy constructor.
~ObjectLoc() - Destructor.
ObjectLoc& operator=(const ObjectLoc& r) - Assignment operator.
void init(long arrayID, int recordNum, ByteArray* array) - Initialize an instance by setting all the state variables.
long arrayID() const - Return the array ID.
int recordNum() const - Return the record number.
ByteArray* array() const - Return a pointer to the byte array.
bool valid() const - Tests whether the state of the object is valid; i.e. represents a valid location.
ObjectRec is a descendent of ByteArrayRecord containing the information needed to distinguish one instance of an object type from another. The read()/write() member functions of ByteArrayRecord allow defined member variables to be automatically written to a byte array. The defineMembers() function implemented by this class defines the member variables to be written.
ObjectRec
ObjectRec() - Default constructor.
ObjectRec(const std::string& name, long instanceID, int recordNum) - Main constructor specifies an object name, an instance ID, and boot record array record number.
ObjectRec(const ObjectRec& r) - Copy constructor.
~ObjectRec() - Destructor.
ObjectRec& operator=(const ObjectRec& r) - Assignment operator.
void init(const std::string& name, long instanceID, int recordNum) - Used to initialize an instance with the given variables.
const std::string& objectName() const - Return the object name.
long instanceID() const - Return the instance ID.
void instanceID(long id) - Set the instance ID.
int recordNum() const - Return the record number.
void recordNum(int recordNum) - Set the record number.
void upToDate(bool flag) - Sets the update status of the record. The record is up to date if it does not need to be written to the array in which it will be recorded.
bool upToDate() const - Checks the update status of the record.
void defineMembers() - Implemented virtual function of the ByteArrayRecord class. Defines member variables to be written to a byte array record.
TypeRec is a descendent of ByteArrayRecord containing the information needed to list object types. The read()/write() member functions of ByteArrayRecord allow defined member variables to be automatically written to a byte array. The defineMembers() function implemented by this class defines the member variables to be written.
TypeRec
TypeRec() - Default constructor.
TypeRec(const std::string& tname, int bootrecsize, long typeID, StorageSet* set) - Main constructor specifies a type name, a boot record size, and a type ID. Specification of the storage set allows a ByteArray object to be instantiated for the object records.
TypeRec(const TypeRec& r) - Copy constructor.
~TypeRec() - Destructor:
TypeRec& operator=(const TypeRec& r) - Assignment operator.
void init(const std::string& tname, int bootrecsize, long typeID, StorageSet* set) - Used to initialize a new record.
const std::string& typeName() const - Return the type name.
long typeID() const - Return the type ID.
int numInstances() const - Return the number of instances of this type.
void bootRecSize(int size) - Set the boot record size of this type.
int bootRecSize() const - Return the boot record size.
long objectRecsArrayID() const - Return the ID of the byte array containing the object records for this type.
void objectRecsArrayID(long id) - Set the object record byte array ID.
ByteArray* objectRecsArray() const - Return a pointer to the object records array for this type.
void objectRecsArray(ByteArray* array) - Set the pointer to the object records array.
long bootRecsArrayID() const - Return the boot record array ID for this type.
void bootRecsArrayID(long id) - Set the boot record array ID.
ByteArray* bootRecsArray() const - Return a pointer to the boot record array for this type.
void bootRecsArray(ByteArray* array) - Set the pointer to the boot records array.
void addObjectRec(const ObjectRec& rec) - Adds an object record to the vector maintained by the type record as a new instance.
void pushObjectRec(const ObjectRec& rec) - Pushes an object record onto the instance vector without incrementing the number of instances. Used to restore existing instance information.
ObjectRec* lookup(long instanceID) - Looks up an object record by instanceID.
ObjectRec* lookup(const std::string& objectName) - Looks up an object record by object name.
void upToDate(bool flag) - Sets the update status of the record. The record is up to date if it does not need to be written to the array in which it will be recorded.
bool upToDate() const - Checks the update status of the record.
void clear() - Clears the state of the type record removing all object instance records.
void defineMembers() - Implemented virtual function of the ByteArrayRecord class. Defines member variables to be written to a byte array record.
ObjectSetRec is a descendent of ByteArrayRecord containing the information needed to begin the process of restoring an object set that has been previously stored. The read()/write() member functions of ByteArrayRecord allow defined member variables to be automatically written to a byte array. The defineMembers() function implemented by this class defines the member variables to be written. There is only one such record in a file recorded in a byte array with just one element. This byte array is identified by name rather than by ID.
ObjectSetBootRec
ObjectSetBootRec() - Default constructor.
ObjectSetBootRec(int numTypes, long id) - Main constructor specifies the essential attributes.
ObjectSetBootRec(const ObjectSetBootRec& r) - Copy constructor.
~ObjectSetBootRec() - Destructor.
ObjectSetBootRec& operator=(const ObjectSetBootRec& r) - Assignment operator.
void numTypes(int num) - Set the number of types in the object set.
int numTypes() const - Return the number of types.
void typeRecsArrayID(long id) - Set the type record array ID.
long typeRecsArrayID() const - Return the type record array ID.
void defineMembers() - Implemented virtual function of the ByteArrayRecord class. Defines member variables to be written to a byte array record.
template <class T> StorageAdapter
The following is a description of the generic storage adapter class template <class T> StorageAdapter. This class must be specialized in order to store or retrieve any particular object type.
template < class T> StorageAdapter
StorageAdapter() - Default constructor.
~StorageAdapter() - Destructor.
long store(ObjectSet* set, T& t, const std::string& objectName) - Store the instance t as a persistent object with the given name.
int retrieve(ObjectSet* set, T& t, long id) - Retrieve the object given its ID.
int retrieve(ObjectSet* set, T& t, const std::string& objectName) - Find and retrieve the object given its name.
The following is a generic description of how to write a storage adapter. First of all, one must create a specialization of the template described above. That is for class Foo we must have:
template<>
class StorageAdapter<Foo>{ ... };
This specialization must declare then define all of the functions described for the generic interface above. That is it must declare.
StorageAdapter()
~StorageAdapter()
long store(ObjectSet* set, const T& f, const std::string& objectName)
int retrieve(ObjectSet* set, Foo& f, long id)
int retrieve(ObjectSet* set, Foo& f, const std::string& objectName)
Its variables should include these standard members:
std::string typeName_m - Identifying type name established by the adapter.
int recSize_m - Required size of boot array record.
One would need to include:
#include "IO/Adapters.h"The boiler plate for writing a Foo store() function would be as follows:
#include "IO/IOUtils.h"
#include "Utilities/PAssert.h"
// Other includes as needed
template <>
long StorageAdapter< Foo>::
store(ObjectSet* set, const Foo& f, const std::string& objectName){
// boiler plate for getting a valid object location
ObjectLoc loc= set->nextObject(typeName_m, objectName);
if(!loc.valid()){} // the type is unrecognized; register as new type, then get valid loc
set->addType(typeName_m, recSize_m);
loc= set->nextObject(typeName_m, objectName);
PInsist((loc.valid()),}
"StorageAdapter< Foo>::store(): location invalid");
// get the boot array from the location
ByteArray* bootarray= loc.array();Perform all storage operations here
// instanceID is the same as the record number by convention
// return instanceID
return loc.recordNum();
The boiler plate for the Foo retrieve function is:
template <>
int StorageAdapter<Foo>::
retrieve(ObjectSet* set, Foo& f, long id){
// boiler plate for obtaining the object location}
ObjectLoc loc= set->findObject(typeName_m, id);
PInsist((loc.valid()),
"StorageAdapter<Foo>::retrieve(): could not find object ID");
// get the boot array from the storage set
ByteArray* bootarray= loc.array();perform all retrieval operations here
// if we made it this far, return 0
return 0;
The best way to implement the retrieve function that finds an instance by name is to expoit the one that retrieves by instance ID and to use the lookup service of ObjectSet to obtain it.
template <>
int StorageAdapter<Foo>::
retrieve(ObjectSet* set, Foo& f, const std::string& objectName){
// boiler plate for obtaining the object location}
ObjectLoc loc= set->findObject(typeName_m, objectName);
PInsist((loc.valid()),
"StorageAdapter<Foo>::find(): could not find object ID");
// use the ID (same as the record number) to retrieve the array
long id= loc.recordNum();
int status= retrieve(set, a, id);
PInsist((status==0),
"StorageAdapter<Foo>::find(): could not retrieve array");
// if we made it this far, return 0
return 0;
To facilitate reading and writing the boot record information it is recommended that the implementer exploit the services of the ByteArrayRecord class. For example, suppose that the boot information required for the Foo class consists of two Foo data values int a and long b.
One should implement a descendent of ByteArrayRecord such as:
class FooBootRec : public ByteArrayRecord{
public:
FooBootRec(int a, long b);
virtual void defineMembers();
private:
int a_m;};
double b_m;
The implementation of the defineMembers() function would be as follows:
void FooBootRec::defineMembers(){
defBegin();}
addMember("a", "int", sizeof(int), &a_m);
addMember("b", "long", sizeof(long), &b_m);
defEnd();
A call to defineMembers() must be included in any constructor for FooBootRec:
FooBootRec::FooBootRec(int a, long b){
a_m=a;}
b_m=b;
defineMembers();
Having done this, the implementer of the adapter for Foo can use the services of the ByteArrayRecord base class. For example, the required boot record size is obtained from FooBootRec::recSize(). Writing the record to the boot record array is then simply accomplished by:
bootRec.write(bootRecArray, recordNum)
where bootRec is an instance of FooBootRec, bootRecArray is a pointer to the boot record array for Foo and recordNum is identical to the instance number being stored. Similarly, the information can be retrieved from the boot record by:
bootRec.write(bootRecArray, recordNum)