Shop

Introduction

Shop is an Eiffel library that allows sharing and storing persistent objects. Shop stands for Shared Objects that Persist. The main goal of this library is to provide the developer a set of persistent classes that help him develop business systems using ordinary classes and class structures together with persistent ones in an as much transparent way as possible.

The main purpose of Shop is to allow to create multiuser application sharing a single object system.

Many Eiffel object oriented application have to deal with persist ency and sharing. Very often these facilities are provide by an imposed relational data base. But, if such a data base is not imposed how to provide such facilities. Some data base system exists who claim to be object oriented. But first the meaning of OO is very large and some time permissive. Second most of these solutions are C++ oriented or impose to use specific language to define any object type which could be stored and this not only for type which would be explicitly be declared as beeping persistent by also for all their transitive closure.

Shop preserve the Eiffel programmer who want to preserve its Eiffel method and himself from tedious external technologies constraints.

In shop, basically on wrote only one application with may run as server, client or event in standalone mode. One process play the role of object warehouse server. The other are clients. They communicate through TCP/IP sockets. The server first load the data from storage file and the start listening for client requests. Clients cache shared objects they needs. They only load data on demand. From the application program, except at the early beginning, it's no difference. One original and recommended approach, consist in writing only one application with switch to start either in single mode, server mode or in client mode. This ensure that the set of class is known in each cases.

Sharing

In the current context, we concept of sharing means sharing between separate processes. This not the usual concept of sharing object by references. Any sharable class must be a heir of the class PRESISTENT. This class has only one attribute: the pid (persistent identifier). This attribute is just an integer. All registered object, also called "alive" object have a positive pid. A newly created persistent object has a null (0) pid until it is registered in the warehouse.

Further, any descendant from this class may include any type of attributes. Non sharable attributes are privates. In contrast all persistent attributes are normally sharable. The privacity of non sharable object is mandatory. If some non sharable attribute is already seen by an other persistent object, this attribute will be cloned accordingly to the persistence closure rule a the next duplication. Thus, the previous external reference any more to the body of that sharable object.

The persistent cluster include also some classes to store structures. Basically this cluster contains persistent lists and tables. Those structure are compatible with usual lists an table but of course they have no cursors and the manipulation are subjects to some conditions. One could say that they are extendibles and prunable only if the client owns a valid reservation on that structure instance. Any reservation is only valid for one operation.

Persistent closure

The persistent closure is the set of objects reachable from a persistent instance, but limited to the other persistent objects. In this set, all non persistent objects are private to the including persistent instance. The duplicata is the result of a duplication process of a persistent object. The duplicata and its persistent source, have disjoint sets of non-persistent attributes but do share the persistent attributes. The non-persistent attributes are deep clones limited to other persistent objects. A duplication is of course not alive (pid <= 0).

The duplication mechanism is used to work properly on persistent object in a modification session, but also in the exchange mechanism between warehouses.

Accessing attributes

A basic persistent is said not safe. That applies either for ordinary objects or structures. That means that the access to a persistent attribute is direct like for any other Eiffel attributes. As a consequence when a client process requires an unsafe persistent object it will retrieve the transitive closure bounded to the safe persistent objects. To be safe, an object should only provides an indirect access to the persistent attributes. That indirect access allows to implement an on-demand access mechanism. If none of the persistent classes are safe the first access to a persistent root of a persistent system will poll the whole system. This is of course unacceptable for performance and even for sharing reasons. That is why, the instenciable lists and tables provided in the library have been made safe. Those classes are actually heirs of HASH_TABLE from the EiffelBase library made and maintained by Eiffel Software Inc (ESI-ISE). But it really easy to make the same type of adaptation for table from libraries. The instenciable lists looks like ARRAYED_LIST but should not be used as an "ACTIVE" structure.

Cluster

Basically there are only two clusters. The first include all persistent classes, the second include all the warehouse support classes.

Warehouse

A warehouse is a class that contains a persistent system. There are two type of warehouses: the storable one and the virtual one.

The virtual warehouse is a local cache of a real storable warehouse. Warehouse caches only work in conjunction with a storable warehouse.

The storable warehouse and the cache communicate through the network.

The storage is an array of persistent object. This array has 1 as lower bound and is open on the other end. The pid of a persistent object actually is the index of this object into the storage array. The pid allocation process recycles all the unused slots. Therefore if the persistent system stay limited in number of persistent object, the maximum pid is kept stable after a certain running time.

The storable warehouse is the only responsible for allocating pids.

The storable warehouse also includes a persistent garbage collector (pgc). This garbage collector works incrementally at each insertion of a new instance. But, some mechanisms are provided to stop and restart the pgc. Normally, the storable warehouse completes the garbage collection cycle before storing data in file.

The cache acts as a mirror of the storable warehouse but only includes the called pids. Normally, the root of the persistent system is always loaded into the cache at startup. The root pid is 1. It's normally a safe structure. That persistent root is thus loaded whit out the referred persistent items. These items are replaced by PERSISTENT_PROXY instances.

PERSISTENT_PROXY is a simple class which only contains the pid value of the real object. These objects are the only called if needed. The cache drives the communication with the server of the storable warehouse. But the server may interleave events with the answers to the requests. However, the cache has a polling mechanism to retrieve pending events when it has nothing to ask to the server. The timing of that poll is adjustable.

Persistent

Any persistent class must be a heir of the class PERSISTENT. This class has only one attribute: the pid (persistent identifier). This attribute is just an integer. All registered object, also called "alive" object have a positive pid. A newly created persistent object has a null (0) pid until it is registered in the warehouse.

Further, any descendant from this class may include any type of attributes. Non Persistent attributes are private. In contrast all persistent attributes are normally shareable. The private export status of non persistent object is mandatory. If some non persistent attribute is already seen by another persistent object, this attribute will be cloned according to the persistence closure rule (see hereafter).

The persistent cluster also includes some classes to store collections. Basically this cluster contains persistent lists and tables. Those structures are compatible with usual lists an tables but they have no cursors and their manipulation is subject to some conditions. One could say that they are extendibles and prunable only if the client owns a valid reservation on that structure instance. Any reservation is valid for only one operation.

Sharable object are also persistent. That means that the warehouse used by server is able to store data on file in order to restart the next session with these saved data. That the reason for the current naming of sharable object classes. Amount persistent descendant, the collections are of a particular interest.

An ordinary persistent is said unsafe. That means that when such an item is transfered either from the server to clients from a client to the server, all it's persistent closure must be transfered. By contrast, some PERSISTENT_COLLECTION are said safe. That means that it's possible to send only a part of such a structure. Ruffly spoken, such a structure are send without the real items. Those items are replaced by PERSISTENT_PROXY which just content the pid of the real item. Without that method, calling for the root collection of the sharable system, would lead to download every things. By building safe only the real part needed by the client are transmitted on demand. When an item is asked to such a collection, that collection first check that this item is really present. If a proxy is found instead, the cache warehouse request through the network.

Networking

In this context, the two main components are the STORE_SERVER and the STORE_CLIENT.

Both are heirs of more general classes called SHARED_SERVER and SHARED_CLIENT. Most the concepts of sharing data between a server and some clients are in these ancestors.

As depicted in the first figure, the STORE_SERVER performs the persistent clipping before

sending data to the network and reconnect data to the persistent system when it receives a clipped packet from the network. This approach allows using the storable warehouse in a standalone mode. Therefore, it is possible to make an application which works either in single user mode or in multi user mode. In contrast, the cache does not do the clipping itself. It does not make sense to use the cache in standalone mode.

Scanners

This is a set of tools which allow to walk through the object transitive closures. The PERSISTENT_SCANNER is a set of functions dedicated to check the quality of shared objects. These functions are mainly used in contracts. Therefor they are not optimized. It's implementation is based on agent technics. The same scanning function is used with several agent functions according to the entry point function purpose. By contrast, the most important classes for the system performance are : PERSISTENT_ATTACHER, _DUPLICATOR and _CLIPPED_DUPLICATOR. These classes have only one purpose in order to faster direct function calls.

Journaling

Shop include journaling facilities. When required the STORE_SERVER file all successfully executed command (who may modify the persistent system) requests in a file. That allows to replay those command after a crash, powerfail, etc. Normally all commands will be save except, may be the last one. Of course, the replay process will stop when fall back on the software bug which did trigger the crash!

The journaling was experimented up when including more than 100.000 commands mixed with persistent garbage collections sequences. It works find except that it's a little bit too slow. But This is not the journaling by itself which is to slow. This more the clipping/attachment process which is still to slow. This should be optimized specially for some persistent collections like tables.

Example

An example is provided with this library. It allows to experiment and test this library. This example is a sharable address book. This is a EiffelVision2 application. The ISE Ace file are supplier for Linux. The example include a small predefined address book named addresses.dmp. to start the server simple type:

> addresses 10000 addresses.dmp

To start a client type:

> addresses localhost 10000 jean

ToDo list

The following things should be done as soon as possible.

Replacing references to persistency by reference to sharing. (??)

A true documentation

Class Interfaces

In the same directory as the present document, you should find a subdirectory containing a file index.html which includes the documentation of the Shop library classes.