New Article - Dynamic method calls and versioned data stores

[AthenaOfDelphi]

Hi everyone,

Just to let you know I've added another article to my series (Tripping the class fantastic).

This one is a programming article introducing you to calling methods dynamically (using their name at runtime) using a simple versioned data store class.

Any comments or questions, please feel free to drop me a mail or post a comment on the article.

[AthenaOfDelphi]

A interesting idea. But the implementation... Um... The horrors like IntToStr() and a performance do not get along too good.

With all my respect to the author of this article, I believe I did solve the same problem better (see https://sourceforge.net/projects/chepersy/ ). It's a custom replacement for the Delphi streaming system. Needs to list all the object's field in a special method, but everything else is automatic. *and* it's optimized for speed.

Currently I am working on the forward compatibility, so that the older programs will be able to read the newer data files that contain new object classes unknown to them.


Hi Chebmaster,

I'd really just like to say that the article is intended to illustrate the use of TObject.methodAddress. This can be a pretty useful little trick that a lot of people may not have encountered before.

The vehicle for this is my basic versioned data store. It is simple and effective and above all illustrates the use of TObject.methodAddress quite well. Whether it provides an optimal solution depends on what qualifies as optimal... in the context of a game where it is used to load game data, map data and game saves, its load and save times are more than acceptable... the files sizes are quite small because it has no additional type/class information and if your idea of optimal involves code size and complexity then I would suggest its right up there because its quite small and easy to understand... even for beginners.

With regards to other solutions to versioned data storage... there are other options (a fact I stated in the article), but... I've tried one or two and I didn't get on very well with them. So, I wrote my own and have used it as the basis for this article with the intention of helping others learn a little more about our favourite language.

Thanks for reading and thanks for the comments.

[AthenaOfDelphi]

Robert has asked for a bigger example of how to use the basic data store from my article. The example code below will hopefully give you a better idea of how it can be used.

This example is more closely related to the base class I use which includes one or two helper functions that cater for including string length checks and loading and saving streams within your object. These methods are as follows:-



procedure saveString(writer:TWriter;srcData:string);
procedure loadString(reader:TReader;var dstData:string);

procedure saveStream(writer:TWriter;src:TStream);
procedure loadStream(reader:TReader;dst:TStream);
The exact implementations of these aren't relevant at this time, but they exist as protected methods of TStreamedObject (refer to the article - page 3 - for the example code I provided).

You will undoubtedly notice the clunkiness of loading/saving streams. This is mentioned in the article.

So, lets consider a simple map (TMap)... the maps height and width are variable. The map consists of 3 layers. The first layer (TGroundLayer - Descended from TTileLayer) contains the base tile for each cell, the cells movement costs. The other two layers are TTileLayer objects that simply tell the engine what tiles to draw to create overhanging trees for example.



unit unitExample;

interface

uses streamedObject, classes, SysUtils;

type
PWord = ^Word;

TTileLayer = class(TStreamedObject)
protected
fTileData : PWord;
fWidth : integer;
fHeight : integer;

function getAddress(base:PWord;x,y:integer):PWord;

function getTile(x,y:integer):word;
procedure setTile(x,y:integer;value:word);
public
constructor create;
destructor destroy; override;

procedure setSize(width,height:integer); virtual;

property tile[x,y:integer]:word read getTile write setTile;
published
procedure load_ver1(reader:TReader); virtual;
procedure save_ver1(writer:TWriter); virtual;
end;

TGroundLayer = class(TTileLayer)
protected
fMovementData : PWord;

function getMovementData(x,y:integer):word;
procedure setMovementData(x,y:integer;value:word);
public
constructor create;
destructor destroy; override;

procedure setSize(width,height:integer); override;

property tile;
property movementData[x,y:integer]:word read getMovementData write setMovementData;
published
procedure load_ver1(reader:TReader); override;
procedure save_ver1(writer:TWriter); override;
end;

TMap = class(TStreamedObject)
protected
fGroundLayer : TGroundLayer;
fLayer1 : TTileLayer;
fLayer2 : TTileLayer;
fName : string;

fWidth : integer;
fHeight : integer;
public
constructor create;
destructor destroy; override;

procedure setSize(width,height:integer);

property groundLayer:TGroundLayer read fGroundLayer;
property layer1:TTileLayer read fLayer1;
property layer2:TTileLayer read fLayer2;

property name:string read fName write fName;

property width:integer read fWidth;
property height:integer read fHeight;

published
procedure load_ver1(reader:TReader);
procedure save_ver1(writer:TWriter);
end;

implementation

(*---TTileLayer------------------------------------------------------------------------*)

function TTileLayer.getAddress(base:PWord;x,y:integer):PWor d;
var
offset : cardinal;
temp : PWord;
begin
if (x>=0) and (x<fWidth>=0) and (y<fHeight) then
begin
offset:=y*fWidth+x;
temp:=base;
inc(temp,offset);

result:=temp;
end
else
begin
raise exception.create('Coordinates out of range in '+
self.classname+'.getAddress('+intToStr(x)+','+intT oStr(y)+
'). Should be in the range (0..'+intToStr(fWidth-1)+',0..'+
intToStr(fHeight-1)+')');
end;
end;

function TTileLayer.getTile(x,y:integer):word;
begin
result:=getAddress(fTileData,x,y)^;
end;

procedure TTileLayer.setTile(x,y:integer;value:word);
begin
getAddress(fTileData,x,y)^:=value;
end;

constructor TTileLayer.create;
begin
inherited;

fTileData:=nil;
end;

destructor TTileLayer.destroy;
begin
if (fTileData<>nil) then
begin
freeMem(fTileData);
end;

inherited;
end;

procedure TTileLayer.setSize(width,height:integer);
begin
if (fTileData<>nil) then
begin
freeMem(fTileData);
end;

getMem(fTileData,(width*height*sizeOf(word)));

fWidth:=width;
fHeight:=height;
end;

procedure TTileLayer.load_ver1(reader:TReader);
var
x,y : integer;
temp : PWord;
begin
// You MUST ensure the size is set before attempting to
// load the data

temp:=fTileData;

for y:=1 to fHeight do
begin
for x:=1 to fWidth do
begin
temp^:=word(reader.readInteger);

inc(temp);
end;
end;
end;

procedure TTileLayer.save_ver1(writer:TWriter);
var
x,y : integer;
temp : PWord;
begin
if (fTileData<>nil) then
begin
temp:=fTileData;

for y:=1 to fHeight do
begin
for x:=1 to fWidth do
begin
writer.writeInteger(temp^);

inc(temp);
end;
end;
end
else
begin
raise exception.create('Layer not initialised in '+self.className+'.save_ver1');
end;
end;

(*---TGroundLayer------------------------------------------------------------------------*)

function TGroundLayer.getMovementData(x,y:integer):word;
begin
result:=getAddress(fMovementData,x,y)^;
end;

procedure TGroundLayer.setMovementData(x,y:integer;value:wor d);
begin
getAddress(fMovementData,x,y)^:=value;
end;

constructor TGroundLayer.create;
begin
inherited;

fMovementData:=nil;
end;

destructor TGroundLayer.destroy;
begin
if (fMovementData<>nil) then
begin
freeMem(fMovementData);
end;

inherited;
end;

procedure TGroundLayer.setSize(width,height:integer);
begin
inherited setSize(width,height);

if (fMovementData<>nil) then
begin
freeMem(fMovementData);
end;

getMem(fMovementData,(width*height*sizeOf(word)));
end;

procedure TGroundLayer.load_ver1(reader:TReader);
var
x,y : integer;
temp : PWord;
begin
inherited load_ver1(reader);

temp:=fMovementData;

for y:=1 to fHeight do
begin
for x:=1 to fWidth do
begin
temp^:=word(reader.readInteger);

inc(temp);
end;
end;
end;

procedure TGroundLayer.save_ver1(writer:TWriter);
var
x,y : integer;
temp : PWord;
begin
inherited save_ver1(writer);

temp:=fMovementData;

for y:=1 to fHeight do
begin
for x:=1 to fWidth do
begin
writer.writeInteger(temp^);

inc(temp);
end;
end;
end;

(*---TMap------------------------------------------------------------------------*)

constructor TMap.create;
begin
inherited;

fGroundLayer:=TGroundLayer.create;
fLayer1:=TTileLayer.create;
fLayer2:=TTileLayer.create;

fName:='';

setSize(1,1);
end;

destructor TMap.destroy;
begin
fGroundLayer.Free;

fLayer1.free;

fLayer2.free;

inherited;
end;

procedure TMap.setSize(width,height:integer);
begin
fGroundLayer.setSize(width,height);
fLayer1.setSize(width,height);
fLayer2.setSize(width,height);

fWidth:=width;
fHeight:=height;
end;

procedure TMap.load_ver1(reader:TReader);
var
temp : TMemoryStream;
begin
// Set the size
setSize(reader.readInteger,reader.readInteger);

// Read the name
loadString(reader,fName);

// Create a temporary stream
temp:=TMemoryStream.create;

// Load the stream containing the layers
loadStream(reader,temp);

// This is important- Return the position to 0 once you have loaded the stream
temp.position:=0;

// Load the layers
fGroundLayer.loadFromStream(temp);
fLayer1.loadFromStream(temp);
fLayer2.loadFromStream(temp);

// Get rid of our temporary stream
temp.free;
end;

procedure TMap.save_ver1(writer:TWriter);
var
temp : TMemoryStream;
begin
// Save the size
writer.writeInteger(fWidth);
writer.writeInteger(fHeight);

// Save the name
saveString(writer,fname);

// Create a temporary stream
temp:=TMemoryStream.create;

// Save the layers
fGroundLayer.saveToStream(temp);
fLayer1.saveToStream(temp);
fLayer2.saveToStream(temp);

// Save the temporary stream
saveStream(writer,temp);

// Get rid of our temporary stream
temp.free;
end;

end.
A note about TGroundLayer.load_ver1 and TGroundLayer.save_ver1... this arrangement is not optimal since you will iterate through the map twice (once in the inherited load/save from TTileLayer and then once in TGroundLayers routines), however it does mean that if you add/remove data to TTileLayer you only have to update TTileLayer and the changes will be reflected without having to change the code for TGroundLayer. You could reimplement the code from TTileLayer in the TGroundLayer routines like this:-



procedure TGroundLayer.load_ver1(reader:TReader);
var
tempTile : PWord;
tempMove : PWord;
x,y : integer;
begin
tempTile:=fTileData;
tempMove:=fMovementData;
for y:=1 to fHeight do
begin
for x:=1 to fWidth do
begin
tempTile^:=word(reader.readInteger);
tempMove^:=word(reader.readInteger);

inc(tempTile);
inc(tempMove);
end;
end;
end;
This would be quicker since you only iterate through the layers data once, BUT, the two lots of data become mixed and you must then take care to ensure that the load/save routines in TGroundLayer reflect any changes you might make to TTileLayer. This is easy with only a small number of fields and a single version... imagine a lot of fields and multiple versions and you could quickly get in a mess. So, I would advise that (whilst it is not optimal from a speed point of view) you keep the data seperated into their classes as I have done in the example (this is optimal from the point of view of ease of maintenance... not to mention the OOP paradigm).

So how would you use these objects...



var
myMap : TMap;
begin
myMap:=TMap.create;
myMap.name:='Test Map';
myMap.setSize(10,10);

myMap.groundLayer.tile[0,0]:=1;
myMap.groundLayer.tile[0,1]:=2;

// I'm sure you get the idea...

Then loading and saving your entire map is as simple as using TMap's loadFromStream and saveToStream routines.

A map that is 10 x 10 (all data set to 0) with the name 'Test Map' occupies 832 bytes on the disk. Consider that the raw data we are storing is 10 x 10 x 2 x 4 (width x height x sizeOf(word) x (ground tile + ground movement + layer 1 tile + layer 2 tile)) = 800 bytes... its pretty good at keeping the size of your data stores down because it doesn't add too much baggage. In reality if all of your map cells (tiles and movement data) were set to 256, then an approximate size would be 10 x 10 x 3 x 4 (width x height x data size x number of layers/data sets) = 1200 bytes. The reason that the data size is 3 (as stored in the stream) rather than 2 (as stored in memory) is that TReader and TWriter store a byte that indicates the size of the data and then the actual bytes that represent the data, but this is based on the actual numerical value and NOT the variable size, so 0 will always be represented by 2 bytes whether it is stored by your object as a byte, integer, word or cardinal. 255 is represented as 2 bytes, but 256 thats 3 bytes... 1 for the size and 2 for the data itself.

Hope this helps clarify the usage of the base class.

[Robert Kosek]

Um, wow! You really came through in spades on my question. Now I truly see the helpfulness of that versioned saving system! Thanks so much, Athena!

[AthenaOfDelphi]

Glad it helped and I'm glad you found the article informative :-)

[Robert Kosek]

The whole series is interesting and very informative. Only recently, well two months back, I began to incorporate a more object oriented approach to my game programming and found the basics of it quite flexible.

I look forward to your subsequent articles, so that I may learn more about object orientation and the various ins and outs of it.

I've heard about "RTTI" and using it to save the properties of forms and elements, but could there be something similar for game objects and saving a game?

[AthenaOfDelphi]

I did investigate (briefly) implementing a system similar to Delphi's component streaming system using RTTI... much like Chebmaster has implemented, but without the requirement to list fields manually in software as his system has.

I had great difficulties obtaining a list of properties.. although there is example code in the classes unit, I couldn't find a great deal of information about the routines and data structures I needed to use, so I gave up (largely due to time constraints).

The biggest problem with this approach is that you have to rely on the list of properties provided by the RTTI for the object. So, everything has to be published (I don't think public will do... this means of course you can't store array parameters as they can't be published) and if you remove a field, you are left with that field in the stream, so you have to take that into account when loading the data. This is where Delphi's built in mechanisms fall over.

Whilst the idea is great, I've pretty much decided against going down that road because of the implications with adding data to and removing it from the objects and its limitations in terms of what you can store (mainly simple types).

With regards to future articles, I have a couple in the pipeline at the moment, although they aren't all specifically about programming.

[technomage]

I did a set of streaming classes a while ago that use RTTI to export and import the published properties. It worked well, but the code was not straight forward (this was in Delphi 5). But it can be done.

You are right you cannot publish arrays but you can publish objects so you would need to write a TCollection style object for your arrays and code for that specifically when exporting and importing the data.

[grudzio]

I just wanted to add that it also works with FPC, only M+ switch must be used to allow declaring methods in published section.

P.S. Sorry for the empty comment on article. I dont know why but nothing showed up when I posted it.

[AthenaOfDelphi]

Thanks for trying it with FPC Grudzio