Formats

[xGTx]

Well, another issue i've been having I realize i may be able to get help or at least a better understanding at these wonderfull forums.

Currently, in my RPG map editor, i save maps by:

1st) Load all data in a string

In the string i load data in a fashion like:
MapName|MapInfo|MapInfo|MapInfo then a line break. After that i scroll through all the tiles and save each tile data like: TileX,TileY,MoreData,...| Until its done. So my string roughly looks like:

MapName|MapInfo|MapInfo|MapInfo
TileX,TileY,MoreData,...|TileX,TileY,MoreData,...| TileX,TileY,MoreData,...|

2nd) And then i simply save it to a file.

This way when i go to open the file. I split the string by that line break, and then split the first half by the | character and now i have all that data. Then i go through the second half extracting all the tile information and loading it into my type.

My question is... Is there any easier, maybe more professional way of going about this?

[Harry Hunt]

You should use a binary format instead i.e. use file streams, create a record for your file header and then use FileStream.Write to save everything to a file.
There are some cases in which an ASCII-format may be helpful, e.g. if you want to be able to edit the files by hand. But in general, binary will be faster and easier to use.

[xGTx]

Example on doing this please? Is there a way to save a record?

[Harry Hunt]

type
TMyRecord = record
X, Y: Integer;
B: Boolean;
end;

var
FileStream: TFileStream;
MyRecord: TMyRecord;
begin
MyRecord.X := 100;
MyRecord.Y := 150;

FileStream := TFileStream.Create('C:\MyFile.dat', fmCreate);
FileStream.Write(MyRecord, SizeOf(MyRecord));
FileStream.Free;
end;


Please also check the Delphi help file. Most objects/components are really well documented there.

[Mrwb]

Just a small side note; when saving records this way, it's a good idea to use a packed record instead of a regular record, since a packed record always has the same size, so the data is less likely to get currupt when loading/saving. ;)


type
TMyRecord=packed record;

[xGTx]

Ok well i set up everything to save like that. Now, i get access errors when I try and load. Here's what i'm doing to load:

Var TheMap : GTRPG_MAP_VER1;
FileStream : TFileStream;
I, X : Integer;
begin
//- New Load -
Try
FileStream := TFileStream.Create(FileName, fmOpenRead);
FileStream.Read(TheMap, SizeOf(TheMap));
FileStream.Free;
ms_Name.Caption := TheMap.NAME;

[Harry Hunt]

what is GTRPG_MAP_VER1? Please post it here.

[xGTx]

Ok these all inter-twine with eachother:

type
GTRPG_MAP_NPC_VER1 = record
NAME : STRING;
SPRITE : STRING;
MOVES : BOOLEAN;
ATTACKS : BOOLEAN;
ATTACK_MODE : INTEGER;
ATTACKABLE : BOOLEAN;
HP : INTEGER;
MP : INTEGER;
SPD : INTEGER;
STR : INTEGER;
ATTACK_1_TYPE : INTEGER;
ATTACK_1 : INTEGER;
ATTACK_2_TYPE : INTEGER;
ATTACK_2 : INTEGER;
VIEW_DISTANCE : INTEGER;
WORK_TOGETHER : BOOLEAN;
ROAMS : BOOLEAN;
BEHAVIOR : INTEGER; // 1 Coward - 2 Normal - 3 Heroic
PATH : STRING; // 1 UP - 2 RIGHT - 3 DOWN - 4 LEFT
DROPS : STRING; // Item_ID Item_ID..
end;


GTRPG_MAP_BORDER_VER1 = record
BORDER_UP : STRING;
BORDER_DN : STRING;
BORDER_LF : STRING;
BORDER_RT : STRING;
end;

GTRPG_MAP_TILE_ANIMATION_VER1 = record
ACTIVE : BOOLEAN; // Animation Active?
TILES : TSTRINGLIST; // Tiles in animation
INDEX : INTEGER; // Current Tile Displayed
end;

GTRPG_MAP_TILE_VER1 = record
GROUND : Integer;
MASK : Integer;
ANIMATION : GTRPG_MAP_TILE_ANIMATION_VER1;
MASK2 : Integer;
ABOVE : Integer;
ABOVE2 : Integer;
ATTRIBUTE : Integer;
ATTRIBUTE_PROPERTIES: String;
end;

GTRPG_MAP_VER1 = packed record
NAME : STRING;
ZONE : STRING;
MUSIC : STRING;
SKILL : STRING;
BORDER_LOCATIONS : GTRPG_MAP_BORDER_VER1;
MAP_NPCS : ARRAY OF GTRPG_MAP_NPC_VER1;
MAP_TILES : ARRAY[0..15, 0..11] of GTRPG_MAP_TILE_VER1;
end;

[Harry Hunt]

The problem is that your records contain strings which are variable-length

Example:
'Hello World' = 11 Bytes
'DGDev kicks a**' = 15 Bytes

writing them to a file is no problem, but when you write them, you will only write the strings themselves and no information on how long they are (whereas with Integers for example, you always know that they're 32 bytes long).

The solution is to use fixed length strings which you declarare like this



TMyRecord = record
Blah: array[0..255] of Char;
end;


These are called null-terminated strings. You can work with them just like you would with a normal string but you have to restrict yourself to whatever length you chose.

[Harry Hunt]

GTRPG_MAP_TILE_ANIMATION_VER1 = record
ACTIVE : BOOLEAN; // Animation Active?
TILES : TSTRINGLIST; // Tiles in animation
INDEX : INTEGER; // Current Tile Displayed
end;

I just saw this bit of code and this will also cause you some problems. String lists just like strings themselve have a variable length depending on how many entries they have. To store this record in a stream I suggest you do something like this:

GTRPG_MAP_TILE_ANIMATION_VER1 = record
ACTIVE : BOOLEAN; // Animation Active?
INDEX : INTEGER; // Current Tile Displayed
TILELISTSIZE: INTEGER; // <--- New stuff
TILELISTOFFSET: INT64; // <--- New stuff
end;

Note that this won't save the string list yet. The idea is that instead of storing the tile list itself in the record, you only store the size of the tile list and the offest (which is the position of the the list in your file).

So you'd have to find out how big all your records are (if you convert all your strings to null-terminated ones you can use the SizeOf function to get the size).
The next thing you have to do is find out how big your tile lists are. The SizeOf operator won't work for that (it will only return the size of the pointer which is always 4 bytes). Here's a very dirty solution for that problem:


var
MyStringList: TStringList;
MyMemoryStream: TMemoryStream;
begin
MyStringList := TStringList.Create;
FillThis(MyStringList); // Just some bogus procedure that fills the string list
MyMemoryStream := TMemoryStream.Create;
MyMemoryStream.Clear;
MyMemoryStream.Seek(0, soFromBeginning);
MyStringList.SaveToStream(MyMemoryStream);
end;


To obtain the size of the memory stream, use the size property of the memory stream (MyMemoryStream.Size).

With the size of the records you can calculate an offset and store it in the GTRPG_MAP_TILE_ANIMATION_VER1 record.
After you have written all records to your file, you can then write the stringlists to it one after the other.


This sounds like a lot of work and as matter of fact it is :D
Unless you completely re-structure your records, I don't see how you could do it differently though.

Another solution of saving structures of arbitrary length to a stream which involves some "abuse" is to turn all your records into TComponents and to then use the SaveComponentRes method of TFileStream... it works really well and will save you some trouble.
Note that only published properties will be saved to the stream...
If you need to store lists of records try using TCollections instead.
It works but of course it's a hack...[/b]

[Useless Hacker]

Have a look here (viewtopic.php?p=8379&highlight=#8379) for how to write and read variable-length strings to and from streams.

[cairnswm]

The other alternative is to use the XML Document and add each item with its property into the XML document.

This is quite a lot of work but makes it easy to edit by hand if you want to change something on the fly.

[M109uk]

If it helps i have a class i use when writting my formats


uses
Classes;

type
TknStream = Class(TFileStream)
public
procedure WriteChar(const Value: Char);
procedure ReadChar(var Value: Char);
procedure WriteString(const Value: String);
procedure ReadString(var Value: String);
....
end;

....

procedure TknStream.WriteChar(const Value: Char);
begin
Write(Value, SizeOf(Char));
end;

procedure TknStream.ReadChar(var Value: Char);
begin
Read(Value, SizeOf(Char));
end;

procedure TknStream.WriteString(const Value: String);
var
i: Integer;
begin
For i := 1 To Length(Value) Do WriteChar(Value[i]);
WriteChar(#0);
end;

procedure TknStream.ReadString(var Value: String);
var
c: Char;
begin
Value := '';
Repeat
ReadChar(c);
If c <> #0 THen Value := Value+c;
Until c = #0;
end;


More value types are easy to put in, just copy what is in the read/write char procedures, but instead of SizeOf(Char) change it to SizeOf(YourType).

Hope this helps.

[Harry Hunt]

The other alternative is to use the XML Document and add each item with its property into the XML document.

This is quite a lot of work but makes it easy to edit by hand if you want to change something on the fly.

Good point! In general I'd say binary format files are better suited for game data, but I think XML is a cool format.
If you want to save your data in XML format, you might want to check out my XML parser component (see "My Projects")

[xGTx]

Wow guys, I'm kinda overwhelmed. I've coded a whole lot of my game based off my current structure. :(. I don't wanna go to XML because the point of doing it binary is to keep people from "hacking" the game. That class above, will work for my problem?

[Harry Hunt]

The class M109uk posted is a nice little helper but it won't solve your problem directly. You still need to work around the variable-length record problem as explained in my posts...

[xGTx]

I don't quiet understand that memorystream thing. And before I lose all hope, what way do you suggest I restructure the type so it'll save cleanly? And do Strings save? Because those can be variable length right? Do i have to change all : Strings into : Array[0..255] of Char ?

[Harry Hunt]

Because I'm such a nice guy :roll: , I wrote a little example program that saves 101 string lists to a file and loads them again. Create a new app, add a button to your form and paste this code into your code window:


unit Unit1;

interface

uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, StdCtrls;

type
TMyFileHeader = record
Title: array[0..255] of Char;
SomeInteger: Integer;
NumElements: Integer;
end;

TMyElementInfo = record
Desc: array[0..255] of Char;
AnotherInteger: Integer;
Size: Int64;
end;

TMyElement = record
StringList: TStringList;
Blah: Integer;
end;

TForm1 = class(TForm)
Button1: TButton;
procedure Button1Click(Sender: TObject);
end;

var
Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.Button1Click(Sender: TObject);
var
Header: TMyFileHeader;
ElementInfo: array of TMyElementInfo;
Elements: array of TMyElement;
I, J: Integer;
Temp: TMemoryStream;
FileStream: TFileStream;
begin
// Fill the header
Header.Title := 'This is a test';
Header.SomeInteger := 42;
Header.NumElements := 101;

SetLength(Elements, 101);
SetLength(ElementInfo, 101);

// Fill the elements first
for I := 0 to 100 do
begin
Elements[I].StringList := TStringList.Create;
for J := 0 to 9 do
Elements[I].StringList.Add('BlahBlahBlah');
Elements[I].Blah := 12;
end;

// Fill the element info array
for I := 0 to 100 do
begin
ElementInfo[I].Desc := 'Element';
ElementInfo[I].AnotherInteger := 14;
Temp := TMemoryStream.Create;
Temp.Clear;
Temp.Seek(0, soFromBeginning);
Elements[I].StringList.SaveToStream(Temp);
ElementInfo[I].Size := Temp.Size;
Temp.Free;
end;

// Save
FileStream := TFileStream.Create('C:\TestTest.dat', fmCreate);
FileStream.Write(Header, SizeOf(Header));
for I := 0 to 100 do
FileStream.Write(ElementInfo[I], SizeOf(ElementInfo[I]));
for I := 0 to 100 do
begin
Elements[I].StringList.SaveToStream(FileStream);
FileStream.Write(Elements[I].Blah, SizeOf(Integer));
end;
FileStream.Free;

// Load
FileStream := TFileStream.Create('C:\TestTest.dat', fmOpenRead);
FileStream.Read(Header, SizeOf(Header));
SetLength(Elements, Header.NumElements);
SetLength(ElementInfo, Header.NumElements);
for I := 0 to Header.NumElements - 1 do
FileStream.Read(ElementInfo[I], SizeOf(TMyElementInfo));
for I := 0 to Header.NumElements - 1 do
begin
Temp := TMemoryStream.Create;
Temp.Clear;
Temp.Seek(0, soFromBeginning);
Temp.CopyFrom(FileStream, ElementInfo[I].Size);
Elements[I].StringList := TStringList.Create;
Elements[I].StringList.LoadFromStream(Temp);
Temp.Free;
FileStream.Read(Elements[I].Blah, SizeOf(Integer));
end;
FileStream.Free;
end;

end.


The basic concept of this is to separate fixed-length and variable-length data carefully and store information about the size of the variable-length data either in the header or in a separate record.

[xGTx]

Wow, your awsome. Right now im leaving for a bit but im gonna get home tonight and try and sort everything out in my game. I'll let you know how it works out. Thanks again! (btw. What projects are you actively working on? You're obviously a great coder and i'd like to see more)

[xGTx]

Ok i've had a chance to mess around with all that coding and concepts... One major problem however. File size. This way, the files are coming out to around 75KB... Thats horrible, concidering this is a MMORPG. The server has to send the map files to all the users every time they move out of the map to a new map. It looks like I may have to stick to my sloppy way of doing it :(

[cairnswm]

XML can be encrypted or compressed to make it binary. Thats how I store my game config now.

While Array[0.255] of Char will work writeing String[255] is easier. Look up ShortStrings in the Delphi Help.

There are a number of nice component sets out there for zipping and encrypting files: look at the DCP Components http://www.cityinthesky.co.uk/cryptography.html and the
dzdel.zip compression libraries http://swiss.torry.net/vcl/compress/std/dzdel.zip

If you have your information in a Memory Stream you can then encrypt the stream and save to disk. This allows you to save and load encrypted data. (My data gets save with a key of about 100 bytes long - then the key is added to the config XML which is then encrypted using another 32byte binary key).

[cairnswm]

I thought that I should quickly do a demo on how to encrypt your stream information so I did one in the Tutorials section:

http://terraqueous.f2o.org/dgdev/viewtopic.php?p=8561

Have a look.

[Paulius]

In any multiplayer game it?¢_~s a very bad idea to send the whole map, make players download maps before joining the game. You should only send changing data like object A moved to position B, and avoid sending strings, send strings like names only once and enumerate them(Make a list of them and use a number to look them up).

[xGTx]

Oh I do, I think I didnt explain clear enough.

In my MMORPG, it isnt a scrolling map. So when a player walks out of the current map, the next map data is sent.