Loading Maps

[Gadget]

I have hit a problem, and I am not sure the best way around the problem.

I have map files:-

Header of about 100 bytes, followed by up to 1000 x 1000 tiles. Each tile is 19 bytes in size.

I need to load at least 2 bytes of the map file into memory for each map, by my reckoning thats 2000000 bytes (2 Meg) of data. There could be up to 255 maps... Now this seems like an awful lot of data to be storing in memory!

The word will contain information relating to the tile, or a pointer to a player object stood there.

Can anyone think of a more practical solution to this? I don't fancy having to store all these in memory, an alternative is to read the relevant offset in each map file every time a player moves, and find another way of working out quickly where players are stood. File access will be too slow I think, especially when you might get 500 players running around one map!

[Alimonster]

What does your tile record/class/whatever look like?

[Gadget]

What does your tile record/class/whatever look like?

[pascal]
type TTile = Record
SmallLib: SmallInt; {library file}
SmallTile: SmallInt; {tile number}
LargeLib: SmallInt;
LargeTile: SmallInt;
ItemLib: SmallInt;
ItemTile: SmallInt;
Frames: Byte; {for animation}
Data: Byte; {1 = no walk / walk, 2 = safe / not safe}
Sound: Word; {track number}
SoundControl: Byte; {1 = start / stop, 2 = repeat, 3 = random, bits 4,5,6,7,8 are volume}
Music: Byte; {track number}
MusicControl: Byte; {1 = start / stop, 2 = repeat}
end; {19 bytes}
[/pascal]

I know this can be optimized, For example, using a bitmask I can merge the lib data into one byte, instead of using 2 bytes each. Potentially I can merge the sound and music data into 1/2 or 1/4 of the size as well. The only data the server needs to be aware of is the walk / don't walk data. This is stored in one bit. I suppose that means I could reduce the stored tile data by 8...

[Harry Hunt]

I guess the best aproach to this problem is to only load one segment of the map at a time and load the other segments "on demand", e.g. if the player comes close to the boundaries of a map.
Pretty much all tile-based games I know work this way. If you optimize your loading routines and use caching, you can create a map, virtually unlimited in size (that is split up into many smaller maps) with no noticeable load-times in between.

A caching system would automatically load let's say nine of the segments into system memory. Now if the player moves, it would unload certain maps and load new maps into the cache. If you do the whole thing thread-driven, you won't notice any of that.


Stupid question: Why does your map have to be so friggin huge?

[Gadget]

I guess the best aproach to this problem is to only load one segment of the map at a time and load the other segments "on demand", e.g. if the player comes close to the boundaries of a map.
Pretty much all tile-based games I know work this way. If you optimize your loading routines and use caching, you can create a map, virtually unlimited in size (that is split up into many smaller maps) with no noticeable load-times in between.

A caching system would automatically load let's say nine of the segments into system memory. Now if the player moves, it would unload certain maps and load new maps into the cache. If you do the whole thing thread-driven, you won't notice any of that.


Stupid question: Why does your map have to be so friggin huge?

Sorry I missed out the important bit... It's a massively multiplayer online RPG, and the map check is done server side, so the server needs to be able to check walk positions on maps for any player anywhere!

[ggs]

Just load it all into memory.

Each of your maps is only ~18mb, which on a modern system is almost nothing.

Now, you cant have all 255 of those maps loaded at a time (the page file would thrash like there is no tomorrow). But you could cache the last 2-3 lasted used maps. Just let them sit in the swap file (thats what it is there for!)

[ggs]

Sorry I missed out the important bit... It's a massively multiplayer online RPG, and the map check is done server side, so the server needs to be able to check walk positions on maps for any player anywhere!

You can still use the title cachin system then. Just have a list of loaded titles, and then a reference count on the number of players which are near them and require them in memory. Then the reference count of a title drops to zero, then you can remove it. Otherwise it would be identical to caching system Harry Hunt proposed.

On requiring a title, check the loaded title list (most rescently used 1st). If it is in the list, then increase its reference count and use it. Otherwise laod the title and set the reference count to 1. Then when the player moves off a title, decrease the referance count (this should be a method).

When the reference count drops to zero, you can free it immidetly or add it to a list of loaded, but unused titles. Then when you need to load a new title, it will kick off the oldest loaded but unused title. Then it is up to use to choice a good unused but loaded title count to use.

[Gadget]

I implemented a quick and easy dynamic array using 1 bit for each tile, 0=walk, 1=dont walk. Using a very fast bitmask routine it works perfectly and is very compact

[Alimonster]

I implemented a quick and easy dynamic array using 1 bit for each tile, 0=walk, 1=dont walk. Using a very fast bitmask routine it works perfectly and is very compact

Compare it to TBits for speed, since that does exactly the same thing . Incidentally, I posted code for a bit array in another thread (you might also want to have a ToggleBit function which uses xor), unless you're not talking about the same thing as I am...

[Zanthos]

You can also use enums for bit manipulation:

[pascal]

type
TBitEnum = set of (bit0,bit1,bit2,bit3,bit4,bit5,bit6,bit7);

var
TestBitEnum : TBitEnum

procedure Test;
begin
TestBitEnum := [bit1,bit2,bit7];
if (bit1 in TestBitEnum) then
begin
MessageBox(0,PChar('Bit 1 Set'),PChar(''),0);
end;
end;

[/pascal]

Obviously setting the number of entries in the bit mask to whatever you want. This method evidently will work horribly if you're using bit array manipulation stuff But for the walk/not walk flags, its viable [/pascal]