Pascal Game Development - Tiled CSV map loading (with ZenGL)

I got a request recently to do a tutorial on how I load Tiled maps, and well, how could I refuse? Tilemaps are some of the most basic systems for maps, and relatively painless to implement. There are unfortunately, a number of caveats and some issues which this specific tutorial will not cover.

What is covered:

Parsing some basic XML without an XML unit
Loading multiple layers
Basic usage of ZenGL to draw the map
Object Orientation

What is not covered:

Advanced ZenGL usage (sprite engine, etc.)
Scrolling maps
Efficient maps
Advanced map loading (Objects, layer names, other XML attributes)

A quick list of the tools that we'll use: Lazarus .9.30, FPC 2.4.2, ZenGL .2.1, Tiled QT .6

Final Result:

Getting Started
This tutorial isn't really meant to be a "Let's do this together, right now!" tutorial, but is more of an extensive commenting of the example code I'm going to provide. If you want to brave following along, see the bottom for a basic idea of how to get ZenGL to work with Lazarus, and look at demo05 from ZenGL to see how you can compose a basic rendering, initializing, etc. program. Really, I recommend you simply open the example project I made and follow along that code with this article.

So, what do we need?

Code: [View]

uses
  zglHeader, SysUtils {for some string handling}, Classes {for TStringList}
  ;

We have ZenGL of course, but also we're going to do some extensive string manipulating, so we need the units SysUtils and Classes to accomplish this. You can get by with SysUtils only if you want to shave 100kb of your executable, as you can simply form your own method for parsing the CSV data (which is all the string list is used for).

Objects

Code: [View]

type
  TTile = class
    public
      index: integer;
  end;

  TTileMap = class
    public
      tile: array of TTile;
  end;

I already mentioned we're going to use objects, but this is a great example of super simple object orientation in Pascal.

Variables

Code: [View]

var
  f: zglPFont;
  tex: zglPTexture;
  mapW, mapH: integer;
  layer: array of TTileMap;

f is our base font, which uses the font included with ZenGL's demos. I was too lazy to generate my own font, and it is only being used to display the FPS. tex is the texture for the tilemap. mapW / mapH are going to hold the width and height of the tilemap (in terms of tiles). In our case, this is 20x15. layer is an array of TTileMap, because I thought it would be easier to have each TTileMap be its own layer, and I wanted to show how to use multiple layers.

Initialization
This is going to be the last section of the source file (in case you're following along).

Code: [View]

begin
  zglLoad(libZenGL);

  randomize();

  zgl_Reg(SYS_LOAD, @Init);
  zgl_Reg(SYS_DRAW, @Draw);

  zgl_Enable(APP_USE_UTF8);

  wnd_SetCaption('Tiled CSV Example');

  wnd_ShowCursor(TRUE);

  scr_SetOptions(640, 480, REFRESH_MAXIMUM, FALSE, FALSE);

  zgl_Init();
end.

When our application is created, we load the ZenGL library (because we used dynamic linking), and perform numerous initializations (pretty self explanatory).

procedure Init
We told ZenGL to call the Init procedure when it loads, and here is what we have for it:

Code: [View]

procedure Init;
begin
  f := font_LoadFromFile('font.zfi');
  tex := tex_LoadFromFile('tiles.png', $FF000000, TEX_DEFAULT_2D);
  tex_SetFrameSize(tex, 32, 32);
  loadMap('map_csv.tmx');
end;

Here we are simply loading some files.

Code: [View]

tex_SetFrameSize(tex, 32, 32);

This is a ZenGL function for sprite strips. In our case, the tilemap consists of 32x32 sprites. ZenGL's going to do the heavy lifting regarding how to draw sections of this image.

Just in case

Code: [View]

procedure debug(output: string);
begin
  WriteLn(output);
end;

I always make some sort of simple function like this, so if you notice it in the code... well, it's just a quick way to output to the terminal. Ideally you wouldn't have the terminal visible in Windows (for instance) in your final build, which is why I consider it debug.

The Map Loading (loadMap)
As you might expect, this one's a doozy. First, the entirety of the function:

Code: [View]

procedure loadMap(fname: string);
var
  mapfile: TextFile;
  s, s2, beginning: string;
  llen, i: integer;
  csv: TStringList;
begin
  AssignFile(mapfile, fname);
  Reset(mapfile);

  while(not(EOF(mapfile))) do
  begin
    ReadLn(mapfile, s);
    s := LowerCase(Trim(s));

    beginning := LeftStr(s, 5);

    if(beginning = '<map ') then
    begin
      // Do some basic XML parsing / hacking to get dimensions of map

      // Sets s2 = the string starting from the width
      s2 := Copy(s, Pos('width', s)+7, 10000);
      // Final parsing, copying to the double quote, we now have a number
      mapW := StrToInt(Copy(s2, 1, Pos('"', s2)-1));

      // Sets s2 = the string starting from the width
      s2 := Copy(s, Pos('height', s)+8, 10000);
      // Final parsing, copying to the double quote, we now have a number
      mapH := StrToInt(Copy(s2, 1, Pos('"', s2)-1));

      debug('Map dimensions: '+IntToStr(mapW)+'x'+IntToStr(mapH));
    end else
    if(beginning = '<laye') then
    begin
      // Wee, we have a new tile layer full of delicious CSV tile data

      // Initialize objects and arrays to the map dimensions
      llen := Length(layer); // Going to be using this a lot, so make a var
      SetLength(layer, llen+1);
      layer[llen] := TTileMap.Create;
      SetLength(layer[llen].tile, mapW * mapH);
      for i := 0 to mapW * mapH do
      begin
        layer[llen].tile[i] := TTile.Create;
      end;
      debug('layer '+IntToStr(llen)+' objects initialized');

      // Read until we hit the CSV data
      while(not(s = '<data encoding="csv">')) do // This is the last line before
      begin
        ReadLn(mapfile, s);
        s := LowerCase(Trim(s));
      end;

      csv := TStringList.Create;
      s2 := '';
      // Read CSV data until no more
      while(not(s = '</data>')) do
      begin
        ReadLn(mapfile, s);
        s := LowerCase(Trim(s));
        s2 := Concat(s2, s);
      end;
      s2 := Copy(s2, 1, Length(s2)-7); // </data> would otherwise be appended
      debug(s2);

      // CSV split into a TStringList
      csv.StrictDelimiter := true;
      csv.Delimiter := ',';
      csv.DelimitedText := s2;
      debug('-----');

      // Tile data populated
      for i := 0 to csv.Count-1 do
      begin
        layer[llen].tile[i].index := StrToInt(csv[i]);
      end;

      debug('');
    end;
  end;

  CloseFile(mapfile);
end;

Code: [View]

AssignFile(mapfile, fname);
  Reset(mapfile);

  while(not(EOF(mapfile))) do
  begin
    ReadLn(mapfile, s);
    s := LowerCase(Trim(s));

    beginning := LeftStr(s, 5);

We start off this procedure by opening our TMX file, fname (the argument of this function), into the TextFile mapfile. Then we kick off a loop to read until we hit the end of the file. You can see I'm careful how data is fed into the string s, it's trimmed of all whitespace and is lowercased, just in case someone edits it by hand and changes the capitalization. The variable beginning is notable because it only checks the first 5 characters of every line.

Parsing Map Dimensions

Code: [View]

if(beginning = '<map ') then
    begin
      // Do some basic XML parsing / hacking to get dimensions of map

      // Sets s2 = the string starting from the width
      s2 := Copy(s, Pos('width', s)+7, 10000);
      // Final parsing, copying to the double quote, we now have a number
      mapW := StrToInt(Copy(s2, 1, Pos('"', s2)-1));

      // Sets s2 = the string starting from the width
      s2 := Copy(s, Pos('height', s)+8, 10000);
      // Final parsing, copying to the double quote, we now have a number
      mapH := StrToInt(Copy(s2, 1, Pos('"', s2)-1));

      debug('Map dimensions: '+IntToStr(mapW)+'x'+IntToStr(mapH));
    end else

Why on earth can we only check the first 5 characters? Well, with the way Tiled's XML structure is currently, all of the tags are rather unique. Remember our mapW / mapH variables? We are now reading their values from the XML. This is a short example of how you can do some basic XML parsing, and I made this example project load the value from the XML in case you decide to make the map bigger.

Code: [View]

<map version="1.0" orientation="orthogonal" width="20" height="15" tilewidth="32" tileheight="32">

The above is the line we are parsing: ripping out the width and height attributes from this tag.

Code: [View]

// Sets s2 = the string starting from the width
      s2 := Copy(s, Pos('width', s)+7, 10000);

Here we are taking a substring of "s", starting at the position where "width" is first encountered, then adding 7 to that position so the string starts with the width.

Code: [View]

// Final parsing, copying to the double quote, we now have a number
      mapW := StrToInt(Copy(s2, 1, Pos('"', s2)-1));

Using s2, we begin copying it until we hit the next double quote, and convert it to the integer. The above two processes are exactly the same for getting the height.

Side note: If you notice, checking for a <map> tag is done every time a line is read. Currently, Tiled stores 1 map per file, although its not inconceivable to concatenate files yourself. I have no idea if this code will work well in those situations, but you're a programmer! (hopefully)

Dealing with new layers

Code: [View]

if(beginning = '<laye') then
    begin
      // Wee, we have a new tile layer full of delicious CSV tile data

      // Initialize objects and arrays to the map dimensions
      llen := Length(layer); // Going to be using this a lot, so make a var
      SetLength(layer, llen+1);
      layer[llen] := TTileMap.Create;
      SetLength(layer[llen].tile, mapW * mapH);
      for i := 0 to mapW * mapH do
      begin
        layer[llen].tile[i] := TTile.Create;
      end;
      debug('layer '+IntToStr(llen)+' objects initialized');

Once again, in the first line you can see we check for one tag to decide we have a new layer to parse.

Code: [View]

llen := Length(layer); // Going to be using this a lot, so make a var
      SetLength(layer, llen+1);
      layer[llen] := TTileMap.Create;
      SetLength(layer[llen].tile, mapW * mapH);

Here I created the llen variable, because I foresaw I would be using the length of the layer array often. Next, we have to create a new TTileMap object after resizing the array. Next, we set the tile array of the layer we just created to have a size equal to the width x the height, which is 300 in our case.

Code: [View]

for i := 0 to mapW * mapH do
      begin
        layer[llen].tile[i] := TTile.Create;
      end;

Those with a quick eye noticed we set the size of the tile array, but never actually instantiated the objects. That is all this for loop does.

Almost to the CSV!

Code: [View]

// Read until we hit the CSV data
      while(not(s = '<data encoding="csv">')) do // This is the last line before
      begin
        ReadLn(mapfile, s);
        s := LowerCase(Trim(s));
      end;

I wasn't sure what other <data> tags were possible (and indeed this might be a bad idea, what if there are more attributes that can be applied to this one specific tag, rather than my un-based assumption that Tiled would put them in new <data> tags). At any rate, we just read lines until we hit this line exactly. The next line is the beginning of the layer's CSV data!

Loading the CSV into a single string

Code: [View]

csv := TStringList.Create;
      s2 := '';
      // Read CSV data until no more
      while(not(s = '</data>')) do
      begin
        ReadLn(mapfile, s);
        s := LowerCase(Trim(s));
        s2 := Concat(s2, s);
      end;
      s2 := Copy(s2, 1, Length(s2)-7); // </data> would otherwise be appended

First, I quickly initialize csv for later use. In this <data> tag are multiple lines of CSV data. "</data>" marks the end of the data, so we loop reading the line until we come across that line, continuously concatenating to s2 every line as we go. Due to the way the loop is coded, after we exit the loop we still have that one extra line containing "</data>" appended to s2, so we chop it off.

Splitting up the CSV data

Code: [View]

// CSV split into a TStringList
      csv.StrictDelimiter := true;
      csv.Delimiter := ',';
      csv.DelimitedText := s2;
      debug('-----');

About time! So we have the CSV data loaded. It's in the structure of something like "0,0,0,0", where each "0" is a tile number. For dealing with this, I used the handy TStringList, which has built in support for delimiting. We simply set it to perform delimiting, set the delimiter (","), and set the text we want delimited, s2. Bam. Now we can access each individual tile by getting a different index of the array csv.

Putting that data in our objects and finishing up

Code: [View]

// Tile data populated
      for i := 0 to csv.Count-1 do
      begin
        layer[llen].tile[i].index := StrToInt(csv[i]);
      end;

Great, we have each tile in its own element of csv, and here we loop through placing each value into the tiles. Remember, llen is the current layer we are working with (since we have multiple, presumably). The CSV in the Tiled map file is structured like it is visually. There (were) rows and columns from top left to bottom right with their visual representations visible in Tiled. This makes it simple, because we can also load and draw in this manner, from top left to bottom right.
(There's some more code that ends that procedure)

Code: [View]

     debug('');
    end;
  end;

  CloseFile(mapfile);
end;

Holy cow, we're done.

Drawing that Map
We just spent forever making that map load itself into a bunch of objects, and now it's off to draw them. Remember, we told ZenGL to use the Draw; procedure for drawing, so here it is:

Code: [View]

procedure Draw;
var
  i, i2, i3, c, c2: integer;
begin
  batch2d_Begin();  
  c := 0;
  c2 := 0;
  for i := 0 to mapH-1 do
  begin
    for i2 := 0 to mapW-1 do
    begin
      for i3 := 0 to Length(layer)-1 do
      begin
        if(not(layer[i3].tile[c].index = 0)) then
        begin
          // tex, x, y, w, h, angle, index
          asprite2d_Draw(tex, i2*32, i*32, 32, 32, 0, layer[i3].tile[c].index);
          Inc(c2);
        end;
      end;
      Inc(c);
    end;
  end;

  text_Draw(f, 0, 0, 'fps: ' + u_IntToStr(zgl_Get(RENDER_FPS)));
  batch2d_End();
end;

A for loop within a for loop within a for loop? No problem, boss. The initial for loop is for looping down each row, and within that is the loop that says while we're in each row, let's loop through each column.

Code: [View]

 for i3 := 0 to Length(layer)-1 do
      begin
        if(not(layer[i3].tile[c].index = 0)) then
        begin
          // tex, x, y, w, h, angle, index
          asprite2d_Draw(tex, i2*32, i*32, 32, 32, 0, layer[i3].tile[c].index);
          Inc(c2);
        end;
      end;
      Inc(c);

You'll notice in this one, the third for loop is for drawing every layer's tiles on top of each other in one go. First, we check if the tile is not = 0, because if it was we're wasting resources drawing a non-existant tile, or we're crashing because there isn't an index 0. I provided the parameters for this sprite drawing function so you could make sense of it. We draw our texture, which is the tilemap, then the x value, which is i2, and multiply it by 32 for spacing, as we have 32x32 tiles. Same for the y value, and again the tiles are 32x32 so we set the width and height params to that, with 0 rotation, and the index of tile. You'll see the variable c is used as a counter variable to determine the total number of tiles we've cycled through so far. Actually, that's a bit misleading. It's the number of tile positions we've drawn so far, so in the end it'll max out at 300. c2 counts the total number of tiles we've actually drawn.

The general structure of these loops is:
We're on a row -> Go through each column in this row -> draw every tile in this row,column (in every layer), switch to next column, etc, -> next row, etc.

Tiled's nice file structure is what allows us to use so little space to draw so many tiles. It puts the lowest leveled layers first in the file, which is extremely handy because this is also how we want to draw the layers: from lowest to highest (for proper z order)

End of drawing

Code: [View]

text_Draw(f, 0, 0, 'fps: ' + u_IntToStr(zgl_Get(RENDER_FPS)));
text_draw(f, 0, 26, 'tiles: '+ u_IntToStr(c2)); 
  batch2d_End();
end;

FPS and tile count is drawn

Comparison of Tiled and our program:

You're Done, see attachment for code and such

Using the Example Code Provided
I created the example project with Lazarus, but it should be easy enough to use Delphi or FPC only. The entire example is contained in one file, the .lpr file.

Assuming you're using Lazarus, be sure to also download ZenGL from http://zengl.org. Once downloaded and extracted, in Lazarus go to Project -> Project Options. Click on "Compiler Options", and in "Other Units -Fu", change the line to point to the path you extracted ZenGL to the headers directory. Example: ZenGL_Base_Install/headers/ or C:/ZenGL/headers/

If you're on Windows, the example should now be able to run. If you're on a different OS, see the ZenGL wiki for compilation instructions. Note that this tutorial uses dynamic linking, because static linking is a little tricky to get working properly, and I wanted the simplest experience. Once this library is compiled, copy it to the bin directory.

About the Code and I
This code is kind of hackish in parts, notably the XML parsing. I'm currently a University student doing this in my spare time, for use in my own personal projects, so this does not matter. While I did try to implement some future-proofing into the loading code, if you're doing this for an important commercial project I would highly suggest just implementing an XML unit to parse everything -- it is the only way to be 100% sure if the TMX file format changes, you will still hopefully have access to your layer data. It also makes much cleaner and more readable code

But alas, I will never feed my projects a different TMX structure, nor do I care about readability in this procedure (for loading the maps), and I do care about executable size, which can be driven up by some XML units. Sure, it's not a drastic amount in the long run, but why have it if you don't need it?

Final Notes

Sorry if I screwed up between CSV and CVS, happens.
If you move the Tiled maps, you'll have to hand edit them to point to the new location of the tileset image or Tiled won't load the map.
Sorry I'm kind of inconsistent with capitalization. Ex, I use camel case for Pascal and FPC procedures, but not for keywords, variables, or my procedures.
Again, this should work just fine on Win/Mac/Lin Delphi/Lazarus, provided you have the proper know-how.
I plan to do examples for XML and base64 (non z-lib) versions of TMX files in the future
I've been busy with a bunch of stuff lately :|
ZenGL doesn't require a lot to get working! As such, it's very easy to just pick apart the Tiled loading code.
The code and my silly tileset are licensed for the public domain. Do what you want
I don't utilize anything fancy with ZenGL. Partly because I was lazy, and partly because I didn't want to confuse the over all concepts. ZenGL offers more managed ways to draw a tilemap, instead of drawing every individual tile yourself.
Tiled and ZenGL are both awesome and open source. If you currently use neither, check them out.
Even if you don't use Tiled (or ZenGL) in your projects, you can easily rip this tutorial and example code apart for its CSV parsing
Making a full tileset is extremely time consuming, so I didn't do it. You can easily tell where there could be more tiles to make it look better, this wasn't meant to be beautiful.
Oh how I wish we had a Pascal highlighter..... ... .. .
Sorry about the in-post horrible indentation. I couldn't be bothered at the moment to reformat it from the source file. I'll probably get around to fixing it later.

9 Comments

Traveler - 04-04-2011, 09:17 AM
- Reply
Nice one! I'm currently at work so I haven't completely gone over it yet, but I will when I get home.
Are you going to do a second tutorial, for the more advanced topics?

dazappa - 04-04-2011, 12:07 PM
- Reply
Yeah, I plan on releasing an XML loader, which afterward I'll merge with this tutorial to show how to do it more professionally, then I'll release an XML + base64 Tiled loader.

As for objects and other attribute reading, that will probably come as a last tutorial, or maybe I'll throw it in with the general XML reading tutorial. I think in most future tutorials or examples, I'll just list this article as "required reading" to avoid trying to explain a lot of stuff again, but they'll probably be different enough to require a lot of explanation anyway.

I'll have another one up by next weekend, at the least. School's a killer

Andru - 04-04-2011, 01:59 PM
- Reply
Nice! But tileset, which are used here, is not good for position with floating point values or for scaling the map by camera(when linear filtering is enabled). Seems, there is a need to write another tutorial about how to prepare proper tileset, which will be rendered correctly in any cases

dazappa - 04-04-2011, 03:12 PM
- Reply
Yes, I used a very simple method for drawing so it would be easier for people to port code to other engines. There are some interesting things with ZenGL I would like to try in the future, such as using the sprite engine and this mysterious function: tiles2d_Draw (going to have to see if it does what I think it might).

Unrelated, but I've been doing a basic performance test; how many 32x32 sprites can engines render at 60fps on my computer?
love: 6100 // Love 2d; OpenGL; interpreted scripting
zgl: 20200 // ZenGL; OpenGL; compiled
gm7: 3100 // Game Maker 7; DirectX; interpreted scripting
enigma: 10500 // C++; OpenGL; compiled
java: 18100 // OpenGL; "compiled" to Java bytecode
flash: 2000 // supposedly hardware accelerated but I don't know
This is a super rough estimate, as there may be other areas each of these excels at, and I didn't do any specific bottleneck checks between cpu / gpu, although I wouldn't be surprised if ZenGL was limited by my GPU currently. I also didn't check whether anything besides flash did dirty rectangles automatically (because I'm drawing all the sprites at the same x,y coords)

I think you deserve a pat on the back, Andru Of course there are hundreds of other libraries and engines, so take these with a grain of salt!

code_glitch - 04-04-2011, 04:46 PM
- Reply
Might I put forward the suggestion of Prometheus? I have been working quite hard on getting speed up there lately... I doubt it will get ahead of Zengl, but it might be interesting to see how good of a job I did

Oh, I'm working on a slightly better looking bunch of code that does the same thing based on this myself so I might put that up too...

And for the record, I am hella biased so yay. Nice tut though dazappa.

Andru - 04-04-2011, 05:16 PM
- Reply
Originally Posted by dazappa

this mysterious function: tiles2d_Draw (going to have to see if it does what I think it might).

If I have a free time I will try to describe it in wiki. This function can increase speed of rendering for tiles, because it uses faster clipping and no need to call asprite2d_Draw many times, that decreases CPU using.

code_glitch - 04-04-2011, 07:52 PM
- Reply
Currrently in the process of writring an article for the alternate method of loading data from those TileD tmx files based on this... TBH there is still some work to be done on my program but its getting there quite nicely. To tell the truth, I will also include this into Prometheus_Core and etc; thats my bias for you. When you're developing a library why not make it as good as it could be right?

For the sake of all the others I will include the code as a standalone console application. On the brighter side you dont need lazarus, just plain FPC with classes, crt and sysutils. If you dont need the clrscr and delay calls for debugging then its just sysutils and classes so yay. And even then it should work without either

wagenheimer - 14-08-2013, 12:34 PM
- Reply
Any news on this?

I'm thinking in implement full support to Tiled.

WILL - 14-08-2013, 05:09 PM
- Reply
I like Tiled. Was playing with it sometime last week. As long as you organize your assets well it can be used rather effectively.