Smoothening tile edges

[User137]

I'm working on a tile-based game at the moment, and could use some hints on making it look better. Rendering is using GLSL pixel shader for these 6-sided hexagon tiles. I let there be 4 texturemaps for the terrain, so i can blend them without too much performance loss. Basically fragment shader does:

  if (color.w > 0.5) {
    gl_FragColor = color * (
      ts.x * texture2D(tex0, texCoord) +
      ts.y * texture2D(tex1, texCoord) +
      ts.z * texture2D(tex2, texCoord) +
      ts.w * texture2D(tex3, texCoord) );
  } else {
    gl_FragColor = vec4(0.0);
  }

So i can "break" 1 edge that is exposed to air, to make it look flatter as seen in the attached screenshot. I render 2 dirt tile and 1 stone tile on right side of them. The top-left corner use alpha 0.2 and bottom mid-left corners are set alpha -0.2, that's variety i can let there be... But it didn't work as well as i hoped. Instead of having 1 sharp edge, there now is 2 smaller ones. Ok that looks better than that 1 big sharp edge, when zoomed further out, but is there anything i can do to make it rounder/flatter? I could propably modify color.w component to start alpha 0 from that edge at 0.5, scale it differently. But then it wouldn't be solid edge anymore.

edit: Scaling was easy to do so attached comparison what it would look like

vec4 col = vec4(color.xyz, (color.w-0.5)*2.0);

[phibermon]

Don't forget that in order to draw the hexgons, they're deconstructed into triangles and you're essentially setting the alpha level on a shared vertex between two triangles which causes exactly what you describe. if you render the scene in wireframe that should make it more obvious.


You don't have to shade your fragments in the classical sence, IE you can control the behaviour with properties that don't come along with the vertices.

for example, if you wanted to fade the edge of a hex out or blend it etc following the edge, you'd instead control the blending as a function of the distance to that line.

Not the only solution but in a really basic way :

Pass the two points that make up a hex edge as uniforms into your shader.


In your pixel shader you can then do some simple maths to work out the fragments distance to the line and then shade it accordingly, You can then get any result you like.


You could also tesselate the hexes for more than 6 triangles, say spitting those 6 into 3 each, this will give you a greater vertex density bringing you closer to the look of per-pixel but signifigantly quicker (note any situation where a shader's main bottleneck is the fragment shader, you should look to see if you can move certain things to the vertex/evaluation/control shaders and then come closer to the per pixel result by evenly tesselating your source geometry. Check out 'Opacity Maps', specifically a similar optimization made with opacity mapped particle systems)

You could also render all hexes on the outside as textured circles of radi equal to that of the hex, with half of the circle fading opacity to zero. the circles would then be aligned (by querying the surrounding blank hexes) so that the faded side points out, this would give a feathered but much smoother looking edge when 'zoomed' out by larger factors.

You could also 'mask' by rendering over the edges of the hexes, like a 'fog of war' on lots of rts titles. Then you don't have to worry about smoothing the hexes, instead worry about drawing a smooth border. This is actually what I'd do because you could quickly identify long rows of hexes in the map by walking around it's edge, then you could draw one big overlay/mask over that group, giving an even smooth all the way down etc

Alternativley you could render the map as a low resolution, filtered, alpha texture (where 0 is no hex etc) Then as you render your actual map, you translate the map coord to this texture and then sample it, using the result as the alpha value in your hex render. the filtering from a low resolution texture will greatly smooth out the edge. If you do this you should scale the co-ordinates slightly so the mask is referenced slightly smaller than the actual hexes, this would be like 'feathering' the alpha edge, encroaching the blend further into the visible hexes (if you get me)


Edit : For anybody reading unfamiliar with shaders, here's me practicing some tutorial writing again - Fragment Shader = Pixel Shader. Control and Evaluation shaders are new additons in GL4.0, seperate from Geometry shaders you may of heard of. Control+Evaluation lets you take arbitary source vertices (called a 'patch' in this context) apply the tesselation hardware and get varied (yet still subject to the hardwired tesselation routines) output containing a higher number of vertices. You can control all per vertex data that gets pumped in and effect new, greater number of vertices that pop out of the tesselation. Geometry shaders are vaugely similar in that you can generate or bypass output via code but they are a lot more relaxed and varied in what they can do. They can also be used to tesselate although its very slow, hence the tesselation control/evaluation shaders which are specifically intended for the task of subdividing surfaces. An example of the difference is that a geometry shader could output cubes if you input points, or output 3D fishes if you input lines where as the tesselation shader couldn't. It could however take a simple cube and output a smoothed cube (smoothed edges or offset the new points into a sphere). So can the Geometry stage with some trickery (but not as quick),

[User137]

Isn't it out of options to have unique uniforms passed per tile? I am rendering hundreds of tiles on single call. Tessellation is nice idea, but my knowledge on the subject is non existent It might require some higher level graphics card, or high OpenGL version. Also doubling the triangle count might double the rendering time, depending on implementation, graphics card etc. Would be nice to have high compatibility even with the poorest laptops with integrated cards.

I'm not sure if i can use opacity map. Firstly it would have to be massive texture for all tiles on screen at once, and secondly it needs to be changeable at realtime. Current terrain textures are size 256x256 that repeats, and the hexagon is made of 6 triangles connected to the middle vertex. I would guess the resolution of opacity map would have to be at least 6x6 pixels per tile... Well, propably even 512x512 would be enough for that, although the realtime requirement would not be met. 6000x6000 texture or so for whole tilemap would be out of question, but i still haven't set any world sizes in stone.

I think i figured a nice alternative, that i will simply not render those outside triangles, and doing this triangle elimination on the CPU. I could remove 3 or 4 triangles, 3 on straight, 4 on corner. This will let me have really flat surfaces on all 6 sides, and i could make even big circle shapes with no holes in the edges. It will make interesting math for collision with these "half-tiles" though. I can also get rid of all alpha-related special treatment in fragment shader, making rendering even faster.

Oh, there is yet another option... I could add 1 triangle to fill the gap between 2 hexagons. Brilliant...

[User137]

And i must say that over 700 views in less than 24 hours is unexpected I don't know if you even realize how much that is on any forum scale. Stop lurking and start posting, now! There are no bad replies.

[Rodrigo Robles]

And i must say that over 700 views in less than 24 hours is unexpected I don't know if you even realize how much that is on any forum scale. Stop lurking and start posting, now! There are no bad replies.

The subject is very interesting but, most people, like me, doesn't have sufficient knowledge about shaders to help.

[pstudio]

And i must say that over 700 views in less than 24 hours is unexpected I don't know if you even realize how much that is on any forum scale. Stop lurking and start posting, now! There are no bad replies.

I'm not even sure what effect you are aiming for :?
Do you want all the edges of the whole map to be smooth or every tile edge? Or is it something else?

[User137]

I'm making a circle shaped world. I'm not far enough to show the idea with game yet, but this should give you the idea:
http://upload.wikimedia.org/wikipedi...Order7.svg.png
Center of gravity is in the middle of the planet, and player feet are always towards it. Imagine walking on that "spiky" surface, there are these "holes" on all 6 sides of it. And i believe this is solved by adding triangles outside the hexagons to fill the gaps.

And not just the planet surface, but some caves inside. If not natural caves, then digged ones. Idea similar to
http://i146.photobucket.com/albums/r...fullscreen.jpg

Hmm.. know what, i would get much easier by doing this with square tiles. I could add triangles between edges with it aswell. And it might even look much better, not to mention easier math around it.

1 tile from 4 triangles and i could also leave 2 triangles hidden depending on corner, instead of adding new ones between tiles. This would also increase the rendering performance. I have to consider the options...
http://i.stack.imgur.com/ssaaS.png