Fog of War is common in most RTS games because vision introduces information asymmetry. In Skymagi, vision is valuable enough to have a sigil dedicated to it — the Scrying Orb (or Divination Sigil).

The player wants to understand the state of their own castle (has anyone boarded? are my sigils healthy?) and the state of their enemy (are their mages hurt? out of mana? are they casting a spell at me? what spell?). The Scrying Orb allows players to gather information, so long as they invest resources.

Before I can create the orb, though, I need the baseline: your vision must be limited.

What is the granularity of vision?

• RTS games like Warcraft chose to have tile-based fog of war, with time-lapsed snapshots of things like buildings.
• Tactics games like FTL based vision on a per-room basis, though typically the entire ship is illuminated unless in a Nebula or the sensors are broken.

Skymagi is a mix of these two. There is no strict room grid and units are freeform (like warcraft). If a player is out of their castle, I can’t expect the concept of a room to always exist, such as in a town square. I need a fog of war solution that can work indoors or outdoors.

Hm. Why not just use my lighting system?

Fog of War with raycasting

Lighting is expensive, but with the few number of lights and units, why write a performant fog of war system if their concepts are coupled anyway? I’ll wait until it becomes a problem, then solve it. Preoptimization is a mistake, and all that.

So I wrote a FogOfWarSystem that used rayHandler.pointAtLight(x, y) for every entity that had a flag component FogOfWar attached.

The logic for the system was dead-simple:

• Filter for any units that are not allied, disable their light radius.
• For each enemy unit, if they are within the rayHandler light, show them. Otherwise, add hidden component.

Why a hidden component, why not just set opacity to 0?

This flag component allowed me to quickly fix a bunch of problems:

• You can’t target a hidden enemy with autoattacks or spells
• You should not be able to right click “chase/attack” a hidden enemey
• Hidden enemies should not display a health/mana bar

Thanks to Ashley, it was as simple as an .exclude call:

// get targetable entities for autoattacks / spell targets
Family.all(AllowCombatTag.class, Position.class, Faction.class).exclude(Hidden.class).get();

By reusing lighting, I had fog of war working in less than an hour!

Unfortunately, there could be nasty flickers near boundaries.

Fade it out

I decided to add a fade with a debounce. If you lose vision, you maintain it for a brief time period before it begins to fade.

Not only would that solve my flicker issue, but it also is much smoother for combat and noticing mages around corners.

Fantastic!

I showed it to Piro, pleased with my result, and Piro said ‘I like the blue effect you added, it feels magical.’

… What blue effect?

Time for a rabbit hole about framebuffers

Gamedev is full of rabbit holes.

Last post, I mentioned the background/sky lighting system problem, where the lightmap was applied to the entire background.

In between that post and this, I managed to find a solution to the issue (or so I thought).

I wanted my lightmap to apply only to objects in the scene, and then composite it with the background. That way I could control the ambience of the skies independently from game entities.

Since box2dlight’s lightmap applies a shader to the entire display, I needed a way to have two different views that I could combine. Unlike the UI, the background still needs to be drawn first. Enter framebuffers.

A framebuffer is effectively a drawable texture on the GPU. Up until now, I only needed a single framebuffer (default) for all my draw calls. Under the covers, the lightmap texture is also a framebuffer that multiplies with the current buffer to achieve the lighted result.

So, in order to selectively apply the lightmap, I chose to…

1. Create ‘background’ and ‘scene’ framebuffers on init
2. Render background objects onto the background buffer
3. Render the scene, tiles, and objects onto the scene buffer
4. Apply the lightmap to the scene buffer only
5. Composite the background and scene buffers together, then draw the UI on top.

Immediately, I ran into issues. Rayhandler.render() applied to the default framebuffer only, and it would not apply to my scene framebuffer.

// swap from 'default' framebuffer to 'scene'
sceneBuffer.begin();
// draw my scene, tiles, objects to the buffer
renderSystem.render(dt);
// apply lighting to scene
rayhandler.render(); // does not apply to sceneBuffer
sceneBuffer.end();

Rayhandler has to render to its own lightmap framebuffer internally, which means my sceneBuffer context ended:

sceneBuffer.begin();
// ...
rayhandler.render();
// underneath the covers, this means...
lightmap.begin(); // swaps the current framebuffer to the lightmap

Luckily, box2dlights planned for this use case. They have functions that split out drawing the lightmap and applying it:

• rayHandler.prepareRender() Renders the lightmap to its frame buffer.
• rayHandler.renderOnly() Applies the lightmap to the current buffer.

And like that, I thought I was done and moved on.

The mysterious blue fade

OpenGL provides blend functions for blending draw calls. I used to Libgdx default:

batch.setBlendFunction(GL20.GL_SRC_ALPHA, GL20.GL_ONE_MINUS_SRC_ALPHA);

While this blending function works for my individual scenes, during compositing, the alpha values of transparent objects in the scene were multiplied with sky background even though the floor under them has no transparency.

Though I had not noticed earlier, all of my shadows had a blue tint as well.

After several hours of reading about blending functions, the end result was simple:

batch.setBlendFunctionSeparate(
  GL20.GL_SRC_ALPHA, GL20.GL_ONE_MINUS_SRC_ALPHA,
  GL20.GL_ONE, GL20.GL_ONE_MINUS_SRC_ALPHA
);

This blend function uses a premultiplied alpha when drawing to the scene and background frame buffers, so when the layers are composited together, their alpha values properly overlap.

And with that, fog of war with fade is fixed!

(to which Piro replied “but dang, that bug did give it a vibe tho”)

spooky ghost castle from my various attempts to fix my blend mode. I gotta use this effect later.