metal-experiment: The value of underengineering

18 June 2024

Half a year ago I embarked on the quest to build a cross-platform, general purpose game engine. I was working on asynchronous asset loading, a reflection system, an ECS for representing the scene, serialization, a graphics API wrapper for Metal and Vulkan, but barely got some simple rendering tests on top of these systems.

My next steps were to get cross-platform shader compilation working, and to further improve my reflection and serialization implementation.

One thing that bothered me however, was the sheer slowness of building correct, production ready software. I wanted to experiment more and understand how to write a performant application on top of these abstractions I built. Not spend weeks adding required functionality to my abstraction to perform a simple experiment.

Then it dawned on me: I had been overengineering. The abstractions I had been building did not arise from need, they arose from planning ahead and thinking they were the right thing to build. However, the whole point of leaving existing game engines was to stay closer to metal, and not have annoying abstractions in between me and the hardware. Now I was reintroducing these same exact abstractions I had fought so hard to leave!

So I decided to do a little experiment: write code so simple, it would seem a beginner had written it (note that I still consider myself a beginner). No abstractions until absolutely necessary. In this line of thought, even functions and data structures are considered abstractions. Hardcoding is good.

At first, I had serious withdrawal symptoms, immediately identifying an abstraction after one case of duplicated code, but I managed to suppress the urge. Funnily, the abstraction I thought I had properly identified was actually wrong.

And so I continued, and managed to focus on graphics programming techniques, rather than endlessly engineering a software framework I did not need nor want, with the worst offender being my ECS implementation.

The results speak for themselves. See the source code (https://github.com/arjonagelhout/metal-experiment) and progress pictures:

Day 1: Text rendering using a simple texture atlas, sprite importer and renderer
Day 1: Rendering axes in 3D with a camera and perspective projection. Here I also added a Cocoa view on the right to suppress the urge to build my own UI framework
Day 2: Procedural terrain. This simply generates a terrain mesh between two coordinates and given subdivisions.
Day 2: Flat terrain is pretty boring, so I added some perlin noise.
Day 3: Now, let's light up the scene! But there are no shadows yet?
Day 3: Shadow mapping to the rescue! But it looks weird?
Day 3: Now it looks pretty :) I had to flip the y-axis because of Metal texture coordinates, and add an offset to avoid precision issues in self-intersections when comparing depth of the depth map (top left) and the depth of the sampled fragment.
Day 3: Better colors!
Day 4: Let's add some fog and colored shadows!
Day 4: I added trees! Now it's starting to look like something I would want to walk around in. Alpha blending is an issue, so we should sort the trees by depth.
Day 4: The shadows should also use alpha blending. Currently we just replace the fragment shader with a simple "return half4(0, 0, 0, 1)"
Day 4: Look how happy the sun is in this new terrain!
Day 4: Water and improved shaders for the trees. I'm now reusing the shader logic for calculating lighting for terrain and the trees.

And some videos:

I lined out the things I could try implementing at the top of my main.mm file:

// shading (PBR, blinn phong etc.)
// normals (calculate derivatives for perlin noise terrain)
// skybox
// lens flare / post-processing
// fog
// foliage / tree shader
// water shader
// frustum culling
// LOD system
// collisions (terrain collider, box collider)
// chunks
// scene file format
// scene / level editor
// 3d text rendering
// erosion

Till next time!