The memory leak in Omeganaut on Android 64-bit, in the ZGameEditor engine, has finally been located and fixed after nearly a year of research and testing. A dark story of trampolines...

On Android 64, Omeganaut was crashing completely at random. I took the engine apart piece by piece to narrow down the possible causes and try to pinpoint the problem. But after months of trying to get a crash dump that was any clearer than:

********** Crash dump: **********
Build fingerprint: 'google/cheetah/cheetah:15/BP1A.250305.019/13003188:user/release-keys'
#00 0x000000000000001

I decided to go back to basics and write down in a notebook the state of my game every time it crashed. The foundation of any good bug report:

• What happened?
• What did I press?

It took a few months, at a rate of one or two play sessions from time to time...

From these observations, I noticed that the game crashed mostly during level transitions. The only difference during these transitions was the appearance of hundreds of speed lines, anime style.

After removing them and running new tests, the game no longer crashed during these transitions, but it was still crashing when shooting or during an explosion!

Interesting. The speed lines, the lasers, and the explosions all share the same base: the rendering of a semi-transparent low-poly 3D model. In my project, transparency is not handled via the ZGameEditor Material options, but directly with OpenGL code:

glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
@UseMaterial(Material:BasicMaterial);
@RenderMesh(Mesh:SpaceShip);
glDisable(GL_BLEND);   

So I created a test, generating 100 colored cubes per frame, each with a lifetime affecting its transparency.

100 cubes per frame means 6000 cubes per second, which greatly increases the chances of triggering a crash quickly. And as expected, I was able to reduce the code down to a single instruction and still get a crash: glEnable(GL_BLEND);

That doesn’t necessarily mean this specific instruction is at fault, but it confirms that OpenGL library calls were causing a memory leak.

Ville, the creator of ZGameEditor, suggested a few more tests to check OpenGL behavior when the command is applied to the entire scene, rather than to each object individually. That allowed him to rewrite the trampoline management:

ZGameEditor generates at runtime a small piece of code called a trampoline, which makes it possible to call external functions by preparing their arguments correctly. Until now, a trampoline was generated for every single call, even for clones of models. Thousands of trampolines for thousands of cubes. This increased the risk of bugs, especially on Android 64-bit, because the cache memory wasn’t always properly cleaned up, unlike in the 32-bit version. From now on, a single trampoline is shared for each function, which makes the system more stable and efficient. One trampoline for thousands of cubes.

The engine of my game is purring, and you have no idea how relieved I am.

I’m putting the December version back online for Android, compatible with both 32- and 64-bit. The current version has its hood open, ready to go into production and receive new content.