Tag Archives: cpp

TDME2 – The port of my JAVA based 3D engine to C++11 using J2C

1. Introduction

This is a short (success) story of porting my 3D engine, named TDME, from JAVA 1.6 to C++11.

To introduce myself a bit I can tell that I became in 2010 jobwise and by accident part of a 20-25 man power team that created browser games.

I did this for 3+ years, while playing a key role doing backend stuff, and came to the conclusion that creating games was the most interesting, challenging and also most fun thing to do.

For some reasons I decided in 2012 that I wanted to dig deeper. My interest ranged from multiplayer network code to 3D realtime rendering and such as alternative efficient data storage than relational databases.

Then I just started to develop TDME as a hobby project, which means doing a lot of research, doing a lot of trial and error and just do programming with a strong hands on mentality. This way I got a small JAVA based 3D visualization engine with some basic physics system built in which included e.g. collision detection.

On the backend side a friend of mine helped out and ported parts of the JAVA based TDME engine like collision detection to C++. Later I added network code. TDMECPP was born.

Yes, my luck was that I had some mates with me all the time that were kind enough to support me. Some of them still do and I am very thankful for that. They provided art assets, did game design and helped out gaining ideas for tools and such.

Then we actually decided to create a 3D realtime multiplayer game which type is known as MOBA.

Soon we had a game prototype that even had a multiplayer game mode included.

And now we get to the point. It turned out that doing such kind of game with JAVA in frontend (and C++ in backend) was not the best idea.

2. The issues with TDME/JAVA

2.1 Garbage collection

I had to fight garbage collection which blocked the frontend unpredictable for
even a second sometimes which is bad for such kind of game.

There was this one idea to fix this by allocating all the memory required on game session start up and never release it until the game session was finished, which improved the situation but lead to e.g. ugly code. At the end I decided that this approach was not good enough.

2.2. JIT

One minor thing was also that you could watch getting logic being handled by JIT step by step. Means logic would take some more time if being executed first and later be faster.
Not that big issue but still an issue. The idea to work around that was to have a warm up run or something similar. But I never tried.

2.3. Multi threading

There were some other hacks required because this way getting feasible multi threading ready logic was harder to implement as this kind of code was not renentrant by default even if it could be in C++ easily.

The game was multithreaded. It had one thread for visualization engine, one for game logic plus physics and one for path finding.

2.4. Network latency

Another issue was that frontend network code had too high network latency which I had no influence on as I was just using JAVA functionality. Using JNI and roll something own was not an option for me for various reasons.

2.5. SIMD

Another very valid point is that JAVA does not allow to do SIMD explicitly.

2.6. JAVA and C++ in the same project

Last point is that having C++ in backend and JAVA in frontend means that you can not share code between them thus having game logic twice. And I just did not wanted to do JNI or such.

3. The C++ port

I decided to try a complete C++ port of TDME. Meanwhile TDME had approximatively a 2 MB big codebase. So I searched the web for a JAVA to C++ translator and found J2C. I also checked the option to compile JAVA to native executables but found nothing feasible.

3.1. What does J2C do by default?

J2C is a Eclipse plugin for JAVA 1.6. Basically It is able to parse the sources of a JAVA project into a AST / syntax tree and prints this tree as C++11 into a new project. The generated project also includes a working makefile as well as JRE classes with empty definitions that are required by the project being ported.

You need to know first that during the whole process the top priority was to get a part running first, then fix or improve things, and test if it still works. This way the port might have taken a bit longer but it was the safest way.

3.2. What did I do to adapt J2C to suite the needs

  • parse JAVADOC and add it into method declaration in header files
  • do not use full namespace in definitions or declarations, always import classes in header of files
  • .hpp -> .h
  • removed null pointer checks (and thus throwing NullPointerExeption)
  • some other minor things

3.3. Port pre process

  • Namespace root simplification net.drewke.tdme -> tdme
  • use as less as possible JAVA classes
    • use own ArrayList, HashMap, Console, … classes
  • Rename some classes to avoid name collision

3.4. Port post process

  • not 100% of the generated code did compile, there were problems with a few files, but they were easy to fix
  • remove JOGAMP
    • use OpenGL and OpenAL instead
    • replace it with GLUT for Window management, GL context initialization and HID events
  • implement required generated JRE classes/methods, because they get not implemented by J2C!
  • import code from TDMECPP like
    • jsonbox
    • tinyXML
    • …
  • get rid of JAVA API usage for accessing JSON and XML files and use the above
  • replace JAVA data containers with STL data containers like std::vector, std::map
  • get completely rid of java::lang::String and friends, rather use std::wstring
  • remove dependency of each class from java::lang::Object and it methods
  • refactor left required classes from JRE into tdme::utils and use them instead of JRE ones
  • get completely rid of JRE classes and remove them
  • refactor static class initialization methods to static variable definitions
  • convert engine to use std::string instead of std::wstring
  • use call/return by reference and constness for various classes like std::string, tdme::math::*
  • convert J2C constructor (structure) to ordinary C++ constructors
  • do memory management
    • put variables on stack instead of heap where appropriate
      • J2C creates everything on heap
    • implement destructors to release memory eventually
  • Include and use 3rd party libraries for certain JAVA functionality, e.g.
    • libpng for loading textures
    • vorbis/ogg for audio package
    • pthreads for multi threading
    • …
  • do inlining where appropriate
  • convert JAVA protected package modifier, which has been transformed to C++ by just using C++ public modifier, to be implemented via friend class relatationship
  • manually transfer TDME code comments to TDME2 because Eclipse AST / syntax tree does not contain them thus J2C can just not do it
  • …

4. Where are we now?

TDME2 is the result of TDME(-JAVA) port to C++11 and TDMECPP. Check its GitHub site.
It already has lots of stuff build in. But please note that TDME2 had no release yet.
A bit of stuff is still left to finish the port. SIMD is missing still too.
But I already continued development on TDME2, as requirements were added, like adding FBX model file support with FBXSDK and much more.

One very good side effect is that now I can now port the engine to other platforms where JAVA is not available or parts of its eco system like JOGAMP.
E.g. I plan to port the engine to Haiku and maybe NetBSD. Lets see.

Yes. I love open source software and open source operating systems.
TDME2 already runs on MacOSX, Windows and Linux.

Now I use the engine for a new game that I am developing with our team. It already works great!!!

I am btw. currently hired to port a medium sized Flash game to OpenFL/Haxe.
Somebody seems to have faith in my porting abilities. 🙂

FYI: This whole process took about 4 months!!!

5. Links