Life of a programmer: particle systems

Life of a programmer: particle systems

Hi Everybody ! I haven't written a dev journal for a good while (oh my!), but I think it's a good opportunity to do a small review of our current status from an engine/technical standpoint, especially as we're getting closer to our Kickstarter launch.

In the past months, we've been (re)working a lot of engine modules. It's not unusual, and as you may know (or more likely, not know), the life of an engine programmer is similar to Sisyphus: always rewriting, always improving entire blocks of code. In a way, it's a never-ending process. For example, this month I've been working on our particle systems.

"But, but, but... Flavien, you already had particles in the Infinity Combat Prototype ( ICP ) back in 2007 ! Why do you need to reimplement particles now ?". Ah, see, I've finally started to learn mind reading ! Here's the thing: yes, we had an old particle system, but it was simplistic, lacking a tons of functionalities we need now, and wasn't flexible. In fact, most of it was hard-coded. Meaning that all parameters worked in a very strict, specific manner and that you couldn't modify it without touching code. Take the size of a particle: it was a simply linear interpolation between a minimum and maximum size based on the lifetime of the particle. Want a non linear interpolation ? Screwed. Want some randomness in particle sizes ? Screwed. Want the size to be a function of some other parameters ? Screwed.

So, yeah, this is just an example, but it basically explains why we've spent so much time reworking entire engine systems. Our new particle system now does a lot of things we weren't able to do previously, including:

• lit / normal mapped particles
• physics-based particles
• lookup functions for particle parameters
• various particle alignment modes
• custom particle material shaders
• attaching lights to particles
• attaching particles to other scene graph entities ( like a ship )
• instancing particle systems or emitters

Lit particles: each sphere is in fact a single quad billboard, affected by lights

The new particle system design is, in addition to all these new functionalities, immensively more powerful. I've introduced the notion of particle behaviors, where each behavior affects how a particle behaves or how some of its parameters change over time. Typical examples include:

• newtonian physics behavior ( positions move based on velocities, velocities change by applying accelerations )
• spawn behaviors ( spawn in a box or sphere shape, at a certain rate per second etc.. )
• physics forces ( attraction to a point, collision with a plane/sphere, friction.. )
• texture coordinates behaviors ( for animating particle sub textures, like packing sprites in an explosion effect )
• color / size behaviors

It is possible to create / design a particle system once and instanciate it at different locations/orientations, and even override the behaviors/parameters. At the moment we're lacking a proper particle editor, which means that all particle systems are loaded/saved into XML files.

Is my work on particles systems finished ? Hell no. There are still a lot more behaviors that I could implement ( but we might not need them for our Kickstarter video, so it's kindda low priority at the moment ). And there's no doubt in my mind that this design might have to be partially rewritten in a couple years. When we programmers write code, we try to balance immediate needs with future flexibility, but there's only so much we can anticipate. For example, our current particle systems work entirely on the CPU. And aren't multithreaded. "But, but, but.. Flavien, how do you dare writing non-optimized code ?", I hear you ask. Well it's not that unoptimized. Every time I write a game system, I profile / benchmark it, so I can evaluate which areas need immediate optimizations and which areas would need future optimizations ( because optimizing from the go would take too much time ). In the case of particle systems our code can already handle tens of thousands of particles per frame ( at > 100 fps ) on the CPU alone. In addition, the part that we can multithread ( applying the particle behaviors ) isn't currently a bottleneck. Writing the particles data to GPU buffers is. And that cannot be easily multithreaded. Which means that even if I spent a week optimizing our current code it wouldn't go faster.

That doesn't mean particle systems can't be optimized in the future, though. As I just said it is possible to multithread the particle behaviors, which will be marginally faster. Of course marginally faster is better than not going faster at all, so it's always a gain. It's just not worth spending the time at the moment when there's still a lot of polishing / bug fixing to do. Another future optimization would be to add support for GPU particles; basically utilizing the GPU to simulate and render the particles. Unfortunately, doing this would require a tremendous amount of work and would only be limited to specific particle systems/behaviors, those that can be uplifted to the GPU. There's also the fact that we're working at spatial / galactic scales ( unlike, say, Unreal Engine 4 or Unity ) and that particle systems are supposed to render fine even at millions of kilometers away from the scene's origin. And trust me: that is not a problem to be under-estimated.

Meanwhile, Kimmo has been working on new post-processing effects. A couple weeks ago he finished lens-flares ( not via sprites though, but actual post-processing flares. Meaning that even bright emissive textures or specular highlights can generate flares ). He's now working on motion blur, which already looks pretty good, and improved anti-aliasing. He's also experimenting with temporal anti-aliasing, which is frankly one of our biggest concerns image-quality wise. It's more or less in the research phase so there's no guarantee that we we'll be able to fix all AA problems at this stage, but we still want to try because sometimes it looks really bad.

On the art side, most of the assets for our Kickstarter video are in the finalization stage. This means that the assets are modeled, UVed and textured but we're reviewing some textures, tuning the physical-based materials ( poor Andre has it rough, he's redone some textures multiple times now. I think he should write a "Life of an Artist" blog one day, heh ). Gene has been doing a lot of lighting work ( turns out that adding 3000 lights to an asset in 3ds max... takes some time.. ). More has to be done like attaching sockets to ships ( for hardpoints or thruster effects ). Jan is still busy working on curves for scene entities, cameras and particle effects. Others are working on finalizing other misc. assets. Technically, on the programming side we're seeing the light at the end of the tunnel too. What's mostly remaining are a couple of additional effects ( like refraction, for the cockpit glass effect ) and a lot of polishing and bug fixing. I want to emphasize that a bit: we need to adjust a ton of parameters left & right, and there are still a lot of bugs remaining; some of them are critical ( crashes ), some are related to visual quality ( like popping ), and some are minor but still need tuning ( like the intensity of some material colors ). We're still at least a month away from being able to record the final KS video and after that we still have a lot to do related to the new website content for Battlescape, the interviews, the KS campaign content/stretch goals, the marketing, etc..

A lot of people have been wondering how much we worked on game design for Battlescape. The answer is: a bit but it's not finalized yet. We've already had multiple meetings about it and produced a pretty long game design doc early in the process ( more than a year ago ), but since then this doc has become messy and needs major updating. For most of it the team's vision / concepts of Battlescape are pretty clear for everybody but a lot of the details are fuzzy. We don't necessarily want everything to be set in stone before the KS though; I personally believe that game design should be a very dynamic process, constantly re-iterating, experimenting with concepts and seeing what works or not. For this reason we've tackled all the concepts from a top-down approach: we've defined everything we want to have in the game in general terms, but we don't know the details yet. For example we've defined all the categories of weapons we want; and we defined a first list of weapons that we think would be interesting to implement. But we haven't defined the relative stats / power of each weapons compared to each others and we'll definitely add/remove weapons during testing. And we'll most likely delay all these details and balance decisions until after we have a first prototype done and ready to go in the hands of the players.


In my next blog I'll be speaking about Elite: Dangerous. I don't think it'll take too long to release ( maybe tomorrow ? ). As a developer it's kind of a sensitive topic since any criticism I might have on the game will necessarily be seen as arrogance / an attack on Elite's devs; but on the other hand there's been a lot of new space-based games in the past years and E:D is the first to release, so I think a lot of people might be interested in my views about that game, especially in relation to my vision of an "Infinity MMO".