Since December 2008 I've been working on exodus, my latest project. It is an attempt to create an open-ended RTS\RPG which will feature a 'real' dynamic universe.
It will be a mix between civilization and freelancer. You will play as a character with the possibility to affect the strategic balance in the game, without scripted action (unlike freelancer).

There will be several planets with resources, with nations who'll exploit resources. Several types of resources exist (raw materials, consumers, shelter ec.). From raw materials other resources can be created (for example, a house can be created from wood and stone, which are both raw materials, while a house is shelter). Resources are produced and consumed by the population. With advancing technology, resources may be produced more efficiently, or from other materials (irrigation, masonry, etc.).

At this moment, the population model and resource model needed to 'run' a planet are nearly finished (== it produces the output I want).

So how does it work!
A planet is an object defined in an XML file, with regions, which can in turn contain resource locations. Nations must claim regions to be able to use their resources.
The resource model is defined in XML. It defines the available resources, of what type they are, how they 'grow' (a forest behaves different from an iron mine), and how population reacts to them. If I want my population to eat rocks, it's a matter of configuration. Very flexible.

The game engine is distributed (implemented!), which means that I can start up as many planets simulators as I want, anywhere I want! Each planet will run as a seperate process, making optimal usage of multicore systems. The simulation data is written in a distributed database (realtime low-latency opensource DDS middleware). Exodus applications will subscribe and publish data in the database. This inherently enables LAN multiplayer, where each player could host it's own set of planets, instead of having one host.

Haven't really started on the graphics yet, chances are big that I'll use XNA though. It seems a great deal more efficiently to me than using the DirectX API from what I've seen.

For those of you interested, I will post updates here from time to time ^^

---

Hapiness isn't hapiness without a violin playing goat

Last weeks I've had some time to work on the infrastructure layer, so that will be the topic for now.

At this moment I have designed three services which will do most of the work. These applications are:
1. The planet simulation
2. The user interface (commandline for now)
3. A transaction manager (not implemented)

The planet simulation will do the basic game logic, process the populations needs for each nation, calculate resource amounts etc. It will calculate the data one day at a time.

The user interface will display the data from the planet simulation, as well as processing user input and publishing this in the distributed database.

The transaction manager will process the user input and make mutations on the distributed data, which will be kept in sync by the planet simulation. An example of user input could be conquering a region from a nation, which could trigger a transfer_region (nation1, nation2) transaction.

Now these processes need to be kept in sync. You can imagine that you don't want to update the user interface with new data when the planet simulation is in the middle of writing data to the database, as this could lead to corrupt or incorrect data. So I figured I need some kind of scheduling mechanism.

I introduced a new application that publishes a clock signal (with adjustable accuracy in msec.). The clock's max. signal is 1 second, after which it will reset to zero. For example, a clock with an accuracy of 200msec will loop the following values:
(0, 200, 400, 600, 800, 0, ...)

Services will update or sync with the distributed database following a schedule that is published by the clock application. This schedule specifies for each service for which clock "ticks" it should trigger. The schedule also specifies for how long the application is allowed to update\synchronize (timeslice).

A schedule for the three applications described above could look like this:

Tick Service Slice
0 planet_earth 0.1
0.2 gui 0.2
0.4 t_manager 0.2
0.6 gui 0.2
0.8 t_manager 0.2

Now the planet simulation for planet earth will be updated once a second, while the gui and transaction manager will be updated twice in the same period.

The schedules for the services are specified in an xml file, which is monitored by the clock application. When this file changes, the schedule is updated (and through infra-signals the services are notified that they should update their schedule). Without a schedule or a clock signal, the services won't run.

Note that services are able to run simultaniously (that's kinda the whole point of having more than one process), but the moments that they need to interact with the distributed data must be synchronized.

So much for the infrastructure for now, there's a lot more, but I'm running out of space =)

Next time something about the planet simulation.

---

Hapiness isn't hapiness without a violin playing goat

Very interesting. How would you handle two segments of the game engine 'branching' by being completely disconnected from each other while still running, with users? When the connection is re-established, something would need to be done to merge the branches, right?

A possible solution:

Every system has a seperate clock running, with schedules for the local processes. These clocks have to be kept in sync, so you can detect when one system (or a group of systems) is disconnected.
When it does, backup the disconnected processes through cold\hot standby to prevent the data from getting behind on schedule.

Offcourse, this won't solve the branching problem yet, as both systems will start backing up processes. The next step is to determine which system is "right".

The easiest way to do this is assign a "master" clock process. The segment that contains this process would be leading. You'd have a scalability problem however when your system contains many processes distributed over many machines. With this solution, when only the master process is disconnected, the system which runs it would start backing up every other running process. I've seen systems where this strategy would lead to absolute disaster.

A more intelligent strategy would be to determine which segment is the largest, and dynamically assigning that one as the master. Or determine which segment has more processing power. Or which has more processing power available. Or which segment is, based on historical data, most likely to have most processing power available..

These solutions can become complex, and require impressive amounts of framework code, which would be a project on itself. Although it's a real problem, I think it is beyond the scope of this project (for now).

Besides, at this moment I'm doing the game-logic, graphics and UI of a tool for the game, which is new for me and infinitely more fun =)

---

Hapiness isn't hapiness without a violin playing goat

I was just wondering if you had an interesting solution already in place, since it's a hard problem.

I don't really like the idea of a master clock, since that's in conflict with the system being distributed. Your other idea is pretty nice, since it'll work in general.

A third possibility is to keep disconnected segments of the same game disjoint ingame. If two network segments become disconnected you could disallow the gameplay in each segment to try to affect something in the other segment (e.g. the interstellar trade lane is out of order ). Then when the segments are connected again you can merge the game state without any conflicts. Obviously this can only work for a specific kind of gameplay, but if each process is its own planet I can see how it might work. (Define network locality isomorphic to gameplay star system locality)

Although surely more elegant than re-allocating processes on other machines, it has some disadvantages. Take for example the more "casual gamer", who's planets would advance a lot slower compared to it's die-hard counterparts.
Another problem would be concepts like trade routes (if the game will have any). Suppose planet A depends on resources from planet B, but then B goes offline. Trade route's gone, planet A will be very unhappy (although there are solutions to this problem, like stubbing the planet with it's last known parameters, which I think is somewhat artificial).

On the other hand you could consider leaving the planet's simulation to itself for too long undesirable for it might spin out of control (growing population, resource shortages etc.). So if planets would still be "alive" while the player is not, you'd want to have some kind of auto-management system that could handle basic tasks.

Which approach is best will eventually depend on the actual gameplay,for which the future is still uncertain.. :-)

Off-topic (but not quite): is attaching pictures to posts not allowed anymore?? =(

---

Hapiness isn't hapiness without a violin playing goat

whatsInAname8:

Which approach is best will eventually depend on the actual gameplay,for which the future is still uncertain.. :-)

Indeed.

whatsInAname8:

Off-topic (but not quite): is attaching pictures to posts not allowed anymore?? =(

I can't either. I suppose there's always imageshack.

The last couple of weeks I dived into the Qt graphical framework, and started working on some tooling for exodus. I've posted a picture (=D) of the progress so far.

You can see the exodus explorer, from which different views can be started (nation, region and resource location so far). From the explorer services can be started and monitored.

Below the explorer, the clock service and planet simulation is running. On the upper-right side a nation monitor is running, and below it a region monitor.

These three processes share distributed data using DDS middleware.

Some more information about the population model later (but the laptop is seriously overheating now =)

---

Hapiness isn't hapiness without a violin playing goat

The project started as a "simple" population model experiment. Simple is not what I'd use to describe it now though. When you're creating a simulation, complexity appears to be a lineair function of the time spent on it.

At this moment, every new feature or small change is a balancing act to keep the model in control. Scope is the only constraint on the complexity.

So how does it work. Basically the simulation can be split up in three parts. First part is the simulation of the planet itself. Growing resources is the focus here. The second part is concerned with the resource production. After an evaluation of the population's needs, resources are collected or produced.
The last part processes the civilization. This is where the population's growth is calculated, along with it's parameters like health, resource security, resource safety etc. Most of the complex code is located here. Since there's quite a lot to say about these steps, I'll split up in three posts. Starting with:

Step 1: Growing resources
Resources grow in resource locations. A region on a planet can host multiple resource locations. A nation can control a number of regions. Not all resources are "grown". A house doesn't grow, neither does a fruitcake or an airplane. But a house is build with stone and wood (for example), and these can be "harvested".

There is a number of attributes which controls the way resources grow. For example, is a resource is harvested or constructed, or how many workers are required to harvest or construct it. For some resources the population's daily needs can be configured (food). For other resources, the need is determined by the nation's economic, social and political situation (tanks, transport vehicles etc.). The resource type determines which attributes are mandatory for a resource, and how the resource model will react to it.

An important factor in harvesting resources is soil fertility. Fertility influences the total amount of resource units one can harvest from the land. Fertility decreases over time when the land is intensively cultivated, and will restore over time when resources are no longer harvested.

Some resources may automatically claim the unclaimed surrounding land. An example of such a resource would be wood (forests).

Research can unlock resource upgrades. Upgrades improve specific attributes for a resource. For example, resources could be harvested more efficiently, or the harvesting consumes less soil fertility (f.e. crop rotation). Sometimes, an upgrade means that the resource is created using different materials. A wooden house could upgrade into a stone house for example, or a combination of both. The upgraded house could then in turn provide more protection.

The image shows an example of how resources can be configurated.



Hope you didn't lost me here! Next time: what does it take to actually produce resources =)

---

Hapiness isn't hapiness without a violin playing goat

Looks like a fun project.

---

[Serdan dot EU][Emotiv]

00din, It certainly is!

Graphics!
Before I'll get to resource production, I'd like to mention that work has started on the 3D graphics engine! Not in XNA, as I stated earlier, but in OpenGL, since support for it in Qt is great. And it'd be nice too if all of exodus will be portable to other platforms! Expect some screens soon

Okay, on to the next subject:

Step 2: Resource production
I explained how resources are "grown". However, before resources can actually be used, they need to be harvested, and just like in real life, harvesting requires a workforce. So what is this workforce exactly, and how can you get it to do something? (Note that where I use the word "harvesting", I could also write "produce" since not all resources are harvested in the traditional meaning of the word)

First of all, there is no such thing as "a worker". As opposed to games like Age of Empires you do not create a worker, unit or any of the like. Instead, you have a whole society which can contain millions of people! They get born, drink and get drunk, eat, educate themselves, give birth, work, age, die and try to have an overall time in between. Now before you'd send them all into battle, think twice.

You see, real people have real needs. And although this holds not always true for their virtual counterparts, it certainly does for the exodus-kind. Some needs are critical, others are of less concern, but may add to the comfort. Some of these needs can be satisfied through, yes, resources.

To get to these resources, people are needed to do the harvesting. Firstly, the society's need for a particular resource is evaluated. Based on this need, an exact amount of that resource is harvested, provided that both enough resources and workers are available. A population is actually quite capable at evaluating and satisfying it's own needs. So it will harvest resources, nicely display exponential growth and be prosperous overall..

Until the moment arrives (and arrive it will) that a critical resource gets exhausted. At that point disaster breaks out, as the population is no longer capable of meeting it's own demands. Shortages will rise, diseases break out, health declines and eventually the population's growth will come to a halt (and maybe even a decline). At such a moment, the need for something less substantial like healthcare or an army may rise.

So that's for the needs of your population, but how about you? It turns out that the population does not have a monopoly on demands. External factors can influence the demands of a population. Take for example trading. When two nations are trading, this influences the demandpatterns for both nations. Or when your ever friendly neighbour has massed a huge army and you'r still using sticks and stones, it might be worth it to invest in something other than decorative flowers..

That's it for resources and demand, next time some more insiders details about the exodus demographics engine!

---

Hapiness isn't hapiness without a violin playing goat

I've had some opportunity to work on the distributed 3D engine, and I'm about to show some of the horsepower that distributed computing has to offer!

Seperate GUI from processing
Firstly, the main design concern for the engine is to seperate the game logic as much as possible from the GUI code. This keeps code for the engine clean, and the result will be quick, responsive code. The game logic is processed by other applications which can be scheduled on different\other cores (we'll have enough of these in the near future anyway).

With this thought kept in mind, I layed down the foundations for a distributed engine, where the sole goal of the application running the 3D GUI is to display the distributed data. That's right, a 3D engine completely depending on runtime distributed data, telling it what to draw and how to draw it.

Implementation details
At the technical level, every 3D object inherits from a low-level object class. Objects of this class are created with a LOCAL or DISTRIBUTED flag, which indicates wether the object should be distributed or not. When the object is distributed, it will update it's position, rotation and scale automatically at each render-pass in the distributed data. On top of that, classes that inherit from this class can add additional metadata (for example, a light-object could add extra data for the light intensity, color etc.).

Test setup
For testing purposes, I created an application that has a "server" mode, which will distribute it's objects, and a "viewer" mode which will read all available objects and draw them to the screen with all graphic settings set to max (although "max" is really only texturing and lighting at the moment).

Note that seperating distributed from local\read objects is not enforced, you can perfectly have an application that mixes distributed objects with local objects.

Screenshot
Below you see the 3D application in server and viewer mode. You can see that the servers have some graphic options turned off for better performance. Additionally, you see that the viewing application takes all published objects from the server applications. In other words: each new server application adds new objects to a scene. As much as ]b]1500 objects at 50Hz can be synchronized[/b] across the applications, which is quite a sight, and pretty impressive!



The cubes in the scene are constantly bumping off the surface, and rotate faster at higher movement speeds. All of the position and rotation processing for the cubes is located on the server side, so all the viewing side has to do is displaying the objects and applying the right transforms!

Enough technical details for now, no fancy graphics, but still nice results for two weeks of work!

---

Hapiness isn't hapiness without a violin playing goat

I've got some (new!) features to show in this post. A lot has changed, but not all visible, so I'll show you

only the most interesting =)

Content filtering
It is now possible to specify a filter in the viewer application, by which the user can tell the application from which service he\she wants to receive objects from. This allows for multiple scenes to be published at the same time, without interfering each other!

Objects can add metadata
Previously an object was only able to publish it's position, rotation and scale (matrixs transformations) in the distributed database. Now it is also possible for objects to specify additional metadata, which can be any property of the object (like for example specular color for a light, speed-vector for an physics object, state data, render options etc.).

Beam class
This week I added a beam class, which is capable of rendering lightning\plasma\neon like effects, depending on runtime configurable properties. These properties are shared between services as metadata in the distributed database.



Screenshot showing the beam class, and lots and lots of bouncing cubes. The still image doesn't do justice to the actual beams, with movement they look so much better =)


This shows just how flexible a distributed environment is. What gets rendered by the user interface depends solely on what services are running at that moment, and what kind of objects they are publishing. The number of services is virtually unlimited.


The beam class up-close. You see two different configurations for a beam here, the a lightning variant, with lots of turbulence and many segments, and the slower moving plasma-like beam, which has not so much segments, and low turbulence.


Here you see the content filtering at work. Two views, each with their own contentfilter are placed in the background, with on the foreground the services that are publishing the objects, and the contentfilter configuration window. You can see that all active services can be added to a filter, although not all services do actually publish objects. The view service at the top is only showing objects from services with name 3D_1, the view service at the bottom of services with name 3D_2..

So that's it, let me know what you think of it so far!

---

Hapiness isn't hapiness without a violin playing goat