Hi! Currently making an MMORPG in C# using IOCP, and I just popped in (haven't been here for a while - noone probably remembers me) because I'm wondering if anyone has any good design theories (tested or untested) for multithreaded IOCP code. Thanks in advance!

From googling I gather that IOCP is an I/O Completion Port, but I'm not sure what it does. Can you (or anyone) give a five-line description? I reckon you're trying to build a client-server architecture, but are we talking about the server or the client here?

Talking about the server. The IO Completion Port is a port that basically allows threads (or threads in a threadpool, normally) to communicate between themselves. By staying connected to the port, they can post messages (in the form of objects - or, in the case of servers, usually sockets) to the port which will automagically get picked up by other threads. My current server IOCP approach is to have one thread in a threadpool listen for connections, and then post sockets to the IOCP when anyone connects. The other threads in the threadpool will then work with the sockets on a per message basis by getting a socket from the IOCP, try to read, and then determine what to do based on a select-case structure. If no data could be read asynchronously, the socket will obviously be posted back to the IOCP again. Any input on this approach? Is it likely to work, or could it (will it?) have bottlenecks?

I don't see why you would want to keep posting sockets to the IOCP and then reading them back again. I would create a single listening thread, and each time it gets a connection it grabs a thread from the threadpool and passes the socket to it, then continues listening. The new (or reused) thread then keeps that socket until it closes, and is responsible for communicating with that client. If the client doesn't send anything, the thread simply waits, and other threads get a chance to run. The client service threads don't really need to communicate, except through the global game state, so I don't see why you would need IOCPs at all in this case.

The reason why it's handy (I.E most effective) to use IOCPs is because otherwise you need a huuge threadpool to serve many players.. at least with your approach anyway. AFAIK, sockets wait on the IOCP until a thread is ready to use them if there aren't available threads at the time of posting. Could anyone confirm this? This is correct, no?

What about synchronization? deadlocks? starvation? Do IOCP calls block the entire application?

---

Programmer at Triumph Studios

Not the last time I tried! Mind though, that I'm using the Managed IOCP library (http://www.codeproject.com/csharp/managediocp.asp), so it's possible that it might handle such things. I personally think that blocking IOCP calls would be a bad idea though, since IOCP is (AFAIK) used very much in high performance server apps, and I don't think that Microsoft would have made it that way.

But what's wrong with having a lot of threads? Most of them will be blocked on IO at any point in time, and the amount of memory overhead for a thread is small. The whole point of threads is that you can have multiple tasks active at the same time, and that if one of them is waiting for something, the others can use the processor to do something useful. In your design, there is the kernel sharing the processor out to a number of threads, and each of those threads is doing multiple tasks. You'd need to write your own scheduler to make sure that all the sockets get a thread regularly, and all that work has been done in the OS already. I imagine sending a message to an IOCP is just a method call on the IOCP object? How about receiving messages though, do you poll the IOCP, or do you call a blocking "wait for messages" method?

What's wrong about having alot of threads based upon my research, is not the memory overhead, which is small, but the CPU overhead. Especially in performance critical applications like the one I'm making, that is critical. Some sources even seems to suggest that one shouldn't spawn more threads than one has CPUs, plus maybe one more. To get an object from an IOCP, you call a blocking method. I like your idea about writing a scheduler, I'll look into it. However, I'm not sure if it's going to be neccessary. Because once a thread is done with a socket, it's passed back to the IOCP and the threads listens to the IOCP once more. And AFAIK, it will always get an object so long as there are objects (in this case: sockets) available. And with maybe 10, 000 sockets and 5 - 10 threads, there will ALWAYS be sockets available. Also, I'm not sure what you mean about "each of those threads doing multiple tasks"? In my design, each thread will only be doing one thing at a time; Either listening to the IOCP, receiving data or sending data. Again, thanks for ideas! Any more input?

Yes, but what data? Is your protocol entirely stateless? If not, then where do you store the per-connection data? As for large amounts of threads being inefficient, AFAIK Apache2 on Windows uses a thread per connection and the default number of threads is 64. It probably won't handle 10000 simultaneous connections on a single box, but I somewhat doubt that that's achievable anyway. The threads are blocked waiting on I/O most of the time anyway (if your server has 1000 connections and is fast enough to handle the load, then each thread must be idle 99.9% of the time, on a single processor), and the kernel will only context switch to threads that actually have some data available to them. In your design, the CPU is continuously polling for data from sockets that probably have no data anyway, which is rather pointless, and continuously context switching between a whole bunch of active threads. I'm not an expert on server programming, or on Windows programming, but I'd still put my money on a one-thread-per-connection design with each thread using blocking I/O and spending most of its time blocked.