MMORPG server with indy

[andrew2110]

Hi. At the moment the server for my MMORPG accepts connections with a TIdTcpServer component and then holds a thread for each connected player. I've done a little more reading around the subject and have found that:

A) I shouldnt use a thread per player, with many players in the same area affecting the same area, extra threads would cause blocking on locks. The solution to this according to GameDev is to:

using select() and reading out what's there, then processing it, is recommended.

But the problem I have with this is that when I dont keep a record of the active clients in threads, I wouldnt be able to send new data back to a different client when new information comes in..

B) The second problem is that I should assign a different port number for each connection. I dont know if the TIdTCPServer does this automatically or not but I assume it doesnt and I couldnt find away to do this. If it does do this automatically, is there a way to generate all necessary sockets at the start of the program as I've heard generating sockets on the fly is quite time consuming...

Can anyone think of any possible solutions to either of these to issues?

[Setharian]

A) well all depends on the number of players you want the server to support....I'm not familiar with the internal workings of the Indy component pack, but I guess they are using blocking IO (that's why a thread per connection)....and in MMORPG there are a lot of receives/sends, so the threads will be active most of the time making context switching consume quite a lot of CPU power...even select() is not enough when dealing with hundreds of connections....if you plan the server to work only on the Windows platform, I advise you to use I/O Completion Ports....with only a handful of threads you may handle thousands of connections.....

B) not sure what you read here....the client must know the port he should connect to, so a different port cannot be assigned to each client...maybe they ment the so-called local port that is connection-specific and gets assigned automaticly....

when it comes to socket-reuse, it is doable but only when using Overlapped I/O (including I/O Completion Ports)....the idea behind the concept is that creating sockets is an expensive operation....you may create many sockets ahead in a pool and then use those....this works only under Windows as I know (maybe I'm wrong) by using the AcceptEx/DisconnectEx/TransmitFile/TransmitPackets calls....these are not ordinary functions and are not exported by ws2_32.dll....the function pointers must be obtained at runtime using WSAIoctl call....

const
  { Extension function pointers' GUIDs }
  WSAID_ACCEPTEX: TGUID = '{B5367DF1-CBAC-11CF-95CA-00805F48A192}';
  WSAID_CONNECTEX: TGUID = '{25A207B9-DDF3-4660-8EE9-76E58C74063E}';
  WSAID_DISCONNECTEX: TGUID = '{7FDA2E11-8630-436F-A031-F536A6EEC157}';
  WSAID_GETACCEPTEXSOCKADDRS: TGUID = '{B5367DF2-CBAC-11CF-95CA-00805F48A192}';
  WSAID_TRANSMITFILE: TGUID = '{B5367DF0-CBAC-11CF-95CA-00805F48A192}';
  WSAID_TRANSMITPACKETS: TGUID = '{D9689DA0-1F90-11D3-9971-00C04F68C876}';
  WSAID_RECVMSG: TGUID = '{F689D7C8-6F1F-436B-8A53-E54FE351C322}';

type
  LPACCEPTEX = function(sListenSocket: TSocket; sAcceptSocket: TSocket; lpOutputBuffer: Pointer;
    dwReceiveDataLength: DWORD; dwLocalAddressLength: DWORD; dwRemoteAddressLength: DWORD;
    lpdwBytesReceived: PDWORD; lpOverlapped: POverlapped): BOOL; stdcall;
  TAcceptEx = LPACCEPTEX;

  LPDISCONNECTEX = function(hSocket: TSocket; lpOverlapped: POverlapped; dwFlags: DWORD;
    reserved: DWORD): BOOL; stdcall;
  TDisconnectEx = LPDISCONNECTEX;

  LPCONNECTEX = function(s: TSocket; const name: TSockAddr; namelen: Integer;
    lpSendBuffer: Pointer; dwSendDataLength: DWORD; lpdwBytesSent: PDWORD;
    lpOverlapped: POverlapped): BOOL; stdcall;
  TConnectEx = LPCONNECTEX;

  LPGETACCEPTEXSOCKADDRS = procedure(lpOutputBuffer: Pointer; dwReceiveDataLength: DWORD;
    dwLocalAddressLength : DWORD; dwRemoteAddressLength: DWORD; var LocalSockaddr: PSockAddr;
    LocalSockaddrLength: PDWORD; var RemoteSockaddr: PSockAddr; RemoteSockaddrLength: PDWORD); stdcall;
  TGetAcceptExSockAddrs = LPGETACCEPTEXSOCKADDRS;

  LPTRANSMIT_FILE_BUFFERS = ^TRANSMIT_FILE_BUFFERS;
  TRANSMIT_FILE_BUFFERS = record
    Head: Pointer;
    HeadLength: DWORD;
    Tail: Pointer;
    TailLength: DWORD;
  end;
  TTransmitFileBuffers = TRANSMIT_FILE_BUFFERS;
  PTransmitFileBuffers = LPTRANSMIT_FILE_BUFFERS;

  LPTRANSMITFILE = function(hSocket: TSocket; hFile: THandle; nNumberOfBytesToWrite: DWORD;
    nNumberOfBytesPerSend: DWORD; lpOverlapped: POverlapped; lpTransmitBuffers: PTransmitFileBuffers;
    dwFlags: DWORD): BOOL; stdcall;
  TTransmitFile = LPTRANSMITFILE;

  LPTRANSMIT_PACKETS_ELEMENT = ^TRANSMIT_PACKETS_ELEMENT;
  TRANSMIT_PACKETS_ELEMENT = record
    dwElFlags: DWORD;
    cLength: DWORD;
    case Boolean of
      False: (
        nFileOffset: LARGE_INTEGER;
        hFile: THandle;
      );
      True: (
        pBuffer: Pointer;
      );
  end;
  TTransmitPacketsElement = TRANSMIT_PACKETS_ELEMENT;
  PTransmitPacketsElement = LPTRANSMIT_PACKETS_ELEMENT;

  LPTRANSMITPACKETS = function(hSocket: TSocket; lpPacketArray: PTransmitPacketsElement;
    nElementCount: DWORD; nSendSize: DWORD; lpOverlapped: POverlapped; dwFlags: DWORD): BOOL; stdcall;
  TTransmitPackets = LPTRANSMITPACKETS;

  LPWSAMSG = ^WSAMSG;
  WSAMSG = record
    name: PSockAddr;
    namelen: DWORD;
    lpBuffers: LPWSABUF;
    dwBufferCount: DWORD;
    Control: WSABUF;
    dwFlags: DWORD;
  end;
  PWSAMsg = LPWSAMSG;
  TWSAMsg = WSAMSG;

  LPWSARECVMSG = function(s: TSocket; lpMsg: PWSAMsg; lpNumberOfBytesRecvd: PDWORD;
    lpOverlapped: LPWSAOVERLAPPED; lpCompletionRoutine: LPWSAOVERLAPPED_COMPLETION_ROUTINE): Integer; stdcall;
  TWSARecvMsg = LPWSARECVMSG;

function WSAGetExtensionFunctionPointer(s: TSocket; const gdExFuncGuid: TGUID): Pointer;
var
  lwOut: Longword;
begin
  Result := nil;
  WSAIoctl(s, SIO_GET_EXTENSION_FUNCTION_POINTER, @gdExFuncGuid, SizeOf(gdExFuncGuid),
    @Result, SizeOf(Result), @lwOut, nil, nil);
end;

the GUIDs used to get the function pointers are not declared in the Winsock unit shipped with Delphi nor in JEDI WinApi Header Translation units....so I posted them here along with the function prototypes and a small function which obtains one of those function at runtime (requires a valid socket descriptor)....the AcceptEx function accepts a new connection after you must call GetAcceptExSockAddrs to get the local/remote addresses....DisconnectEx with TF_REUSE_SOCKET disconnect a client so the socket can be used again with the AcceptEx call...note that DisconnectEx is available only under WindowsXP and higher - to support earlier Windows versions, use TransmitFile with all params 0/nil and the TF_REUSE_SOCKET flag specified....

I guess I got carried away a bit ...hope this helps and answers your questions....

[andrew2110]

Wow, thats great, Thanks!

I did a little further reading and stumbled across I/O Completion ports but couldnt find any good Delphi examples until now! Will have a play around with this and see how it goes!

[andrew2110]

I've now learnt that its possible to create a IOCP server with indy 10. After upgrading to indy 10.1.5 and installing the 'Supercore' package. However after installing the supercore there are still no components available that would support the creation of an IOCP server. Has anybody had experience with the supercore in indy?

Any help would be much appreciated

[jdarling]

IOCP isn't fully integrated into Indy yet. We are working on it, and hope to have it done some time before the .NET version of Indy completes. Of course right now things have been slow moving on the group. Many of the Core members (myself included) just don't have alot of time to work on Indy any more .

Actually the last few messages I've seen going around were promising that ground is being moved, I just don't know how fast. Hopefully one of these days I will have enough open time to get back into active development along with the others.

I did attempt to create an IOCP demo in D6 w/o supercore, but never completed it. The real problem with IOCP and Indy is the fact that Indy attempts to be cross platform. Running in Lazarus, Kylix, Delphi, and Delphi.NET.

Thats enough out of me for now, I'll look through my hard drive and see if I can find the IOCP demo for Indy 10.1.5, if I do I'll post it up for you to take a look at if you need it.

[andrew2110]

ah thanks for the update! I'm now toying with the idea of making a sort of P2P MMORPG, after reading up about IOCP and deciding im a bit too thick to work out how to do it within the next week or 2!