multiplayer (and LNet)

[laggyluk]

Anyone has experience with lnet library and udp?
I just got to implementing multiplayer part of my game project. And that's a first time I'm working with udp ::)
Game architecture is event/message based so I had almost no problem with sending input through network to the 'server' part and controlling the actor. Which feels pretty good after going through all that event hassle ;p
Now, every frame input state is sent to server. I'm pretty sure I should drop packets that are older than last one received. But is this info encoded somwhere in the packet header or should I add 'frame stamp' my self?
And other thing, can server/sending socket get 'overflown' with send/receive requests?

[SilverWarior]

Anyone has experience with lnet library and udp?

No I'm afraid that I don't have direct expirience with mentioned library.


Now, every frame input state is sent to server. I'm pretty sure I should drop packets that are older than last one received.

If as an input state you mean which keys are pressed and mouse position then you definitly should not drop any packets. Droping packets in such scenario could lead to dropping information about player pressing certain button which would afect actor movment and its actions.
For instance: Player presses Left arrow button three times which is sent to server by three seperate packets and you discard one becouse newer packet has arrived before older was processed. This would result in server registering only two Left arrow key presses instead of three.
But if you are sending current state (actor position, actor actions, etc.) then you probably might get away with dropping of packets as you only require newest game state.

Also if you are using sending input states to server using of UDP might not be best idea. Why? UDP protocol does't have packet deliver confirmation system built in like TCP/ip does. So it is posible that some UDP packets get los or damaged in transit. This could lead to unexpected behavior.
So for sending input states it would be better to use TCP/IP becouse the built in delivery confirmation system will make sure that the packet is delivered intact (the packet will be automatically resent if necessary) and finally it will raise error message if the packet can't be sucsessfully delivered. And yes becouse of this delivery confrimation system TCP/IP does require more network bandwich.

I have seen a few games which actually use combination of both TCP/IP and UDP at the same time. Most data is sent through UDP to lower network bandwich. But at regular intervals whose purpose is for clinet/server synchronization they use TCP/IP becouse with it they can verify that data reached all clients sucsessfully.

[Super Vegeta]

What kind of connection to use and how to send your data depends quite much on what kind of game are you writing.

TCP is a nice protocol, because it makes sure that no data is lost, and that it arrives in order. API-wise, it is quite often represented simply as a stream of bytes, with all the packet-related hassle hidden from the programmer. TCP is connection-based: hosts must first establish a connection, and then they exchange data over it. Should the connection be lost, they will try to reconnect, which may take some time. Also, checking whether the packets are all in order also introduces some overhead.

UDP, on the other hand, is connectionless - you just hurl packets out into the net and hope they will reach their destination. There is no guarantee packets won't be lost nor will they arrive in order - so yes, you must include your own frameno/timestamp data somewhere in the packet. But because UDP is much simpler than TCP, it doesn't have that much overhead and is usually much faster.

So, regarding the type of game you're writing:

With turn-based games, you could even go for TCP - most of the time the players will be waiting for their turn, and unless you incur some kind of turn length limit, a loss of few seconds because of packets resend/sort or reconnecting won't really matter.
With some kind of RTS, a good choice is going for UDP and simply postponing executing player actions. When a player issues an order to the unit, you keep it in memory and postpone its execution for X game cycles. Unless latency is bad, the other players should receive this info before the scheduled game cycle, so everything will be perfectly synchronised. You can check packets dropped simply by requesting a delivery report on each packet and resending it if no report arrives in some amount of time.
With some fast-paced games like shooters, things start to get tricky. If you plan only on local play, then yeah, you could go with sending input states every frame and only advancing the game once all players send their data. But on the web, you have latency, packets dropping and all that stuff. So it's simpler to just send state changes to the server and other players, and simply try to calculate as much as possible client-side. This article (https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking) explains some quirks related to networking in fast-paced environments.


Obviously I'm no expert, since I haven't written much network code, but well, that's what I know. Hope you'll find my answer useful.

[laggyluk]

It's gonna be fast paced shooter (with Mass Efect 3 multiplayer mode as huge inspirration ;) so tcp is not an option, maybe useful for chat or sth but not for gameplay. I started with sending input because it was a simplest step but as SilverWarrior mentioned it's not gonna do because dropping some packets will break the gameplay. I guess I'll have to pack world state in a packet like that steam article suggests.

[phibermon]

Your networking model sounds similar to mine - synchronization of actor properties using thread safe messages by using the network connection as a message relay. I'm currently using both TCP and UDP for communication. Individual properties are marked as either 'critical' or not along with a priority. critical, low priority messages through TCP (game state control, player chat etc) non-critical, high priority messages through UDP (rotation/heading without projectiles) and a blend between them for all other things. You can do packet confirmation on critical messages sent thru UDP for both speed and reliabilty but not bother with confirmation for things that don't really matter or would otherwise quickly get superseded.

For example for the position of a player, you can send one critical update to the position each second and inbetween those as many non-critical updates as the bandwidth allows for. So you could theoretically only get the critical update each second. Then you can use prediction combined with lerping like Quake3 net code etc to make the motion seem smooth for other players.

The nice thing about this is that marking a packet as non-critical / low-priority allows you to dump it or move it down the queue when a new critical/high-priority message comes in for sending. It's a total win to implement as theoretically your stream of messages could suffer such disruption anyway over UDP, all you're doing is juggling to squeeze out the real-time performance factor.

accurate timing sync + timestamps are critical for the setup, I recommend that you make timestamps an inherent part of your message structure. I connect my server messenger to my client messenger internally without networking and introduce artificial delays to the pumping of the queues and random message drops in order to simulate something approaching an unreliable internet connection. If you do this then your net code is no longer important, you can do the hard part of dealing with sync/delay in your own ecosystem and when it's done, just stream your messages through whatever TCP/UDP you fancy and job done.

To put simply, the networking code in this context becomes a message relay with a fast,unreliable path and a slower,reliable path. You could just use UDP and have a system for re-transmitting lost packets, and re-ordering them and while this can be faster than TCP in some scenarios, that is the fundamental difference between the two, TCP is basically UDP with packet re-transmission and re-ordering. There's no performance hit at all to using both TCP/UDP and the TCP path means any UDP packet reliabilty code can be non-existant or just very simple.

[laggyluk]

thanks for some ideas :D working on this feels refreshing

[phibermon]

You're welcome, I've given a lot of thought to this messaging model of net code, always happy to share ideas.

I actually have nearly every part of my engine as a messenger along with clientID's and routing. you can do bizzare things like run your sound manager on a different computer and have it recieve the sound messages from local or network requestors. My GUI is the same, mousemouve/onclick/hover etc messages. I can actually do something similar to X and have a window/controls on one computer sending all the messages back to the actual app instance on another computer.

The messenger/actor model is very powerful! well worth any performance penalties!

EDIT : Just in case anybody doesn't know, UDP packets don't always arrive in the same order you sent them and sometimes they don't even arrive BUT if they do arrive, it is garunteed that the data they contain is complete and CRC checked.

[pitfiend]

I guess that the best approach for multiplayer games, is to build the game around an IRC server/client architecture. One thing I was wondering about, is to do something around bittorrent as it may grant an option to build a serverless multiplayer game. Also have to be aware that any multiplayer game will suffer from lag (it's not a secret), no matter how well you code your network layer or the protocol/transport you use. Anyway you may timestamp your packets or set some way to tell priorities or number of packets before any action happens. Just my opinion.

[SilverWarior]

One thing I was wondering about, is to do something around bittorrent as it may grant an option to build a serverless multiplayer game.

"something around bittorent" are you serious? You do know that torrent protocol is just a Per to Peer protocol which alows spliting large files into smaller packets to make data sharing a bit easier.
Using torrent protocol for multiplaye is out of the question as it is optimized for sharing static data between multiple sources. In multiplayer game you have anything but static data.

But yes using of Peer to Peer protocol you can achieve serverless multiplayer. This approach has been used for long time. DirectX was first masievly used framework which alowed that.
But in the end it all depends on your multiplayer inplementation.

[phibermon]

Peer to Peer gaming has never been much of a success outside of a LAN, keeping any game state synchronized across such a model in real-time is difficult enough but when it comes to decision points it's nigh on impossible. Say you had 4 players, player one shoots player two. From players 2 state, player 1 didn't have line of sight for the shot, nor did they from player 3's state, but they did from player 4's.

So did player 1 shoot player 2? well without a server you either have to take player 1's word for it (so you're opening the door to cheaters) or you have to take a vote, if the majority say the shot connected, it did etc

But then you have a very quick action (a shot) requiring a quick decision from all or most of the peer to peer clients, which is only going to complete as fast as the slowest clients ergo slowing the game down for everybody.

Yes lag differences between clients can cause similar issues in some FPS games, but the issue would be *greatly* magnified in a peer to peer topology.

Of course you could just take players 1's word for it, but then they could either cheat, or player two is going to uninstall your game because they keep on getting excessivly killed by players with a lower ping who weren't even in the room (from their perspective) as they died.

A server is more than just a physical hub for game clients, it's an essential 'mutex' in the concurrency problem that is network gaming.

Obviously there are slower paced games / mode of play that might be better suited to a peer to peer model but then at the same time, they'd also be even more suited to a client/server model.

Plus you can get away without UPnP for opening up ports on a players router etc in a server model as you're routing out. The moment you rely on UDP / incoming comnnections, you realistically need UPnP code unless you want your forum filled with "can't connect, help!" messages.

[laggyluk]

'serverless' would be nice but I don't really see how this could work. I'm planning to have a central server for match making and some verification but game will be hosted on player's machine. co-op against bots so hopefully cheaters won't be such a game breaking factor

[SilverWarior]

Yes the biggest problem in Peer to Peer multiplayer is keeping game state synchronized between all clients. So you need active mechanizm which makes sure that game states are synchonized between all clients. This of course means that the slowest computer with worst connection will cause the most lag to all other players.
But the biggest advantage of Peer to Peer multiplayer is that you don't require dedicated server for it (no aditional expesnes for dedicated servers) becouse anyone can host a game at any time.
Another advantage is that using Peer to Peer you can distribute processing between different clients. This in the end alows using of more complex game mechanics.
Unfortunately this introduces huge chances for players to cheat with modified clients.

But if you wanna make a multiplayer game where noone could cheat all I can say is good luck. Why? There is no easy solution to this.
For instance I remember a cheat mod for Far Cry which alowed you to easily shot at other players. All you needed was to have another player in sight and mod automatically calculated needed trajectory for bulet. This means you didn't even have to aim and was still making perfect kills.
When PunkBuster service got implemented they prevented this mod to work until it was updated to work even with PunkBuster protection.

So the only way to prevent players from cheating in multiplayer games is making games so that client simply sends player imputs to server and server simpy returs visual and audio information back to player. There is absolutely no processing made on client. So you need extreemly powerfull servers for this. But a viable solution is actually moving server into cloud to get enough processing power.
And since all processing would be made on server side this would also alow you to play extremly demmanding games even on crappy computer (no need for advanced graphics cards, no need for large amount of RAM, no need for large HDD space as you would require only tiny application on your computer). All it is necessary is for that computer to be able to smotly play any streamed movie with desired resolution.
But for this to work you need excelent network connection with minimal lag.

[pitfiend]

I'm aware that p2p multiplayer games are something hard to code. My idea takes in account that you have nodes/supernodes for each client/server. So, when you came into some specific area where a supernode is already working, game client switchs to it as main server, later all supernodes syncs between them to keep data integrity, or as soon as you move to another supernode or became one. Anyone in that network scenario could became a supernode, as any client could upscale as needed.

On the pros side, you could play solo or with your friends (multiplayer is an option).

On the cons side, you may need humongous disk space for user data validation, once you became a supernode. A house keeping mechanic must be implemented to make this viable. A tracker server may be needed to be able to sync nodes/supernodes.

This is just a project on planning stage. I know there are lots of things/condition to foresee/consider.

[phibermon]

A distributed model will work but you'll hit limits pretty quickly and it'd be an absolute nightmare to debug/optimize. You might want to make a virtual network with artificial delays, either using VM software or by mechanism in your code, it'll be a bit of work to get it set up but well worth it if you plan on making something robust enough for real-world use. Would love to hear about progress so please keep us updated!

[pitfiend]

As far my schedule permits it, I will, unfortunately this is a project I work on spare time.

[laggyluk]

btw, probably I could use p2p to share world data between users in bit torrent manner and limit server bandwidth usage. Anyone done pascal torrenting? :P

[pitfiend]

Torrenting in pascal is the tricky part of my project, as there are not any component ready to use. Some of the available are just wrappers for external libraries with really dark interfaces. There's a torrent library in Ares Galaxy, but still trying to understand how it works, as I'm looking for a way to separate it from the main core. To be honest I found it a complicated task to achieve, working on spare time.

[SilverWarior]

I remember a Pascal Torrent project a few years ago. It was quite stable but unfortunately it didn't support all the features of latest torrent protocol. It had no DHT support, it had no Peer to Peer seds exchange, no encripted transfers.
As far as I know Pascal Torrent didn't depend on external libraries but it was trying to reimplement whole torrent protocol using Sockets.
In the end I belive that orignal authors abandoned this project.

[pitfiend]

I remember a Pascal Torrent project a few years ago. It was quite stable but unfortunately it didn't support all the features of latest torrent protocol. It had no DHT support, it had no Peer to Peer seds exchange, no encripted transfers.
As far as I know Pascal Torrent didn't depend on external libraries but it was trying to reimplement whole torrent protocol using Sockets.
In the end I belive that orignal authors abandoned this project.

The one I found inside Ares Galaxy, seems to have all the torrent protocol features. It's available on sourceforge here (http://aresgalaxy.sourceforge.net/), as you can see it is announced as a free open source bittorrent and chat program. My trouble, I can't found any documentation (or maybe I'm looking in the wrong place).

[phibermon]

You don't have to adhere to the bit-torrent protocol, you can do multi-part sharing of your own accord, split into arbirary parts, get the hash of each part (MD5 etc) and use it to confirm each part as it's transferred. A simple list of confirmed aquired parts (or fully downloaded etc) and you're onto a winner. I think bit-torrent does some funky hashing that allows it to CRC check individial parts from just having the entire file/torrent hash (some funky maths going on there) but you really don't need that. just store a file similar to a .torrent that contains a list of all the parts, their start-end offsets and their CRCs. You can flag the bits you've got in the hashing file so you can resume downloads. A process that calculates the CRC of the parts from the file you have can be used to determine what's good in the instance of a program crash / unclean shutdown.

[pitfiend]

You don't have to adhere to the bit-torrent protocol, you can do multi-part sharing of your own accord, split into arbirary parts, get the hash of each part (MD5 etc) and use it to confirm each part as it's transferred. A simple list of confirmed aquired parts (or fully downloaded etc) and you're onto a winner. I think bit-torrent does some funky hashing that allows it to CRC check individial parts from just having the entire file/torrent hash (some funky maths going on there) but you really don't need that. just store a file similar to a .torrent that contains a list of all the parts, their start-end offsets and their CRCs. You can flag the bits you've got in the hashing file so you can resume downloads. A process that calculates the CRC of the parts from the file you have can be used to determine what's good in the instance of a program crash / unclean shutdown.

Seems logic, but why to invent the wheel if it already exists? One thing I want to use from bittorrent protocol is a new feature that allows to stream data the way you want. Actually it's focused to allow video stream over a p2p network, no one said it's the only use it. Also want to build a chat system over bittorrent or something alike.

[phibermon]

Cool :) yeah the streaming thing isn't anything amazing, it simply prioritizes parts to come as close to a stream as possible. A stream supporting video player connects to utorrent for example and parts are prioritized that are within the requested buffer size of the player. I find it only really works well enough to stream something with lots of fast seeds. However there's all kinds of load balancing implemented by clients to ensure an even distribution of data across the pool of peers, while it's possible to request specific parts (as in streaming) there's clients such as Vuze that will actually block clients for making too many such requests, the streaming ability of utorrent has overall had a negative impact on the health of torrents as all the shared video files (especially brand new popular torrents) have an part availability bias towards the start of the file, damn near negating the usefulness of many peers in the swarm.


I know that clients such as VUZE give you a chat functionality but I don't think that's part of the bit-torrent protocol, probably a custom type of bit-torrent packet marked as 'chat' that can be picked out. It's only going to be the same as a standard UDP chatting program but routed thru the shared bit-torrent port to save opening and routing a second port. There's nothing inheriant about the bit-torrent protocol that lends any advantages to chat as far as I can see and since a chat is normally one on one, a chat program that shares files with bit-torrent wouldn't really get any advantages, it'd work just fine thou, one seed, one peer.

Fingers crossed you can find a pascal implementation of Bit-torrent of sufficient quality to use in your projects :D

Writing your own bit-torrent seems like it would be more work than writing your own bit-torrent like protocol as you've got standards to adhere to and years of anti-leeching rules to adhere to unless you want your client blocked by other clients all the time.

Definitely a fun project with lots to get your teeth into! :D