PDA

View Full Version : Game Network Engine



technomage
22-05-2006, 09:17 PM
Hi People

I have a large scale network project I want to start working on. One requirement is a reliable Networking system. Something along the lines of RakNet (http://www.rakkarsoft.com) or Torque Network Library (http://www.garagegames.com/products/27).

I think it would be nice to have a native networking system, as I don't think either of the libraries mentioned can be used from Delphi or Free Pascal, they export C++ classes which is always a bit tricky and I don't think they have a flat API for languages such as delphi.

So I have a proposal. I want to start a project to implement a solid game networking system on top of Indy which as of now supports Free Pascal. Now I'm going to need somehelp on this one. So if anyone has experince of implementing Networking systems or ideas on design let me know.

If we keep it on this thread others can contribute.

Dean

JSoftware
22-05-2006, 09:42 PM
I found the indy components to be pretty bad working with. Why not start all lowlevel? If you keep it in Berkeley sockets then you should easily be able to make it portable to all systems

WILL
22-05-2006, 10:00 PM
So basically you want to make something of an Open platform for a Battle.net-type (by Blizzard) of server and client API, but cross platform?

Well I've not been one that has had great success doing much network programming, but what I have done was kinda fun. :)

And since I was considering making a dedicated server for my latest project, it would only support my own needs. So... you got my support. ;)


Questions Galore:

How dependant on extra libraries did you want to get? I'd like it to support even console-based games aswell. And what options are best for cross-platform?

Would it be feesible to try to allow the use of either SDL_Net, Indy, DirectPlay and such as library options? Or would that be a bad idea and make optimization a royal pain in the arse?

technomage
22-05-2006, 10:29 PM
I found the indy components to be pretty bad working with. Why not start all lowlevel? If you keep it in Berkeley sockets then you should easily be able to make it portable to all systems

I have played with Indy for a few projects and to my mind they work pretty well. Allot of the Hard work has been done already. Going low level basically mean starting from scratch in which case we might as well just take the source for RakNet and convert it to pascal... just a thought.. it has very good reviews..

I would prefer 0 external libraries except those you would find on an OS, so that limits this to Sockets, no SDL_Net no DirectPlay. Indy is based on normal sockets and supports Windows and Linux (so far) but with the recent port to FPC I can see Mac OSX on the horizon.

What I thik the goal should be is a native pascal networking system that can be used to "most" games with a few modifications. The kind of services we should be providing is

:arrow: Fast UDP only implementation
:arrow: Implement REliable UDP packets, the engine should detect errors and resend.
:arrow: Distributed Objects. eg. an object can update itself across the net, the engine should handle this so if you derive from a proxy class all the published properties will be sent to it's twin on the server.
:arrow: A Stream based interface (get away from those horrible bit arrays)
:arrow: ...<insert your requirements here>

We could use SDL_Net at a push as that is low enough, the only reason I'm not for that is it's not thread safe,

WILL
22-05-2006, 10:47 PM
Hmm ok. Well some advanced features I'd love for it to have in the library are:

:arrow: an Account Management System
:arrow: Chat Rooms with logging and wisper capabilities
:arrow: some kind of Host/Join Game System with Password, player limit suppot and other customizable options while keeping it 'open' for customization to a specific game's required options.
:arrow: Ping/Lag functions to detect a player/server's network strength.

Thats all I can think of right now...

aidave
22-05-2006, 11:34 PM
I've built a native Pascal multiplayer net engine using Indy (called AIR).
It runs well but not perfect... there are alot of issues to deal with. It gets complicated really fast.

It features:
- List server for clients to find servers
- Passwords
- Admin password / remote server control
- Sync only changing objects
- Sync creation/deletion of obj
- Client resyncs questionable obj
- Client can control any object
- See users name on obj and camera position
- Server authority of game state (users cant cheat)
- Moderate Smoothing (its playable but you can feel the lag)
- Simple chat
- etc

Unsolved issues:
- Compression of packets
- Hacker prevention
- Perfect smoothing
- Lag Compensation
- Perfect Physics Prediction
- etc

Although I havent OpenSourced it yet, (not 100% sure I want to).
It would be really nice to have a community or a team working on a networked engine.

In any case we'll be releasing a test of Blocked PVP multiplayer, in the coming weeks.

aidave
22-05-2006, 11:47 PM
BTW A thread-based approach is the best way to go.
All the in/out sending is done in a thread while the game loop runs without interference from the network.

Indy is great for this and includes alot of thread-safe variables and data structures.

WILL
22-05-2006, 11:50 PM
Well noone says you can't offer some of your engines functions and routines for public use. :)

Might even be able to borrow from this one aswell...

aidave
23-05-2006, 12:24 AM
True! I'll definitely help out with this project.

What I am more interested in though, is a commercial venture. Either closed source, or some kind of licensed open source (free for free use, or % of profits). It is fun and all to make game engines but wouldnt it be cooler to do it for a living? :wink:

WILL
23-05-2006, 12:42 AM
Well I might be going off topic here, but I've always felt that libraries should be free and let the games be commercial.

I think that this is truely a new enough concept to presue in the opensource realm, provided that it builds on top of the basic network library and make an adaptable dedicated server API out of it. Otherwise there is Indy, DirectPlay, SDL_Net, and so on...

cairnswm
23-05-2006, 08:08 AM
When I did the networking for Run-A-War last year I first tried to do it with the Indy components (UDP if IRC) - however the Indy components used all the processing power on the PC monitoring the Ports that my game didn;t work. :(

Thanks to Technomage I got the networking working with SDL_Net.

I would love to see something like this getting off the ground but I agree with WILL it should be free for use in the same way SDL is available to everyone.

Where possible I'd like to help.

Sly
23-05-2006, 08:11 AM
Take a look at enet (http://enet.bespin.org/). It is written in C, so it would be fairly easy to port or just compile as a DLL.

technomage
23-05-2006, 12:12 PM
That looks like a nice little library there Sly. I think a reliable UDP protocol is the way to go. If we did use Indy we would have to build this ontop of the current systems.

I think it might help if I gave a little background for my motivation for this prioject. I am working on a system that will require 1000-2000 connections to a server, so what ever libray is used it the server system will need to be able to handle that. I think Indy can accomodate this, I'm not sure SDL_Net can. But this is one of my requirements.

my second is that it needs to be very stable, AFAIK Indy is very stable, it has been around for years and I have very rarely found a problem in the underlying classes.

aidave - I would be interested in more details comments on the problems you had with Indy in AIR.

cairnswm - also you experiences would be usefull.

WILL- Is there any problem with having the discussion for this project here on PGD? Perhaps a section in the Game Libraries and components.

{MSX}
23-05-2006, 12:46 PM
I just want to say that there is this great library:
http://www.ararat.cz/synapse/
It's very complete and easy to use. It's portable and it's all pascal. Delphi, FPC and other systems supported.
Also, i've used SDL_Net and it's very very easy to use. I suggest it if you want something up and running in short time.

bye :P

WILL
23-05-2006, 01:14 PM
Not at all Dean. It has been too quiet in the Networking forum anyhow. ;) That it's being used it great to see!

If this project becomes something new then ok, sure create a new thread in it's name, but no rush for that.

WILL
23-05-2006, 01:15 PM
Not at all Dean. It has been too quiet in the Networking forum anyhow. ;) That it's being used it great to see!

If this project becomes something new then ok, sure create a new thread in it's name, but no rush for that.

Hmm... perhaps a quick rundown of what the difference between TCP and UDP ports are would help those not initiated. :)

technomage
23-05-2006, 01:42 PM
TCP - This is a network protocol that ensures that if you send your data it will get there. Using this protocol is like having a phone conversation. You are connected directly to the remote site and data is send back a forth between the two. Because it is reliable, TCP is considered to be slower than UDP, most internet protocols (HTTP, FTP etc) user TCP



UDP - This is a conectionless network protocol. The best description I have found is UDP is like putting a letter in the post, it has an address and you send it, but you have no idea if it will get there. On LAN's UDP is quite reliable, over the internet it's not as reliable as TCP. However allot of games use UDP because it is faster and has a lower overhead than TCP.

aidave
23-05-2006, 03:39 PM
Indy is blocking, which means unless you use it in threads, it is going to slow down your game. It is made by alot of networking experts, who know what they are doing, so why reinvent the wheel?

There are some issues with Indy that I found to be really annoying. One was trying to find the UDP port of connecting clients (still cant figure that out). Another is disconnecting. Indy makes it seem like it supports disconnection detection but only in some cases. For the rest you have to detect yourself with timeouts etc.

If you decide to go with Indy I can teach u more.

cheers
dave

savage
23-05-2006, 03:40 PM
Just to add a little more info about TCP. When TCP information is sent, it knows in what order and how much information should be received. If some information is missing ( a packet is lost during transmission ), it will re-request that bit of information again and again until it receives it. Which obviously generates more network traffic and thus can slow things down.
TCP guarantees delivery while UDP does not, so any data loss via UDP needs to be handled by the application and not the protocol.

I hope I have not muddied the waters.

technomage
23-05-2006, 05:22 PM
Indy is blocking, which means unless you use it in threads, it is going to slow down your game. It is made by alot of networking experts, who know what they are doing, so why reinvent the wheel?


Quite true, this is my main reason for going with Indy




There are some issues with Indy that I found to be really annoying. One was trying to find the UDP port of connecting clients (still cant figure that out). Another is disconnecting. Indy makes it seem like it supports disconnection detection but only in some cases. For the rest you have to detect yourself with timeouts etc.


UDP is connection less so there is no way of knowin g when a UDP client is no longer sending data. Only with TCP can you detect droped connections etc.

Getting the remote UDP port is trickly in most netowrk libraries (I know it's a pain in SDL_Net) I always got round this by sending on a known port but firewalls don't always make that easy, or send the port to send info back on to the server in the first "handshake" mesages.

Some thing else I do is to use TCP to connect to the server and login, then use UDP for all the time critical information. Taht way if the TCP connections drops you can handle that and remove that client from the UDP client list as well.



If you decide to go with Indy I can teach u more.
dave

That would be cool. :D

aidave
23-05-2006, 07:14 PM
UDP is connection less so there is no way of knowin g when a UDP client is no longer sending data. Only with TCP can you detect droped connections etc.

Getting the remote UDP port is trickly in most netowrk libraries (I know it's a pain in SDL_Net) I always got round this by sending on a known port but firewalls don't always make that easy, or send the port to send info back on to the server in the first "handshake" mesages.

Some thing else I do is to use TCP to connect to the server and login, then use UDP for all the time critical information. Taht way if the TCP connections drops you can handle that and remove that client from the UDP client list as well.


I do the same thing with TCP, and for UDP ports of clients (the client sends their port via TCP). Plus if the client cant use UDP for some reason, it reroutes traffic to TCP.

Indy doesnt fully detect TCP disconnects (thats what i meant at first). Even if you ask it "TCPClient.Connected" it can return true while the connected has been broken or timed out. I purchased an Indy support package (recommended) and they told me that we have to send keep alives and queries to verify TCP connection.

8)

technomage
23-05-2006, 07:39 PM
I got the Indy Support Documentation, it was very helpful. :D

So is it a good idea to have this new engine use TCP to handle the initial handshakes/logins/important messages but to use UDP for messages that are very frequent :?:

I did like the feature list of you AIR engine, especially the sync object stuff.

I have been thinking about how to make objects that automatically send changed data over the net to it's twin on the server. I did some tests using RTTI to serialize published properties to XML (binary would be better) and then rebuild the object from the serializer data, but I'm not sure if it would be as quick as a virtual merthods that is just overridden in the decendent classes.

aidave
23-05-2006, 09:15 PM
I got the Indy Support Documentation, it was very helpful. :D

So is it a good idea to have this new engine use TCP to handle the initial handshakes/logins/important messages but to use UDP for messages that are very frequent :?:



Well, im not sure if its a good idea or not! ;) There is a case for and against it. If i startched from scratch I would probably just go for UDP. This is my first attempt at network coding.



I did like the feature list of you AIR engine, especially the sync object stuff.

I have been thinking about how to make objects that automatically send changed data over the net to it's twin on the server. I did some tests using RTTI to serialize published properties to XML (binary would be better) and then rebuild the object from the serializer data, but I'm not sure if it would be as quick as a virtual merthods that is just overridden in the decendent classes.

I'm using virtual methods, I find from a coding point of view its quite easy. Plus there may be some data or complex logic you want to sync that isnt possible from a property. Also consider some objects might only have certain data that needs to be changed alot vs just on the initial sent. The position changes quite frequently but you only need to send the color (or whatever property) of it once.

For every object I put a Handle (unique integer) that is used for the clients to match obj with the server. If the client recieves a packet with a handle it doesnt know about, then the client requests a full update about that object. Thats a good way to make sure all your obj are being sent over UDP. I've avoided completely the sequential numbering of UDP packets and checking for out of order or missing packets, nor storing packet history on server, because basically take the assumption that any UDP data is not important because it may be overwritten by the next packet anyway. And if it isnt, the client will notice something fishy and request a new packet. Each object gets a time stamp from the last server update, plus a change counter. If the object has Changed and the time stamp is getting old, then the object is changing on the client improperly (although we do want to make sure the obj can change for prediction purposes), so the client again asks for a resync via the obj handle. This happens alot if you bump into a stack of boxes or some other unpredictable physics circumstance. I'm not sure what kind of network you want, but with 3d realtime physics this is the way I went to sync it.

Big drawback if you use virtual methods then you are restricting because your users have to inherit from your code (and there is no multiple inheritance in OOP)

heres an outline of the network stuff in AIR:


AIRSyncObject = Class&#40;AIRObject&#41;
private
fSyncStamp&#58; integer;
fChange&#58; integer;
fPredictable&#58; boolean;
fExtraChanges&#58; string;
protected
function GetChanged&#58; boolean;
public
Constructor Create; override;
Destructor Destroy; override;

// time of last sync &#40;for client/server synchronizing&#41;
property SyncStamp&#58; integer read fSyncStamp write fSyncStamp;
// changes?
property Change&#58; integer read fChange write fChange;
// changed<>0
property Changed&#58; boolean read GetChanged;
// extra change info for packets
property ExtraChanges&#58; string read fExtraChanges write fExtraChanges;
// is this client-side predictable? &#40;wont send updates if true&#41;
property Predictable&#58; boolean read fPredictable write fPredictable;
// returns true if the engine should network this &#40;default&#41;,
// false if not ... such as smoke effects, menus, etc
function Networked&#58; boolean; virtual;
// network
function FullPacketOut&#58; string; virtual;
procedure FullPacketIn&#40;const Packet&#58; TStrings&#41;; virtual;
function ChangedPacketOut&#58; string; virtual;
procedure ChangedPacketIn&#40;const Packet&#58; TStrings&#41;; virtual;
// change data
procedure ChangeIncrease; overload;
procedure ChangeIncrease&#40;aAmount&#58; integer&#41;; overload;
procedure ChangeIncreaseMax;
procedure ExtraChange&#40;const ExtraMsg&#58; string&#41;;
end;


hope that gives u some ideas.

8)

technomage
24-05-2006, 08:56 AM
That is really helpful, the Sync objectsd looks quite comprehensive.

I think it should be possible to not force users to inherited from TSyncObject, does Free Pascal Support Interfaces on all platforms? If so that might be a good way around the problem, as objects that want to be sync'd just need to support the ISyncOBject interface.

So who would use this engine if it was based on Indy? I got the impression some people are in favour of writing this engine from scratch, which is a fair point but I for one am not a Networking expert so using Indy which has already tackled most of the platform specific issues seems to me like a good idea.

So where do we go from here....

I will attempt to draw up a class heirachy on top of Indy. For example a "TGameServer" for example will derive from TIdUDPServer etc. Then we can split the work up into well defined packages and those of you who want to help out will work on a package (which will probably contain 1 or 2 classes)

I will also try and think of a good name for this "project" :D

WILL
24-05-2006, 09:44 AM
Thanks for the quick 101 on the TCP/UDP issue. ;)

Perhaps only using TCP for that initial 'handshake' would be best. Just to get all your network configuration options set. From there on UDP all the way it seems, is best for network play. I'm sure it's users will be happy for the extra speed because of it. Of course there are options and settings you can put in for some of their own specific preferences.

A visual chart or the initial heirachy would be nice to get our bearings. :) I'm a visuals kinda guy. Like to 'see' everything.

Oh, and I'm all for using Indy. Heck we can go nuts with all the extra options is has. mini-HTTP server to host game stats, mini-FTP server for game patch updates, IRC channels for chat rooms in the multiplayer lobby areas, etc...

savage
24-05-2006, 11:36 AM
Might be worth have a look at this as well - http://www.gamedev.net/reference/programming/features/shavingping/

WILL
24-05-2006, 11:53 AM
:lol: You know... I was looking at that just the other day. Was thinking hmm... I should post this in PGD's Library. :P

{MSX}
25-05-2006, 10:17 AM
So who would use this engine if it was based on Indy? I got the impression some people are in favour of writing this engine from scratch, which is a fair point but I for one am not a Networking expert so using Indy which has already tackled most of the platform specific issues seems to me like a good idea.


Uhm, i wouldn't use it if it depends on Indy. It's too big for a dependancy.. Also, it's component based which is something unuseful and wasteful for most games.. I'll be happy to use SDL_Net becouse it's very easy and becouse i already use SDL, but of course i can't hope you all agree or use sdl :P For most of you a DLL is not welcome i think :P
There's also the library i posted, synapse. Anyone gave it a shot? It's like indy but more lightweight and class-based instead of component-based.. Maybe we could just use the UDP classes.
Or we can develop a base UDP library. The socket api are easy enought to use, i've used TCP sockets directly for a long time, and udp are easier.

Anyway, my needs for a network library are:
- sending normal UDP packets.
- sending reliable UDP packets.

for the second point i think we could implement some very simple aknowledge system. Note that just very rare packets will need to be reliable (for game setup), so there's no need to have ultra performances.. Just a simple acknowledge system would do.

If you want to implement more advanced features, such as chat rooms and game setups, that's good, but it should be possible to just use the basic sending/receiving capabilities. They could be like two separated libraries, one on top of the other.

These were my two cents :P

technomage
25-05-2006, 11:42 AM
Can you post the link for synapse here? Is it thread safe?

I have to admit I have played with SDL_Net, my main reason for avoiding it at the moment is not because of the extra DLL but becasue of the threading issue, SDL_Net is not Thread Safe. I also wouldn't want to tie anyone wanting to use this library to an external dll, people using DirectX might not want to have to ship SDL_Net and SDL with the app (SDL_Net depends on SDL).

The Threading issue is the biggest issue though. Any high performance networking system needs to be multi threaded.

My personal preference is a library that is pascal based to there are no external dependancies. Indy can be used in a class based mode (I use it like that all the time).

I guess we could start from scratch, but my knowledge of dealing with sockets on the various plattorms (windows, linux, FreeBSD0 is very limited.

aidave
25-05-2006, 11:46 AM
Indy doesnt have to use drag-n-drop components, you can create everything dynamically. There is overhead tho

{MSX}
25-05-2006, 12:04 PM
[quote="technomage"]Can you post the ]
Already posted but here you are :P
http://www.ararat.cz/synapse/

I don't know if it's thread safe..

Btw, afaik the socket api is by itself thread safe, isn't it?

JSoftware
25-05-2006, 12:24 PM
I guess we could start from scratch, but my knowledge of dealing with sockets on the various plattorms (windows, linux, FreeBSD0 is very limited.
It's precisely the same as long as you use berkeley sockets. they are implemented with the same names. socket, listen, bind, connect, send, recv, etc

technomage
25-05-2006, 12:28 PM
Btw, afaik the socket api is by itself thread safe, isn't it?

I don't know.... anyone know?

technomage
05-06-2006, 01:06 PM
I've had a good think about Indy and the other solutions. Now I agree Indy is a large component suite, but it can be used in a non component mode, it is also very stable, thread safe. It's server components are solid and easy to use. All in all I think this is a good basis for a game network.

Speed it the only thing that is an unknown at this point , I don't have much experience with Indy and how it handles large volumes of traffic, so if anyone has any input on Indy from a performance point of view please coment.

I should be able to start with a basic design in a few weeks. Then once people have seen the design they can signup and i'll share out the tasks :)

tanffn
05-06-2006, 06:11 PM
I wrote a game network demo/(start of a network engine) using synapse a while back, if you're interested I can look it up.

The demo shows UDP Client/Server (using broadcast as well) and TCP/IP Client/Server.

technomage
05-06-2006, 06:32 PM
Sounds interesting :D

aidave
05-06-2006, 08:16 PM
Check the latest version of Blocked (0774) www.blocked.ca
It is now multiplayer capable!

:batman:

tanffn
05-06-2006, 08:54 PM
Is there an FTP server in PGD I can upload examples to?

cairnswm
13-06-2006, 08:21 AM
I came accross this article while doing some Game Engine design research:
http://www.cs.cmu.edu/~ashu/papers/nsdi2006.pdf

Might be an interesting read to someone that actually understands it :)

WILL
13-06-2006, 08:56 PM
Is there an FTP server in PGD I can upload examples to?

Sorry, unfortunately we cannot offer such services. PGD does it's best to offer as much as we can, however bandwidth and filespace does remain an issue up to a certan level. We are able to host forums, articles, links the reference library and of course news items with nice pretty graphics, but an FTP server would put us well over the top as it would require a much higher amount of bandwidth and file space from our host. At least this will remain the case while not being publicly funded or financially assisted.

tanffn
24-04-2008, 08:53 AM
Any news on the network engine endeavor?
Its not has fun as AI or 3D rendering but, sadly, it has to be done :)

I want to (re)start working on my RTS-TBS (http://www.pascalgamedevelopment.com/viewtopic.php?t=3643&highlight=) game and for that I need reliable TSP/IP based networking.

It could have been nice to have WCF/Remoting style in Delphi, is there something like that available?
Any ideas/request for approaching this?
I was thinking about a simple approach, send command records with a known format:
TBasePacket = packed record
ID: cardinal;
Size: cardinal;
End;
TDoSomthingPacket = packed record
ID: cardinal;
Size: cardinal;
NewX, NewY: integer;
...
End;
This way the software reads the stream, with the ID it knows what type of packet it is (for casting).

arthurprs
24-04-2008, 05:32 PM
variant record?

technomage
24-04-2008, 08:34 PM
I dropped the idea of building a Net work engine in favour of building a 3D game engine InfintEngine (http://www.pascalgamedevelopment.com/viewtopic.php?t=3672) for the InfiniteSpace-Online (http://www.infinitespace-online.net) project.

I have recently started using ENet as a basis for the Network engine as it provides Reliable UDP support. However I am using Indy as well but this detils will be wrapped up under one simple interface.

tanffn
25-04-2008, 07:35 AM
arthurprs: no, simple record.
You read a stream of data, you know that the 1st word has the Type ID. so you can read the next SizeOf(OurRecord) directly into the record's pointer.
Nothing smart, I don't understand the data I simply dump it into the correct record type.

technomage: Sent you a PM.

cairnswm
25-04-2008, 08:58 AM
What I have done before for networking was through a use of a case inside the record. So the recieving side can always load it directly into a record because every message is the same size:

type
TShapeList = (Rectangle, Triangle, Circle, Ellipse, Other);
TFigure = record
ID : TShapeList
case TShapeList of
Rectangle: (Height, Width: Real);

Triangle: (Side1, Side2, Angle: Real);
Circle: (Radius: Real);
Ellipse, Other: ();
end;


And then you dont need to do fancy tricks reading off the stream as every record is the same size.

tanffn
25-04-2008, 11:13 AM
That’s a nice idea but it works for simple objects, as the more complex are the object the more overhead you’ll get.
but its a good idea to keep in mind :)

arthurprs
25-04-2008, 05:16 PM
cairnswm code is a variant record, whats the problem with then?

of course they are not good for complex objects but they are very simple and easy to manage

tanffn
26-04-2008, 08:13 AM
No problem with them, simply not what I want to do :)
I am reading a few articles about networking. I’ll post my finding in a separate post.

Brainer
26-04-2008, 10:04 AM
I'm not a networking guru, but I'd like to share my knowledge with you. :)

In past few years I worked with two ways of dealing with the networking. The first way was based on strings. Consider this:


Packet Type|User ID|Param Count|Param0|Param1...|Paramn

Using strings like that you can transfer data pretty easily. Look at this example:

const
PACKET_TYPE_LOGIN = 'LOG';
PACKET_TYPE_LOGOFF = 'OFF';
PACKET_TYPE_CHAT = 'CHT';
PACKET_TYPE_MOVE = 'MOV';
PACKET_TYPE_SHOOT = 'SHT';

var
Msg: ShortString;
UserID: Integer;
begin
UserID := 20;
Msg := PACKET_TYPE_LOGOFF + '|' + IntToStr(UserID) + '|' + '0';
end;

This is an example of the message send to the server when user tries to log off. As for the server, it could use something like that to read the data:

const
PACKET_TYPE_LOGIN = 'LOG';
PACKET_TYPE_LOGOFF = 'OFF';
PACKET_TYPE_CHAT = 'CHT';
PACKET_TYPE_MOVE = 'MOV';
PACKET_TYPE_SHOOT = 'SHT';

procedure Split(const AOutput: TStrings; const AText, ASeparator: String);
var
S1, S2: string;
begin
AOutput.Clear();
S2 := AText + ASeparator;
repeat
S1 := Copy(S2, 0, Pos(ASeparator, S2) - 1);
AOutput.Add(S1);
Delete(S2, 1, Length(S1 + ASeparator));
until (S2 = '');
end;

var
RcvdMsg: String;
SL: TStringList;
begin
// before calling this, RcvdMsg has to be filled with data
Split(SL, RcvdMsg, '|');

// check if the packet type is logging in
if (SL[0] = PACKET_TYPE_LOGOFF) then
begin
if not UserList.UserConnected(SL[1]) then // user is connected
exit;
if (StrToInt(SL[2]) <> 0) then // invalid paramater count
exit;
UserList.Delete(StrToInt(SL[1]));
end;

end;

I think this way is good for small games.

Another idea is to use variant records. I think someone has mentioned an example of a variant record before, yet I'm doing it again:

type
{ .: TPacketType :. }
TPacketType = (ptLogin, ptLogoff, ptChat, ptMove, ptShoot);

{ .: TPacket :. }
TPacket = packed record
PkgType: TPacketType;
UserID: Integer;
case TPacketType of
ptLogin: (Login: String[15]; Password: String[15]);
ptChat: (Destination: String[15]; ChatText: String[255]);
ptMove: (X, Z: Single);
ptShoot: (StartX, StartZ: Single);
end;

Personally, I'd stick with variant records, because the size of the record is always the same, no matter what type it is.

Now, let's compare the bandwidth used by both solutions. With strings, it varies, because it depends on the packet type. But let's assume that each packet is 255 bytes. It's around 21,5 MB/day for one person. Similar situation is with variant record. Its size is 277 bytes and the bandwidth is around 23,3 MB/day. It's quite a lot, but with records, you can reduce the size by using Word type instead of Integer.

Correct me if I'm wrong.

paul_nicholls
30-04-2008, 11:43 PM
Hi guys,
I am definitely not a networking guru either, but I have whipped up a quick example of a possible way to do the packets which should be very flexible too.

see the code below:



Unit packet_types;
&#123;$IFDEF fpc&#125;
&#123;$MODE DELPHI&#125; &#123;$H+&#125;
&#123;$ENDIF&#125;
Interface

Type
&#123;................................................. .............................&#125;
TPacketType = &#40;
ePacketType_Login,
ePacketType_Logoff,
ePacketType_Chat
&#41;;
TPacket = Packed Record
PacketType &#58; TPacketType;
End;
TPacket_Login = Packed Record
PacketType &#58; TPacketType;
Username &#58; String&#91;15&#93;;
Password &#58; String&#91;15&#93;;
End;
TPacket_Logoff = Packed Record
PacketType &#58; TPacketType;
Username &#58; String&#91;15&#93;;
Password &#58; String&#91;15&#93;;
End;
TPacket_Chat = Packed Record
PacketType &#58; TPacketType;
Msg &#58; ShortString;
End;
PPacket = ^TPacket;
PPacket_Login = ^TPacket_Login;
PPacket_Logoff = ^TPacket_Logoff;
PPacket_Chat = ^TPacket_Chat;
&#123;................................................. .............................&#125;

Implementation

End.




Unit packet_classes;
&#123;$IFDEF fpc&#125;
&#123;$MODE DELPHI&#125; &#123;$H+&#125;
&#123;$ENDIF&#125;
Interface

Uses
packet_types;

Type
&#123;................................................. .............................&#125;
TPacketReader = Class
Private
Protected
Function ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Virtual;
Public
Function ReadPacket&#40;Var APacket &#58; Pointer&#41; &#58; Boolean;
Procedure FreePacket&#40;Var APacket &#58; Pointer&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
TPacketWriter = Class
Private
Function WritePacket&#40;Const APacket &#58; PPacket&#41; &#58; Boolean;
Protected
Function WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Virtual;
Public
Function WriteLoginPacket &#40;Const AUserName,APassWord &#58; AnsiString&#41; &#58; Boolean;
Function WriteLogoffPacket&#40;Const AUserName,APassWord &#58; AnsiString&#41; &#58; Boolean;
Function WriteChatPacket &#40;Const AMsg &#58; AnsiString&#41; &#58; Boolean;
End;
&#123;................................................. .............................&#125;


Implementation

&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter.WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Begin
Result &#58;= False;
&#123;Write ADataASize bytes from AData to the network and return true if successfull&#125;
&#123;override this in descendent classes&#125;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketReader.ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Begin
Result &#58;= False;
&#123;fill AData with ADataSize bytes from the network and return true if successfull&#125;
&#123;override this in descendent classes&#125;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketReader.ReadPacket&#40;Var APacket &#58; Pointer&#41; &#58; Boolean;
Var
DataSize &#58; LongInt;
Packet &#58; Pointer;
Begin
Result &#58;= False;
If Not ReadData&#40;@DataSize,SizeOf&#40;DataSize&#41;&#41; Then Exit;
GetMem&#40;Packet,DataSize&#41;;
If Not ReadData&#40;Packet,DataSize&#41; Then
Begin
FreeMem&#40;Packet&#41;;
Exit;
End;
APacket &#58;= Packet;
Result &#58;= True;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure TPacketReader.FreePacket&#40;Var APacket &#58; Pointer&#41;;
Begin
If APacket = Nil Then Exit;
FreeMem&#40;APacket&#41;;
APacket &#58;= Nil;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter.WritePacket&#40;Const APacket &#58; PPacket&#41; &#58; Boolean;
Var
DataSize &#58; LongInt;
Begin
Result &#58;= False;
Case APacket^.PacketType Of
ePacketType_Login &#58; DataSize &#58;= SizeOf&#40;TPacket_Login&#41;;
ePacketType_Logoff &#58; DataSize &#58;= SizeOf&#40;TPacket_Logoff&#41;;
ePacketType_Chat &#58; DataSize &#58;= SizeOf&#40;TPacket_Chat&#41;;
Else
Exit;
End;
If Not WriteData&#40;@DataSize,SizeOf&#40;DataSize&#41;&#41; Then Exit;
If Not WriteData&#40;APacket,DataSize&#41; Then Exit;
Result &#58;= True;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter.WriteLoginPacket&#40;Const AUserName,APassWord &#58; AnsiString&#41; &#58; Boolean;
Var
LoginPacket &#58; TPacket_Login;
Begin
LoginPacket.PacketType &#58;= ePacketType_Login;
LoginPacket.Username &#58;= AUserName;
LoginPacket.PassWord &#58;= APassWord;
Result &#58;= WritePacket&#40;@LoginPacket&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter.WriteLogoffPacket&#40;Const AUserName,APassWord &#58; AnsiString&#41; &#58; Boolean;
Var
LogoffPacket &#58; TPacket_Logoff;
Begin
LogoffPacket.PacketType &#58;= ePacketType_Logoff;
LogoffPacket.Username &#58;= AUserName;
LogoffPacket.PassWord &#58;= APassWord;
Result &#58;= WritePacket&#40;@LogoffPacket&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter.WriteChatPacket&#40;Const AMsg &#58; AnsiString&#41; &#58; Boolean;
Var
ChatPacket &#58; TPacket_Chat;
Begin
ChatPacket.PacketType &#58;= ePacketType_Chat;
ChatPacket.Msg &#58;= AMsg;
Result &#58;= WritePacket&#40;@ChatPacket&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
End.


Example usage:



Type
&#123;................................................. .............................&#125;
TPacketReader_Stream = Class&#40;TPacketReader&#41;
Private
FStream &#58; TStream;
Protected
Function ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Override;
Public
Constructor Create&#40;Const AStream &#58; TStream&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
TPacketWriter_Stream = Class&#40;TPacketWriter&#41;
Private
FStream &#58; TStream;
Protected
Function WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Override;
Public
Constructor Create&#40;Const AStream &#58; TStream&#41;;
End;
&#123;................................................. .............................&#125;

Constructor TPacketReader_Stream.Create&#40;Const AStream &#58; TStream&#41;;
Begin
Inherited Create;
FStream &#58;= AStream;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketReader_Stream.ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Type
PByte = ^Byte;
Begin
Result &#58;= FStream.Read&#40;PByte&#40;AData&#41;^,ADataSize&#41; = ADataSize;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Constructor TPacketWriter_Stream.Create&#40;Const AStream &#58; TStream&#41;;
Begin
Inherited Create;
FStream &#58;= AStream;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter_Stream.WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Type
PByte = ^Byte;
Begin
Result &#58;= FStream.Write&#40;PByte&#40;AData&#41;^,ADataSize&#41; = ADataSize;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure WritePackets&#40;Const APacketWriter &#58; TPacketWriter&#41;;
Begin
APacketWriter.WriteLoginPacket &#40;'UserName','PassWord'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#1'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#2'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#3'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#4'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#5'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#6'&#41;;
APacketWriter.WriteLogoffPacket&#40;'UserName','PassWo rd'&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ProcessLoginPacket&#40;Const APacket &#58; PPacket_Login&#41;;
Begin
WriteLn&#40;'Login Packet&#58;'&#41;;
WriteLn&#40;' ',APacket^.UserName&#41;;
WriteLn&#40;' ',APacket^.PassWord&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ProcessLogoffPacket&#40;Const APacket &#58; PPacket_Logoff&#41;;
Begin
WriteLn&#40;'Logoff Packet&#58;'&#41;;
WriteLn&#40;' ',APacket^.UserName&#41;;
WriteLn&#40;' ',APacket^.PassWord&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ProcessChatPacket&#40;Const APacket &#58; PPacket_Chat&#41;;
Begin
WriteLn&#40;'Chat Packet&#58; '&#41;;
WriteLn&#40;' ',APacket^.Msg&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ReadPackets&#40;Const APacketReader &#58; TPacketReader&#41;;
Var
Packet &#58; Pointer;
Begin
While APacketReader.ReadPacket&#40;Packet&#41; Do
Begin
Case PPacket&#40;Packet&#41;^.PacketType Of
ePacketType_Login &#58; ProcessLoginPacket &#40;Packet&#41;;
ePacketType_Logoff &#58; ProcessLogoffPacket&#40;Packet&#41;;
ePacketType_Chat &#58; ProcessChatPacket &#40;Packet&#41;;
Else
&#123;unknown packet so do error or something&#125;
End;
APacketReader.FreePacket&#40;Packet&#41;;
End;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Var
PacketReader &#58; TPacketReader;
PacketWriter &#58; TpacketWriter;
MemoryStream &#58; TMemoryStream;
begin
MemoryStream &#58;= TMemoryStream.Create;
PacketWriter &#58;= TPacketWriter_Stream.Create&#40;MemoryStream&#41;;
PacketReader &#58;= TPacketReader_Stream.Create&#40;MemoryStream&#41;;
WritePackets&#40;PacketWriter&#41;;
MemoryStream.Position &#58;= 0;
ReadPackets&#40;PacketReader&#41;;
MemoryStream.Free;
WriteLn&#40;'Finished!'&#41;;
ReadLn;
End.


It does work (I tried it on a Stream packet reader writer descendants as a test), and should be easy to modify.
It should be fairly easy to make TPacketReader/Writer descendants to read and write from sockets too (synapse, indy, etc).

cheers,
Paul

arthurprs
30-04-2008, 11:59 PM
i don't think that it will work with udp :? or will?

paul_nicholls
01-05-2008, 12:18 AM
i don't think that it will work with udp :? or will?

Why not?
You can read and write data via UDP the same as TCP, it just isn't guarenteed to arrive at its destination, or in the correct order.

the read/write bit should still work if you make a TPacketWriter/Reader descendant to write/read data using UDP, let's say via Synapse for example.

cheers,
Paul

paul_nicholls
01-05-2008, 03:39 AM
Hi arthurprs,
I just wrote a quick project containing the stream reader/writer I already presented, but now also has synapse UDP reader/writer classes too :)

It has a test for both types of reader/writer combos, and both tests work for me (including the UDP)...



program reader_writer_test;
&#123;$APPTYPE CONSOLE&#125;
uses
SysUtils,
packet_types,
packet_classes,
Classes,
blcksock;

Type
&#123;................................................. .............................&#125;
TPacketReader_Stream = Class&#40;TPacketReader&#41;
Private
FStream &#58; TStream;
Protected
Function ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Override;
Public
Constructor Create&#40;Const AStream &#58; TStream&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
TPacketWriter_Stream = Class&#40;TPacketWriter&#41;
Private
FStream &#58; TStream;
Protected
Function WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Override;
Public
Constructor Create&#40;Const AStream &#58; TStream&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
TPacketReader_Synapse_UDP = Class&#40;TPacketReader&#41;
Private
FSocket &#58; TUDPBlockSocket;
Protected
Function ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Override;
Public
Constructor Create&#40;Const ASocket &#58; TUDPBlockSocket;
Const APort &#58; AnsiString&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
TPacketWriter_Synapse_UDP = Class&#40;TPacketWriter&#41;
Private
FSocket &#58; TUDPBlockSocket;
FIP &#58; AnsiString;
FPort &#58; AnsiString;
Protected
Function WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean; Override;
Public
Constructor Create&#40;Const ASocket &#58; TUDPBlockSocket;
Const AIP, APort &#58; AnsiString&#41;;
End;
&#123;................................................. .............................&#125;

Constructor TPacketReader_Stream.Create&#40;Const AStream &#58; TStream&#41;;
Begin
Inherited Create;
FStream &#58;= AStream;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketReader_Stream.ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Type
PByte = ^Byte;
Begin
Result &#58;= FStream.Read&#40;PByte&#40;AData&#41;^,ADataSize&#41; = ADataSize;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Constructor TPacketWriter_Stream.Create&#40;Const AStream &#58; TStream&#41;;
Begin
Inherited Create;
FStream &#58;= AStream;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter_Stream.WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Type
PByte = ^Byte;
Begin
Result &#58;= FStream.Write&#40;PByte&#40;AData&#41;^,ADataSize&#41; = ADataSize;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Constructor TPacketReader_Synapse_UDP.Create&#40;Const ASocket &#58; TUDPBlockSocket;
Const APort &#58; AnsiString&#41;;
Begin
Inherited Create;
FSocket &#58;= ASocket;
FSocket.Bind&#40;'0.0.0.0',APort&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketReader_Synapse_UDP.ReadData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Type
PByte = ^Byte;
Begin
FSocket.RecvBuffer&#40;AData,ADataSize&#41;;
Result &#58;= FSocket.LastError = 0;
If Not Result Then WriteLn&#40;'TPacketReader_Synapse_UDP&#58; ',FSocket.LastErrorDesc&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Constructor TPacketWriter_Synapse_UDP.Create&#40;Const ASocket &#58; TUDPBlockSocket;
Const AIP, APort &#58; AnsiString&#41;;
Begin
Inherited Create;
FSocket &#58;= ASocket;
FIP &#58;= AIP;
FPort &#58;= APort;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Function TPacketWriter_Synapse_UDP.WriteData&#40;Const AData &#58; Pointer; Const ADataSize &#58; LongInt&#41; &#58; Boolean;
Begin
FSocket.Connect&#40;FIP,FPort&#41;;
FSocket.SendBuffer&#40;AData,ADataSize&#41;;
Result &#58;= FSocket.LastError = 0;
If Not Result Then WriteLn&#40;'TPacketWriter_Synapse_UDP&#58; ',FSocket.LastErrorDesc&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure WritePackets&#40;Const APacketWriter &#58; TPacketWriter&#41;;
Begin
APacketWriter.WriteLoginPacket &#40;'UserName','PassWord'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#1'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#2'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#3'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#4'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#5'&#41;;
APacketWriter.WriteChatPacket &#40;'Message#6'&#41;;
APacketWriter.WriteLogoffPacket&#40;'UserName','PassWo rd'&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ProcessLoginPacket&#40;Const APacket &#58; PPacket_Login&#41;;
Begin
WriteLn&#40;'Login Packet&#58;'&#41;;
WriteLn&#40;' ',APacket^.UserName&#41;;
WriteLn&#40;' ',APacket^.PassWord&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ProcessLogoffPacket&#40;Const APacket &#58; PPacket_Logoff&#41;;
Begin
WriteLn&#40;'Logoff Packet&#58;'&#41;;
WriteLn&#40;' ',APacket^.UserName&#41;;
WriteLn&#40;' ',APacket^.PassWord&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ProcessChatPacket&#40;Const APacket &#58; PPacket_Chat&#41;;
Begin
WriteLn&#40;'Chat Packet&#58; '&#41;;
WriteLn&#40;' ',APacket^.Msg&#41;;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure ReadPackets&#40;Const APacketReader &#58; TPacketReader&#41;;
Var
Packet &#58; Pointer;
Finished &#58; Boolean;
Begin
Finished &#58;= False;
While Not Finished Do
Begin
If APacketReader.ReadPacket&#40;Packet&#41; Then
Begin
Case PPacket&#40;Packet&#41;^.PacketType Of
ePacketType_Login &#58; ProcessLoginPacket &#40;Packet&#41;;
ePacketType_Logoff &#58; ProcessLogoffPacket&#40;Packet&#41;;
ePacketType_Chat &#58; ProcessChatPacket &#40;Packet&#41;;
Else
&#123;unknown packet so do error or something&#125;
End;
If PPacket&#40;Packet&#41;^.PacketType = ePacketType_Logoff Then
Finished &#58;= True;
APacketReader.FreePacket&#40;Packet&#41;;
End;
End;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure RunTest_Stream;
Var
MemoryStream &#58; TMemoryStream;
PacketReader &#58; TPacketReader;
PacketWriter &#58; TPacketWriter;
Begin
MemoryStream &#58;= TMemoryStream.Create;

PacketWriter &#58;= TPacketWriter_Stream.Create&#40;MemoryStream&#41;;
PacketReader &#58;= TPacketReader_Stream.Create&#40;MemoryStream&#41;;

WritePackets&#40;PacketWriter&#41;;
MemoryStream.Position &#58;= 0;
ReadPackets&#40;PacketReader&#41;;
MemoryStream.Free;
PacketReader.Free;
PacketWriter.Free;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Procedure RunTest_UDP;
Var
ClientSocket &#58; TUDPBlockSocket;
ServerSocket &#58; TUDPBlockSocket;
PacketReader &#58; TPacketReader;
PacketWriter &#58; TPacketWriter;
Begin
ClientSocket &#58;= TUDPBlockSocket.Create;
ServerSocket &#58;= TUDPBlockSocket.Create;

PacketWriter &#58;= TPacketWriter_Synapse_UDP.Create&#40;ClientSocket,'Loc alHost','12000'&#41;;
PacketReader &#58;= TPacketReader_Synapse_UDP.Create&#40;ServerSocket,'120 00'&#41;;

WritePackets&#40;PacketWriter&#41;;
ReadPackets&#40;PacketReader&#41;;
ClientSocket.Free;
ServerSocket.Free;
PacketReader.Free;
PacketWriter.Free;
End;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;
Begin
WriteLn&#40;'*********************'&#41;;
WriteLn&#40;'**** stream test ****'&#41;;
WriteLn&#40;'*********************'&#41;;
RunTest_Stream;
WriteLn&#40;'******************'&#41;;
WriteLn&#40;'**** UDP test ****'&#41;;
WriteLn&#40;'******************'&#41;;
RunTest_UDP;
WriteLn&#40;'Press Enter to continue!'&#41;;
ReadLn;
End.

cheers,
Paul

arthurprs
01-05-2008, 03:51 AM
i don't think that it will work with udp :? or will?

Why not?
You can read and write data via UDP the same as TCP, it just isn't guarenteed to arrive at its destination, or in the correct order.

the read/write bit should still work if you make a TPacketWriter/Reader descendant to write/read data using UDP, let's say via Synapse for example.

cheers,
Paul

since the packets have diferent sizes
how do you will receive then (you don't know the type of package, any package can arrive anytime in anyorder) :? ?

paul_nicholls
01-05-2008, 04:02 AM
i don't think that it will work with udp :? or will?

Why not?
You can read and write data via UDP the same as TCP, it just isn't guarenteed to arrive at its destination, or in the correct order.

the read/write bit should still work if you make a TPacketWriter/Reader descendant to write/read data using UDP, let's say via Synapse for example.

cheers,
Paul

since the packets have diferent sizes
how do you will receive then (you don't know the type of package, any package can arrive anytime in anyorder) :? ?

When I do get packets (including the packet type), I get the whole packet, it is just the order of the received packets that could be out of order (Not LAN, but over internet or possibly WAN too)...

If I want to worry about that issue, I would put in a time-stamp on the packets and disregard out of order packets (if not super important)

Also it doesn't matter that the packet sizes are different, the send/receive code takes care of that for me regardless of the reader/writer class being used.

I hope this makes sense,
cheers,
Paul

arthurprs
01-05-2008, 06:11 AM
i don't think that it will work with udp :? or will?

Why not?
You can read and write data via UDP the same as TCP, it just isn't guarenteed to arrive at its destination, or in the correct order.

the read/write bit should still work if you make a TPacketWriter/Reader descendant to write/read data using UDP, let's say via Synapse for example.

cheers,
Paul

since the packets have diferent sizes
how do you will receive then (you don't know the type of package, any package can arrive anytime in anyorder) :? ?

When I do get packets (including the packet type), I get the whole packet, it is just the order of the received packets that could be out of order (Not LAN, but over internet or possibly WAN too)...

If I want to worry about that issue, I would put in a time-stamp on the packets and disregard out of order packets (if not super important)

Also it doesn't matter that the packet sizes are different, the send/receive code takes care of that for me regardless of the reader/writer class being used.

I hope this makes sense,
cheers,
Paul

uhm, you don't need to specify the amount of data you want to receive?

paul_nicholls
01-05-2008, 06:33 AM
uhm, you don't need to specify the amount of data you want to receive?

If you look at the code I posted, then you can see that I do.


Function TPacketWriter.WritePacket&#40;Const APacket &#58; PPacket&#41; &#58; Boolean;
Var
DataSize &#58; LongInt;
Begin
Result &#58;= False;
Case APacket^.PacketType Of
ePacketType_Login &#58; DataSize &#58;= SizeOf&#40;TPacket_Login&#41;;
ePacketType_Logoff &#58; DataSize &#58;= SizeOf&#40;TPacket_Logoff&#41;;
ePacketType_Chat &#58; DataSize &#58;= SizeOf&#40;TPacket_Chat&#41;;
Else
Exit;
End;
If Not WriteData&#40;@DataSize,SizeOf&#40;DataSize&#41;&#41; Then Exit;
If Not WriteData&#40;APacket,DataSize&#41; Then Exit;
Result &#58;= True;
End;

You can see that I send the size of the packet first, then the packet itself...

On the receiving end I read the datasize first, then use that datasize to read the packet itself.


Function TPacketReader.ReadPacket&#40;Var APacket &#58; Pointer&#41; &#58; Boolean;
Var
DataSize &#58; LongInt;
Packet &#58; Pointer;
Begin
Result &#58;= False;
If Not ReadData&#40;@DataSize,SizeOf&#40;DataSize&#41;&#41; Then Exit;
GetMem&#40;Packet,DataSize&#41;;
If Not ReadData&#40;Packet,DataSize&#41; Then
Begin
FreeMem&#40;Packet&#41;;
Exit;
End;
APacket &#58;= Packet;
Result &#58;= True;
End;

As you can hopefully see, this does not depend on the type of Packet reader/writer descendant you write :-)

I hope this helps :-)
cheers,
Paul

arthurprs
01-05-2008, 04:06 PM
great =D

now i understand

paul_nicholls
01-05-2008, 10:49 PM
I'm glad I could help :)
BTW, I have made a small change to the code to make it a bit nicer.

In the packet_classes.pas file I have added a cPacketSizeArray Constant.



Implementation

Const
&#123;................................................. .............................&#125;
cPacketSizeArray&#58; Array&#91;TPacketType&#93; Of LongInt =
&#40;
SizeOf&#40;TPacket_Login&#41;,
SizeOf&#40;TPacket_Logoff&#41;,
SizeOf&#40;TPacket_Chat&#41;
&#41;;
&#123;................................................. .............................&#125;

&#123;................................................. .............................&#125;


and changed the TPacketWriter.WritePacket() method to use this instead of the case statement:


Function TPacketWriter.WritePacket&#40;Const APacket &#58; PPacket&#41; &#58; Boolean;
Var
DataSize &#58; LongInt;
Begin
Result &#58;= False;
DataSize &#58;= cPacketSizeArray&#91;APacket^.PacketType&#93;;
If Not WriteData&#40;@DataSize,SizeOf&#40;DataSize&#41;&#41; Then Exit;
If Not WriteData&#40;APacket,DataSize&#41; Then Exit;
Result &#58;= True;
End;


The code is a bit better now I think :-)

cheers,
Paul

paul_nicholls
07-05-2008, 02:05 AM
Thanks to some ideas from Pyrogine, I have now re-written my packet stuff so the packets are now Classes instead of records :)

The code is a bit more complex now, but I think it is more flexible in the long run.

You still create 'concrete' packet reader and writer classes to read/write the packets to wherever you want (Stream, UDP, TCP, etc).

You create new 'concrete' packet types by descending from TPacket_Base, overriding the WriteToStream and ReadFromStream methods so you control which fields in the class get written and read, and add a new cPacketID_ constant for that packet type.

You also override the Create method to set the PacketID to the correct type.

The base packet class even has its own Assign method that uses the 'concrete' packet classes' own WriteToStream and ReadFromStream methods so you can assign one packet to another of the same type without adding any extra code :)

You register the all the packet types you are going to use with the PacketReader instance you are using, and 'away you go' :)

So, if anyone is interested, here is the new code http://fpc4gp2x.eonclash.com/downloads/packet_reader_writer_test.zip

Enjoy :)

cheers,
Paul

arthurprs
07-05-2008, 05:04 PM
what happens if the packet with the size of the record is lost ?

paul_nicholls
09-05-2008, 03:35 AM
what happens if the packet with the size of the record is lost ?

Sorry, but I'm not sure what you mean...

cheers,
Paul

arthurprs
09-05-2008, 05:30 PM
what happens if the packet with the size of the record is lost ?

Sorry, but I'm not sure what you mean...

cheers,
Paul

opz, sorry my english.

let me try again,

what happens if the size of the record (that will be send after its size) is lost?

ps: im saying this cuz i read somewhere that UDP can lose some data

Andreaz
11-05-2008, 07:27 AM
opz, sorry my english.

let me try again,

what happens if the size of the record (that will be send after its size) is lost?

ps: im saying this cuz i read somewhere that UDP can lose some data

It's not common that udp looses some random bits in the packet, either the packet is lost or not.

To avoid some nasty crashes with lost packets make shure that you keep the data amount in each packet to less then the UDP framesize (sadly it varies depending on the network type), then you wont have to wory about getting to little data.

If you want to send larger packets you have to code around this, make shure the length of the incomming data is larger then the length of the stream. If it's not you have to append the next packet that arrives to the stream.

What you probably going to need then is some kind of sequencing so you can detect when the 2nd packet is lost, else you will get incorrect data for the next packet.

Writing theese algoritms is big task, i would recommend using something like ENet where theese things are already written

arthurprs
11-05-2008, 03:57 PM
opz, sorry my english.

let me try again,

what happens if the size of the record (that will be send after its size) is lost?

ps: im saying this cuz i read somewhere that UDP can lose some data

It's not common that udp looses some random bits in the packet, either the packet is lost or not.

To avoid some nasty crashes with lost packets make shure that you keep the data amount in each packet to less then the UDP framesize (sadly it varies depending on the network type), then you wont have to wory about getting to little data.

If you want to send larger packets you have to code around this, make shure the length of the incomming data is larger then the length of the stream. If it's not you have to append the next packet that arrives to the stream.

What you probably going to need then is some kind of sequencing so you can detect when the 2nd packet is lost, else you will get incorrect data for the next packet.

Writing theese algoritms is big task, i would recommend using something like ENet where theese things are already written

thanks =)

User137
14-05-2008, 02:49 PM
A little inspired by this topic started making my own network "wrapper". I'm using WSockets 1.20 http://www.delphi32.com/vcl/2142/ for it, to support TCP and UDP server and client in a single class. Constructing 1 of this class makes 1 connection which does few of following (still in construction ofc):
- Unified server and client code, making it less significant for programmer to deal with
- TCP or UDP you choose, engine don't care :wink:
- String masked whole packet encryption (optional custom encryption in addition if you want to do it hard/more secure way)
- List of clients who joined the network properly, seen by all connections
- Queue system to verify no lost packets (this is handled within Update function similar to DelphiX direct input). Packets are stored and resent when needed.
- Sending from 1 connection to other or broadcasted to all, including self
- Ping checking
- Optional Checksum for making sure packets don't get damaged, maybe useful for file transfer

As for mechanics how TCP and UDP can use same kind of packets, i use a large, say 40000 bytes+header structure (noone should ever use that big) in memory where WSockets writes only the amount of incoming data through onData event. It has a ReadBuffer() function which returns number of bytes received. This temporary packet is a variable within class to prevent slowing that would happen if packet was created within procedure every time. Found out this is the logic how TUDPClient read dynamic strings too. If i had sent packet size first and packet after it in separate packets, i'd risk them coming in different order or disappearing on the way, which could entirely hang the network (imagine 4 clients send the 2 packets same time).

And as any other of my projects, they either finish or not, without any time limits... it is far already though.