Creating Reliable UDP Messages

Blitz3D Forums/Blitz3D Programming/Creating Reliable UDP Messages

GIB3D(Posted 2009) [#1]
I'm finding it rather annoying trying to convert my game from using BlitzPlay Lite to RottNet. I was thinking about just putting what I need from RottNet into BlitzPlay, the Reliable UDP Messages but I'm not completely sure what all I would have to bring over.


_Skully(Posted 2009) [#2]
To create reliable UDP you need a outgoing message queue and an aknowledgement timeout that will resend the packet once surpassed.


GIB3D(Posted 2009) [#3]
How will you know if it's been sent though once the other person finally gets it? Wouldn't that mean they would have to keep resending a message back to you telling you they got it which means that you will have to send a message back to them again and it would ping pong back and forth?


_Skully(Posted 2009) [#4]
Nope.. because if you send back the reply and its not received by the server the packet will just be resent again initiating the response again... if you sequence the packets the client will know it had already received it and just ignore the data.


GIB3D(Posted 2009) [#5]
Well how should the steps look like?
Like...
1. Server sends reliable message
2. ...


_Skully(Posted 2009) [#6]
2. Server queues message
3. Client receives message and sends response
4. Server gets response and clears message from the queue

4a. If server doesn't get a response, it re-sends packet after a time-out period


Gabriel(Posted 2009) [#7]
You can't write it in order like that because the server and the client are both doing multiple things at once.

Essentially, if the server wants to send a message to a player, it sends the message, and then makes a copy of that message - along with the player it sent it to - and puts it on the unconfirmed list. It does this for any messages it sends out.

When the client receives a message, it should immediately send a received notification. The received notifications are marked as such so that the server does not think this is a new message from the player to the server.

The server is constantly monitoring for new received confirmations. When it receives one, it removes the corresponding message from the unconfirmed list. It also checks for messages which have been on the unconfirmed list for longer than a certain period of time. (Period of time is up to you but should allow for two messages to be sent in different directions and should account for adverse connection issues. ) As soon as a message has been on the unconfirmed list for that period of time, the server resends the message. It doesn't do anything with the unconfirmed list as it needs to await confirmation again, so it just sets the time for this message back to zero again so that it doesn't send any more resends for this message until the time is up all over again.

The server should also be aware that it may - under some circumstances - receive two confirmations for the same message. (EG: when a message takes longer than expected to be confirmed and it has already been resent before the first confirmation arrives. ) So the server should always check that the message is still on the queue before trying to remove it from the unconfirmed list.


Note that all you have now is a reliable UDP messaging system. You do NOT have ordered, reliable UDP. If you want your messages to arrive in order, you will have to do considerably more. (EG: the client will have to keep messages which are received out of order and not act upon them until the previous messages arrive, apart from confirming receipt, of course. )


GIB3D(Posted 2009) [#8]
So you're saying RottNet does not put the messages in order? What about BlitzPlay?


Gabriel(Posted 2009) [#9]
I have no idea what RottNet does. I'm saying that all you'll have if you do what I've outlined above is a system of reliable UDP messaging, but you won't have them ordered without additional work.

BlitzPlay Pro/Plus/whatever it was called had reliable, ordered UDP. I don't know what Lite has/had.


GIB3D(Posted 2009) [#10]
BlitzPlay Lite doesn't have reliable UDP. I don't know if it has them ordered or not either.


Gabriel(Posted 2009) [#11]
If they're not reliable, they won't be ordered. Ordering is something you only do if the messages are reliable.


GIB3D(Posted 2009) [#12]
Well RottNet has reliable messages but there are things I'm not liking about RottNet. One of them being that I have to create a server and then create a client and keep track of both of them. I don't have to do that with BlitzPlay. That's why I want the reliable message functionality of RottNet with the easiness of BlitzPlay. RottNet has groups but those are easy, I can make that myself in probably less than a minute.


Mahan(Posted 2009) [#13]
Don't always try to reinvent the wheel.

Try eNet.

It handles it's own queuing and ordered delivery and also provides for reliable messages. All over UDP. Written for gaming!

Here is the userlib:
http://www.blitzbasic.com/Community/posts.php?topic=49076

eNet has been out for years and is considered by many as rock-stable. For me it would have taken months to write, test and debug a network lib to this state, and I'm sure for many others too. I've said it before but I repeat: Don't reinvent the wheel. Especially when you already got "another wheel" accessible when you sit there in your comfy-chair :)

EDIT: forgot to add another important point: eNet does it's own "packaging". This means that it does not matter much if you send 100 10-byte messages or 1k, 10k or 100k in one package. eNet will repackage it internally but you will always receive the packages as you sent them on the other end.

EDIT2: And I also forgot to mention that it got it's own automatic (configurable) bandwidth limitations, client-connected/disconnected state etc. built in too.


RifRaf(Posted 2009) [#14]
I read the features list, it looks really nice. I especially like the channeled sequencing.


GIB3D(Posted 2009) [#15]
@ Mahan
You said not to reinvent the wheel, but with eNet I feel like I will have to try even harder to do just that. With eNet, there are no functions (or variables) for getting certain things like...

Getting the Player ID (I would need that or else I can't send messages)

Getting the Player Name (eNet doesn't even let you set player names)

Seeing if you're still online.

Seeing if you're the host.

...I know I can code that myself but that will mean I have to "reinvent the wheel". Another thing I was used to from using BlitzPlay was using a For, Next to loop through a Type which stored all of the messages like
msgData
msgFrom
msgType

Once again, I could do that myself. If there is no other way, I will try eNet.


All I was trying to was take a tiny chunk of the RottNet wheel and glue it into the unfinished BlitzPlay wheel.


GIB3D(Posted 2009) [#16]
I was just about to start doing something with eNet and then I realized a few things.

1. Does the server automatically create a player/user for you? (BlitzPlay does but RottNet doesn't)
2. If #1 is False, then can you have client and server in one file?
3. If #2 is True, then do you have to have separate banks for client and server?


Mahan(Posted 2009) [#17]
@RifRaf: The only downside with the channels is that the Userlib does not expose that functionality iirc (that's why I left that out from my description). Maybe some day, but only if my current project really requires it, I might rewrite that lib to include channel support. Or someone else gets to it before me :-)

@GIA_Green_Fire_:

The thing is that eNet is somewhere between raw UDP on one hand and BLitzPlay and RottNet on the other. I only mentioned eNet because I saw some discussion above about the unholy matters of raw UDP-stream programming and that is just so meaningless when things like eNet exist.

With eNet all the serialization and messaging mechanics still needs to be written but all that insane boring stuff of getting low-level networking to play is gone.

With eNet you get a stable boilerplate. The groundwork that just needs to be in place anyway before all the application programming around the network takes place.

I'll try to answer the questions:

1) No the server does not create anything. You start the server with some functions and then you manage incoming messages by some function calls. You can send a packet from anywhere in you code with a sender function.

2) Sure you can. You can have the same program being both client and server at the same time, if you feel like it. Just depends on what functions you call to start them up.

3) (Point 3 seems somewhat recursive in it's premise. I'll fall back to assume you meant "#2" :-) :

eNet looks to the program like functions accessing incoming and outgoing queues (sender queue/receiver queue).

You use banks in the function-calls (of the userlib) to receive the data of a package that arrived, and then that data must be interpreted from that bank by your program. Since banks are just used as a temporary storage area while you read out the package information, I don't see why you couldn't use the same bank as long as you interpret the data from one receiving call before you call it again (regardless if you call receive for the server or for the client).

edit: spelling/grammar


GIB3D(Posted 2009) [#18]
3) (Point 3 seems somewhat recursive in it's premise. I'll fall back to assume you meant "#2" :-) :
AHH I just KNEW something looked wrong with that when I typed it. I edited it.

I don't see why you couldn't use the same bank as long as you interpret the data from one receiving call before you call it again

Ah I see. I guess it really wouldn't matter as long as you don't fill it up(this can either be a question... or just a thought)


Mahan(Posted 2009) [#19]

Ah I see. I guess it really wouldn't matter as long as you don't fill it up(this can either be a question... or just a thought)



Yes, and to not fill it up, you just have to create a bank large enough for your largest messages. But since you (the programmer) is in charge of how large messages you send, that should not be a huge issue.

In my implementation I plan on using a single bank of a few hundred k's so I don't risk "overflows".


GIB3D(Posted 2009) [#20]
I was trying to make my own include file for eNet but gave up after I realized that I was going to have to pretty much make everything myself. So I'm going to try not being lazy about learning RottNet and I have some questions about it.

In RottNet, for a multiplayer game, you have one server and usually clients (I don't see why not). When the host creates the server, does he then have to make a client for himself? I'm now thinking that the host does not HAVE to create a client for him to play, only the other players.

BUT! When the host creates the server, he does not have a name, so therefore the players won't know who he is because his name and other stuff is not stored. So now I'm thinking he will need to create a client too... very confusing. I didn't have to worry about this stuff with BlitzPlay :(


_Skully(Posted 2009) [#21]
I think thats up to you... I wouldn't recommend putting the local player through the network pipeline though... since there is no need to do so. What I did with a little multiplayer space game I made (which I have lost the source for) is to channel local player communications into the server post arrival queue directly... that was where all player input data was processed.


RifRaf(Posted 2009) [#22]
Perhaps you are thinking blitzplay has some magic player system.. Its not that magical. CreatePlayer makes an internal type with name, and network ID among other things.

I guess nobody has been able to contact Surreal in some time? is his paypal address still valid ? or his purchase link on his old site?


GIB3D(Posted 2009) [#23]
For me there is no "CreatePlayer" on BlitzPlay. I guess the guy changed a few function names for BlitzPlay Pro. For hosting and joining I have...
BP_HostSession(HostName$,MaxPlayers%,GameType%,LocalPort%,TimeoutPeriod%)
BP_JoinSession(ClientName$,LocalPort%,strHostIP$,HostPort%)

And I just found in the BP_HostSession, not hard to find...
nInfo.NetInfo = New NetInfo
nInfo\IP = BP_My_IP
nInfo\Port = BP_My_Port
nInfo\Name = HostName
BP_My_Name = HostName
BP_My_ID = 1
nInfo\net_id = 1

in BP_JoinSession, it creates a spot in NetInfo too...
nInfo.NetInfo = New NetInfo
nInfo\Name = ClientName
nInfo\net_id = BP_My_ID
nInfo\IP = BP_My_IP
nInfo\Port = BP_My_Port
nInfo\Alive = True


How the server updates his own player the same as the other players, I'm not sure of, but BlitzPlay does it. I'll have to look deeper into the BP_UpdateNetwork function.


GIB3D(Posted 2009) [#24]
I'm in the process now of changing a lot of stuff in BlitzPlay to work with eNet. Most of it is pretty similar so this shouldn't be too hard.


GIB3D(Posted 2009) [#25]
I've got it to work! Yes! Awesome! Using eNet inside of BlitzPlay works! Should I post it somewhere?