Networking (again I know...)

Monkey Forums/Monkey Programming/Networking (again I know...)

Kaltern(Posted 2016) [#1]
So I've been trying to wrap my head around the absolutel basics of networking, specifically a TCP server/client connection.

I already have a TCP server written in VB.NET, ready to send and receive commands. I want to have a Monkey client, which is always actively listening for a TCP command, and to be able to send them as and when.

I have tried (unsucessfully) to understand the TCPEcho monkey code, but it just isn't making sense to me - and I didn't think I was quite so dim :(

As I'm not asking for anything really taxing - just the ability to send/receive a simple line of text - Could someone please point me in the right direction?

I have tried looking at Transport, but that really confuses me, and I can't get the Regal module to work at all. I would probably be much happier using the built-in brl sockets module, as I understand this to be the easiest way to use it on multiple platforms.

Any help would be graciously received!


Kaltern(Posted 2016) [#2]
After coffee and sleep, I seem to be understanding how the Regal networking module works, and I think will probably suit my needs well. I tried using Transport, but I seem to get a lot of garbage data, and I suspoect this system may not be ideal for me.

So, I'll go away and not mention Networking again :P


k.o.g.(Posted 2016) [#3]
What are you not unterstand with the TCPEcho example? Explain your problem and we / me can help you :D


Some simple Steps:
- You need a Receive Buffer
- Make all task async
- to send data over network, you need your data in a DataBuffer object


ImmutableOctet(SKNG)(Posted 2016) [#4]
Hi. Please don't feel as though your topics or posts aren't valid because there's been previous threads about the subject. If you have problems with my modules, or simply want to drum up a discussion about something, there's nothing wrong with that. Similarly, if you're having trouble understanding something, don't feel bad about it, that's what we're here for.

Anyway, from your specific description, you're definitely looking for 'regal.transport' over 'regal.networking', as you may have already guessed. 'regal.networking' is not a low-level transfer API, it has its own protocol behavior and formatting on top of the transport scheme. Supporting this "specification" in another language is asking for headaches. Plus, I might end up optimizing it later down the road, anyway.

The 'regal.transport' module doesn't have this bloat, but that also means it isn't as "feature rich". For raw transfers like this, it's perfectly fine, though. 'regal.transport' also has no dependencies other than the official modules, so it's well suited for this kind of thing. If you're still interested in using raw sockets instead of 'regal.transport', there's nothing stopping you, though.

So, you said you were getting "garbage data", what are we talking about here? I imagine the connection was made just fine on the Visual Basic side of things, and the Monkey code reflects this? If not, be sure the "Basic_IO" Monkey example is working correctly before anything else. The module's not perfect, it could be on my end.

Assuming that it's making the connection, I need to know what you're doing to read and write. Are you reading lines from the Monkey client, or are you sending raw data like integers or floats? I need a bit more context. Maybe you could post the source code, or a suitable test program, so I could take a crack at it? I'm a bit rusty with VB, but I could probably help debug it.

P.S. If you have any questions about installing the Regal Modules, feel free to ask me here. For 'regal.transport', it should be installed in "modules/regal/transport", where "modules" is your preferred/configured modules-directory. With that said, 'regal.transport' is entirely portable, so you could install it however you like.

I'll be going to bed shortly, so it may take a while to get back to you. (It's 3:00 in the morning here)


Kaltern(Posted 2016) [#5]
Hi guys thanks for responding :)

I am about to go cook a delicious steak and chicken dinner, so I'll eat that and come and try and explain my issue (bear in mind I've been away from anything remotely resembling code for 2 years!)


ImmutableOctet(SKNG)(Posted 2016) [#6]
@Kaltern: Just a heads up, it's a decent hour over here, so once I get a few things done I'll be available to help.

Speaking of chicken, I've got to throw some in the crock-pot myself, shortly.


Kaltern(Posted 2016) [#7]
Right, let me see if I can explain exactly what I'm trying to achieve,, and the issues I'm facing.

VB server code seems to be working, I can send/receive TCP messages with no issues that I am able to tell, so just for the moment, lets assume that's just fine.

The sequence of events for the game I'm writing would be as follows.

Player clicks 'Start'
Game sends message to server in form of a split string (for example: "#|123.234.32.23|LOGIN")
Server reads message, does some backend database stuff and sends back a message "!|ACK|19384"
Game then uses this info for various game-related things, and then sends back another message such as "?|19384" (I'm oversimplifying everything, but this is roughly how it'll work)
The game then waits for the server to send a gamestart message.
The game then sends a request to the server.
The server then send a response to this request.
The game acts upon this response, and waits for the player.  In the meantime, more server messages may come in without any further requests, which the game would act on accordingly.
This cycle continues until game end.

Now, I already have the server side working to send and receive messages sent from Packet Sender, so I know that code is working.
I am just having a lot of problems wrapping my head around the easiest way to implement this into Monkey. Using the TCPEchoServer example, I honestly cannot separate the echo from receiving external data.
Transport seemed to be working, but it seems the databuffer was not clearing after each message and leaving garbage data in the buffer. Unfortunately, I'm not clever enough to figure out how to close/clear the buffer.

Thats pretty much it I think, but I'll try to explain in more detail if needed.

And my steak and chicken was lush :P Although crockpot cooked chicken... mmmm now that would be nice too...


Kaltern(Posted 2016) [#8]
This is the server code I'm using curently:

Const ADDRESS:String = "127.0.0.1"
Const PORT:= 5029'

Method Reset() ' this is what is called at the beginning of each game to reset all variables etc
...' irrelevant code...

			CreateContainers()
			Host = New Server(PORT, Self) ' Server.PORT_AUTO
			OpeningServer = True
End

Method AddClient:Client()
	Local C:= New Client(ADDRESS, Host.Port, Self)
	Clients.AddLast(C)
	Return C
	End
	
	Method SendMessage(msg:string, frm:NetworkUser)
		Local P:= Host.AllocatePacket()
		P.WriteString(msg)
		Host.Send(P, frm)
		
		Host.Free(P)
		SendFromHost = Not SendFromHost
	End

	Protected
	
	Method CreateContainers:Void()' 
		
		Clients = New List<Client>()
		Users = New List<NetworkUser>()
		Return
	End
	
	Method CanSwitchParent:Bool(CurrentParent:NetApplication, NewParent:NetApplication)
		Return False ' True
	End
	
	Method OnPacketReceived:Void(Data:Packet, Length:Int, From:NetworkUser)
			
		Local Message:= Data.ReadLine()
		If (Message <> LatestMessage) Then
			
			Print("Message contents:")
			Print(LatestMessage)
			
			If LatestMessage = "LOGIN" Then
				SendMessage("ACKNOWLEDGED", From)
			EndIf
			
			LatestMessage = Message
			Message = ""
		EndIf
		Return
	End
	
	Method OnUnknownPacket:Void(UnknownData:DataBuffer, Offset:Int, Count:Int)
		Print("Unknwon packet data found: " + Offset + ", {" + Count + "}")
		Return
	End
	
	Method OnServerBound:Bool(Host:Server, Port:Int, Response:Bool)
		OpeningServer = False
		If (Not Response) Then
			Print("Failed to bound server socket on port " + Port + ".")
			Return False
		Endif
		
		Print("Server socket bound on port " + Port + "; everything checks out.")
		Print("Attempting to connect a 'Client'...")
		AddClient()
		Return True
	End
	
	Method OnServerUserAccepted:Bool(Host:Server, User:NetworkUser)
		Users.AddLast(User)
		
		' Always agree to more potential clients.
		Return True
	End
	
	Method OnClientBound:Bool(C:Client, Port:Int, Response:Bool)
		If (Not Response) Then
			Print("Failed to bind client socket on port " + Port + ".")
			
			Return False
		Endif
		
		Print("Client socket connected to port " + Port + ".")
		
		Print("Allowing messages...")
		
		' Tell 'C' to accept messages.
		Return True
	End


I mainly just copied all the methods from the example IO code, and modified to be more what I require. I think that the adding clients code is probably not needed, as there will only ever be one 'client' which would be the server.

When I run this, the game listens for connections as expected, and when it receives data, it receives a garbled version of it. If I resend something, it shows the previous data, and other garbage characters. I'm pretty sure I just need to flush the databuffer somehow, but I'm afraid I'm pretty poor with this whole stream code :(


Kaltern(Posted 2016) [#9]
Hmm, I just discovered that I forgot to add a '\n' to the end of my packetsender data, that seems to solve that immediate problem.

SO I can currently receive and send from the game, the only thing puzzling me now is how to initially send to the server - I know it uses the
P.WriteLine(" FROM: CLIENT")

For Local C:= Eachin Clients
   C.Send(P)
Next
code, but I don't know how to initally open a connection to an external server. I think if that can be solved, I'll be good to go.

EDIT: So steak and chicken and chips seems to help the brain I think. Finally realised I was trying to bind 2 servers to 1 port on localhost - which obviously doesn't work :| So I think I have it working now, it's probably cumbersome and inefficient as I'm using code designed for multiple clients when I only need one, but it works.

I used the code exactly as above, except for Addclient, I simply added the address/port for my VB server, which seems to be working.


Kaltern(Posted 2016) [#10]
Ok, so from Monkey I cannot send an initial message. I can connect to the server, but I can't figure out how to use the P.Send() method to send. Whenever I use the command Local P:= Host.AllocatePacket(), I get a memory allocation error, which points to packetmanager.monkey. I'm guessing it's because I'm not actually acting as a host that it is causing this particular issue.


ImmutableOctet(SKNG)(Posted 2016) [#11]
Hey, I'm writing an example to showcase the two classes ('Server' and 'Client') as individual programs. This should help you understand how to apply them to this situation. It'll be done shortly, and when it is, I'll post a link to it here.

For packets, though; both 'Server' and 'Client' classes work as packet managers/pools. You can find the 'PacketManager' class in "regal/transport/packetmanager.monkey", if you'd like to handle them separately. With that said, however, you really don't need to do that here.

From any 'Client' object, you can use the 'AllocatePacket' method to allocate a packet handle, and the 'Free' method to release the handle. This behavior isn't specific to the 'Server' or 'Client' classes, they both can do this.

When a packet handle is released, it gives the module the right to restore the object whenever it deems necessary. This means that if you send synchronously ('Send'), the application will be blocked the needed amount to ensure the connection is stable, then to copy the data, then give you back control of the 'Packet'.

So, if you're using 'Send', you just need to ensure you use your object's 'Free' method (Or manage 'Packet' objects yourself). Once again, the object in question can be a 'Client' or 'Server' object.

Now, the behavior I just described is in contrast to 'SendAsync', which is a contract to the object ('Client' or 'Server') that you will no longer use the 'Packet' object in question. This means if you call 'SendAsync', you're saying that you won't call 'Send', 'SendAsync', or a similar method on that 'Packet' object again. Of course, you could always write and allocate new 'Packet' objects while the other one is in transit, but the original 'Packet' handle is no longer valid.

This is why the 'Basic_IO' example uses 'Send' instead of 'SendAsync', because it's sending to many end-points ('NetworkUser' handles) one after another. If it was doing this asynchronously ('SendAsync'), it would be required to allocate a new 'Packet' object for each 'NetworkUser' it sends to. Though that may be effective in some situations, in the example, we can just send to everyone synchronously, cutting out ownership problems.

My 'regal.ioutil.repeater' module may be useful if you intend to write to multiple 'Packet' objects at the same time, by the way. It's a default module if you have the others installed. It's not directly related to 'regal.transport', though.

At any rate, to restore a 'Packet' object's internal position, simply call the 'Reset' method. Calling the standard 'Seek' method will also work with this class. This is done automatically if you use 'Free' and 'AllocatePacket', so you won't have to worry about the object when using those. Just keep the 'SendAsync' behavior in mind (Don't call 'Free' on an asynchronous 'Packet').

Sorry for the late response, I had to look through the code and remember what I did when I wrote this. The example I mentioned before will be done shortly. It's not going to be grandiose, but it should help.


Kaltern(Posted 2016) [#12]
I think splitting the Client and Server examples will be enormously helpful!

I can pretty much get my head around the code. I think seeing the aforementioned examples will help my poor brain understand what I'm doing wrong, as I'm obviously mixing up Client and Server stuff to achieve the same result.

Thanks for doing this, my head would seriously implode without some external help - and I thought I was reasonably switched on :P


ImmutableOctet(SKNG)(Posted 2016) [#13]
Whew, that took forever to do. I also managed to add to the API a bit, making disconnections a real thing and all that fun stuff.

You can find the new examples here, along with the updates required in the repository.

To make things simpler, just click here to download everything.

I hope this helps you understand the API better. The examples are pretty well commented, so it shouldn't be too hard to understand. If you have any other questions, or if you find a bug, please let me know.

P.S. I forgot to mention this before, but because this uses the official 'DataStream' implementation, that class's version of the 'Eof' method isn't effective for this kind of usage. I made 'EndOfPacket' to help with this issue.


Kaltern(Posted 2016) [#14]
Fantastic stuff, I shall go absorb it and hopefully everything will become clear :P

Thanks for doing that, I've no doubt it'll benefit many others too :) I'll report back once I've played about with it all!


Kaltern(Posted 2016) [#15]
So I've had a play and a read and it all makes pretty good sense to me now. The thing I was failing to take into account was ensuring that the data connection was open before I sent anything, and using the WhileConnected() method allowed me to use that to do all of my network processing in that one place.

I think I'm good to go and actually finish the game I started about 2 years ago now! Thanks for your help, it was invaluable!