Sharing memory between BlitzMax processes?
BlitzMax Forums/BlitzMax Programming/Sharing memory between BlitzMax processes?
| ||
So, what I'm trying to do is to do some load balancing for a CPU-intensive game. Shared memory would be my method of choice to perform some tasks in parallel, as BlitzMax lacks the ability to do real multithreading. Here's the deal: I have a game project with potentially a lot of background calculation going on at all times, stuff that doesn't need to be calculated all the way through every single frame. Currently I have implemented a crude way to spread the workload between frames: go through a small part of an array during every frame and let the game run while a large amount of data gets processed during a period of 100 or so frames. I'd love to see a way to do it in a more elegant way, despite the fact I've never accomplished anything similar in any programming language. I need some advice on how to spawn another process that takes its "work orders" from the main game process through memory buffers, chews on the the data on its own, and finally returns the results back to the main game through an output buffer. Or something like that. Is there any way to create and access shared memory areas between two processes? |
| ||
Yes although it wouldnt be cross platform what platform are you interested in? Doug Stastny |
| ||
Ah, it would be Windows XP. |
| ||
Memory is shareable between threads but not between processes. You will have to dublicate it or keep as much as possible dynamic and sent it back and forth through UDP. |
| ||
Memory is shareable between threads but not between processes Totally untrue. Memory is ALWAYS shared between threads, it is BY DEFAULT not shared between processes, but can be EXPLICITELY made shared betwwen processes.For this you can use either file mappings, or shared segments. From BlitzMax your best bet is to use file mappings. A file mapping (Windows specific) lets you access to a shared area in the virtual memory space. Type "createfilemapping" in google and from there you should find the needed information. As Budman said, that's totally platform-specific. |
| ||
memory is shareable but i dont know the trick . (winhex/rameditor) |
| ||
Ding Ding Ding! Koriolis gets the prize :) Doug Stastny |
| ||
Ill be nice here is some code ' Simple program to demonstrate sharing memory between process ' compile as console program and run one or more times the first run sets the ' shared memory ' additonal runs read it SuperStrict Extern "Win32" Function CloseHandle:Int(Handle:Int) Function MapViewOfFile:Byte Ptr(hFileMappingObject: Int,dwDesiredAccess: Int, dwFileOffsetHigh:Int, dwFileOffsetLow:Int, dwNumberOfBytesToMap:Int) Function UnmapViewOfFile:Int(lpBaseAddress: Byte Ptr) Function CreateFileMappingA:Int(hFile: Int, lpFileMappingAttributes: Byte Ptr,flProtect:Int, dwMaximumSizeHigh:Int, dwMaximumSizeLow:Int, lpName$z) Function GetLastError:Int() End Extern Const PAGE_READWRITE : Int=4 Const FILE_MAP_WRITE : Int=2 Type TSharedMemory Field _Handle : Int Field _Name : String Field _Size : Int Field _Owner : Int Field _Data : Byte Ptr Method Delete() Close End Method Method Open:Int(Name: String, Size: Int) Close() _Name = Name _Size = Size ' CreateFileMapping, when called with $FFFFFFFF For the hanlde value, creates a region of shared memory } _Handle = CreateFileMappingA($FFFFFFFF, Null, PAGE_READWRITE, 0,_Size,_Name) If _Handle = 0 Then Return False _Owner = GetLastError() = 0 ' We still need To map a pointer To the handle of the shared memory region _Data= MapViewOfFile(_Handle, FILE_MAP_WRITE, 0, 0, _Size) If Not _Data Close Return False End If Return True End Method Method Close() If _Data UnmapViewOfFile(_Data); _Data=Null End If If _Handle CloseHandle(_Handle) _Handle=0 End If End Method End Type Print "Test Share Mem" Local mem:TSharedMemory = New TSharedMemory mem.Open("TESTMEM",30) If mem._Owner Print "Setting Title" MemClear mem._Data,30 Local title:String="Budman Rocks" Local pTitle:Byte Ptr=title.toCString() MemCopy mem._Data,pTitle,title.length+1 MemFree pTitle Else Print "Getting Title" Local title : String = string.FromCString(mem._Data) Print "The Title is "+title End If Input mem.Close Have fun Doug Stastny |
| ||
Aren't you basically asking to be able to provide a multitasking feature within your single process that uses as much `leftover time` as is available for background tasks and doesn't get in the way of the important stuff you need to accomplish? Why not just do the stuff you must get done at the start of the frame then with remaining time just do a loop where you do a bunch of background stuck and check the Millisecs() every now and then to see if the frame time has run out yet? |
| ||
There are somethings you cant with Timeslicing, due to the fact they block and you can not measure time slice. Which is why people want threading so bad. Examples. 1. Networking, you really want to handle that as fast as possible not wait and poll. 2. Database work. Lets say you want to save player states for a MUD to a Database, you can wait for your interaction for the database to complete, as you might timeout network connections to the players. 3. You want to connect your game/app to a internet server but still be responding to user in realtime. These are types of things that simple timeslicing cant resolve. Now if your doing everything in your own code time-slicing can be done, but been there done that, its a pain in the rear, to code. Me I would prefer threads however Shared Process memory can help a feature limited BMAX. Doug Stastny |
| ||
Everyone says max is lacking features when their little obscure feature happens not to be supported ;-) But anyway, yeah you are probably right. Threads would be nice especially to make use of the trend towards multiple core cpu's. |
| ||
Thank you all for your input, and thanks Doug for the snippet you provided (although I was prepared to do the research myself ;) ) The Windows API solution seems to work and is worth looking into. =) |
| ||
No problem Vilu, I just translated some delphi classes, I had used is a project. My only concern would be speed issues. Look forward to reading what you might do with this. Doug Stastny |
| ||
Hello, another possbile solution could be using anonymous pipes (windows). I used them (in Delphi) to send/receive info from different processes :-) --Alessandro |
| ||
@Budman Hi, I tried your source code a few days ago and converted it into my own class also included critical section support (Not threw API calls just a counter in the shared memory). It seemed to work realy well for a while but after like 40 runs of the program or more my computer started grinding to a hault and the application would randomly quit. It was made even worse when in debug mode. Perhaps this method is realy unstable in BlitzMax? I didnt change much from your source only added to it. I would post the code but it dont have it with me sorry. Just thought you mite be interested. |
| ||
@Pantheon Hello, Nothing really special about BMax and this code should cause a problem. I suspect if you dont call the close method and ensure you Free the Shared memory you will run into problems. You need to make sure that if the Application Terminates the Close Method has been called as that would leak API resources. My demo was really simple example, but I would use a Global Var for the Shared Memory section and write an Onexit handler to ensure that the close method is invoked. If you just call "END" in BMAX you will have problems. The one nuanices of these types of resources is they are avaialbe across processes so you need to be real sure your application cleans them up. Now your code might be faulting due to both procesess accessing the Shareed memory at the same time. For that it really needs a real OS MUTEX/CRITICAL Section to ensure only one process is accessing the Shared memroy at that same time. My example was trivally stupid and just to show it can be done. Its not really a full blown piece of code just the bare minimum. If you want to share your code, Ill take a look at it and see if I can see what might cause your problem. I suspect it needs OS Mutex. Doug Stastny |
| ||
When I get back to my computer then I will post some code. You say that the two processes accessing shared memory at the same time can cause error? That would be the most likely case because I implemented two functions, Lock( ) and Unlock( ). The shared memory size is increased by an integer and that becomes a variable to say how many times the memory has been locked. The lock call was blocking by using a while loop to check if memory was locked. I had a small delay in the loop and I guess I ran into trouble when I brought the delay time down. If you can make sense of that then it all seems to pan out. I'll put the code up as soon as just for peoples intrest. Thanks |
| ||
Hi good afternoon. I once changed your program. Write 1x filemapping Read 1x file mapping First, the listener then Start the evening program. greeting BlitzMax-Filemapping-Write-Integer: SuperStrict Graphics 300,200 Global timer:TTimer = CreateTimer(60) global wert:int Extern "Win32" Function CloseHandle:Int(Handle:Int) Function MapViewOfFile:Int Ptr(hFileMappingObject: Int,dwDesiredAccess: Int, dwFileOffsetHigh:Int, dwFileOffsetLow:Int, dwNumberOfBytesToMap:Int) Function UnmapViewOfFile:Int(lpBaseAddress: Byte Ptr) Function CreateFileMappingA:Int(hFile: Int, lpFileMappingAttributes: Byte Ptr,flProtect:Int, dwMaximumSizeHigh:Int, dwMaximumSizeLow:Int, lpName$z) Function GetLastError:Int() End Extern Const PAGE_READWRITE : Int=4 Const FILE_MAP_WRITE : Int=2 Type TSharedMemory Field _Handle : Int Field _Name : String Field _Size : Int Field _Owner : Int Field _Data : Byte Ptr Method Delete() Close End Method Method Open:Int(Name: String, Size: Int) Close() _Name = Name _Size = Size ' CreateFileMapping, when called with $FFFFFFFF For the hanlde value, creates a region of shared memory } _Handle = CreateFileMappingA($FFFFFFFF, Null, PAGE_READWRITE, 0,_Size,_Name) If _Handle = 0 Then Return False _Owner = GetLastError() = 0 ' We still need To map a pointer To the handle of the shared memory region _Data= MapViewOfFile(_Handle, FILE_MAP_WRITE, 0, 0, _Size) If Not _Data Close Return False End If Return True End Method Method Close() If _Data UnmapViewOfFile(_Data); _Data=Null End If If _Handle CloseHandle(_Handle) _Handle=0 End If End Method End Type Local mem:TSharedMemory = New TSharedMemory mem.Open("MyFileMapping",30) Repeat cls delay(300) wert=wert+100 if wert > 20000 then wert=0 drawtext "write integer : "+ wert,10,10 MemClear mem._Data,30 Local pTitle:Byte Ptr=varptr wert MemCopy mem._Data,pTitle,4 MemFree pTitle Flip WaitTimer(timer) Until KeyHit (KEY_ESCAPE) mem.Close end BlitzMax-Filemapping-Read-Integer: SuperStrict Graphics 300,200 Global timer:TTimer = CreateTimer(60) global wert:int global text:string Extern "Win32" Function CloseHandle:Int(Handle:Int) Function MapViewOfFile:int Ptr(hFileMappingObject: Int,dwDesiredAccess: Int, dwFileOffsetHigh:Int, dwFileOffsetLow:Int, dwNumberOfBytesToMap:Int) Function UnmapViewOfFile:Int(lpBaseAddress: Byte Ptr) Function CreateFileMappingA:Int(hFile: Int, lpFileMappingAttributes: Byte Ptr,flProtect:Int, dwMaximumSizeHigh:Int, dwMaximumSizeLow:Int, lpName$z) Function GetLastError:Int() End Extern Const PAGE_READWRITE : Int=4 Const FILE_MAP_WRITE : Int=2 Type TSharedMemory Field _Handle : Int Field _Name : String Field _Size : Int Field _Owner : Int Field _Data : int Ptr Method Delete() Close End Method Method Open:Int(Name: String, Size: Int) Close() _Name = Name _Size = Size ' CreateFileMapping, when called with $FFFFFFFF For the hanlde value, creates a region of shared memory } _Handle = CreateFileMappingA($FFFFFFFF, Null, PAGE_READWRITE, 0,_Size,_Name) If _Handle = 0 Then Return False _Owner = GetLastError() = 0 ' We still need To map a pointer To the handle of the shared memory region _Data= MapViewOfFile(_Handle, FILE_MAP_WRITE, 0, 0, _Size) If Not _Data Close Return False End If Return True End Method Method Close() If _Data UnmapViewOfFile(_Data); _Data=Null End If If _Handle CloseHandle(_Handle) _Handle=0 End If End Method End Type Local mem:TSharedMemory = New TSharedMemory mem.Open("MyFileMapping",30) Repeat cls If mem._Owner wert = mem._Data[0] drawtext "read integer : "+wert,10,10 End If Flip WaitTimer(timer) Until KeyHit (KEY_ESCAPE) mem.Close end |