Rare behavior?
BlitzMax Forums/BlitzMax Programming/Rare behavior?
| ||
| Hi I don't sure where to post this message, here or in bug section, so I decide to put it here. I have this code:
SuperStrict
SetGraphicsDriver D3D9Max2DDriver()
Graphics 800,600
Global fullScreen:Int = False
Global img:TImage = LoadImage("test.png")
Global list:TList = New TList
Type entity
Field x:Float
Field y:Float
Field _link:TLink
End Type
For Local i:Int = 1 To 400
Local e:entity = New entity
e.x = Rand(1, 800)
e.y = Rand(1, 600)
e._link = list.addlast(e)
Next
While 1
Cls
For Local e:entity = EachIn list
DrawImage(img, e.x, e.y)
Next
If (KeyDown(KEY_SPACE))
For Local e:entity = EachIn list
e._link.remove()
e = Null
Next
list = New TList
GCCollect()
End If
If (KeyDown(KEY_ESCAPE))
End
End If
If (KeyDown(KEY_LALT) And KeyDown(KEY_ENTER))
If (fullScreen = True)
Graphics 800,600
fullScreen = False
Else
Graphics 800,600, 32
fullScreen = True
End If
End If
Flip
Wend
Note: test.png is any png image with 32x32px Here I have two situation First: I create 400 images, and put it on a Tlist, then draw all the images in Tlist, so when I press space all the images must be erased. I see the RAM occupied by the executable using window Task Manager and then press space, and I no see the memory goes down, indeed the memory rise up a bit. Is some thing wrong or this behavior is normal? Second: On windows 7 using directx9 graphic driver when I press the Alt+Enter keys combination the program change from windows mode to full screen without problem but the memory goes up considerably and that happen each time I change the resolution. This not happen if I use directx7 graphic driver. I'm doing wrong the change screen procedure? All the test are in release mode. I'm using the last BMax version. Thanks. |
| ||
| hmm usually i don't go as far as using the OOP commands for working with lists, i'm quite satisfied using Clearlist and listremove. but i just tried your code and noticed there is a tiny increase in memory after removing all the types from the list. |
| ||
| This looks fine to me and seems like the garbage collector is grafting as it should. I've changed it a little, press A to set everything up, then Space to clear the lot, or keep on hitting A then Space (Or even just hit A repeatedly) and you'll see (well here it does) the memory moves a bit, but after a few goes it calms down to a steady fluctuation, with no apparent 'leak', which would be more worrying, but alas, all seems rosey:-
SuperStrict
SetGraphicsDriver D3D9Max2DDriver()
Graphics 800,600
Global fullScreen:Int = False
Global img:TImage
Global list:TList = New TList
Type entity
Field x:Float
Field y:Float
Field _link:TLink
End Type
While 1
Cls
If img <> Null
For Local e:entity = EachIn list
DrawImage(img, e.x, e.y)
Next
End If
If KeyHit(KEY_A)
For Local e:entity = EachIn list
e._link.remove()
e = Null
Next
img = Null
list = New TList
GCCollect()
For Local i:Int = 1 To 400
Local e:entity = New entity
e.x = Rand(1, 800)
e.y = Rand(1, 600)
e._link = list.addlast(e)
Next
img = LoadImage("D:\backup\Blitz\NagsDarts_RC1_0\Media\gfx\180.png")
End If
If (KeyDown(KEY_SPACE))
For Local e:entity = EachIn list
e._link.remove()
e = Null
Next
list = New TList
img = Null
GCCollect()
End If
If (KeyDown(KEY_ESCAPE))
End
End If
If (KeyDown(KEY_LALT) And KeyDown(KEY_ENTER))
If (fullScreen = True)
Graphics 800,600
fullScreen = False
Else
Graphics 800,600, 32
fullScreen = True
End If
End If
Flip
Wend
Thing is you've got to remember that the GC sometimes doesnt instantly clear the resources, so you'll probably witness memory building up then snapping back down to a steady rate. Dabz |
| ||
| @Dabhand(givesup) I think the problem is not to free the resource var, but the memory size of 400 created entities. I monitor the memory of the program for 10 minuts after make the entity delete procedure and the memory never goes down from original size. |
| ||
| The GC could be allocating blocks of memory and retaining them? Does the memory stay constant even when re-allocating more objects, as this could indicate the GC is reusing previously allocated memory? |
| ||
| I'm noticing a retention problem in my project as well. I haven't touched the areas concerned in a while (maybe a month...), but I have been updating bmax. I noticed 2 days ago that my threaded loading routine doesn't properly garbage collect any more. If I call GCCollect() it frees properly, but auto collecting doesn't clean up, my memory builds and builds until it locks up. This is on Mac btw. I've tried rolling some modules back but it takes so long to re-compile everything on my system that I just don't have time to roll and recompile a million versions to try to figure out where/when this started. Further I know the objects are being removed (I put a print in their delete emthod) and that by extension they are calling delete on their memory blocks... it just hangs around. It does go up and down a little indicating that SOME collection is occuring but without a manual call to GCCollect() it will build and build... |
| ||
| Taking another look into the GC it seems for me atleast on mac that the alloc size test is failing in the auto collect routine (gc_alloced-alloced)>heap_size/3 line 207 in blitz_gc_ms.c |
| ||
| Looking into a bug/delay in my latest game, it records data as the game progresses and then uses the data to show the user graphs at the end of each turn, it uses a very simple timestamp object to record a value of a parameter in a list. Now in theory the game (Save The Silly Earthlings [Alpha]) could end up storing 30 pieces of info for every hour in 100 years (or about 26,298,000 elements). Anyway though the slowdown could be caused by the current Alpha version clearing out 5 years worth of data (about 5m elements). Stress testing that now and it would appear that the game in single threaded mode is not GC'ing the cleared data, just using a simple Nulling of the lists at the moment? Windows Vista Ultimate 64bit! |
| ||
SuperStrict
While(Not AppTerminate())
Delay(10)
Local pixm:TPixmap = LoadPixmap("sample2.png")
wend
give that a large (multi-meg) png or jpeg to load and watch memory usage... in threaded or non threaded compile I watch the memory go up up up, it goes down tiny bits here and there but generally it's steadily climbing. since the pixmap is local it's getting released every loop... if I use a small png or jpeg (under 100kb) it will climb to between 11 and 54mb before clearing back down, which is the normal cycle for the GC... I don't know what it means but it explains why I'm getting climbing to death in my project since I'm mainly working with large pictures. IMPORTANT note, if you use FreeImage, which overrides the loadpixmap (or if you load image instead of using loadpixmap), OR if you load freeimage objects instead of strait pixmaps you get the same behavior, so it isn't loadpixmap per-se. Can anyone else confirm they see the same climbing of memory as me? have only tested on my mac so far. [update] same results under windows for me, just grows and grows. also loading on child threads at the same time causes all kinds of havok, here's a sample to play around with, just toggle some comments on and off to see how it goes wrong. loading JUST in a child thread just causes the same memory climb though.
SuperStrict
'Import bah.freeimage
Function tload:Object(in:Object)
While(True)
Local pixm:TPixmap = LoadPixmap("sample2.jpg")
Delay(10)
Wend
End Function
'CreateThread(tload, "")
While(Not AppTerminate())
Delay(10)
Local pixm:TPixmap = LoadPixmap("sample2.png")
'Local image:TImage = LoadImage("sample2.jpg")
wend
|
| ||
| Wierd behaviour here, Build 30 lists of 5*24*365 data objects containing an int and float, Release build single threaded (ST)... Creates the 30 lists filling with random values in 465 ms. * Clearing the 30 lists using list[i].Clear() takes 3695 ms * for the first pass, subsequent passes take about 200 ms? Ditto for debug (ST), the first clearing takes 10+ times longer? Odd no? |
| ||
| It appears to be faster in this test case using Multi-threading (MT) and the first clearing pass is as fast as all subsiquent passes? |
| ||
| Can anyone else confirm they see the same climbing of memory as me? have only tested on my mac so far. [update] same results under windows for me, just grows and grows. Does the memory build, spike, then drop off, and I mean, take a load before it dumps and simmers at a decent (expectant) level? Dabz |
| ||
| Further reading:- http://www.blitzbasic.com/Community/posts.php?topic=89969#1023084 Which in laymens terms mean, if your app goes should rocket memory, crash your OS, or run it to a crawl then yep, theres something wrong. If you can chaps, stress your apps and keep an eye on memory. Dabz |
| ||
| P.S. systems specs go a long way to! ;) Dabz |
| ||
| Re dab: with a small picture it builds and then falls in a normal cycle (as apps in the past always have) with a large picture I think it tried but the peak before it would drop and reset seems too high... So it just builds till it runs out of ram and crashes. |
| ||
| Re dab: with a small picture it builds and then falls in a normal cycle (as apps in the past always have) with a large picture I think it tried but the peak before it would drop and reset seems too high... So it just builds till it runs out of ram and crashes. How large is the picture, can you post it? Dabz |
| ||
| 5.8mb sample2.jpg It's important to note though that this happens with any large picture, on mac, or PC, with or without using freeimage. This means it's not the file (any picture) it's not the OS (mac/win) and it's not the loading/unloading mechanism (freeimage/standard). This only leaves the garbage collector not auto collecting properly, which can be demonstrated by doing a manual GCCollect(), this causes the memory to stay low. So the collector CAN collect it. I suspect it would auto collect it eventually but it allows it to rise to a point that it crashes before it tries to collect (this is based on comparing the behavior of a small picture to a large picture) |
| ||
| cpu time: 14 seconds, memory usage ~700mb. It is independend of the usage of "globals" or locals...nulling is useless too. Watching my system monitor, it shows a memory usage decrease for one time...after 7-8 seconds, afterwards the memory climbs until even swap file is excessively used. This just happens if the picture is a bit smaller (width 1280, your example rotated clockwise by 90°) but doesn't happen every run. The larger picture has a linear increase of memory over time ... so NO garbage is collected. Compiled under ubuntu lucid, bm 1.36 and up edit: just resaved the jpg as tga and voila... gc seems to work again, so i think it has to do something with the image loader not the collector. bye MB |
| ||
| I've tried it with png with the same result. Also note that if you force a garbage collect it doesn't climb forever... it's that the auto collect isn't getting triggered. I put a printf in the auto collect to watch it and it doesn't get called due to a failed test (see above regarding line 207), if you comment out the test so it always auto collects it won't climb. (again because it's always collecting). And again to note, this happens if you use freeimage as well, which replaces the standard load routines with it's own... interesting that you're getting different results with a TGA however... how does a small TGA compare to a small JPEG? do they both cycle the same, or does the JPEG have a larger swing in it's collect cycle? |
| ||
| Maybe this is related http://www.blitzbasic.com/Community/posts.php?topic=91360 Looks like a leak but followup looks like it stabilizes after enough iterations... |
| ||
| I have had let the code above run for about 30 seconds... after that i had my ram 100% used, swap getting bigger and bigger. Using TGA-Files the GC seemed to work flawlessy, no rising memorycurve in sys monitor. PNG and Jpeg are both highly compressing the images, tga is using a kind of lzha/zip mechanisms. bye MB |
   |