Memory leak with String[]? (reposted)
Monkey Forums/Monkey Bug Reports/Memory leak with String[]? (reposted)
| ||
| Got some legacy code along these lines: Local file:String[] = app.LoadString(filename).Split("~n") file = file[1..] 'Skip header For Local dataline:String = EachIn file If dataline.Contains("//") Then Exit Local objstr:String[] = dataline.Split("~t") If objstr[0] = objid Return CreateMyObject(objstr) End Next Not especially pretty, but I've found that every time it's called, allocated memory increases by around 2mb and never goes back down. I'm aware that GC only kicks in every 8mb, but if I set the code to manually trigger by tapping the screen, I can tick up allocated memory indefinitely until the device crashes. I've checked in the profiler and all the memory is being allocated by Split/Slice. Anyone else encountered this? NB: the contents of objstr are NOT being retained by the CreateMyObject function, which is in any case only invoked once per call and could only ever retain a reference to a handful of bytes. Update: I changed the loop to only split the line that matched the id - no change in behaviour. I removed the 'eachin' and replaced it with a simple numerical counter - no change in behaviour. |
| ||
| Can you post some runnable/testable code? |
| ||
| Ok, think I'm getting a handle on what's going on here. The short version is: string rep allocation bypasses the GC. The long version: Loading the string and Split()ing it allocates ~3mb of string rep data handled by a single string array. The memory for the string reps is directly malloc'd, NOT obtained through the GC, so the GC does not realise memory is being consumed. gc_new_bytes does not get increased (apart from the allocation of the array object), so if all you're doing is allocating string arrays, it can be a LONG time before GC is triggered. Meanwhile, hundreds of megs of memory can get tied up in string reps, and the game can crash. If you're very lucky, other allocations will trigger GC (putting the arrays on the free list) AND request enough memory afterwards for gc_flush_free to work its way through and free up the arrays. In practice, that rarely happens. Instead, other things from the free list get deleted to make room for another 3MB string array the GC thinks is only a few bytes in size. Technically, it's not a memory leak, because everything is accounted for. It's just that the GC's idea of the memory situation can get very out of step with the actual situation. |
| ||
To see this happening, create a blank mojo game with a large text file in the data folder, and add this line:
Method OnUpdate()
Local file:String[] = app.LoadString("bigfile.txt").Split("~n")
End
Run this and watch TaskManager. Memory consumed will tick up until the GC starts freeing off array objects, then it stabilises. Edit: On reflection, I'm not sure why things are worse in a real application than that test example. The worst case OUGHT to be a solid 8mb of string array objects, as in the example above, where they all get GC'd to the 'free' list and then the top one gets recycled every update. However, it's definitely possible to exceed the memory bloat of that test app in a real game, and I don't see how... Duh. I forgot that when I clicked on the task manager, updates would stop and memory wouldn't go up any further. I thought it stabilised at 130mb but it actually caps out at 570-ish, which matches our experience in-game loading and splitting the exact same file. |
| ||
| Ok, there's an experimental fix for this now up at github... https://github.com/blitz-research/monkey/blob/develop/modules/monkey/native/lang.cpp Seems to work here - can you verify? |
| ||
| I'll pull that version in as soon as I can. On lockdown at the moment for submission. |
   |