Efficient method for pinpointing memory leaks
BlitzMax Forums/BlitzMax Programming/Efficient method for pinpointing memory leaks
| ||
| Hi guys, What's the best and most efficient way of pinpointing memory leaks with multithreaded BlitzMax apps? (and one that will work on Mac OS X as well) It seems all the tricks I learned in academia (e.g. use Valgrind, use Instruments) have proved fruitless. The ultimate problem being those programs work best when they can access the C-level source, or at least all the code/compiler symbols. I get no useful inspection when I analyze a BlitzMax binary running in release, and the program doesn't even really work when running in debug. Additionally, using Valgrind on Snow Leopard barely even works, or at least I've encountered problems running binaries through it. (Maybe you know how to?) Using Instruments while running my BlitzMax/MiniB3D app just causes it to hang whenever my app garbage collects --- which is similar to how the GCCollect() currently is slow even in a typical run (it's just super super slow when using Instruments/MallocDebug). It'll show a few leaks here and there, but I can't understand what code the culprits are referring to. For example, MallocDebug will output the following: Identifier: main.debug.mt Version: ??? (???) Code Type: X86 (Native) Parent Process: bmk [33846] Date/Time: 2012-02-28 03:02:45.640 -0800 OS Version: Mac OS X 10.6.8 (10K549) Report Version: 7 leaks Report Version: 2.0 Process 33857: 52567 nodes malloced for 1176314 KB Process 33857: 2 leaks for 1104 total leaked bytes. Leak: 0x1875800 size=1024 zone: DefaultMallocZone_0xafc000 0x00000000 0x00000000 0x00000000 0x00000000 ................ 0x00000000 0x00000000 0x00000000 0x00000000 ................ 0x00000000 0x00000000 0x00000000 0x00000000 ................ 0x00000000 0x00000000 0x00000000 0x00000000 ................ 0x00000000 0x00000000 0x00000000 0x00000000 ................ 0x00000000 0x00000000 0x00000000 0x00000000 ................ 0x00000000 0x00000000 0x00000000 0x00000000 ................ 0x00000000 0x00000000 0x00000000 0x00000000 ................ ... Leak: 0xc44c40 size=80 zone: DefaultMallocZone_0xafc000 sound C++ main.debug.mt 0x0075f748 0x018e3600 0x00000001 0x00000000 H.u..6.......... 0x00000000 0x00000000 0x2d2d2d31 0x3e544d3c ........1---<MT> 0x6f4c205d 0x64656b63 0x74754d20 0x00010000 ] Locked Mut.... 0x00010000 0x10001000 0x00000000 0x00000000 ................ 0x0000000b 0x0000000b 0x0000676e 0x00050005 ........ng...... The leak mentioned at the bottom of the dump (0xc44c40) I believe refers to one of my runtime logging methods (in fact, you can see part of the string for "Locked Mutex" one of the log messages the binary generates). From this alone though I find it difficult to deduce what that's from, aside from "perhaps String data is leaking...". I use Strings all over the place. And probably not correctly since they seem so lightweight, I typically don't think too much about their usage (now of course, more so). My program is very large, and I've done what I can to null out old memory, clear out old TLists, but obviously there is some stuff missing. I'm thinking I have more (string?ogg?) data trapped in TLists somewhere that I'm not aware of yet. Instead of grepping my code base for CreateList()s and New()s, what's a more strategic, "down to the function" way of identifying the source of leaks? Ideally something more than printing out GCMemAlloc()'s everywhere. Any ideas? Thanks a bunch Last edited 2012 Last edited 2012 |
| ||
| Haven't got a clue, but you say you're using Strings everywhere, in your BMax code are you using "MyString".ToCString() at all? as this will require a MemFree after you've finished with it. Last edited 2012 |
| ||
| Hi col, We're not using the ToCString() function, so my guess is that's not the issue. What I found out it was was two fold: a) The JSON parsing library we've been using takes a JSON-encoded string and creates a TMap out of it (with a bunch of other TMaps and TLists nested within it) b) BlitzMax in multi-threaded mode does not recursively clear out these data structures automatically during garbage collection, so I had to come up with some custom mechanisms to do so I ended up creating a Valgrind-like module for BlitzMax which I then hooked into brl.mod to help me count where each CreateList() / CreateMap() is occurring. I found a whole bunch I wasn't aware of (including some in MiniB3D where lists were getting created each frame). By implementing an object pool and replacing all direct calls to CreateMap() & CreateList(), the code now doesn't cause as much memory fragmentation which was occurring previously (ultimately resulting in malloc() failing). |
| ||
| Very interesting... this sounds like the basis of a threaded crash that's been haunting me for a long time now... do you have any points to look at relating to the GC failing to clear all structures in lists? Sounds to me like that's the failing clear in the collect calls that I've lost so much sleep/hair over... See bug report: http://www.blitzbasic.com/Community/posts.php?topic=91117#1037368 Last edited 2012 |
| ||
| @ima747: I pretty much don't rely on garbage collection to get rid of data stored in TMaps and TLists (in multithreaded apps). Before calling .clear() on the collection directly, I iterate through its contents and free up whatever it was holding. I'm 99% given up on multithreaded MiniB3D and have instead been exploring interprocess communication where I can separate the MiniB3D process and the MT process that handles background tasks because the GC in MT is just too wonky. It's become a long-term headache to debug GC-related gameplay performance issues. Have you had any success? Last edited 2012 |
   |