Possible bug in MT memory management.

Archives Forums/BlitzMax Bug Reports/Possible bug in MT memory management.

Corum

(Posted 2010) [#1]

Hi,
I'm working (very slowly) on a game project.
To speedup AI processing, I was considering to approach MT features.
My project is working perfectly in standard "single threaded mode".
As a start, I just recompiled all the mods in MT version and did a simple build of my sources - no changes at all - with MT feature enabled.
My dev environment is:
Leopard 10.5.8 - XCode 3.1.3 - BlitzMax 1.36

My program crashes, giving as output the following malloc double free error, for about 20 times:

Executing:main.debug.mt
main.debug.mt(824,0xa07be720) malloc: *** error for object 0x51ed60: double free
*** set a breakpoint in malloc_error_break to debug
...
...
...
main.debug.mt(8226,0xa07be720) malloc: *** error for object 0xc0000003: Non-aligned pointer being freed
*** set a breakpoint in malloc_error_break to debug

The Mac OSX crash report states the following:

Code Type:       X86 (Native)
Parent Process:  bmk [791]

Interval Since Last Report:          4619 sec
Crashes Since Last Report:           11
Per-App Interval Since Last Report:  0 sec
Per-App Crashes Since Last Report:   11

Date/Time:       2010-01-11 14:44:31.524 +0100
OS Version:      Mac OS X 10.5.8 (9L31a)
Report Version:  6

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000072657355
Crashed Thread:  0

Thread 0 Crashed:
0   ???                           	0x72657355 0 + 1919251285
1   main.debug.mt                 	0x0006d652 2219 + 11
2   main.debug.mt                 	0x0006ba9f 1832 + 8
3   main.debug.mt                 	0x001fded9 188 + 0
4   main.debug.mt                 	0x0020a00a bbObjectFree + 28
5   main.debug.mt                 	0x00204211 collectMem + 667
6   main.debug.mt                 	0x00203c96 allocMem + 182
7   main.debug.mt                 	0x00204539 bbGCAllocObject + 61
8   main.debug.mt                 	0x0020832d bbStringNew + 66
9   main.debug.mt                 	0x00208c8b bbStringSlice + 59
10  main.debug.mt                 	0x000039dd 19 + 22
11  main.debug.mt                 	0x00003c14 354 + 10
12  main.debug.mt                 	0x00004062 75 + 34
13  main.debug.mt                 	0x00004731 435 + 38
14  main.debug.mt                 	0x00004b55 128 + 0
15  main.debug.mt                 	0x00004d39 476 + 17
16  main.debug.mt                 	0x00005106 497 + 12
17  main.debug.mt                 	0x0007365d _ifsogui_gui_ifsoGUI_Base_Load9Image + 166
18  main.debug.mt                 	0x00069156 2056 + 121
19  main.debug.mt                 	0x000696b4 _ifsogui_scrollbar_ifsoGUI_ScrollBar_SystemEvent + 98
20  main.debug.mt                 	0x0007b970 5450 + 24
21  main.debug.mt                 	0x00006ba2 _bb_TGUIWrapper_InitGui + 424
22  main.debug.mt                 	0x0000566a 34 + 8
23  main.debug.mt                 	0x00002f3a 4 + 25
24  main.debug.mt                 	0x00003245 run + 82
25  main.debug.mt                 	0x0000347d -[BlitzMaxAppDelegate applicationDidFinishLaunching:] + 353
26  com.apple.Foundation          	0x9343042a _nsnote_callback + 106
27  com.apple.CoreFoundation      	0x90ec847a __CFXNotificationPost + 362
28  com.apple.CoreFoundation      	0x90ec8753 _CFXNotificationPostNotification + 179
29  com.apple.Foundation          	0x9342d680 -[NSNotificationCenter postNotificationName:object:userInfo:] + 128
30  com.apple.Foundation          	0x93436ed8 -[NSNotificationCenter postNotificationName:object:] + 56
31  com.apple.AppKit              	0x9376cdf2 -[NSApplication _postDidFinishNotification] + 125
32  com.apple.AppKit              	0x9376cd01 -[NSApplication _sendFinishLaunchingNotification] + 77
33  com.apple.AppKit              	0x936e681b -[NSApplication(NSAppleEventHandling) _handleAEOpen:] + 284
34  com.apple.AppKit              	0x936e6014 -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] + 98
35  com.apple.Foundation          	0x93455a9f -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] + 655
36  com.apple.Foundation          	0x934557af _NSAppleEventManagerGenericHandler + 223
37  com.apple.AE                  	0x94f4a648 aeDispatchAppleEvent(AEDesc const*, AEDesc*, unsigned long, unsigned char*) + 144
38  com.apple.AE                  	0x94f4a57e dispatchEventAndSendReply(AEDesc const*, AEDesc*) + 44
39  com.apple.AE                  	0x94f4a425 aeProcessAppleEvent + 177
40  com.apple.HIToolbox           	0x972bc981 AEProcessAppleEvent + 38
41  com.apple.AppKit              	0x936e38e9 _DPSNextEvent + 1189
42  com.apple.AppKit              	0x936e2f88 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 128
43  com.apple.AppKit              	0x936dbf9f -[NSApplication run] + 795
44  main.debug.mt                 	0x000036c9 main + 582
45  main.debug.mt                 	0x0000288f _start + 209
46  main.debug.mt                 	0x000027bd start + 41

Thread 1:
0   libSystem.B.dylib             	0x9668c46e __semwait_signal + 10
1   libSystem.B.dylib             	0x966b6dcd pthread_cond_wait$UNIX2003 + 73
2   libGLProgrammability.dylib    	0x90491b32 glvmDoWork + 162
3   libSystem.B.dylib             	0x966b6155 _pthread_start + 321
4   libSystem.B.dylib             	0x966b6012 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x696e7265  ebx: 0x165e9268  ecx: 0x72657355  edx: 0x6f442f61
  edi: 0x16602a58  esi: 0x00025b70  ebp: 0xbfffe6c8  esp: 0xbfffe68c
   ss: 0x0000001f  efl: 0x00010246  eip: 0x72657355   cs: 0x00000017
   ds: 0x0000001f   es: 0x0000001f   fs: 0x00000000   gs: 0x00000037
  cr2: 0x72657355

What's wrong, in your opinion?
Thanks.

Brucey

(Posted 2010) [#2]

Ooooh... double free... that's not so good.

Does it always fall over with a similar stack-trace?

Corum

(Posted 2010) [#3]

Hi Brucey! :)
You had the right intuition about the crash report creativity. :-)
The answer is no. Sometimes it crashes on certain calls, sometimes on others, in an absolutely arbitrary fashion.
Mmhh I suppose it is bad. It sounds sooo bad. :-)

This time was:

Thread 0 Crashed:
0   main.mt                       	0x0002b6ed unzCloseCurrentFile + 61
1   main.mt                       	0x00023174 _ifsogui_gui_TZipStream_Close + 31
2   main.mt                       	0x000228cf _ifsogui_gui_TBufferedStream_Close + 20
3   main.mt                       	0x0011939a 188 + 0
4   main.mt                       	0x0011c0ff collectMem + 591
5   main.mt                       	0x0011c27f allocMem + 335
6   main.mt                       	0x0011c50c bbGCAllocObject + 44
7   main.mt                       	0x0011d783 allocateArray + 195
8   main.mt                       	0x0011d977 bbArrayNew1D + 23
9   main.mt                       	0x00120959 bbStringSplit + 217
10  main.mt                       	0x00025265 1738 + 31
11  main.mt                       	0x00010940 _ifsogui_slider_ifsoGUI_Slider_LoadTheme + 28
12  main.mt                       	0x00010dd1 501 + 0
13  main.mt                       	0x00027db4 255 + 0
14  main.mt                       	0x000037cb _bb_TGUIWrapper_InitGui + 78

Otus	(Posted 2010) [#4]

Do you do anything complicated in any of your Delete methods?

Corum

(Posted 2010) [#5]

@Otus: the sources come from an external mod, and are covered by copyrights, so I cannot be more specific..
But IMHO the problem is related to GC.
Looking at the library code, in every method, the temporary objects are allocated as Local, and often set to Null in the end.
There are no Delete or similar methods occurrences.
I talk about temporary initialization object because application doesn't even starts, but crashes during early steps of init phase...

NOTE: the library author did a try under windows (same version of Bmax I'm using on Mac), with an example source that actually crashes under Mac environment, with no issues at all.

Corum

(Posted 2010) [#6]

Ok, I restricted the field to a module wrapped from C.
If I use to load a theme without the zipstream feature, but straight from filesystem, my MT app doesn't crash anymore.
For what I know about it, it loads the .zip, extracting it to memory and opening a stream from there. Many chances to break something.
I'm going to investigate further.
Thanks. :)

Brucey

(Posted 2010) [#7]

I looks like there's stuff going on in that ifsogui module that is causing your problems.

Does it always crash with a double free?

TaskMaster

(Posted 2010) [#8]

Brucey, the failure actually occurs within the zipstream module which makes some standard C calls. It works in multi-threaded if he doesn't use a zip file and it works with a zip file if he doesn't use multi-threading.

The weird thing about it is that it works fine in Windows, I do not have a mac to debug it with. When I get a chance (hopefully tonight), I am going to try it on Linux. And I do have a mac VM I might be able to try it on.

Brucey

(Posted 2010) [#9]

Which zipstream module?

I can test it if I know what I need to replicate it in a simple example.

Corum

(Posted 2010) [#10]

The one from Koriolis.
And of course, always double free issue, and it has problems under windows too.
Sadly, if you take the module and build'n run the basic examples, cannot reproduce the messy behaviour.

marksibly

(Posted 2010) [#11]

Hi,

Are you actually using threads at all?

Corum

(Posted 2010) [#12]

Hi Mark. :)
No, as I said in OP, I simply made an MT build of my working project, with future plans to parallelize several AI stuff by using MT features.
If I rollback to 1.34, I can safely run the same project with MT option enabled.

Bye.

TaskMaster

(Posted 2010) [#13]

That is the same result I am getting. And it turns out that it does do it in Windows, so it is not just a mac problem. When I first tested, I was not actually using the zipstream.

So, to explain it clearly: We take a program in Single Thread, using the koriolis.zipstream and it works fine. Then we recompile with multi-thread and it fails. The failure occurs when using LoadImage. I step through the app and it makes it 3 or 4 lines past the LoadImage sometimes and sometimes it crashes right at the end of the LoadImage. This is why it seems to be a GC problem.

I do not know if you are familiar with the zipstream module, but it is mostly C code.