Win32 multi-core BMK (NG) Testing

BlitzMax Forums/Brucey's Modules/Win32 multi-core BMK (NG) Testing

Brucey

(Posted 2009) [#1]

I was going to post this in the BlitzMax Programming forum, but given it's chance of blowing up, I thought it better to post here - out of the way :-p

This download consists of a Win32 bmk.exe, and two .bmk scripts. Get it here : bmk_win32mt_test.zip (168kb)
Drop all 3 files into your BlitzMax/bin folder.
Please backup your bmk.exe before trying this - because you know it makes sense!

This particular bmk has been built with "multi-core" support for compiling C/C++ files. This means that it will kick off several compiles at the same time, scaling with the number of cores on your system + 1. So, if you have 1 core, it will attempt to build 2 C/C++ files at the same time... 2 cores, 3 files... etc.

You will only see an improvement in compile times on projects (modules/apps) where there are more than 2 or 3 C/C++ files. Otherwise, it should generally take the same amount of time as before.

Assuming of course you see any improvement on Windows... it works great on OS X and Linux :-)

For timing, I'm not sure the best way to benchmark on Windows. On a *nix box, you can do this :

time ./bmk makemods -a bah.libxml

which times the process and gives you an answer like this at the end :

real	1m11.322s
user	1m19.913s
sys	0m6.309s

which is nice :-)

Anyway... if anything, it should "just work" like the current version.

Thanks in advance for your assistance!

DavidDC

(Posted 2009) [#2]

I don't know if it's just me but I get (XP)

Building untitled1
Compiling:untitled1.bmx
flat assembler  version 1.67.36  (1572863 kilobytes memory)
3 passes, 831 bytes.
Linking:untitled1.mt.exe
C:/Program Files/BlitzMax/lib/libgcc.a(__main.o): undefined reference to `__EH_FRAME_BEGIN__'
C:/Program Files/BlitzMax/lib/libgcc.a(__main.o): undefined reference to `__EH_FRAME_BEGIN__'
Build Error: Failed to link C:/Program Files/BlitzMax/tmp/untitled1.mt.exe
Process complete

When I try to build this:

SuperStrict

Framework BRL.Blitz
Import BRL.System
Import BRL.Standardio 
  
Local start_time:Int = MilliSecs()

system_ "C:\Program Files\BlitzMax\Bin\bmk makemods -a bah.libxml"

Print MilliSecs() - start_time

End

The error goes away when I revert to the BRL bmk

Brucey

(Posted 2009) [#3]

Ah... I see what's up.
When linking "-lmingw32", on a native windows build it needs to be in a different order. For cross-compiling, I found it had to be at the end of the list (of libs). But on native, it needs to be nearer the front (of the list).

Pah!
You'll need to wait til I can get home and fix the code...

Curiously, why are you building your exe with mt enabled? :-p

DavidDC

(Posted 2009) [#4]

Oh right - I got the error without mt, so I tried again with mt in the ludicrous hope it might make a difference.

Brucey

(Posted 2009) [#5]

Heh :-)

Interestingly, for a native Windows build using GCC 4.x (rather than 3.4.5 which we currently use), the "-lmingw32" lib also needs to be at the end of the list (of libs).

All this keeps me on my toes, I suppose.. :-)

Brucey

(Posted 2009) [#6]

Updated. Hopefully that will fix the build when running on native Windows using gcc 3.4.5...

DavidDC

(Posted 2009) [#7]

Average of 3 compiles (Intel Core 2 2.16 Ghz, XP)

BRL: 60,183
Brucey: 34,080

Which makes Brucey... the winner :-)

DavidDC

(Posted 2009) [#8]

Uh oh... this is building my app.

Linking:Mandala.debug.exe
C:/Program Files/BlitzMax/lib/libstdc++.a(eh_alloc.o):eh_alloc.cc:(.text+0xb1): undefined reference to `_CRT_MT'
C:/Program Files/BlitzMax/lib/libstdc++.a(eh_alloc.o):eh_alloc.cc:(.text+0xde): undefined reference to `_CRT_MT'
C:/Program Files/BlitzMax/lib/libstdc++.a(eh_alloc.o):eh_alloc.cc:(.text+0x13e): undefined reference to `_CRT_MT'
C:/Program Files/BlitzMax/lib/libstdc++.a(eh_alloc.o):eh_alloc.cc:(.text+0x205): undefined reference to `_CRT_MT'
C:/Program Files/BlitzMax/lib/libstdc++.a(eh_alloc.o):eh_alloc.cc:(.text+0x21f): undefined reference to `_CRT_MT'
C:/Program Files/BlitzMax/lib/libstdc++.a(eh_globals.o):eh_globals.cc:(.text+0x43): undefined reference to `__mingwthr_key_dtor'
C:/Program Files/BlitzMax/lib/libstdc++.a(eh_globals.o):eh_globals.cc:(.text+0x222): undefined reference to `_CRT_MT'
C:/Program Files/BlitzMax/lib/libgcc.a(unwind-sjlj.o): undefined reference to `__mingwthr_key_dtor'
C:/Program Files/BlitzMax/lib/libgcc.a(unwind-sjlj.o): undefined reference to `_CRT_MT'
C:/Program Files/BlitzMax/lib/libgcc.a(gthr-win32.o): undefined reference to `__mingwthr_key_dtor'
Build Error: Failed to link C:/Code/Amorillo/Mandala.debug.exe

Brucey

(Posted 2009) [#9]

Oops... sorry about that.

Rather than checking the original BMK for the correct place in the list of libs, I moved it too far forward... :-p

Should be fixed now.

Talk about finicky !

34 seconds for ... ?

DavidDC

(Posted 2009) [#10]

34 seconds for...

system_ "C:\Program Files\BlitzMax\Bin\bmk makemods -a bah.libxml

DavidDC

(Posted 2009) [#11]

Should be fixed now.

Yes it works now thanks Brucey.

Brucey

(Posted 2009) [#12]

third time lucky...

BRL: 60,183
Brucey: 34,080

Win32 should be a wee bit more efficient, because it doesn't seem to mind calling multiple system_() in different threads - whereas the other platforms do - so I have to fork() Mac and Linux to avoid that issue, which has an overhead.

Still... better than nothing ;-)

xlsior

(Posted 2009) [#13]

Just got around to trying this -- definitely makes a substantial improvement.

bmk makemods -a bah.libxml

with the original bmk.exe: 62 seconds
with the modified bmk.exe: 37 seconds

Under Vista64 on a Core 2 Duo E6600 (2.4GHz)

Grisu

(Posted 2009) [#14]

My new Core i7 loves your modifications.

Again... well done!

plash

(Posted 2009) [#15]

I have a similar error as DavidDC:

$ bmk makeapp -l win32 -r -a roper.bmx
Compiling:roper.bmx
flat assembler  version 1.67.36  (1296977 kilobytes memory)
4 passes, 80770 bytes.
Linking:roper.exe
/home/tim/blitzmax/lib/libgcc.a(__main.o):(.text+0x4f): undefined reference to `__EH_FRAME_BEGIN__'
/home/tim/blitzmax/lib/libgcc.a(__main.o):(.text+0x73): undefined reference to `__EH_FRAME_BEGIN__'
Build Error: Failed to link /media/data/struct/code/lang/bmax/projects/apc/komiga.assoc/roper/roper.exe

..But this is using Linux -> Windows compiled modules.

Mark Tiffany

(Posted 2009) [#16]

I had this at one point too, but just had to rebuild all modules, making sure I was rebuilding the platform I wanted. I may also have scrubbed all the compiled .i, .a, etc files too just to make sure.

Brucey

(Posted 2009) [#17]

The undefined reference problem is because of a difference of linking requirements between the cross-compiling toolchain and the native Windows MinGW.

The placement of libmingw in the link order appears to be important, so bmk needs to insert it into the correct place depending on whether you are native or cross compiling...

I'm going to release the source shortly, so you can all tinker with it if you want.