BlitzMax 134 now available!

BlitzMax Forums/BlitzMax Programming/BlitzMax 134 now available!

marksibly(Posted 2009) [#1]
Hi,

Please go to the product updates page of the Account section to grab BlitzMax134!

The main feature is a brand new threaded mode garbage collector, which should be faster and more stable than the old one. There are likely to be a few issues with the new GC, but I am ready and waiting to fix anything that pops up!

Versions.txt...

::::: IMPORTANT ::::: This will be the last release to support Win95/98/ME.

* Added SA_RESTART to sa_flags in blitz_thread.c to fix wait(pid) dying when GC stops world.

* Mac OS 10.6 tweaks.

* Removed BDWGC garbage collector entirely.

* Updated FASM to 1.68.

* Fixed debugger string escaping.

* Improved ToLower/ToUpper unicodeness.

* Changed default installation dir to C:\

* Tuned MS GC significantly.

* Added MouseX/Y/ZSpeed() functions to brl.polledinput.

* Fixed fasm2as bug.

* Linux/Macos mutexes now recursive as per Win32.

* Fixed Win32 RequestDir initial directory.

* Restored hwndOwner=GetActiveWindow() to RequestFile/RequestDir;

* Added MS (Mark Sibly) garbage collector!

* Fixed enet_peer_address - was using completely wrong struct offsets

* Fixed Linux StopTimer - was releasing timer multiple times

* Fixed OpenAL delete buffer leak




Jesse(Posted 2009) [#2]
cool. will download as soon I finish seting up my lenux system.


teraku(Posted 2009) [#3]
Thanks Mark for another great release.
Will 134 work on 64 bit linux distros?


H&K(Posted 2009) [#4]
cool.


erm, somthing I dont understand here. BMax is crossplatform cos the assembler that makes the compiler is cross platform, right?

So is the loss of 95/98/me support cos FASM is going to abbandon 95/98/me?
Or is there some other tech reason that I dont need to understand (but would like to know)

Thanks


marksibly(Posted 2009) [#5]
Hi,

> Or is there some other tech reason that I dont need to understand (but would like to know)

It's mostly a unicode thing - latest versions of Blitz have had to provide 2 versions of lots of routines: one for win95/98, one for unicode. There's a 'unicows' dll around that helps with some of this, but by no means all.

Anyway, supporting both unicode AND non-unicode is becoming a bit of a headache, so we're making the leap to 'unicode only' windows.


therevills(Posted 2009) [#6]
Cool!

Noticed that the download link still has RC (rc7) in the name, does this mean that its not the final version of 1.34 or it really just 1.34.7?

Cheers!


Tommo(Posted 2009) [#7]
Wow, a new GC!
That's quite a big improvement for a 0.01 version increment.
Rebuilding modules now, can't wait to try.
Thanks!


JoshK(Posted 2009) [#8]
Is the new GC used in both ST and MT modes? I get these errors (in ST mode):
C:/BlitzMax/tmp/.bmx/untitled1.bmx.gui.release.mt.win32.x86.o: undefined reference to `bbGCEnter'

With DLLs we had to have a call to GCEnter() at the start of the function. Does this still apply?


marksibly(Posted 2009) [#9]
Hi,

The new GC is for multithreaded apps only.

Single threaded apps still use the old (slightly faster) reference counting GC.

[edit]
bbGCEnter/bbGCLeave got zapped during a gc related housekeeping session! They're just tiny, so I'll restore them in rc8. In the meantime, you can replace blitz.mod/blitz_gc.win32.x86.s with this:


...or try this dodgy Max version...




Panno(Posted 2009) [#10]
* Added MS (Mark Sibly) garbage collector! ...... muhaaaaaa


thx for the update
mfg


therevills(Posted 2009) [#11]
Added MS (Mark Sibly) garbage collector!


Could there be any benefit for using this in single-threaded apps? (Either in a new version or compiling a ST app as a MT app?)


Brucey(Posted 2009) [#12]
Could there be any benefit for using this in single-threaded apps?

Probably no benefit. And likely a small speed hit - due to the extra work it is doing low-level (but doesn't need to for a non-threaded app).


ziggy(Posted 2009) [#13]
Could there be any benefit for using this in single-threaded apps?
Yes, this new GC handles ciclic references. (but it's slower). I would say it's the ideal GC for desktop (not realtime like games) applications. If used on games, I supose it has to be used from the begining of the game coding, so the game is coded properly to benefit from nulti-thread speed gain (wich on several cores machines and a proper design) could take advantage of the GC in speed therms.

I'm not sure on a 2 cores machine this is worth it... haven't tested properly yet!


siread(Posted 2009) [#14]
The new GC still isn't fast enough in my racing game. After simply re-compiling as a threaded app (and not creating any threads) the framerate steadily declines.

I've narrowed down the problem to the replay code. Basically a replay of the race is stored by adding replay data to a TList for each car. As the lists get longer during the course of the race the GC gets slower and slower making the game unbearable after about 1 minute (40 updates per sec * 60 secs * 20 cars = 48,000 list items per minute. Each item contains 8 floats and 4 ints.)

Turn off the replay system and everything runs pretty smooth although not quite as smooth as unthreaded.

Despite that, cheers Mark!


_JIM(Posted 2009) [#15]
@siread: Probably not the fault of the GC. TLists are slow with that many items. Maybe you should alter the design a bit.

One example would be: Use arrays or banks and allocate memory for let's say... 32,000 items. If you get past that, double it. Then double again and so on. That way you have 5-6 allocs at most, and it should be fast enough.

You could also dump it to a file (good use for another thread) once every 10 seconds or so, while clearing the memory (good exercise for thread safety :) ).

I usually avoid TLists past 5000 items. Too slow even when just looping through them.


GW(Posted 2009) [#16]
I agree with with _Jim 100%. If you use Tlists your trading convenience for speed. Anything beyond trivial Lists or lists that need to iterated every game loop, should be moved to arrays. I think slicing an array to add another 32k of elements will be faster than iterating a list of that size.


Brucey(Posted 2009) [#17]
I don't think the issue is with iterating a TList - since all he needs to do is add to the end of a list, which should be just as fast regardless of the size.

The issue is most likely the very large number of "live" objects, which the GC is presumably checking if they should be GC'd or not?

That could be a flaw in the GC of course...

One can understand that 48,000 items + TLinks per minute is certainly going to put a load on it.

As Jim notes, perhaps streaming to a file instead of keeping it in memory, could save yourself a lot of headaches.


siread(Posted 2009) [#18]
Yes, I only add to the lists during gameplay, then step through it when watching the replay. It's perfectly quick enough in non-thread mode, but it might be worth streaming it to a file. That would have the added bonus having replay recordings that could be stored and loaded at a later date or even shared between players. :)


JoshK(Posted 2009) [#19]
I doubt there will ever be such a thing as a fast garbage collector that handles circular references.


GfK(Posted 2009) [#20]
Thanks for the update - why the change to the default installation folder, though? Its been C:\Program Files\BlitzMax for ever, and now its C:\BlitzMax?


JoshK(Posted 2009) [#21]
Maybe something like GCDisable(o:Object) and GCEnable(o:Object) could be used to remove objects from the GC cycle? Your example indicates that the number of objects contributes to the CPU usage, even if they are not being collected. That would be safer than something that force-freed them. GCDisable could remove objects from the global list or array or whatever is used, so the collector doesn't even have to cycle through them.


_JIM(Posted 2009) [#22]
Could also use metadata for fields (or entire types) that you don't want checked by the GC.

It would also be nice to give the option of turning the GC completely off and doing your own data freeing. This should be the fastest of them all, right? (its true that it's not the easiest though)


_Skully(Posted 2009) [#23]
Gfk,

Believe it or not that solves all the Vista problems. UAC leaves it alone there. Weird but true


SebHoll(Posted 2009) [#24]
It would also be nice to give the option of turning the GC completely off and doing your own data freeing.

I thought you could already do this using GCSetMode( 2 ).


xlsior(Posted 2009) [#25]
GfK: Default behaviour under Vista and Windows 7 is that it won't let you write .exe's anywhere under c:\program files except when it's done by a 'proper' installer. That means that Blitzmax won't be allowed to create its files there either, which includes compiling the hep snippets and the likes.

While there are workarounds (disabling UAC, manually changing folder permissions, etc.) having the default setup break on most new users PC is obviously a bad idea... Installing outside of the c:\program files hierarchy fixes that problem.


_JIM(Posted 2009) [#26]
@SebHoll: I wasn't referring to that. It still uses the GC because you have to call GCCollect at some point.

I meant manually having to delete variables and allocated memory. Hm... maybe MemFree(VarPtr(variable)) works.


VIP3R(Posted 2009) [#27]
Default behaviour under Vista and Windows 7 is that it won't let you write .exe's anywhere under c:\program files except when it's done by a 'proper' installer... Installing outside of the c:\program files hierarchy fixes that problem.

What's wrong with BlitzMax creating the executables outside of Program Files instead?


dmaz(Posted 2009) [#28]
Could also use metadata for fields (or entire types) that you don't want checked by the GC.
that sounds like an excellent idea!


ImaginaryHuman(Posted 2009) [#29]
All those thousands of items being added to a linked list, that's a TONNE of objects even without threading. If you're going to be filling in storage for that amount of data, where all you're really doing is adding something onto the end/next slot, should be done with a memory bank or an array. You don't want/need thousands of memory allocations regardless of the garbage collector.


Mahan(Posted 2009) [#30]

What's wrong with BlitzMax creating the executables outside of Program Files instead?



Well, the standard is that .exe's are created where compiled. Lots of people just make a subdir to the normal BlitzMax dir and code away, so there you might have a problem.

I agree with BRL making this change so that the default install is as intuitive as possible for newcomers.

Nothing (AFAIK) stops you from still installing BlitzMax under C:\Program Files\ if you run XP and you want it that way, but the default install just tries to make everything as lean and as expected/predictable as possible.


xlsior(Posted 2009) [#31]
What's wrong with blitzMax creating the executables outside of Program Files instead?


It creates the .exe's in the same folder as the source file (necessary for dependencies such as DLL's you may need, media, etc.)

Also, since the samples live in a subfolder under the main blitzmax install folder, those would be affected as well.

either way, you can still manually select c:\program files\blitzmax if you wanted to, nothing is stopping you.


marksibly(Posted 2009) [#32]
Hi,

> As the lists get longer during the course of the race the GC gets slower and slower

Does it happen in BlitzMax 1.33 as well?

Simply using a maxed out float array if possible, eg...

Global replay#[MAX_REPLAY_LENGTH]

...should fix it, as array's of non-objects don't/shouldn't affect GC.

Not a very elegant fix though...


VIP3R(Posted 2009) [#33]
Well, the standard is that .exe's are created where compiled. Lots of people just make a subdir to the normal BlitzMax dir and code away, so there you might have a problem.

It creates the .exe's in the same folder as the source file (necessary for dependencies such as DLL's you may need, media, etc.)

Yes, but putting your project files within the Program Files folder is not a good idea. For example a system restore could restore older versions of your project files if they're within Program Files.

It would be better to compile and run them from the Documents folder in Vista by default, like Visual Studio etc do. The demos could be made to compile and run from the Temp folder too (didn't Blitz3D do that?).


xlsior(Posted 2009) [#34]
It would be better to compile and run them from the Documents folder in Vista by default, like Visual Studio etc do


Personally I prefer the current way, of having everything self-contained within a single directory tree -- splitting it up across program files, documents, etc. it annoying, especially if you happen to be running blitzmax from a thumbdrive and use multiple PC's (in which case the documents folder would be on a different, non-portable drive)


markcw(Posted 2009) [#35]
It would be better to compile and run them from the Documents folder in Vista by default, like Visual Studio etc do. The demos could be made to compile and run from the Temp folder too (didn't Blitz3D do that?).

I think a fairly major rewrite of BlitzMax was needed to be able to use the Documents folder in Vista.

Blitz3D has a subfolder called /tmp which it uses for compiling but it never used C:\temp, if that's what you meant.


VIP3R(Posted 2009) [#36]
@xlsior: Ideally it would use the project folder wherever it is - including your thumbdrive, but not Program Files. Documents would be the default project folder instead of the BlitzMax folder.

@markcw: I meant C:\Windows\Temp, haven't seen C:\Temp for ages :)

I'm not bothered either way tbh as it's optional, I just think installing applications to the root of the hard drive is a bad idea. It's annoying enough with MinGW being picky about where its installed.


Space_guy(Posted 2009) [#37]
The new GC is alot better for my game engine than the old one. But i guess i use a few tlists that i can recode to arrays to get it smoother as its still not as smooth as non threaded. :)


Yasha(Posted 2009) [#38]
Blitz3D has a subfolder called /tmp which it uses for compiling


I suppose this is offtopic, but I always thought Blitz3D compiled directly into memory? (Unless you actually specified "create exe", that is)


JoshK(Posted 2009) [#39]
I am experiencing problems with a BlitzMax dll compiled with 1.34. The nature of the problems indicate a random memory overwrite. I experienced similar problems in earlier versions if GCEnter() was not the first line of an exposed DLL function.

I'm going to reinstall 1.33 and try to verify the BlitzMax version is the only difference. If you know of any potential problems 1.34 might have created with DLLs please say so.

[edit]
Oh, I see your edit. I got it working.


SebHoll(Posted 2009) [#40]
@ Mark: FYI - makedocs bug + fix.


Space_guy(Posted 2009) [#41]
Im just writing again since i tested this version more and did some tweaking to my code and it all goes so much better than it did before :) Great update!


theHand(Posted 2009) [#42]
I doubt there will ever be such a thing as a fast garbage collector that handles circular references.

I would have to agree with Leadwerks. Circular references run through a GC will have it doing more work, and for nothing, if the circular reference is in use.
Does the Garbage Collector work by keeping a list of all the things that it has to go through?
How does GCSuspend/GCResume work, at the lower level? Does it mark things to be skipped, or does it remove them from the GC loop (from the "list")?


Glenn Dodd(Posted 2009) [#43]
Build Modules option does not find any changes i have downloaded from Brucey from SVN.
I used to just click this option from the default MaxIDE after each SVN update.
Since installing BlitzMax 1.34 this hasn't shown any new items to build. I just tried the new MaxIDE 1.34 and there is no difference.

I installed the BMax to c:\BlitzMax134R7.
I haven't changed anything else.

The icon i run from the desktop points to the correct folder and i tried the icon in the folder too.

Any ideas?


Brucey(Posted 2009) [#44]
Build Modules option does not find any changes i have downloaded from Brucey from SVN.

Just make sure that folders like bah.mod are in the right place of your new installation.
Which would be something like : c:\BlitzMax134R7\mod\bah.mod\...


Glenn Dodd(Posted 2009) [#45]
yup.
no problem with that.

I may have found the solution though.
I use TortoiseSVN and it is still updating the previous blitz install folder.

How do people normally "Copy or Move" the svn files to a new folder?

I guess i will just delete the old stuff and reconnect to SVN headers from the new install path, then "rebuild all" but that seems like the hard way to me.


Grey Alien(Posted 2009) [#46]
Thanks for the release. Good to see the OpenAL fix in there officially too. Very important for Vista users who are not using FreeAudio or DirectSound.


Sokrates(Posted 2009) [#47]
shift it in the right forum - sorry


Armitage 1982(Posted 2009) [#48]
Hi

Thanks for this new release Mark :)

To resume it's not necessary important to update right away if you are not using thread ?
Because I will probably distribute a new version of my game soon.

Thanks


xlsior(Posted 2009) [#49]
It depends on your game -- while the threaded garbage collector is probably the most major change, there's also some other enhancements that may or may not affect your game, such as the OpenAL memory leak fix, better unicode support (=foreign characters in strings), some filerequester fixes, etc.

Depending on what your game is using, one or more of these may be significant to you. If not, then there is no real issue with continuing to use the previous release...


Armitage 1982(Posted 2009) [#50]
Thanks Xlsior, I'm thinking the same :)


Scienthsine(Posted 2009) [#51]
Inline?
Was there some reason why we don't have inline functions? Surely it can't be that hard to implement? There are some times when you want the functionality of a function (lol), but inline because it's in a loop or something. I seem to spend alot of time programming something, then going back and manually inlining things in collision detection/response loops. Afterwords the code is much much uglier, but much faster due to not having the function overhead.


Czar Flavius(Posted 2009) [#52]
About the replay list thing, would strings make it any better? You can put the id/name of each command in a string along with any numerical parameters it needs..


The Caffeine Kid(Posted 2009) [#53]
What version of OSX is required to run stuff produced with this?


jkrankie(Posted 2009) [#54]
You need OS X 10.4.x to run the games.

Cheers
Charlie