BlitzMax Threads!
BlitzMax Forums/BlitzMax Programming/BlitzMax Threads!
| ||
| Hi, Ok, who's using the Blitzmax thread stuff? How/What for? Any requests? |
| ||
| Threads? I have missed something! Where is the thread interface? |
| ||
| I haven't been using the threading for anything lately. Not been doing much coding in BMax (Objective-C is too much fun), and when I do it's usually not something I need threading for, so I'm without any examples of what I'm doing with them or how I'm using them. I'd just point to Cocoa's threading and say "that's what I want," since classes like NSOperationQueue seem nice. |
| ||
| Threads? I have missed something! Where is the thread interface? You are joking, right? |
| ||
| Hey Mark. Before BlitzMax had real o/s threads, I could see that being limited to a single thread on a single CPU core was going to be very limiting. I wanted to be able to do several things at once, especially as I am working on a dynamic multimedia environment where I need many things to be happening in parallel. I envisioned being able to create an OpenGL-based or in-game GUI where long-term tasks can be running concurrently, but that's somewhat tricky with a single linear execution flow. The solution I started working on, and for additional reasons, was to implement a script language and a script execution engine. In order to provide a multitasking capability I built my own scheduler which switches execution to other scripts based on timeslices, priorities, timed events and frame flips, etc. And it works well. Just as I finished getting this to work, and noticing how much CPU time is being unused on the other cores of multicore cpu's or multiprocessor systems, you came out with your thread module. I converted my system to implement a thread pool, so as to minimize thread allocation/deallocation. Each thread now runs its own script scheduler and executes script instructions. This is now operational (that sounds like I just announced the full operability of the death star ;) and there is obviously a significant performance increase due to other cpu cores now becoming accessible for the execution of script commands. I have a dual core CPU but I know there are people out there with more, and there will be many more in future. I do believe this is definitely the wave of the future. So I am VERY glad that you implemented support for real threads. While I was really happy with my own virtual solution, combined with real threads and taking advantage of extra cores is really nice. I can now create graphics/multimedia/game software that really takes advantage of the extra power. I am really interested in graphics and eventually will be doing lots of image processing and graphics creation which will be much faster when taking advantage of the available cores. I haven't had many issues with threads - I know it requires one to be very careful with data access and mutual exclusion and all that. The only thing I am slightly concerned about is that the new garbage collector is somewhat slower than your original system. Is it possible to put garbage collection in its own thread? I did see my app taking advantage of the extra cpu core, but while I anticipated a 2x speedup it was more like a 1.6x speedup, and wondered if that was largely due to the garbage collector. But still - it's gotta be safe n'all that. I'd rather we definitely have threads with a slowish collector than no threads at all, and really it's not that bad. I realize I can run multiple threads on a single core, as well ,but I like the idea of a fairly constant thread-pool based more on the number of available cores than on the number of tasks I want to perform, since I am using my own scheduling system for individual tasks. One little problem I had with threads is that I wanted to be able to have a `running within this thread` context within which I could define Globals which are only Global within the thread, so that for example when a thread call some functions those functions can access global values which are defined outside the functions, but `inside` the thread, not shared with other threads. As it stands this is impossible and I am forced to pass a variable in to every function called in the thread. If you would create a new context definition for `thread-local-globals` that would be awesome. The current TLS system is cool but a) it is much too much overhead to call a function just to store and retrieve a variable, and b) you have to have a handle to the data you stored in TLS and end up having to then pass the handle into functions as a parameter, invalidating the benefits of using TLS at all when you might as well just pass the variable. Besides this, I can see that if I wasn't doing all this jiggerypokery I would use threads to be able to load/spool files while doing other things, and to at least split off some logic/ai/graphics/physics processing in separate and possibly redistributable threads. I think threads are a great addition to BlitzMax. |
| ||
| I tried to use threading, but I was also using wxMax and it is not threading capable. So, I had to give up on the threading. Also, without the svn being up, may people do not have access to the threading stuff. Mark, is it possible you can get the svn back up, or another method for us to get updates? Sorry, don't mean to derail your thread. I was all prepared ot use the threading stuff until I got stuck with errors in wxMax. |
| ||
| I will probably start using threads if it is officially released just for the sake of playing around (currently no intentions to use it seriously - but I can see the point with a large project, especially when pretty much everyone is making multi-core processors now). I wont use it initially, so as long as the GC works EXACTLY the same on the Max side (and doesn't break any existing code [when you don't use threads]), I am completely fine with it. |
| ||
| I will be using threads when it's official as well. Massive online games could really benefit from this as well as applications that are not games per say. So please keep us posted, Mark. :) |
| ||
| Hey ImaginaryHuman, Are you actually using the TThread type, or just the low level CreateThread() etc functions? I've spent the day cleaning up much of the thread stuff so it now looks more like the rest of Max. There is now a thin OO wrapper on top of the (now private) low level stuff, and then a procedural version of that, eg you get both this... Type TThread Method Detach()... Method Wait:Object()... Method Running()... Function Create:TThread()... End Type ...and... Function CreateThread:TThread()... Function DetachThread( thread:TThread ) Function WaitThread:Object( thread:TThread) Function ThreadRunning( thread:TThread ) So...thread now looks more like the low level stuff than it used to. I've ditched some of the gnarly qnx like messaging stuff from TThread because I wasn't really confident in it and want to initially start with stuff I *know* works. I also ditched the Wait() timeout in TThread because this was effectively busy looping which is not sexy. Does this sound OK for what you're up to? I know it's a bit different from the SVN version, but it's more Max-ish and I think will make a cleaner base to build on. There's still MainThread:TThread() and CurrentThread:TThread() too... |
| ||
| How about keeping the old GC and the new (thinking of the GC as the threading feature), but allow the user to add a compiler flag telling it which to use? |
| ||
| On behalf of Gfk (who is currently offline and moving house, back in two weeks): Ok, who's using the Blitzmax thread stuff? Gfk! ;-) How/What for? I believe he was loading sounds etc with it Any requests? He really wanted the log file thing fixed |
| ||
| I'm looking forward of using threads in my game project, as it will be quite CPU intensive and has things that can, and should, be executed in parallel. When thinking of today's multicore CPU's I think it is inevitable for programmers (games or applications alike) to start paying more attention to run tasks in parallel. |
| ||
| Personally I've glanced at the threads stuff, but haven't actively used it for anything real yet -- the problem I've run into so far as that the garbage collector required by the threading module is adding a massive slowdown in the application I'm currently working on. Example: My application does hundreds of thousands of string operations. The 'standard' non-threaded build takes 1.564 seconds to complete on average. Without making any changes to the code yet and just switching it to use the thread-safe garbage collector, execution time takes a massive nosedive and all of a sudden the same app takes 40.52 seconds to complete. (while maxing out a single core on my CPU) That's over 25 times slower than the non-threaded build... Being able to use the 2nd core doesn't make up for the difference... :-/ So... I hope you're not planning on abandoning BlitzMax' original garbage collector. That said, the ability to have threads is definitely a welcome one, it just happens that it's not the best solution for every application due to some of the overhead that's introduced. |
| ||
| I dont use them at the moment but Im really happy they are there. Its really just the garbage collector that behaves strangely on occasions. With strings for example as xlsior wrote. |
| ||
| You are joking, right? No, I have been away for a while... I have the latest downloadable version, I see no help topic or interface for thread support. Am I just blind? |
| ||
| I have the latest downloadable version There was a development version with threading available for a while. I believe threading is meant to arrive properly in the next release. |
| ||
| @Brendane: If you have only the latest downloadable version, you can't use threads. Threads were added on the development version, wich was accesible throug SVN some weeks ago. It is no longer available as a download, but Mark mentioned something about releasing an update including this functionality soon. |
| ||
| Ahhhhh, thankyou all. I really did miss something then :) In that case, Mark, one thing I have needed in the past when working with a higher level thread interface is the ability to set/change the thread priority and also create the thread with a different stack size to the default. Are you giving an interface for Events also? ie the windows CreateEvent / SetEvent / ResetEvent / PulseEvent interface... I have made extensive use of this in the past. |
| ||
| Can someone post the interface for me to have a look over please? |
| ||
| http://www.blitzbasic.com/Community/posts.php?topic=83676#944030 It would seem that he is. |
| ||
| I would love using threads particulary when working with Networking applications. But I simply waiting the official update since working on a big project and don't want to complicate my life with two version of blitzMax. I already played with threads in Purebasic in the old days but often crash my application because I wasn't ready enough to use them. But now this would probably be easier and I could certainly use them with parsimony. |
| ||
| Mark, yeah that sounds okay. At the moment I think the only thread commands I actually use are to create a thread, to detach a thread, and to set or release a mutex. I did not use your preliminary Thread Pool type, I preferred to use commands like CreateThread() and then implemented my own thread pool. I'll be interested to see how fast CurrentThread() is if that means i could do CurrentThread.MyThreadLocalGlobal rather than passing MyThreadLocalGlobal as a parameter into every function. Does CurrentThread() just access a type instance variable quickly or does it have to go through some hoops to get that information? I also second the request for some way to define the priority of each thread - it would be really nice to be able to manually manage how much cpu time each thread is getting. Perhaps also it would be useful, for profiling purposes, to be able to find out how long a thread took to execute some given piece of code, perhaps since the previous wait. |
| ||
| Cross-platform thread priority (and possibly processor affinity setting) would be welcome icing on the, um, thread... cake. |
| ||
| definately cross-platform is a must seeing as max is cross platform |
| ||
| No, I have been away for a while... I have the latest downloadable version, I see no help topic or interface for thread support. Am I just blind? Sorry, with the numerous threads about it and it being available for months, I didn't see how somebody could miss it. |
| ||
| I want to use threading for the CPU side of my renderer, which evaluates a hierarchy of objects. It would be pretty simple to branch the code and say "Okay, this thread takes this node and evaluate its hierarchy, and this thread takes this other node and evaluate its hierarchy". When you have thousands of trees in a scene just iterating through them each frame to see what needs to be drawn can be a big task. If I could cut that processing time down by about 75% on a quad core it would be a big advantage. I'm sure Crysis is iterating through every instance in a scene rather than just storing an array of instance matrices, because the LOD for each object changes individually, instead of a group of objects changing at once. I also want to write callbacks for use with Newton physics that can be called from multiple threads and not crash. |
| ||
| Seeing as the SVN is down to get the major benefits from Threading we will have to wait for the next major release to test it. |
| ||
| Oh yeah, I'd also like a command in Blitz which reports how many processor cores are available - I know some partial code was posted in an older thread thread (lol) but it'd be nice for it to be a native command e.g. Processors() or ProcessorCores() or CoreCount() or ThreadCores(). |
| ||
| Leadwerks, how about a bounding volume at each tree note to enclose its children efficiently? |
| ||
| I already do that. I say CountCPUCores(). |
| ||
| While we're at it, how about a cross-platform way to find out the clock speed of the cpu and maybe other details about it e.g. manufacturer, and perhaps also for the GPU? and if possible, the amount of video ram? |
| ||
| what would be the advantages/disadvantages of using processes instead? |
| ||
| Not used it as it was on SVN. Will use it when it comes out as a proper release. Will find it useful for AI (doing A* in different thread) |
| ||
| The main difference between a thread and a process is the isolation of memory space. Each process is like a separate application with its own memory which typically is not shared with other programs/processes. A thread shares all of the memory and resources across the threads, within a process. Threads are much more `open` and let you share all your variables/memory/images/etc across all of the threads in your program, while still being able to run threads on multiple CPU cores like processes. Processes are a bit safer in the sense that they are usually confined in how much memory they can access and can't share memory with other programs/processes without some additional `door opening` message-passing commands. I personally prefer that all my doors start out being open and then have the option to close them later, rather than that they all start out shut and have to force them open. Threads are more efficient and require fewer `state changes` in order to multitask the threads. |
| ||
| Multi-processing with BlitzMax is fun though... and you get to keep the more efficient GC :-) |
| ||
| I'm using it to handle network traffic. Would love to see it pushed out in a full version! |
| ||
| Currently using it with streams + cegui for an updater with a live progress-bar in my game + chat lobby. I would, at one point like to use threads to draw the cursor seperately in client apps, but i'm waiting for a 'stated' stable version to even try it with max. I'm more than happy with the support for it this far - no complaints. Cheers. |
| ||
| Although this is completely trivial, it would be nice to show a rotating or animating progress indicator during a load screen, just to look like you are making huge use of multicore processing. We've finally decided on a game concept, so this kind of thing isn't too far off. |
| ||
| I am using it to create and handle a game server. PLS |
| ||
| I too would use it for networking |
| ||
| I would use it for networking, and possibly physics calculations. |
| ||
| Hmm.. Yes, it would be excellent for networking. I would like to ask this question again.. How about keeping the old GC and the new (thinking of the GC as the threading feature), but allow the user to add a compiler flag telling it which to use? |
| ||
| Plash: That's what it currently does. By enabling 'threaded build' in the IDE it just switches GC's. The catch is that the threading routines won't work if you are using the old GC, but you can use the new GC without actually using threading. |
| ||
| The catch is that the threading routines won't work if you are using the old GC, but you can use the new GC without actually using threading. Yes, but the SVN version had it so when you used the threading GC, the old GC was still in the code (unnecessarily?) |
| ||
| Ah, I see -- I suppose it might be. I just hope that the old GC sticks around for non-threaded apps, since it is a bit faster. |
| ||
| I would also use threads for AI, networking, file scraping summarized into two main areas: optimizing independently divisible tasks large background procedures while maintaining app responsiveness |
| ||
| Networking stuff. After all, I'm now working for a satellite communications company (since Monday, actually). ;-) |
| ||
| Hi, I need it for Networking too ,special for NAT! And so many other cases where it be more as usefull. regards C64. |
   |