BlitzMax Automated Benchmarking Suite
BlitzMax Forums/BlitzMax Programming/BlitzMax Automated Benchmarking Suite
| ||
| It just occurred to me that it would be nice to have a way to automatically run some benchmark tests in BlitzMax - where for each run, it would use a different set of compiler optimizations. After the run, it would have generate a report, summarizing the results. For this suite, what we really need are some fairly good, small, test programs / algorithms that can be run. By "good", I mean that it wouldn't just loop around 1 million times, incrementing a variable. Plus, we could also use C/C++ in the tests, since the compiler optimizations would affect these more - and since there are a few C/C++ based modules around, it might be useful to know how much better they will work when compiler-optimised. So, what I'm asking for, is some help in gathering together some of these algorithms, onto which we can build the suite. They can of course utilise various modules (like LinkedList, Map, pixmaps, etc) if we want to cover many different parts of BlitzMax. - Different tests should thrash different parts of BlitzMax. The automation itself shouldn't be too hard to do, since it's just a case of telling BMK to rebuild modules with specific settings, build some apps, run the apps, gather some stats, create a report. Would anyone like to participate? Is anyone actually interested in the performance of their BlitzMax applications? |
| ||
| Great idea I'd love to participate what about multithreading, I'm keen to learn about Actor based multithreading and this could be a good way to really stress test modules? |
| ||
| Sounds good but is this something that we each would be running or is that all something you will be doing and reporting on? Cus I don't think many people here have a clue that there are even any such things as compiler options or how to use them. |
| ||
| What about a framework like the testbed in Box2D ? Where you'll be able to switch on and off various examples (visually or hard coded). Since it would be a framework anyone in this forum "specialized in his field" would include his little cool example code. Then it would be easy to cover any external modules (like yours for example) without any obligations for the non-users of specific mod. But another question would be : Does BMK optimization will ever be part of the official BlitzMax release ? Then benchmark would probably interest a wider range of Blitz users. |
   |