Speed test
Archives Forums/General Discussion/Speed test
| ||
| ..hey guys..I am interested to hear from you who use to purchase B3D SDK, information about speed B3D SDK with tool you use (C#,PB, whatever) against pure Blitz3D based on same scene..if there is anyone who did that...I am interested to buy this SDK for PB use, but I wouldlike to hear from you guys is there and speed difference between code compiled in Blitz3D and one you did with some other programming tool using B3D SDK?? I'll apprechiate any info regarding this... |
| ||
| I did some test between B3D, Max and VisualC. It was a simple scene: a planet with overlaid clouds and the water has a fake specular map (using fredborg's method). B3D did approx 3000 fps Max did approx 2500 fps VC did approx 2500 fps There is not much in it and with no actual game logic, it is tough to tell which would be faster at the end of a project. Things like processing lists and objects need to be taken into consideration. |
| ||
| so grahpics are not the issue i guess. |
| ||
| You need something more stressful in order to test. Tests which score at thousands of FPS just aren't reliable enough because the margins are too small and FPS are not linear. |
| ||
| Agreed, multiple variable types and equations, some number crunching a standard benchmark type test... I can tell you this now, if you are working with a lot of objects C# will outperform and directly call a specific object a lot fast that iterating through blitz's objects haha. It just depends on what youre doing I suppose as to what is going to be faster... The SDK just gives you so many more possibilities, and between it and C# its a good starting point for moving into C++. Why not take the plunge? |
| ||
| Well, I've tried some stuff, and it seems that, yes, B3D renders fastest, but BlitzMax and C++ are better in the long run because they are much faster when it comes to logic intensive stuff. So in an actual game I would recommend the SDK. |
| ||
| Is C# (managed code) really faster than C++ (gcc or VisualC++) or BlitzMax (or even Blitz3D) in doing things like listing objects ? |
| ||
| No C# is not faster than Cpp in any regard other than the Rapid Application Development. The managed code is bloated. Takes up a lot of resources(1.0mb of memory in Cpp in typical usage in C# is around 20-50mb depending) due to the garbage clean up and the way it allocates memory. I havent worked with with Blitz Max much, but C# does handle objects faster than Blitz3d thats for sure. Where you can gain a huge performance in C# over native blitz is with multi threading... using the thread pool and performing tasks with the background worker. Note that you cannot Multi-thread actually blitz functions such as rendering.. But you can manage your own code so that object manipulation and the data you send to blitz is better managed and threaded. C++ is by far the powerhouse. It can be a royal pain to code in though and atleast with my bad programming practices I get random application crashes and loose memory its just haha. Unless you REALLY know what youre doing C++ for game dev is difficult. Im still curious about C#'s over all speed and resource usage on a large scale game project though. |
| ||
| Also handling of float numbers is way too slow in C#. I think even BlitzMax is faster than C# in floats. The good thing about C++ is that it crashes whenever you make an error, that will teach people to program cleanly. The managed code handler in C# just teaches people to do crappy code, as it watches and correct too many things which should be corrected in the source code in first place. BlitzMax has also some advantages over C++, as it has the SuperStrict mode, which should be used always imho. Not sure if C++ has a similar thing (it has simple Strict at least). I also like the GCSuspend() command in BlitzMax, as you can test with it immediately if you have some memory leaks in your program by turning the GC off. |
| ||
| Here is a simple benchmark comparing c# speed of float to BlitzBasic speed of floats: Blitz goes first. Here is the code: For i = 1 To 5 startTime = MilliSecs() c# = 0 While c# < 20000 c# = c# + .01 Wend Print MilliSecs() - startTime Next WaitKey() Results are as follows: 204 190 193 190 190 189 Now for the really slow c# (using low-res Environment.TickCount function) ;^) static void floatTest() { for (int i = 0; i < 5; i++) { int startTime = Environment.TickCount; float c = 0.0f; while (c < 20000) c = c + 0.01f; Console.WriteLine(Environment.TickCount - startTime); } Console.ReadLine(); } Results are: 15 31 16 16 15 C# is not slow. C# is not bad, and does not teach people to do crappy code. I write and maintain code full time in Delpi, C#, VB.Net, and sometimes C++ (Oh, yeah, and my first language was Commodore 64 Basic back in the 80s). C# is one of the most elegant languages I have ever worked with. However, crappy code can be written in any language. |
| ||
| OOaaaahhh... sorry. I had the debugger enabled in Blitz! I ran the Blitz test again without the debugger: 29 28 32 28 29 Much closer now. Sorry 'bout that. :S |
| ||
| In Emergence BASIC I get: 15 16 15 16 15 Quite satisfactory :) Barney |
| ||
| I did the same test, but counted up to 1 million and got the following results: Visual C++ 2005: 311 320 311 310 321 Visual Basic 2005: 330 330 330 340 330 Visual C# 2005: 341 330 331 330 331 BlitzMax: 586 581 591 582 586 MinGW: 601 601 601 601 591 Blitz3D: 1255 1271 1273 1264 1295 So I was right, C# is slower than C++ and VB in floats. |
| ||
| Power Basic: 12 14 15 15 17 |
| ||
| I chose blitzmax because it is the fastest tool to program games in. A lot quicker than blitz3d as it is a lot more flexible. |
| ||
| Note that the simple usage of "print" will slow your results. I suggest calculate the time taken BEFORE printing the result to make sure you don't calculate the time it takes to actually print the result. |
| ||
| +1 ;-) |
| ||
| I did the same test, but counted up to 1 million and got the following results: Delphi 7: 114 106 130 106 114 Visual C# 2005 Express: 297 250 250 234 250 System specs: Pentium 4 3.2GHZ with 2 GB DDR2. |
| ||
| Umm, guys I think you should do the test at least with 2 languages, else the results are totally incomparable since everyone has a different speed PC :) How about doing the test in addition with your preferred language also with BlitzMax and/or Blitz3D as reference. |
| ||
| "Umm, guys I think you should do the test at least with 2 languages, else the results are totally incomparable since everyone has a different speed PC :)" ha ha ha ha ha ha..... You're right. |
| ||
| So true... Pointless otherwise! |
| ||
| SopiSoft, did you run the .exe for the C# test from a DOS window or from the Visual Studio IDE? From DOS it runs much faster. And did you also compile it as "Release" and not "Debug"? Also did you use "double" data type instead of "float", at least on my PC the 64-bit "double" runs also much faster than the 32-bit "float" (I have 32-bit Windows XP Pro though). I can't believe that Delphi 7 is so much faster than Visual C#, because then it Delphi 7 would be also much faster than Visual C++. But if it is, I'm gonna start using Delphi 7 too. |
| ||
| i've updated the above post with the new Visual C# 2005 results, i ran the test from DOS compiled as a Release. i used floats not doubles. Here are the results again, all ran from DOS and compiled as Release: Visual C# 2005 Express (float): 297 250 250 234 250 Visual C# 2005 Express (double): 235 250 234 234 235 Delphi 7 (double): 98 102 97 99 98 And this is Blitz3D but only counted up to 250,000 because 1,000,000 took forever: 367 387 366 368 382 I don't know what's up with Blitz3D but it doesn't seem to finish when i count up to 1,000,000 and when i click on the window it just says it's not responding. |
| ||
| Wow, Delphi 7 kicks some serious ass! EDIT: Yeah, there's a bug in Blitz3D, you cannot count up to 1 million with 0.01 steps (it just hangs). You have to split the loop into 5x 200000 loops (just another "for x = 1 to 5" loop around it), it gives the same results, as if counting directly to 1 million though. |
| ||
| so that's it, right, let's try your suggestion then. ;-) NEW RESULTS! Blitz3D counted up to 1,000,000: 1575 1592 1822 1851 1867 |
| ||
| OOPS!!!! I just found out that in the Delphi 7 speed test i wasnt adding up 0.01 but 0.1, that's why Delphi was so fast, lol. The real results are actually rather disappointing: Delphi 7 (double): 982 967 967 995 1052 |
| ||
| These tests prove absolutely nothing IMHO. Everyone knows C, C++, C# is very fast. Assembler language being even much faster as it's the closest you'll get to the actual CPU's machine language. Basic (any form of basic language) is written using a C language. So it is obviously much slower than the former. Pascal / Delphi will always be slower than C, but it's still faster than most any Basic language. So, if you, SopiSoft, absolutely want to know the truth and share that truth about language performance, I suggest writing a whole article on the subject and then let us read and take note. Just spitting out a bunch of numbers with little references will not get the message out IMHO. |
| ||
| Basic (any form of basic language) is written using a C language. So it is obviously much slower than the former. Completely wrong. Firstly, there is absolutely no need for any Basic language to be written in C. Secondly, even if it is, the language the compiler is written in is completely irrelevant to the quality of the compiled code. Thirdly, this very site happens to be home to several basic languages, all of which compile to machine language without compiling to C along the way, so C is in no way involved in the compiled code they generate. Pascal / Delphi will always be slower than C, but it's still faster than most any Basic language. The benchmarks above suggest this to be wrong also. For some reason you seem to think that the syntax of a language has anything to do with the speed of it, and that's just not right. Currently Delphi would appear to be slower than C and most modern variants of Basic, but it could just as easily be faster than both, because the syntax has very little affect on the speed of the compiled code. These tests prove absolutely nothing IMHO. Well on that we can agree. Unless you're writing the "count to a million in very small steps" game, benchmarking with a loop of double precision floating point variables tells you nothing worth knowing about the comparison between languages. |
| ||
| Whatever you guys say, this was just a test to see how fast some languages were dealing with floats/doubles, that's all. |
| ||
| Gabriel: Please... ;) The only surprise I have gotten out of ANY Basic language, is how FAST Visual Basic is. The Basic language has been known to be slow, by nature. The only basic I remember which was real fast, was GFA Basic on my old Atari ST, but it still couldn't keep up with C language, nor Pascal. But that doesn't mean I don't enjoy working in Blitz3D ;) This is something I found interesting, talking about benchmarks (for Blitz3D): http://www.blitzbasic.com/codearcs/codearcs.php?code=1998 |
| ||
| It all comes down to how well optimized the compiler is. Not the language it compiles. PowerBasic is a very fine example of an amazingly well optimized compiler. |
| ||
| Kanati, for sure. But where are the extensive benchmark articles, comparing many language and all sorts of results? This is something that would definatly catch my interest. |
| ||
| Gabriel: Please... ;) Please what? Please don't correct your mistakes? But someone might read what you're writing and not realize it's all wrong. The Basic language has been known to be slow, by nature. Repeating your mistakes won't correct them. Syntax and compilation speed are completely unrelated. I don't deny that there are a long line of slow basic languages, but you can't get from there to saying that this is because of the syntax. You're making a huge logical leap, which is leading you to your mistake. It's the same mistake you'd be making if you found that most football teams with relatively few supporters lose more games than they win, and concluded that having relatively few supporters causes teams to lose games. It could be that teams who lose a lot of games don't attract supporters. It could even be a complete coincidence. In this case, the long line of slow basic languages is largely because BASIC was historically an interpreted language. Indeed there are some ignorant programmers who are still completely unaware than it can be a compiled language at all. It's also due in a big way to not being taken seriously, which meant that people capable of writing good compilers weren't writing them for a BASIC syntax. Syntax is not inherently slow or fast. Compilers produce inherently slow or fast code. If you were right, there wouldn't be a huge chasm in speed between Blitzmax and Blitz3D execution speed, because their syntax is practically identical. |
| ||
| For all those who believe that BASIC is a "slow language by nature", they should take a very close look at the executables that POWERBASIC for example creates, especially when it does number crunching. And then those people would be well adviced to remain silent forever afterwards. It's the code the compiler generates and the used (runtime) libraries that cause the performance difference, not the source language itself. But if you need to compare, then BlitzMax can only be compared to a strongly typed language that runs on a runtime system with garbage collection. Basically something like C#.NET, Java or Objective-C 2.0. Pascal, C and C++ are out of the question, because they do not have garbage collection (unless you are comparing the "managed" .NET versions of those languages). By the way, since when is Pascal slower than C? Pascal and C are languages on almost exactly the same level with an almost identical featureset. Pascal just did not have pointer arithmetics, that's all. It did not keep Apple from writing many parts of the original Mac & Lisa operating systems and toolbox in Pascal. And since when are Pascal compilers written in C? The Free Pascal compiler is written in Pascal, as the Mono Visual Basic compiler is written in Visual Basic and the Mono C# compiler is written in C#. The difference is that Free Pascal does not need any runtime systems that are written in a different language. As said above, Pascal is a language that can and has been used to write operating systems. If somebody now wants to say something about the slowness of the good old UCSD Pascal... Then he or she should remember that UCSD Pascal used a virtual machine just like Java and C# do today. The difference being that the UCSD Pascal VM ran well enough on a 1 MHz 6502 CPU, when the first Java VM ran painfully slow even on a Pentium CPU when it was released. But coming back to some of the benchmarks. I think that above two results measured by Lumooja speak for themselves: Visual C++ 2005: 311 320 311 310 321 MinGW: 601 601 601 601 591 Those are both C++ compilers on the SAME machine, compiling the very SAME code in the very SAME programming language. And, interesting enough, BlitzMax even performs better than the GNU C++ compiler on that machine, even though BlitzMax itself was created in GNU C++: BlitzMax: 586 581 591 582 586 But in the end, this entire discussion is just complete useless nonsense. Productivity is much more important than used CPU cycles. Python is one of the most productive programming languages that are currently available, but it is also one of the slowest when it comes to runtime performance. However, that just does not matter in the end: It is still "fast enough" for any job. Programmers are more expensive than computer hardware and CPU cycles., therefor what really matters is that it is much cheaper to get the code written and the job done with Python than it would be with a low level language like C++ or even Java or C#. Python code is also much more maintainable and much better readable than C/C++/Java/C#-code over time. Enough said for tonight. ;-) |
| ||
| ..I'll join Winni and kindly suggest to folks who still thinking that BASIC's is slow, that they may update their knowledge about it...PowerBasic is nice example how one 'crappy' and 'slow' programming language such as BASIC can easy kick ass speed regarding to any other 'shine' languages... |
| ||
| PowerBasic is nice example how one 'crappy' and 'slow' programming language such as BASIC can easy kick ass speed regarding to any other 'shine' languages... While PowerBASIC does indeed kick the pants off of many other compilers, this does come with some problems: While the current compilers generate 32-bit executables, the actual compiler itself is hand-written in 16-bit assembly code... Which doesn't run on a 64-bit OS. You need a 32-bit machine to compile any .exe's, which will then work on 32 and 64 bit platforms. Apparently they're currently working on a rewrite of the compiler so it will run on a 64 bit machine as well -- but considering how much work went into the old 16 bit compiler, one can wonder whether or not the new one will be as optimized as the current version. |
| ||
| xlsior: I can't imagine that (seemingly very) talented coders poring over their previous 16-bit incarnation of a compiler will introduce ugliness when porting to 32-bit. Personally, I would likely spot more scope for optimisation. Granted, I would likely introduce a couple of stupid bugs which would need fixing, but I would make sure things worked better overall in the end :) |
| ||
| Can anyone explain why MAX is slower than B3D using the B3D engine? |
| ||
| It is??? :/ |
| ||
| In the second post it is. |
| ||
| it also has something to do with which programs you are running in the background .. fx. windows automatic updates, etc. which is running and not always taking cpu.. |
| ||
| That doesn't explain the difference between Max & B3D. |
| ||
| Well, he's measuring the FPS in the thousands, and not using any game logic, so to me they're pretty much running at the same speed - as I'd expect. Bmax should only get a speed increase from improved game logic speed, right? |
| ||
| But it's slower. Why would that be? |
| ||
| Not sure, TBH. |
| ||
| There are lots of times/examples were Bmax is slower than b3d, the mandlebrot one thats floating about for example. (although in that case I beleive that its because of plot being a quad in Bmax rather than B3d). In the others examples it comes down to if you have declared the globals/locals in the most optimum way for BMax. Its quite possible in BMax to change a varibale from Global to local (or suprisingly the reverse), and it have quite a dramatic effect on speeding up the example. I seem to be in the Minority in thinking that Bmax isnt in fact faster in general than B3D. I do think that BMax is a better language than B3D, and has such give me (who obviously is the important thing for me), a faster development, but in the majority of Tests Ive seen/run with the SDK, Bmax is slower. However all these benchmarks dont seem to take dev time into concideration, and if I can say so, generaly are so pointless as to be useless. There was one a coulple of months ago that showed that C++ was oneK times faster than Bmax, until it was pointed out that the C++ pre-compiler was simple missing out the pointless variable alocation in a loop because it was subsquently unused. I got Bmax over B3d, not because Bmax was faster, but because it had the things I wanted in a langage that B3D didnt, things that I felt I would make my programs in some way more asteticly pleasing. I am not supprised if the Sdk plus Bmax is slower than B3D. And in fact the fisrt thing I did with the SDK was to OOPwrap it, which made it even slower. After having said all that, and irrelevent of whats been posted, I had a quick go at loading in a terraed map to the SDK, and it wasnt noticably slower than in B3D To sum, I really cannot see the point in most of this. A lot of us started with KHZ CPUs, and now we all have KilloMegaHrtz, it should nowadys all come down to the ease of the language rather than its speed, as whoever said it above said about Python |
| ||
| Why would global/local variables have such an impact on speed? That's scary. |
| ||
| Something to do with local CPU resitor allocation |
| ||
| However all these benchmarks dont seem to take dev time into concideration Your end-user doesn't give a stuff how long your game took to make. If a game runs like a lame pig, then they won't buy it. This is why there is an emphasis on execution rather than development speed. |
| ||
| I agree GFK, but I at least dont "Push" the envelope, and so even if the languge is twice as fast in execution, then it isnt as important to me, cos that means it will only run an a PII800 rather than a PII400. If you are trying to write something at that end of the spectrum that needs all the new cutting edge tech, then fair enough. However thats not normaly the case in the indie market, were normaly "Original" or "memorable" is more important than "Flash". And in a one person Dev team speed of DEV is a lot more important than Speed of exercution. I believe youve published in the order of Twenty progams (May be wrong, please subsitute correct number), which would be better for you, that you had made 10 that ran on a halfspeced computer, or the twenty you have done? |
| ||
| I'm trying to decide whether to to stick with B3D or switch to BMAX for my next project. If BMax is faster then that's a reason to switch. If it's slower (even just a bit) I don't see the point in going through the learning curve and porting tons of code. I'm not interested in the 'better language' argument particularly. The only things that limit what I do in B3D are connected to the graphics engine. |
| ||
| @John, I seem to be the only one who thinks that Bmax is ever slower than B3D. And even I think that normaly thats just because of the greater power given to the programer for varible allocation. I dont think it would take you that long to learn Bmax, theres a "Bmax for B3d users as a sticky, and thats all you need to learn [Aside: And if you dont care about "Better language", could you really go back to Gotos, which are normaly a lot faster than functions. If you do move over to Bmax, after a while you will think the same about BMax Types compared to B3D types] |
| ||
| Personally, the only advantage I currently see to using Blitz3D SDK over Blitz3D, is that the SDK will more than likely be supported for a lot longer than BB3D will. That said, I don't think there's anything massively wrong with BB3D as it is. As for Blitzmax's learning curve, the biggest things I had trouble getting used to, were automatic garbage collection (I didn't trust it at first!), and manual typelist management (which now, I couldn't live without). The rest is largely the same, albeit object-oriented. But you don't have to go down the OO route if you don't want to. |
| ||
| [Aside: And if you dont care about "Better language", could you really go back to Gotos, which are normaly a lot faster than functions. If you do move over to Bmax, after a while you will think the same about BMax Types compared to B3D types] Rather patronising. I'm familiar enough with both languages. I'm saying the 'better language' argument is not part of my decision. That's not the same as saying I don't care about languages at all. |
| ||
| JP, a post by skidracer => Performance answers this question. Most games I think in blitz3d will be spending 80%+ time calling either UpdateWorld or RenderWorld so optimizations from switching language of your project is going to be pretty hard to spot, unless you have something like a complex bb based verlet engine in your code which will get a good boost when ported to bmx. The tweaking I did to improve rendering speed were based on benchmarking the clone army featured in the animtest (and the leading bug in blitz3d bug reports) where ATI cards were stalling on cloned b3d models and ATI and NVidia both showed significant performance boosts when primitives were grouped in an extra sorted list to minimize state changes when using multi surface .x type animated robots. Basically, you'd just be calling the same code to do the same job so I can't understand how one could be faster than the other. |
| ||
| Well my question was why MAX is slower in the second post. Naked War would absolutley benefit from a faster language BTW. It's quite CPU heavy in a couple of spots. |
| ||
| Well my question was why MAX is slower in the second post. Do you mean Big&'s post? If so, it's impossible to say without any look at the source code, but as I said in the fourth post, when you're talking about 2500 FPS, you're talking about fractions of a millisecond, far too precise to measure with a low resolution timer like Millisecs(), so my first guess would be that it wasn't slower at all. You'll note that he also finds VC slower than B3D and even the kindest soul, the most fanboyest of Blitz-fans would be hard pressed to say that B3D is faster than VC++, giving further credence to my assessment that his test was simply doing too little on too much hardware to be accurately measured. |
| ||
| Millisecs() is perfectly fine if you measure over a long enough time. There must be a reason for the difference. |
| ||
| Rather patronising. oh.Thats why I put it in [] and as an aside, I thought you'd take it as one of those stupid things that dont need saying, but that needed to be said, if you see what I mean. Sorry if it was taken in anyother way. I think the only reason to move to Bmax is the language features, and that B3D isnt slower. I would imageine the difference in Bigs example is simply the DLL call/switch time |
| ||
| Millisecs() is perfectly fine if you measure over a long enough time. No it isn't - not at > 2500 fps. It's like trying to measure quantum leaps using a track stopwatch. For each discreet unit of time you have 2.5 occurrences of whatever it is you're trying to measure. That gives you a nice 150% margin of error - which grows linearly proportional to the time it's been running to the point where the numbers become meaningless. It's not entirely by chance that the FutureMark guys design benchmarks that run at low framerates. There must be a reason for the difference. Poor Benchmarking. |
| ||
| which grows linearly proportional to the time it's been running I dissagree, it decreases linearly proportional to the time it's been runningThat is, if after 3Hours one fliped twice as often as the other, I would feel quite confident that one was running twice as fast as the other |
| ||
| Flameduck. If you run the thing for long enough you can measure very accurately with millisecs. Not sure what this margin of error you're talking about is. You seem to be talking about running the test for 1 second which would be daft. |
| ||
| The code in my timing test was basically 99% the same, with just some set up differences between the various languages. The logic to it all is just 3 turn entities inside a timing loop. Its interesting that the two different languages that use Blitz3D as a DLL are running at the same sort of speed. Maybe there is a very small overhead in calling the Blitz3D functions through a DLL rather than it being called natively to the compiled language. Just guessing mind. |
| ||
| The only reason I chose max is its multiplatform nature, if b3d was multiplatform I wouldnt have switched. |
| ||
| The only reason I chose max is its multiplatform nature, if b3d was multiplatform I wouldnt have switched. ...and you didn't want to write a 3d game. ;-) |
   |