uh, Math bug?
BlitzMax Forums/BlitzMax Programming/uh, Math bug?
| ||
Local ItemVal:Int = 1000 Local CRate:Int = 100 Local ItemCost:Int = (ItemVal* 2) * (CRate * 0.01) Print ItemCost Ok, so (1000*2) = 2000 2000 * (100 * 0.01) = 2000 * 1.0 = ...2000. why is it returning 1999 for me? There's simple work arounds for this, but i'm not seeing why this code is creating a problem. |
| ||
There's simple work arounds for this Yep. Use Floats instead. Local ItemVal:Int = 1000 Local CRate:Int = 100 Local ItemCost:Int = (ItemVal* 2) * (CRate * 0.01) Print ItemCost Local ItemValF:Float = 1000 Local CRateF:Float = 100 Local ItemCostF:Float = (ItemValF* 2) * (CRateF * 0.01) Print ItemCostF |
| ||
Of course, if those were important actual monetary calculations, you wouldn't be using Float or Double either. You'd be using some kind of arbitrary precision type instead. :-) |
| ||
Interesting... What is the cause of that though? It would make more sense to me if it was 2001, as i've noticed floats/doubles sometimes hold extra random 0.000001 values, but 1999? i'm confused.. :S |
| ||
It's interesting that: Print (1000 * 2) * (100 * 0.1) actually does print 2000. It's when you use the variables that something happens. |
| ||
Ahh, here's the answer. 0.1 cannot be accurately represented in floating point single precision. It can only do 0.00999999978 which is as close as it can get to 1.0 So when you do 100 * 0.00999999978 you get 0.999999978 instead of 1. This is then multiplied by your 2000 to get 1999.999956, which then rounds down due to storing the result in an Int, to 1999. |
| ||
Only use Ints for your math if you can...Local ItemVal:Int = 1000 Local CRate:Int = 100 Local ItemCost:Int = (ItemVal* 2) * (CRate / 100) Print ItemCost |
| ||
Gets truncated, not rounded down. The reason that it only seems to give problems with variables is that constant expressions aren't evaluated at runtime, the compiler takes those and simplifies them to a single constant. Floats always have little bit of error like that. When comparing floats, always use something like 'abs(Float1-Float2) > .00001' or something similar to see that they are 'almost' the same, due to inaccuracies. It's similar to how we can't accurately store 1/3 in base 10. It's actually .333333-> to infinity. In a different base however, 1/3rd may be exactly representable. Also note that fractions aren't a base, they are incomplete operations. Finally, it's worth noting that when doing math in most programming languages if you use all integers, it will use integers in all the intermediate steps, meaning you get truncated (at best) results in ever step of the operation. If you use a floating point constant (or variable) then your operation is done in floating point. It's pretty easy to accidentally throw an expression out there, and it compile, but things don't work propperly... until you go back and put a decimal point on some constant to force floating point operations. This can also be done by casting ofcourse. Hope that helps. |
| ||
Thanks for the info guys! This really helps clear things up. I was at a total loss haha. |