Check floats/doubles? floating point precision
BlitzMax Forums/BlitzMax Programming/Check floats/doubles? floating point precision
| ||
| Best and neatest way to do this? The reason this is a problem is because a double or float is never exact. If MyDouble + Micro > 10 If MyDouble - Micro < 10 If MyDouble - 10 > 0 'Checks if MyDouble is greater than 10 I've seen something involving absolute values. How do you do? What would the value of Micro be for Floats/Doubles? |
| ||
| I guess you're talking about offering a tolerance value so that if the number is within tolerance then it's considered to be that number. I'm not sure if that's going to work. The floating point format, if I understand it right, changes the amount of precision available depending on in what range the numbers are, so you might have certain ranges of numbers that are more accurate than others. It also seems that the more digits you have to the left of the decimal place, the less you can have on the right. If you need things to be super accurate just use a fixed point format of your own, like one int for the left part and one int for the right. |
| ||
This is the normal method:Const EPS# = 0.00001 If Abs(MyDouble - Micro) < EPS 'numbers are the same(ish) EndIf EPS = Epsilon |
| ||
| @AngelDaniel - No I don't need them super accurate, just better than converting to closest integer. Thanks! Is there any norm for values of EPS? |
| ||
| Yeah on actual systems its around 2^-52 ;-) But the problem is this EPS only holds around 0 ... the larger the values are, the larger are the steps between the possible values you can have. This means if you plan to use large values, you might consider to include a value that modifies the testing epsilon (something that is basing on the float / double maximum value and your actual value) |
| ||
| Is there any norm for values of EPS? Not really, as it depends on the numbers being compared - different EPS are better for different apps. |
   |