Anyone else have problems with String.ToFloat()?

BlitzMax Forums/BlitzMax Programming/Anyone else have problems with String.ToFloat()?

lonnieh(Posted 2010) [#1]
Some results are good, some are weird. For example ".003".ToFloat() returns 0.00300000003. I'm using version 1.41 on linux and Windows XP. Any one else have this problem?


xlsior(Posted 2010) [#2]
This is a well-known limitation of Floats in general: not all numerical values can be represented accurately, and you get some weird 'rounding' errors like the one you see here.

Unfortunately, that's the way they are designed to work, it's not blitzmax-specific.

For more info, see: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems


lonnieh(Posted 2010) [#3]
Thanks for the heads up! Today I learned about floating point accuracy problems...


xlsior(Posted 2010) [#4]
biggest thing to keep in mind when dealing with floats, is that you can't do an exact match: "if x=0.003" will never be true thanks to this issue.

If you are using floats, it means that you may need to compare against a small range instead of a 100% match. (if x>=0.002999999 or x<=0.003000001 then...)


Czar Flavius(Posted 2010) [#5]
If you need perfect percision, try an arbitrary percision maths library (but they are very slow). However, apart from looking bad and annoying you, an inaccuracy of 0.00000000003 is insignificant for most games, so don't worry about it.