Anyone else have problems with String.ToFloat()?
BlitzMax Forums/BlitzMax Programming/Anyone else have problems with String.ToFloat()?
| ||
Some results are good, some are weird. For example ".003".ToFloat() returns 0.00300000003. I'm using version 1.41 on linux and Windows XP. Any one else have this problem? |
| ||
This is a well-known limitation of Floats in general: not all numerical values can be represented accurately, and you get some weird 'rounding' errors like the one you see here. Unfortunately, that's the way they are designed to work, it's not blitzmax-specific. For more info, see: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems |
| ||
Thanks for the heads up! Today I learned about floating point accuracy problems... |
| ||
biggest thing to keep in mind when dealing with floats, is that you can't do an exact match: "if x=0.003" will never be true thanks to this issue. If you are using floats, it means that you may need to compare against a small range instead of a 100% match. (if x>=0.002999999 or x<=0.003000001 then...) |
| ||
If you need perfect percision, try an arbitrary percision maths library (but they are very slow). However, apart from looking bad and annoying you, an inaccuracy of 0.00000000003 is insignificant for most games, so don't worry about it. |