Float and Int Precision After Multiplication

Monkey Forums/Monkey Programming/Float and Int Precision After Multiplication

c.k.(Posted 2012) [#1]
I've got a float value of 9.2, that when multiplied by a const SCORE_PRECISION, which equals 100000000 (one hundred million), is becoming 919999999.9999999.

Is there any way to get this to work properly so 9.2 * SCORE_PRECISION becomes 920000000?


Cygnus(Posted 2012) [#2]
This is normal behavior with floats.

If you use int(your_variable+0.5) then it will round the figure to the nearest integer.


Gerry Quinn(Posted 2012) [#3]
And beware when targeting Android, floats are single precision on it (should be an option to use doubles IMO) and such problems will be worse.


c.k.(Posted 2012) [#4]
I know it's somewhat expected behavior, but is there a work-around? There's got to be a way! :-)

Cygnus, I don't want to round, because the values are sometimes going to be 7 or 8 decimal places. I need to keep that precision and accuracy.

Gerry, I think that's true of the HTML5 target as well, which is where I'm experiencing the problem.


ziggy(Posted 2012) [#5]
If you use int(your_variable+0.5) then it will round the figure to the nearest integer.
This is only valid por positive values.
Int(your_variable+0.5*Sng(your_variable))
works for both positive and negative numbers.


Gerry Quinn(Posted 2012) [#6]
If you really have that many digits it's double precision. Single-precision floats have only about eight decimal digits.

You don't have to round the float itself: just generate a rounded int and use that for scores or whatever.

This issue comes with the territory when using floats. There's no simple fix for it.


c.k.(Posted 2012) [#7]
The only thing I can imagine doing is taking the float and assigning it to a string.

Run this:

[monkeycode]

Function multiply:String(f:Float, p:Int)
Local s:String = String(f)
Local d:Int = s.Find(".")
Local sP:String = String(p)
Local pL:Int = sP.Length()

Local sZero:String = "0000000000"

Local mant:String
Local dant:String

If d > 0 Then ' there's a decimal
mant = s[ .. d]
dant = s[d + 1 ..]
EndIf

Print "original s: '" + s + "'~nmant: " + mant + ", dant: " + dant
Print "precision: " + pL

Local x:Int = (mant.Length() +dant.Length()) -pL
Print "x = " + x
If x < 0 Then
dant += sZero[ .. Abs(x)]
Else

EndIf

Return mant + dant

End

Function Main()
Local myFloat:Float = 9.2
Local myString:String = String(myFloat)
Local prec:Int = 100000000

Print "From " + myFloat + " -> " + prec
myString = multiply(myFloat, prec)
Print "Stringified: " + myString
Print " multiplied: " + myFloat * prec

End
[/monkeycode]

That works for what I need: it guarantees (somewhat) that my floats will be accurately and precisely multiplied by the size factor. I'm sure it can be done better. There are probably bugs. Maybe even Diddy already has this functionality.

What do y'all think, though, as a foundation for accurate number manipulation?


Gerry Quinn(Posted 2012) [#8]
What are you trying to achieve exactly?

If you want arbitrary precision rationals or decimals, you will have to either find or develop a class of objects to represent them. If you search you'll find code for this in a variety of languages, which you will be able to adapt to Monkey (assuming nobody already has). Simple floats and integers in most languages including Monkey have limited precision. They are designed to fit neatly in the arithmetic registers of computer processors.

The downside is that arbitrary-precision numbers are much slower to calculate with. Ask yourself whether your application really needs that much precision.


c.k.(Posted 2012) [#9]
I'm keeping track of scores that are very precise, and need to be very precise to determine rank/placement on a leaderboard.

9.1234567 is a better score than 9.1234566.

Some platforms can handle greater precision, but for the lowest common denominators, I'm having to truncate/round. I need that to be as accurate as possible.

Since I'm only doing the calculation once, right after the game is over, it's not a huge deal, thankfully!

And I don't want a lot of ties on the board. :-)