NewsForumsCodeLogsGallerySpecsSearch

Precision problems ...

BlitzMax Forums/BlitzMax Programming/Precision problems ...

HrdNutz(Posted 2007) [#1]
Hi,

I seem to be running into a weird issue, and i'm trying to figure out if it's my machine or BMAX. The following code prints out:

25.000000000000000
25.000000000000000
0.00000000000000000

which is incorrect. can anyone please run this and confirm if their numbers look ok? I seem to be having precision problems here :/

Local t1:Double = 25.0000002
Local t2:Double = 25.0000001
Local t3:Double
t3 = t1 - t2
	
Print t1
Print t2
Print t3



HrdNutz(Posted 2007) [#2]
when I run :
Local t1:Double = 25.002
Local t2:Double = 25.001
Local t3:Double
t3 = t1 - t2
	
Print t1
Print t2
Print t3



i get :
25.002000808715820
25.000999450683594
0.0010013580322265625


almost looks like my CPU is having problems, can anyone confirm please?

thanks in advance


Gabriel(Posted 2007) [#3]
What's unusual about this? Even with double precision floating point numbers, not every number can be perfectly represented in 64 bits.


HrdNutz(Posted 2007) [#4]
Whats unusual is that i dont know how to treat those numbers ...

Local t1:Double = 25.6000061
Local t2:Double = 25.6000004
Local t3:Double
t3 = t1 - t2

Print t1
Print t2
Print t3



results in :

25.600006103515625
25.600000381469727
5.7220458984375000e-006


5.7220458984375000e-006 isnt the result im looking for


Tachyon(Posted 2007) [#5]
Those are stunning results. (oh, I failed to notice the "e-006")


Perturbatio(Posted 2007) [#6]
5.7220458984375000e-006 isnt the result im looking for

What result are you looking for?

that's 0.0000057220458984375 which is as close to zero as you need:
Local t1:Double = 25.6000061:Double 'numbers with decimals default to floats, force double to keep some precision
Local t2:Double = 25.6000004:Double
Local t3:Double
t3 = t1 - t2

Print t1
Print t2
If t3 < 0.00001 Then
	Print 0
Else 
	Print t3
EndIf



HrdNutz(Posted 2007) [#7]
my problem comes when i try to compare some numbers, i'll clip them manually this time, thanks for the help.


DJWoodgate(Posted 2007) [#8]
As regards the intial post, the problem is that the constants are being defined as single precision floats. So add ! to make them double precision.

Edit. oops. Obviously I did not read the whole thread and missed Perturbatios response.