Bizarre Rounding Issue

BlitzMax Forums/BlitzMax Programming/Bizarre Rounding Issue

zoqfotpik(Posted 2014) [#1]
I'm getting a totally bizarre error in division.

ivar:Float=40
foo:Float = 320.0
fraction:Float = ivar/foo
Print "ivar should be 40:"+ivar
Print "foo should be 320:"+320
Print "ivar/foo should be .125:"+ Float (ivar/foo)


The output is here:

flat assembler  version 1.68  (572428 kilobytes memory)
3 passes, 929 bytes.
Linking:untitled6.exe
Executing:untitled6.exe
ivar should be 40:40
foo should be 320:320
ivar/foo should be .125:0.000000000


Update two minutes later. I just reran it without changing anything.

My results are here:

flat assembler  version 1.68  (1050342 kilobytes memory)
3 passes, 918 bytes.
Linking:untitled6.exe
Executing:untitled6.exe
ivar should be 40:40.0000000
foo should be 320:320
ivar/foo should be .125:0.125000000


WTF IS GOING ON HERE


Derron(Posted 2014) [#2]
Compiling:test.bmx
flat assembler  version 1.68  (32768 kilobytes memory)
3 passes, 3484 bytes.
Linking:test
Executing:test
ivar should be 40:40.0000000
foo should be 320:320
ivar/foo should be .125:0.125000000
fraction should be .125:0.125000000
Kompilierung erfolgreich beendet.


bye
Ron


zoqfotpik(Posted 2014) [#3]
Right. I compiled and ran it again and it worked fine. I hadn't changed anything.

I was having a rounding error in two separate pieces of code, my real code and my test code.

In my test code the error is now no longer happening. In my real code it is giving me 1.000000 instead of 0.000000.


Floyd(Posted 2014) [#4]
You can always tell the type of number displayed by the Print statement. If there is no decimal point then it is an integer. If there is a decimal point then it is float or double, which you can distinguish by number of digits after the decimal point.

Your output implies that ivar and foo are integers. Perhaps you accidentally ran a previous version of the code.


zoqfotpik(Posted 2014) [#5]
If I paste the code that I initially pasted here, that initially delivered erroneous output, it now does not error. My live code containing substantively the same piece of code does error but now the math error is different (getting a 1 instead of zero.)

Am I losing my mind?


Brucey(Posted 2014) [#6]
Have you tried turning it off and on again? (Your Windows box, that is)


zoqfotpik(Posted 2014) [#7]
I am going to reboot.

What I should have done is paste the code to my mac but it runs fine there, and fine on the PC.

Do you have any idea what that might have been?

For whatever it's worth this is indeed an Intel PC...

Well, it's working now. I am going to walk away and chalk it up to a gremlin...


Brucey(Posted 2014) [#8]
Well, the fact that these are different :
flat assembler  version 1.68  (572428 kilobytes memory)
3 passes, 929 bytes.


flat assembler  version 1.68  (1050342 kilobytes memory)
3 passes, 918 bytes.

is rather interesting.

It would have been nice to see the generated asm for them - to compare.


Running here, on Linux :
Building untitled1
Compiling:untitled1.bmx
Linking:untitled1
Executing:untitled1
ivar should be 40:40.0000000
foo should be 320:320
ivar/foo should be .125:0.125000000

Process complete

No fasm output on mine because it doesn't use it...


ziggy(Posted 2014) [#9]
I would guess somehow you where printing a truncated version of the result as an integer.


zoqfotpik(Posted 2014) [#10]
Brucey I had added a line to print fraction, similar to the others.

It's working now, going to go outside for a while.

Mmm, just found I'm running Blitzmax 1.36... that might cause some sort of trouble.


Floyd(Posted 2014) [#11]
This is just confusion about what code was actually run. The posted code could never produce the given output. Here is the relevant part:
Print "ivar should be 40:"+ivar
Print "ivar/foo should be .125:"+ Float (ivar/foo)

ivar should be 40:40
ivar/foo should be .125:0.000000000

The displayed value of ivar is 40, which tells us that ivar is an integer. If it were float there would be a decimal point.
If foo is also integer then ivar/foo is 0, which explains the other displayed value.

It's not difficult to mix up what code you are dealing with. I've even had two instances of the IDE running and somehow forgotten about it.

The name of the program, untitled6, hints that you are doing too many things at once.


zoqfotpik(Posted 2014) [#12]
That makes sense to me but the distinct impression I had was that something very peculiar was happening. I literally pasted it here, went back to the code and it ran. I also found a trojan dropper on my box today (mohodropper D) and I am wondering if that might have poisoned the exe either in memory or at compile time.

I was definitely having trouble understanding what was going on, I had been coding for 6 hours and maybe I had depleted whatever it was in my brain that enables me to comprehend what I'm doing. I had a problem with brain fog so bad two years ago that I could barely do anything, I was barely able to write one function a day. It may very well have been adrenaline fatigue from prodigious amounts of caffeine.