Bizarre Rounding Issue
BlitzMax Forums/BlitzMax Programming/Bizarre Rounding Issue
| ||
I'm getting a totally bizarre error in division.ivar:Float=40 foo:Float = 320.0 fraction:Float = ivar/foo Print "ivar should be 40:"+ivar Print "foo should be 320:"+320 Print "ivar/foo should be .125:"+ Float (ivar/foo) The output is here: flat assembler version 1.68 (572428 kilobytes memory) 3 passes, 929 bytes. Linking:untitled6.exe Executing:untitled6.exe ivar should be 40:40 foo should be 320:320 ivar/foo should be .125:0.000000000 Update two minutes later. I just reran it without changing anything. My results are here: flat assembler version 1.68 (1050342 kilobytes memory) 3 passes, 918 bytes. Linking:untitled6.exe Executing:untitled6.exe ivar should be 40:40.0000000 foo should be 320:320 ivar/foo should be .125:0.125000000 WTF IS GOING ON HERE |
| ||
Compiling:test.bmx flat assembler version 1.68 (32768 kilobytes memory) 3 passes, 3484 bytes. Linking:test Executing:test ivar should be 40:40.0000000 foo should be 320:320 ivar/foo should be .125:0.125000000 fraction should be .125:0.125000000 Kompilierung erfolgreich beendet. bye Ron |
| ||
Right. I compiled and ran it again and it worked fine. I hadn't changed anything. I was having a rounding error in two separate pieces of code, my real code and my test code. In my test code the error is now no longer happening. In my real code it is giving me 1.000000 instead of 0.000000. |
| ||
You can always tell the type of number displayed by the Print statement. If there is no decimal point then it is an integer. If there is a decimal point then it is float or double, which you can distinguish by number of digits after the decimal point. Your output implies that ivar and foo are integers. Perhaps you accidentally ran a previous version of the code. |
| ||
If I paste the code that I initially pasted here, that initially delivered erroneous output, it now does not error. My live code containing substantively the same piece of code does error but now the math error is different (getting a 1 instead of zero.) Am I losing my mind? |
| ||
Have you tried turning it off and on again? (Your Windows box, that is) |
| ||
I am going to reboot. What I should have done is paste the code to my mac but it runs fine there, and fine on the PC. Do you have any idea what that might have been? For whatever it's worth this is indeed an Intel PC... Well, it's working now. I am going to walk away and chalk it up to a gremlin... |
| ||
Well, the fact that these are different :flat assembler version 1.68 (572428 kilobytes memory) 3 passes, 929 bytes. flat assembler version 1.68 (1050342 kilobytes memory) 3 passes, 918 bytes. is rather interesting. It would have been nice to see the generated asm for them - to compare. Running here, on Linux : Building untitled1 Compiling:untitled1.bmx Linking:untitled1 Executing:untitled1 ivar should be 40:40.0000000 foo should be 320:320 ivar/foo should be .125:0.125000000 Process complete No fasm output on mine because it doesn't use it... |
| ||
I would guess somehow you where printing a truncated version of the result as an integer. |
| ||
Brucey I had added a line to print fraction, similar to the others. It's working now, going to go outside for a while. Mmm, just found I'm running Blitzmax 1.36... that might cause some sort of trouble. |
| ||
This is just confusion about what code was actually run. The posted code could never produce the given output. Here is the relevant part:Print "ivar should be 40:"+ivar Print "ivar/foo should be .125:"+ Float (ivar/foo) ivar should be 40:40 ivar/foo should be .125:0.000000000 The displayed value of ivar is 40, which tells us that ivar is an integer. If it were float there would be a decimal point. If foo is also integer then ivar/foo is 0, which explains the other displayed value. It's not difficult to mix up what code you are dealing with. I've even had two instances of the IDE running and somehow forgotten about it. The name of the program, untitled6, hints that you are doing too many things at once. |
| ||
That makes sense to me but the distinct impression I had was that something very peculiar was happening. I literally pasted it here, went back to the code and it ran. I also found a trojan dropper on my box today (mohodropper D) and I am wondering if that might have poisoned the exe either in memory or at compile time. I was definitely having trouble understanding what was going on, I had been coding for 6 hours and maybe I had depleted whatever it was in my brain that enables me to comprehend what I'm doing. I had a problem with brain fog so bad two years ago that I could barely do anything, I was barely able to write one function a day. It may very well have been adrenaline fatigue from prodigious amounts of caffeine. |