"e" numbers

BlitzMax Forums/BlitzMax Beginners Area/"e" numbers

EOF	(Posted 2011) [#1]

I have a database of high precision numbers in a plain text file which I want to re-organise into simpler/shorter equivalents (I don't need the high precision so can afford to drop several digits)

E.g,
From "0.01234546789876" to "0.12345"

That's easy enough, but some of the numbers in the database contain the "e" notation, and I can't quite figure out how to covert these. Example:

2.3432602026631493e-17

My understanding is that the "e-17" bit means the number is multiplied by 10^-17. Correct?

Does that mean 2.3432602026631493e-17 is the same as 0.0000000000000000234 ?

Last edited 2011

col	(Posted 2011) [#2]

Yep, you got it.

Last edited 2011

EOF	(Posted 2011) [#3]

Thanks Dave

Czar Flavius

(Posted 2011) [#4]

Maybe if you convert from a string containing the "e" notation to float, it might just work?? Worth a try.

col	(Posted 2011) [#5]

@Czar Flavius

Print Double("2.2e-4")

prints out

0.00022000000000000001

However this

Print Double("2.3432602026631493e-017")

will still print

2.3432602026631493e-017

The point of printing the exponent as an 'e-' is at e-5 on my machine. e-4 will print all the expected zeros.

I'm sure this just a conversion in the string printing process though ?? The actual value will still be the expected value when used in arithmetic.

Last edited 2011

Czar Flavius

(Posted 2011) [#6]

do you want to convert string to float or float to string? That is how floats with too many decimal places appear when they are converted to strings. If you want to print them as a raw number, you'll need to write your own code.

EOF	(Posted 2011) [#7]

String to float. I solved my issue by checking for "e-" in the string and returning 0.0 if present

Czar Flavius

(Posted 2011) [#8]

i guess that will work if you're not interested in really small or large numbers..