[proFit-list] 6.2.14: rounding issue with large numbers?

Holger Apfel mail at apfel-edv.de
Wed Nov 12 05:59:51 CST 2014


I've to convert some large numbers from hex to decimal. I'm using a script and so far I've succeeded in converting the hex value into a number within the script and to convert this into a text to put it back into the data sheet.

But as soon as the value, ex 

1.415718971000000e+9 (text within a script)

is copied into the datasheet with SetCell(x,y,txt) the number is changed into

1.415718912000e+9 - which is obviously quite different. As the number represents a time in seconds the last digits are quite important.

Did I miss something?

Holger Apfel
Fischerstr. 3

85221 Dachau

08131 338997






More information about the proFit-list mailing list