Truncation and rounding of numbers: Difference between revisions

From Planimate Knowledge Base
Jump to navigation Jump to search
(Importing text file)
(No difference)

Revision as of 09:20, 10 January 2008

The Truncating operation uses the floor(x) function, this returns the highest integer that is less than or equal to x. So trunc(1.5) is 1, trunc(0.5) is 0 and trunc(-0.5) is -1.


Very small -ve values will trunc() to the value below it, just as 1.999999999999 would trunc to 1 but display as 2. The internal precision is about 15 digits.


It all comes down to the application you have for the trunc-ed number, normally "round" is much safer and should be used to convert a number which *should* be integer to an actual integer (with the precision limits of a C "double" that is).


For internal double to int conversion, I actually use a delta of 1.0e-6. Values are display-rouded to 4 decimal places. And values are compared for equality using a dynamic precision based on the highest value, of 10 digits,



idkbase note 68