Floating point addition/subtraction in tcl/python

75 views Asked by At

I am trying to perform subtraction of the numbers

in tcl or python: expr { 202.2246-0.42} = 201.80460000000002

ideally the answer should be 201.8046

One way to solve it would be to convert it to integer and do the subtraction and convert it back but the no of decimal digits are not fixed and I would not prefer to multiply it with 10^9 (int limit) - maybe as a last resort.

The precision is highly important. Why does the result varies from the exact result ? How can I control it in a generic way - not only for these operands?

0

There are 0 answers