Nonintuitive result of the assignment of a double precision number to an int variable in C

by

… why I get two different numbers …

Aside from the usual float-point issues, the computation paths to b and c are arrived in different ways. c is calculated by first saving the value as double a.

double a =(Vmax-Vmin)/step;
int b = (Vmax-Vmin)/step;
int c = a;

C allows intermediate floating-point math to be computed using wider types. Check the value of FLT_EVAL_METHOD from <float.h>.

Except for assignment and cast (which remove all extra range and precision), …

-1 indeterminable;

0 evaluate all operations and constants just to the range and precision of the
type;

1 evaluate operations and constants of type float and double to the
range and precision of the double type, evaluate long double
operations and constants to the range and precision of the long double
type;

2 evaluate all operations and constants to the range and precision of the
long double type.

C11dr ยง5.2.4.2.2 9

OP reported 2

By saving the quotient in double a = (Vmax-Vmin)/step;, precision is forced to double whereas int b = (Vmax-Vmin)/step; could compute as long double.

This subtle difference results from (Vmax-Vmin)/step (computed perhaps as long double) being saved as a double versus remaining a long double. One as 15 (or just above), and the other just under 15. int truncation amplifies this difference to 15 and 14.

On another compiler, the results may both have been the same due to FLT_EVAL_METHOD < 2 or other floating-point characteristics.

Conversion to int from a floating-point number is severe with numbers near a whole number. Often better to round() or lround(). The best solution is situation dependent.

Leave a Comment