Is floating point precision mutable or invariant?

The precision is fixed, which is exactly 53 binary digits for double-precision (or 52 if we exclude the implicit leading 1). This comes out to about 15 decimal digits.


The OP asked me to elaborate on why having exactly 53 binary digits means “about” 15 decimal digits.

To understand this intuitively, let’s consider a less-precise floating-point format: instead of a 52-bit mantissa like double-precision numbers have, we’re just going to use a 4-bit mantissa.

So, each number will look like: (-1)s × 2yyy × 1.xxxx (where s is the sign bit, yyy is the exponent, and 1.xxxx is the normalised mantissa). For the immediate discussion, we’ll focus only on the mantissa and not the sign or exponent.

Here’s a table of what 1.xxxx looks like for all xxxx values (all rounding is half-to-even, just like how the default floating-point rounding mode works):

  xxxx  |  1.xxxx  |  value   |  2dd  |  3dd  
--------+----------+----------+-------+--------
  0000  |  1.0000  |  1.0     |  1.0  |  1.00
  0001  |  1.0001  |  1.0625  |  1.1  |  1.06
  0010  |  1.0010  |  1.125   |  1.1  |  1.12
  0011  |  1.0011  |  1.1875  |  1.2  |  1.19
  0100  |  1.0100  |  1.25    |  1.2  |  1.25
  0101  |  1.0101  |  1.3125  |  1.3  |  1.31
  0110  |  1.0110  |  1.375   |  1.4  |  1.38
  0111  |  1.0111  |  1.4375  |  1.4  |  1.44
  1000  |  1.1000  |  1.5     |  1.5  |  1.50
  1001  |  1.1001  |  1.5625  |  1.6  |  1.56
  1010  |  1.1010  |  1.625   |  1.6  |  1.62
  1011  |  1.1011  |  1.6875  |  1.7  |  1.69
  1100  |  1.1100  |  1.75    |  1.8  |  1.75
  1101  |  1.1101  |  1.8125  |  1.8  |  1.81
  1110  |  1.1110  |  1.875   |  1.9  |  1.88
  1111  |  1.1111  |  1.9375  |  1.9  |  1.94

How many decimal digits do you say that provides? You could say 2, in that each value in the two-decimal-digit range is covered, albeit not uniquely; or you could say 3, which covers all unique values, but do not provide coverage for all values in the three-decimal-digit range.

For the sake of argument, we’ll say it has 2 decimal digits: the decimal precision will be the number of digits where all values of those decimal digits could be represented.


Okay, then, so what happens if we halve all the numbers (so we’re using yyy = -1)?

  xxxx  |  1.xxxx  |  value    |  1dd  |  2dd  
--------+----------+-----------+-------+--------
  0000  |  1.0000  |  0.5      |  0.5  |  0.50
  0001  |  1.0001  |  0.53125  |  0.5  |  0.53
  0010  |  1.0010  |  0.5625   |  0.6  |  0.56
  0011  |  1.0011  |  0.59375  |  0.6  |  0.59
  0100  |  1.0100  |  0.625    |  0.6  |  0.62
  0101  |  1.0101  |  0.65625  |  0.7  |  0.66
  0110  |  1.0110  |  0.6875   |  0.7  |  0.69
  0111  |  1.0111  |  0.71875  |  0.7  |  0.72
  1000  |  1.1000  |  0.75     |  0.8  |  0.75
  1001  |  1.1001  |  0.78125  |  0.8  |  0.78
  1010  |  1.1010  |  0.8125   |  0.8  |  0.81
  1011  |  1.1011  |  0.84375  |  0.8  |  0.84
  1100  |  1.1100  |  0.875    |  0.9  |  0.88
  1101  |  1.1101  |  0.90625  |  0.9  |  0.91
  1110  |  1.1110  |  0.9375   |  0.9  |  0.94
  1111  |  1.1111  |  0.96875  |  1.   |  0.97

By the same criteria as before, we’re now dealing with 1 decimal digit. So you can see how, depending on the exponent, you can have more or less decimal digits, because binary and decimal floating-point numbers do not map cleanly to each other.

The same argument applies to double-precision floating point numbers (with the 52-bit mantissa), only in that case you’re getting either 15 or 16 decimal digits depending on the exponent.

Leave a Comment