Graphics UnitMath FAQ ( Precision ) Last Modified: 2/3/2000

email Examples FAQ Home Sales $$$ What's New Windows demo

What is Precision? TOP

Precision is the exactness ( uncertainty ) of a measurement.

All measurements have uncertainties that range from exact to the really rough "ball park figure". For example there is exactly one item being measured below, but its length is not exact. Due to the unevenness of the object and the courseness of the ruler, the best we can say is the length is about 4.8 inches. It would be misleading to say the object has a length of 4.81950 inches.


Precision is often represented in terms of significant digits.

What are significant digits (figures)? TOP

Significant digits are the number of meaningful digits in a number. More significant digits means the value is know more precisely. For example "5." has one significant digit and implies a value is between 4.5 and 5.5; while, "4.8" has 2 significant digits and implies a value is between 4.75 and 4.85 and is more precise. 4.81950 inches has six significant digits and implies a range of 4.819495 to 4.819505 inches. The following table should clarify this.

Value Low High % { ( high - low ) / low }
5. 4.5 5.5 22.2%
4.8 4.75 5.85 2.1%
4.81950 4.819495 4.819505 2.1/10,000 %

Significant digits are very useful in estimating the precision of a calculation. The rule of thumb for multiplication and division is: the precision of the answer has no more significant digits than the smallest number of significant digits in the equation. Therefore 8.95 * 0.53 ~ 4.744 would have at best 2 significant digits in the answer so 8.95 * 0.53 ~ 4.7.

Is precision in UnitMath just significant digits? TOP

No, significant digits are just one of many ways to describe precision in UnitMath. This makes UnitMath easier to use, and more precise than just using significant digits.

Should I care about Precision? TOP

Yes, in fact you already do.

If you were told the cost of an item is "$100 give or take", you would be very interested in finding out what give or take meant. Is that give or take $1 or $10,000? Give or take is the precision of the $100.

Is more precision always better? TOP

No, the acceptable precision depends on the question being asked. Increasing precision beyond what is needed to answer the question wastes time and effort and can be very expensive.