Ok, it is well known floating point calculations have their limits. But what if a mathematical tool brings yet another result?
Topic is to convert molecular weight into a frequency. The calculations (with German description) and a sample input mass are as in the image, taken from a Maple app file.
Up to today only an Excel sheet exists for this calculations for public usage, which of course has its limitations. So I thought a simple tool using BigNumberMBS would do the job much better. The problem is that BigNumber (and BiggerNumber too) is much closer to the Excel sheet than Maple, but the client sees the latter as his standard. Does anyone know the internal precision of Maple float calculations or can explain where these deviations can come from?
(Dots & Commas in German meaning too – but should be enough for comparisons)
Excel
459.979.504.168.425.000.000.000.000,000
Double
4,59.979.504.168.425.512.207.646.72e+26
BigNumber
459.979.504.168.425.548.972.018.650,56669946
BiggerNumber
459.979.504.168.425.548.972.018.650,56669946
Maple
4,59.979.369.027.860.809.221.583.006*10^26
3 posts - 2 participants