Quantcast
Channel: Xojo Programming Forum - Latest topics
Viewing all articles
Browse latest Browse all 4031

Help explaining a behavior with doubles

$
0
0

I could use the help of somebody who understands the binary structure of doubles better than I do. With the following code:

Var BigNumber As Double = 999999999999999999 // 18 9's

Why does the value in BigNumber become 1000000000000000000? That’s 1 with 18 0’s.

I considered that it could be a formatting issue, so I tried some comparisons:

Var BigNumber As Double = 999999999999999999
Var BiggerNumber As Double = 1000000000000000000

If BigNumber > BiggerNumber Then
  Break
ElseIf BigNumber < BiggerNumber Then
  Break
Else
  Break // This one hits, so they are equal
End If

I’m sure this is some quirk with how they are stored in memory, but I don’t know what it is.

3 posts - 2 participants

Read full topic


Viewing all articles
Browse latest Browse all 4031

Trending Articles