XArrayDB Reference > XArrayDB Object Properties > Precision Property (XArrayDB) |
Precision Property (XArrayDB)
This property sets or returns a double that determines the precision used for floating point comparisons performed during sorting and searching operations.
XArrayDB.Precision= double
Read-only at run time. Not available at design time.
Property applies to XArrayDB object.
By default, this property returns 0, and all floating point values stored within an XArrayDB object are compared as is.
You can set the Precision property to a nonzero value to specify a tolerance for floating point values processed by the Find and QuickSort methods. If the difference between two values is less than or equal to the Precision value, then XArrayDB considers them to be equal for comparison purposes.
Example
The following example initializes a single-column XArrayDB object containing double-precision numbers, then uses the Find method to demonstrate the effect of the Precision property:
MyArray.ReDim 1, 4, 1, 1
MyArray(1, 1) = 198#
MyArray(2, 1) = 150#
MyArray(3, 1) = 200#
MyArray(4, 1) = 250#
' find an exact match for 200#
Debug.Print MyArray.Find(1, 1, 200#) ' prints 3
' change the precision to plus or minus 5
MyArray.Precision = 5#
' find values between 195# and 205#
Debug.Print MyArray.Find(1, 1, 200#) ' prints 1
See Also