Precision Settings

ToFUL exposes two independent precision controls: Calc precision governs the accuracy of internal computations; Display precision governs how many decimal places appear in the UI. They are deliberately separate so you can, for example, compute to 12 significant figures but display only 4.

Calc Precision

Setting: 1 – 15 decimal places. Default: 8.

This value sets the tolerance used by the computation engine:

  • For discrete series, the convergence threshold is \(10^{-\text{prec}}\). A term or partial-sum difference smaller than this is treated as effectively zero.

  • For continuous integration (SciPy quad), both epsabs and epsrel are set to \(10^{-(\text{prec}+4)}\), clamped at \(10^{-15}\) (the float64 floor).

  • Above 12 decimal places, the engine automatically switches from SciPy to mpmath’s tanh-sinh quadrature, which supports arbitrary precision. This is slower but eliminates float64 rounding errors.

Recommended values:

Setting

Use case

Accuracy

4

Quick exploration

~6 significant figures

8

Default — most use cases

~10 significant figures

12

Research / verification

Near float64 machine ε

13–15

Arbitrary precision (mpmath)

15+ significant figures

Display Precision

Setting: 1 – 15 decimal places. Default: 4.

This controls how many decimal places are shown in:

  • The moment metric cards (Moments tab)

  • The statistical summary cards (Statistics tab)

  • All values in the detailed results table (Table tab)

  • Axis labels in distribution plots

It does not affect the accuracy of computation — only the rendering. Setting it higher than Calc precision will show trailing zeros that have no computational significance.

When to Increase Calc Precision

Increase Calc precision when:

  • You see convergence warnings in the Convergence tab even after increasing Max series terms.

  • Moments for different orders vary wildly in a way that seems numerically unstable (common for high-order moments, e.g. r > 10, on broad distributions).

  • You are computing moments of distributions where the PDF or PMF has values very close to zero (e.g. Poisson with λ=0.1 and x > 5).

  • You need to verify a result against an analytical formula to many significant figures.

When to Increase Display Precision

Increase Display precision when:

  • Moments are very small (e.g. in the range 1e-6) and the default 4 decimal places rounds everything to 0.0000.

  • You are copying values to a paper or report and need more digits.

  • You want to verify that two moments agree to more than 4 places.

Float64 Limits

Python floats (float) are IEEE 754 double-precision values with approximately 15–17 significant decimal digits. Even with Calc precision set to 15, results cannot be more accurate than this unless mpmath is active (precision ≥ 13).

For discrete series, the accumulated rounding error from summing hundreds of terms grows with the number of terms. The convergence accelerators (Wynn ε, Aitken Δ²) partially compensate by needing fewer raw terms.

mpmath Mode

When Calc precision ≥ 13, the backend sets:

mpmath.mp.dps = precision + 10

and uses tanh-sinh quadrature for all integration. The extra 10 guard digits protect against internal cancellation during computation.

Discrete series still use NumPy/float64 arithmetic in mpmath mode — only continuous integration benefits from arbitrary precision.

See also