Skip to content

Commit

Permalink
Use muladd in aleph calculation in _quantile to avoid some rounding
Browse files Browse the repository at this point in the history
errors.

In some cases when aleph is an integer (in exact) arithmetic, the
calculation of aleph via m can introduce some rounding errors that
makes the value non-integer. Using muladd avoids this in some cases.
  • Loading branch information
andreasnoack committed Jan 2, 2025
1 parent 793733e commit ee29e57
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 2 deletions.
5 changes: 3 additions & 2 deletions src/Statistics.jl
Original file line number Diff line number Diff line change
Expand Up @@ -1035,8 +1035,9 @@ end
@assert n > 0 # this case should never happen here

m = alpha + p * (one(alpha) - alpha - beta)
aleph = n*p + oftype(p, m)
j = clamp(trunc(Int, aleph), 1, n-1)
# Using muladd here avoids some rounding errors when aleph is an integer
aleph = muladd(n, p, m)
j = clamp(trunc(Int, aleph), 1, n - 1)
γ = clamp(aleph - j, 0, 1)

if n == 1
Expand Down
5 changes: 5 additions & 0 deletions test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -796,6 +796,11 @@ end
@test quantile(v, 0.8, alpha=1.0, beta=1.0) 10.6
@test quantile(v, 1.0, alpha=0.0, beta=0.0) 21.0
@test quantile(v, 1.0, alpha=1.0, beta=1.0) 21.0

@testset "avoid some rounding" begin
@test [quantile(1:10, i/9) for i in 0:9] == 1:10
@test [quantile(1:14, i/13) for i in 0:13] == 1:14
end
end

# StatsBase issue 164
Expand Down

0 comments on commit ee29e57

Please sign in to comment.