Skip to content

Commit

Permalink
Convert fill values correctly from negative to unsigned integers (#165)
Browse files Browse the repository at this point in the history
* Convert fill values correctly from negative to unsigned integers

Positive integers are unaffected by this change anyway.

* Only reinterpret negative integers when decoding fill values to unsigned

---------

Co-authored-by: Fabian Gans <[email protected]>
  • Loading branch information
asinghvi17 and meggart authored Jan 6, 2025
1 parent b52be51 commit 6c8f5c4
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion src/metadata.jl
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,13 @@ Base.eltype(::Metadata{T}) where T = T
fill_value_decoding(v::AbstractString, T::Type{<:Number}) = parse(T, v)
fill_value_decoding(v::Nothing, ::Any) = v
fill_value_decoding(v, T) = T(v)
fill_value_decoding(v::Integer, T::Type{<: Unsigned}) = reinterpret(T, signed(T)(v))
fill_value_decoding(v::Number, T::Type{String}) = v == 0 ? "" : T(UInt8[v])
fill_value_decoding(v, ::Type{ASCIIChar}) = v == "" ? nothing : v
# Sometimes when translating between CF (climate and forecast) convention data
# and Zarr groups, fill values are left as "negative integers" to encode unsigned
# integers. So, we have to convert to the signed type with the same number of bytes
# as the unsigned integer, then reinterpret as unsigned. That's how a fill value
# of -1 can have a realistic meaning with an unsigned dtype.
# However, we have to apply this correction only if the integer is negative.
# If it's positive, then the value might be out of range of the signed integer type.
fill_value_decoding(v::Integer, T::Type{<: Unsigned}) = sign(v) < 0 ? reinterpret(T, signed(T)(v)) : T(v)

0 comments on commit 6c8f5c4

Please sign in to comment.