Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.14 cannot create variable from an Int64 array with missing values #246

Closed
sjdaines opened this issue Feb 9, 2024 · 3 comments
Closed

Comments

@sjdaines
Copy link

sjdaines commented Feb 9, 2024

Describe the bug

With NCDatasets v0.14.1, creating a netcdf variable from a Julia Int64 array with missing values causes an ERROR: InexactError

This used to work on NCDatasets v0.12.x

To Reproduce

julia> import NCDatasets
julia>  ds = NCDatasets.NCDataset("test.nc","c")
julia> NCDatasets.defDim(ds, "mydim", 2)

julia> v = NCDatasets.defVar(ds, "test_array_i64_missing", [missing, 2], ("mydim", ))
ERROR: InexactError: trunc(Int32, -9223372036854775806)

(see full output below)

Expected behavior

Should create a variable v with Datatype: Union{Missing, Int64} (Int64)

Examples that do work as expected:

julia> v = NCDatasets.defVar(ds, "test_array_i64", [1, 2], ("mydim", )) # OK
test_array_i64 (2)
  Datatype:    Int64 (Int64)
  Dimensions:  mydim

julia> v = NCDatasets.defVar(ds, "test_array_i32_missing", [missing, Int32(2)], ("mydim", )) # OK
test_array_i32_missing (2)
  Datatype:    Union{Missing, Int32} (Int32)
  Dimensions:  mydim
  Attributes:
   _FillValue           = -2147483647
julia> Array(v)
2-element Vector{Union{Missing, Int32}}:
  missing
 2

Environment

julia> versioninfo()
Julia Version 1.10.0
Commit 3120989f39 (2023-12-25 18:01 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Windows (x86_64-w64-mingw32)
 CPU: 4 × Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
 Threads: 1 on 4 virtual cores
  • NCDatasets version: v0.14.1
  • Output of using Pkg; Pkg.status(mode=PKGMODE_MANIFEST)

Full output

julia> v = NCDatasets.defVar(ds, "test_array_i64_missing", [missing, 2], ("mydim", ))
ERROR: InexactError: trunc(Int32, -9223372036854775806)
Stacktrace:
  [1] throw_inexacterror(f::Symbol, ::Type{Int32}, val::Int64)
    @ Core .\boot.jl:634
  [2] checked_trunc_sint
    @ .\boot.jl:656 [inlined]
  [3] toInt32
    @ .\boot.jl:693 [inlined]
  [4] Int32
    @ .\boot.jl:783 [inlined]
  [5] nc_put_att(ncid::Int32, varid::Int32, name::String, data::Int32)
    @ NCDatasets C:\Users\sd336\.julia\packages\NCDatasets\mKQDc\src\netcdf_c.jl:564 [inlined]
  [6] defAttrib(ds::NCDatasets.Variable{Int64, 1, NCDatasets.NCDataset{Nothing, Missing}}, name::String, data::Int64)
    @ NCDatasets C:\Users\sd336\.julia\packages\NCDatasets\mKQDc\src\attributes.jl:85
  [7] setindex!(a::CommonDataModel.Attributes{NCDatasets.Variable{Int64, 1, NCDatasets.NCDataset{Nothing, Missing}}}, data::Int64, name::String)
    @ CommonDataModel C:\Users\sd336\.julia\packages\CommonDataModel\SVxJP\src\attribute.jl:112
  [8] defVar(ds::NCDatasets.NCDataset{…}, name::String, vtype::DataType, dimnames::Tuple{…}; chunksizes::Nothing, shuffle::Bool, deflatelevel::Nothing, checksum::Nothing, fillvalue::Nothing, nofill::Bool, typename::Nothing, attrib::OrderedCollections.OrderedDict{…})
    @ NCDatasets C:\Users\sd336\.julia\packages\NCDatasets\mKQDc\src\cfvariable.jl:146
  [9] defVar
    @ C:\Users\sd336\.julia\packages\NCDatasets\mKQDc\src\cfvariable.jl:99 [inlined]
 [10] _defVar(ds::NCDatasets.NCDataset{…}, name::String, data::Vector{…}, nctype::Type, vardimnames::Tuple{…}; attrib::Vector{…}, kwargs::@Kwargs{})
    @ CommonDataModel C:\Users\sd336\.julia\packages\CommonDataModel\SVxJP\src\variable.jl:173
 [11] _defVar
    @ CommonDataModel C:\Users\sd336\.julia\packages\CommonDataModel\SVxJP\src\variable.jl:123 [inlined]
 [12] #defVar#5
    @ CommonDataModel C:\Users\sd336\.julia\packages\CommonDataModel\SVxJP\src\variable.jl:101 [inlined]
 [13] defVar(ds::NCDatasets.NCDataset{Nothing, Missing}, name::String, data::Vector{Union{Missing, Int64}}, dimnames::Tuple{String})
    @ CommonDataModel C:\Users\sd336\.julia\packages\CommonDataModel\SVxJP\src\variable.jl:91
 [14] top-level scope
    @ REPL[10]:1
Some type information was truncated. Use `show(err)` to see complete types.
sjdaines added a commit to PALEOtoolkit/PALEOmodel.jl that referenced this issue Feb 9, 2024
Workaround for issue in NCDatasets v0.14 (and probably in v0.13 as well)
 JuliaGeo/NCDatasets.jl#246
 "v0.14 cannot create variable from an Int64 array with missing values"
@Alexander-Barth
Copy link
Member

Thank you for submitting the detailed bug report.
I am currently testing this fix:
4638a21

@Alexander-Barth
Copy link
Member

All tests do pass. I will make a new release.
https://github.com/Alexander-Barth/NCDatasets.jl/actions/runs/7849851817
Please re-open of the issue persists.

sjdaines added a commit to PALEOtoolkit/PALEOmodel.jl that referenced this issue Feb 10, 2024
Issue in NCDatasets v0.13 (probably) - v0.14.1 caused failure, now fixed in v0.14.2
JuliaGeo/NCDatasets.jl#246
"v0.14 cannot create variable from an Int64 array with missing values"

Update Project.toml [compat] to require either v0.12 or >= v0.14.2
sjdaines added a commit to PALEOtoolkit/PALEOmodel.jl that referenced this issue Feb 10, 2024
Issue in NCDatasets v0.13 (probably) - v0.14.1 caused failure, now fixed in v0.14.2
JuliaGeo/NCDatasets.jl#246
"v0.14 cannot create variable from an Int64 array with missing values"

Update Project.toml [compat] to require either v0.12 or >= v0.14.2
@sjdaines
Copy link
Author

Fantastic ! Thank you !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants