Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

F# class serialization: mutable fields within F# records are duped with @ suffix #557

Open
nodakai opened this issue Oct 1, 2024 · 2 comments

Comments

@nodakai
Copy link

nodakai commented Oct 1, 2024

Library Version

4.23.0

OS

macOS 14.6.1 (23G93)

OS Architecture

64 bit

How to reproduce?

module Main

open System.IO
open Parquet.Serialization

[<CLIMutable>]
type Foo = {
  mutable A: int
  B: float
}

let [<EntryPoint>] main _ =
  task {
    use f = File.Create "test.parquet"
    let! sch = ParquetSerializer.SerializeAsync([|{A=1; B=1.}|], f)
    printfn $"schema:\n%O{sch}"
    return 0
  } |> _.GetAwaiter().GetResult()

<= 4.22.1

schema:
A (System.Int32)
B (System.Double)

>= 4.23.0

schema:
A (System.Int32)
B (System.Double)
A@ (System.Int32)

The columns A and A@ contain duplicated data. The latter seems to come from .field public int32 A@ according to Rider's IL viewer.

% dotnet --version
8.0.401

Failing test

No response

@nodakai nodakai changed the title [BUG]: breaking change: class serialization: F# records with mutable fields are duped with @ suffix [BUG]: breaking change: class serialization: mutable fields within F# records are duped with @ suffix Oct 1, 2024
@nodakai
Copy link
Author

nodakai commented Oct 1, 2024

Hmm maybe F# shouldn't have defined the backing field as public...

Is it possible to add a new field, sth like PropertiesOnly to ParquetSerializerOptions to restore the pre 4.23 behavior?
I can't seem to apply JsonIgnore to the F# compiler-generated backing fields.

@aloneguid
Copy link
Owner

aloneguid commented Oct 2, 2024

Thanks, I need to think about how to do this without impacting performance, as type info is cached at the moment and not discovered on every deserialisation call.

In the meantime you could do something like this:

type Foo = 
    val mutable private _a: int
    val B: float

    member this.A
        with get() = this._a
        and set(value) = this._a <- value

@aloneguid aloneguid changed the title [BUG]: breaking change: class serialization: mutable fields within F# records are duped with @ suffix F# class serialization: mutable fields within F# records are duped with @ suffix Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants