Cannot get vwt.initializeWithCopy to copy Arrays properly?

[NSExceptional]

To recap, I am using Echo to make a JSON object mapper. My library uses the output of JSONSerialization to initialize any type you want. So far it works very well, except for cases like this where COW properties are involved. In this case, I have a Person type like so, and I am trying to decode a Person with one child.

struct Person {
    /// For testing purposes; this is used as the structure of the "one child"
    static let sally = Person(name: "Sally", age: 5, married: false, kids: [], job: nil)
    
    var name: String
    var age: Int
    var married: Bool
    var kids: [Person]
    var job: String?
}

It looks like the array's backing storage is being released out from under it when the kids ivar is assigned in StructMetadata.createInstance() edit: but it is not. To be specific, once the array is assigned to the box in createInstance(), attempting to print or view the array from the box yields a description of 1 element but it does not print the element, and trying to access the element yields a failed force-unwrap. I am still able to print the array from the input dictionary:

This probably would affect other types of COW collections I assume but so far I am only testing Array. The strings I'm using are static as well, so they would never run into this problem. Here are the relevant code segments:

Code

// Helper operator intended to cast from one
// pointer type to UnsafeMutableRawPointer
postfix operator ~
postfix func ~<T>(target: T) -> UnsafeMutableRawPointer {
    return unsafeBitCast(target, to: UnsafeMutableRawPointer.self)
}

extension StructMetadata {
    // Entrypoint for creating struct instance from a property map
    func createInstance(props: [String: Any] = [:]) -> Any {
        // Precondition: props must contain a key for every property
        assert({
            let givenKeys = Set(props.keys.map { $0 as String })
            let allKeys = Set(self.descriptor.fields.records.map(\.name))
            return givenKeys == allKeys
        }())

        var box = AnyExistentialContainer(metadata: self)
        for (key, value) in props {
            self.set(value: value, forKey: key, pointer: box.getValueBuffer()~)
        }
        
        // At this point, the struct is populated correctly, except
        // for array properties which fail to unwrap any elements
        return box.toAny
    }

    func set(value: Any, forKey key: String, pointer ptr: UnsafeMutableRawPointer) {
        // These two are absolutely correct
        let offset = self.fieldOffset(for: key)!
        let type = self.fieldType(for: key)!
        ptr.storeBytes(of: value, type: type, offset: offset)
    }
}

extension UnsafeMutableRawPointer {
    func storeBytes(of value: Any, type: Metadata, offset: Int = 0) {
        // Things get wonky here if I don't use my modified version
        // of this function to avoid double-boxed containers from #49
        var box = container(for: value)
        type.vwt.initializeWithCopy((self + offset), box.projectValue()~)
        // What I was doing before, which also didn't work for arrays
        // (self + offset).copyMemory(from: box.projectValue(), byteCount: type.vwt.size)
    }
}

// MARK: Populating AnyExistentialContainer
extension AnyExistentialContainer {
    var toAny: Any {
        return unsafeBitCast(self, to: Any.self)
    }
    
    /// Calls into `projectValue()` but will allocate a box
    /// first if needed for types that are not inline
    mutating func getValueBuffer() -> UnsafeMutableRawPointer {
        // Allocate a box if needed and return it
        if !self.metadata.vwt.flags.isValueInline && self.data.0 == 0 {
            return self.metadata.allocateBoxForExistential(in: &self)~
        }
        
        // We don't need a box or already have one
        return self.projectValue()~
    }
}

What could be going on here?

Edit: after updating to your latest commit, I now get EXC_BAD_ACCESS when trying to print the fully initialized instance, but the debugger's variable view shows pretty much the same output as my lldb screenshot above. Also, I am able to print the array if I manually dereference the ivar? O_o

[Azoy]

I'm very confused because I'm using your code above and creating my own person instances with kids, and I'm able to see them completely (nothing is wrong on my end).

[NSExceptional]

Oh, weird. I'll do that, thanks.

Switching gears one more time: can I use _openExistential to, say, invoke the initializer for an ExpressibleByStringLiteral type? Or any other type with an associated type

This doesn't appear to compile:

class Pet: ExpressibleByStringLiteral {
    typealias StringLiteralType = String
    var name: String
    
    required init(stringLiteral: StringLiteralType) {
        self.name = stringLiteral
    }
}

func makeFromStringLiteral<T: ExpressibleByStringLiteral>(_: T.Type) -> T where T.StringLiteralType == String {
    return T.init(stringLiteral: "fido")
}

let p: Any = _openExistential(Pet.self, makeFromStringLiteral(_:))

I'd like to be able to construct an enum instance by calling into init(rawValue:), and ideally a generic way to do it for any raw value type without hard-coding checks for several different types

[Azoy]

Could you not just do:

makeFromStringLiteral(Pet.self)

? The way you're using _openExistential does not work/doesn't make sense because there is no existential to open.

[NSExceptional]

Replace Pet.self with a variable of type Any.Type, my bad:

let type: Any.Type = metadata.type
makeFromStringLiteral(type) // nope

Another example:

enum Colors: String {
    case red = "red"
    case blue = "blue"
    case green = "green"
}

let rawValue: Any = ...

func createColor<T: RawRepresentable>(_: T.Type) -> T {
    let value = rawValue as! T.RawValue
    return T.init(rawValue: value)!
}

let type: Any.Type = Colors.self
let i = _openExistential(type, do: createColor(_:))

This yields Local function 'createColor' requires that 'Any' conform to 'RawRepresentable'

[Azoy]

The issue here is that we can't reference those protocols, associated type and Self requirement!!!!, but soon we will be able to make existentials with those protocols and the compiler will happily accept what you're trying to do.

let x = metadata.type as! ExpressibleByStringLiteral.Type
let y = _openExistential(x, do: makeFromStringLiteral(_:))

[NSExceptional]

Ah, gotcha. I will just work around this until existentials are opened up then. Thanks :)

[NSExceptional]

When I created the array I'm trying to assign to kids, it was created as [Any]. The property I'm assigning it to is of type [Person].

Solution: use swift_dynamicCast to perform the conversion :)

Edit: see Azoy's replies above; swift_dynamicCast may not be as good a choice as _openExistential