How to use @Dependency on Codable types: "does not conform to protocol 'Decodable'"

[finestructure]

Apologies if this has been asked/addressed before but I couldn't find anything in the dicussions or issues.

We've got a type Model that uses a dependency, let's say \.date.now, in a method foo(). This type has to be Codable. Attaching the @Dependency like so:

struct Model: Codable {
    @Dependency(\.date.now) var now
    var value: Int

    func foo() -> String { "\(value) \(now)" }
}

fails to compile with

Type 'Model' does not conform to protocol 'Decodable'

Do you have any guidance how to best instrument this use-case?

I can think of a couple of mechanisms:

1. Write a custom codable implementation that skips now:

struct Model: Codable {
    @Dependency(\.date.now) var now
    var value: Int

    private enum CodingKeys: CodingKey {
        case value
    }

    init(from decoder: any Decoder) throws {
        let container: KeyedDecodingContainer<DepTest.Model.CodingKeys> = try decoder.container(keyedBy: DepTest.Model.CodingKeys.self)

        self.value = try container.decode(Int.self, forKey: DepTest.Model.CodingKeys.value)

    }

    func encode(to encoder: any Encoder) throws {
        var container: KeyedEncodingContainer<DepTest.Model.CodingKeys> = encoder.container(keyedBy: DepTest.Model.CodingKeys.self)

        try container.encode(self.value, forKey: DepTest.Model.CodingKeys.value)
    }

    func foo() -> String { "\(value) \(now)" }
}

Even in this simple case it adds a lot of boilerplate that we didn't need before. Moreover, any changes to the set of properties now also requires syncing up the Codable conformance by hand (or by rerunning the Codable-refactor).

In our case Model has 30 properties and the autogenerated Codable conformance is 200 lines of code. That's not ideal to maintain 😅

2. There are some packages that add macros for customising Codable conformances. One of these customisation is typically to ignore certain fields. I have not yet tested this but it might work with the @Dependency property.

The downside is that we'd have to add another depenendency, and for a very critical part of our app.

I've seen there's the @ObservationIgnored macro which deals with a similar problem regarding observation. Is a similar macro as part of swift-dependencies conceivable, to deal with Codable?

3. Put the @Dependency(\.date.now) var now on another type, say DependencyContainer, and inject that as a parameter on the foo() call, as in foo(dependencyContainer: container).

That would be feasible, although it would clutter up some call-sites. I guess it could be made not to be too terrible with default arguments but it still feels quite counter to the elegance of the @Dependency system overall.

Is there a better solution than these three? How are people dealing with @Dependency in the context of Codable types?

[mbrandonw]

Hey @finestructure, I think first and foremost we would advise against having a @Dependency in what should be a "plain data type." Especially codable types. When you encode the type you of course can't encode the dependency resolution, and so when it is decode it its dependencies will go back to whatever is provided by the surrounding context.

However, if you do want to continue doing this you could add a @retroactive conformance of Dependency to Codable that just does the trivial thing (this how ObservationRegistrar is handled too). You can also try moving @Dependency into the foo method, but that will subtly change how dependencies are resolved. It will resolve at the time of invoking foo, whereas currently dependencies are captured at the time of creating Model.

[mbrandonw]

Are you able to get a minimal repro of some bad behavior so that we can take a look?

[finestructure]

I'm trying 😅 I'll have to extend my current test project with more of our setup. It doesn't reproduce there yet!

[finestructure]

Ok, I don't have a fully standalone reproducer yet, but I think I have a lead on what's going on. I've reduced the main function that is missing the @Dependency setup to

    static func analyze(database: Database, repository: Repository) async throws {
        try await database.transaction { tx in
            updateScore(repository: repository)
        }
    }

updateScore is using @Dependency and throws the Unimplemented: @Dependency(\.date) failure. You'll notice that updateScore is not actually using the Vapor db transaction at all - there's no saving happing.

If I remove the transaction wrapper like so

    static func analyze(database: Database, repository: Repository) async throws {
//        try await database.transaction { tx in
            updateScore(repository: repository)
//        }
    }

the error goes away. transaction is part of fluent-kit:

public extension Database {
    func transaction<T>(_ closure: @escaping @Sendable (any Database) async throws -> T) async throws -> T {
        try await self.transaction { db in
            self.eventLoop.makeFutureWithTask {
                try await closure(db)
            }
        }.get()
    }
}

and makeFutureWithTask is from swift-nio:

extension EventLoop {
    @inlinable
    public func makeFutureWithTask<Return>(
        _ body: @Sendable @escaping () async throws -> Return
    ) -> EventLoopFuture<Return> {
        let promise = self.makePromise(of: Return.self)
        promise.completeWithTask(body)
        return promise.futureResult
    }
}

with completeWithTask:

    @discardableResult
    @inlinable
    public func completeWithTask(_ body: @escaping @Sendable () async throws -> Value) -> Task<Void, Never> {
        Task {
            do {
                let value = try await body()
                self.succeed(value)
            } catch {
                self.fail(error)
            }
        }
    }

And that's the problem, isn't it? Isn't @Dependencies using some task local stuff and that's one of the things that unstructured Tasks don't inherit?

[mbrandonw]

Hey @finestructure, ok I think I see the problem. It isn't the unstructured task, those specifically do inherit task locals (only Task.detached does not).

However, escaping closures lose dependencies. So by the time you get inside transaction { … } you have lost your dependencies, and similarly for makeFutureWithTask { … }. In general, escaping closures are the enemy of structured programming, and so therefore also the enemy of task locals or any of the tools in structured concurrency.

Luckily we do have a tool for this though. You can use withEscapedDependencies to escape dependencies in order to propagate them into an escaping closure:

static func analyze(database: Database, repository: Repository) async throws {
  withEscapedDependencies { dependencies in 
    try await database.transaction { tx in
      dependencies.yield {
        updateScore(repository: repository)
      }
    }
  }
}

I don't know if that's the exact place you need this tool, but that's the idea.

[finestructure]

Ah, that's a big relief that there might be a fix we can apply! I'll try this first thing in the morning and report back. Thank you!

[finestructure]

That's amazing - that fixed all our tests. Thanks so much, @mbrandonw !