Proposal: Add fromEnvelope, like Message.fromEncoded

[mattheworiordan]

@paddybyers @SimonWoolf I think we should consider adding a fromEnvelope method to the client SDKs so that developers can reliably process enveloped messages from any Reactor source.

I think it's a convenience for developers that would allow them to safely process messages and know that the format follows a strict structure and will remain the same for that version of the SDK and the Reactor rule.

Are there other envelope types we should consider, or ideally, standardise on so that we can have one method in the SDKs only?

┆Issue is synchronized with this Jira Task by Unito

[SimonWoolf]

the reactor envelope is pretty standardised https://github.com/ably/realtime/blob/develop/common/lib/reactor/reactormessage.js#L41-L49 , except for webhooks which do their own super-nested thing.

know that the format ... will remain the same for that version of the SDK and the Reactor rule

I mean we can't actually promise the first part of that.

Unlike for a realtime connection or rest request, a client lib that's processing reactor envelopes has no way to request a particular version. All it can do is work with what it's given. Which means the sdk version doesn't really have anything to do with the price of eggs -- any changes can only be synchronized to the rule version (and in the case of adding a new field probably not even that).

i.e. having a strongly-typed structure in a client library feels like it's implicitly promising a different guarantee to the one we can actually give (which is just that we won't remove or change any field without an interaction with the rule entry that changes the version)

That said, I can see it'd be useful for languages like java or .net where it's difficult to work with untyped objects, so sure whatever. Tot sure there's much point for ruby/js/python etc. though.

[mattheworiordan]

except for webhooks which do their own super-nested thing.

Sure, would be good if possible to standardise so that a customer can use one method to process the envelope regardless.

a client lib that's processing reactor envelopes has no way to request a particular version. All it can do is work with what it's given. Which means the sdk version doesn't really have anything to do with the price of eggs

I know that :)

What I am suggesting is that we should guarantee that major version X of the client lib can parse major version X of the Reactor rule without breaking. If we introduce new fields that are not part of the type, they won't be made available of course (I think we have a spec item to this effect anyway stating client libs should be tolerant of unexpected fields), but it will continue to work. Ideally, client libs should also be backwards compatible as much as possible, until the types are incompatible.

That said, I can see it'd be useful for languages like java or .net where it's difficult to work with untyped objects, so sure whatever. Tot sure there's much point for ruby/js/python etc. though.

Why is it useful for Java / .NET / typed languages only? A developer, regardless of whether a language is typed or not, benefits from knowing what output to expect from a method. The client SDKs provide that today and thus simplify things significantly for developers (with IDE enhancements & docs).

[SimonWoolf]

Sure, would be good if possible to standardise so that a customer can use one method to process the envelope regardless.

It'd be a shame if we made a breaking change to webhook format but left them at with the current architecture. Paddy's had inchoate plans to change webhooks architecture for ages, which I'd assumed would involve adding a 'new-style webhook' rule type that would use a new mechanism and a new format (which can then use the standard envelope), and leave the old one as is.

Why is it useful for Java / .NET / typed languages only? A developer, regardless of whether a language is typed or not, benefits from knowing what output to expect from a method.

I was imagining the implementation for ruby/js/etc. would just be fromEnvelope = (env) => env. (Well, not quite, it'd run the messages through fromEncodedArray etc., but it wouldn't touch the metadata). What else would you want it to do? I guess it could strip out any fields it doesn't know about, but I don't see the point -- if a customer doesn't want to use at any new fields we add they can just not look at them, actively removing them just seems like it'd inconvenience anyone who does want them, for no benefit. (Not to mention forcing us to do a round of client library updates whenever we add a field).

(also I guarantee that the identity function can parse an envelope from any version of any reactor rule without breaking 😛)

[mattheworiordan]

Regardless of whether we're using a typed language or not, the reason we provide SDKs is to simplify how developers use our service and APIs. Take the history API for example, we could have easily not provide a typed PaginatedResult return value type and simply returned the underlying JSON response from Ably, but we don't do that specifically because we aim to simplify how developers process data and get a "contract" of sorts about types they can deal with.

I am not suggesting we couldn't in dynamic languages, if we so choose, include additional fields if they are provided to them in a Reactor payload, however that is not where the value lies. The value in providing a fromEnvelope envelope method is that a developer can refer to the documentation for that method and the returned type, not have to worry about payload compatibility issues, not have to worry about passing messages or presence messages to a further method to handle decoding, and in theory should also get an obvious guarantee about our commitment to that method working the same across all minor versions of that client library regardless of what we do at Ably to change the Reactor i.e. it's our problem to know they are compatible as opposed to their problem. I do think that's what is fundamentally different - if we provide an API in our SDKs that processes Reactor messages and spits out a typed envelope, developers no longer have to worry about whether that will continue to work as expected or not.

[mattheworiordan]

Related to https://github.com/ably/ideas/issues/316

[sync-by-unito]

➤ Automation for Jira commented:

The link to the corresponding Jira issue is https://ably.atlassian.net/browse/SDK-2798