How to deal with missing/incorrect state updates?

[AlCalzone]

We currently try to verify that a set-type command worked correctly. This is done by issuing a corresponding get (poll) after the set was acknowledged. However, this has its flaws, as can be seen in several issues.

As far as I see it, there are several scenarios we need to support:

• The device reports the change automatically through the Lifeline Association. We shouldn't need to poll, but we do. 😐
• The device does not report the change automatically. The poll returns the new value. ✔
• The device does not report the change automatically. The poll returns the old value or a transitional one (e.g. 82 when transitioning from 99 to 0). ❌
• The device supports Supervision CC. We can use that to get notified when it is correct to poll. We already use that for Multilevel Switch CC, but probably should be extended to more (all?) CCs. ✔

@hanskroner
What would be a good strategy to deal with this? How does the transition time used for the set commands go into this? How do we reliably figure out if the poll is necessary?

[hanskroner]

• The device reports the change automatically through the Lifeline Association. We shouldn't need to poll, but we do.

Don't poll. Or rather, why the need for a poll? See also the next answer.

• The device does not report the change automatically. The poll returns the new value.

How is the library deciding how long after the SET to follow-up with a GET?

• The device does not report the change automatically. The poll returns the old value or a transitional one (e.g. 82 when transitioning from 99 to 0).

As with the above answer, if the library was responsible for sending the SET command that causes the transitional change, then it knows how long before the transition is complete and when it should check the new state with a GET. Devices that support newer Command Class versions that explicitly handle transitions can make this easier to get right in some cases. In other cases, the older Command Classes simply don't provide a way to have visibility into this. There's a reason why a new Command Class version was necessary after all ;)

• The device supports Supervision CC. We can use that to get notified when it is correct to poll. We already use that for Multilevel Switch CC, but probably should be extended to more (all?) CCs.

As with basically everything else, there isn't an easy answer. It's generally a good idea to use Supervision encapsulation for SETs when possible and when it makes sense. It's also important to acknowledge that there is overhead involved, which creates traffic and adds latency. I believe the Z-Wave Plus and Plus v2 Device Types provide some guidance here - but that's all it is: guidance. If a library considers that knowing if a power switch actually changed state is crucial and is willing to accept the overhead of using Supervision, there's nothing wrong about doing it - it's a choice.

A closing remark: There's a lot of emphasis on providing "real state" mirroring. In my experience, after deeper analysis on this, it turns out to be rarely needed - outside of cool demos. This particular application might be different, so understanding why the goals are what they are would be helpful in defining a strategy to accomplish them.

Maintaining an accurate mirror of the state of a distributed network with multiple external change agents is hard so starting with "is it absolutely necessary?" is important. For example, consider what happens if this library goes though major pains to in order to provide the optimal solution to the above, so that library clients have near-realtime notifications, but then the state of a switch (which doesn't do Lifeline reporting) is changed externally: either manually at the switch, or by another Z-Wave device. The library won't be notified of this change until the next poll cycle, which might mean that automations that a client of the library's might have defined using that change as a trigger will be delayed - if the network is large enough that the polling interval must be necessarily high, it might be that that the delay ruins the whole scenario. Knowing that case is real and unavoidable, and that there are lots of others like it, might help add a little perspective to this question.

[jcam]

When using the factory default transition, the get appears to return the duration remaining for my Eaton v2 dimmers. Is this something that can be relied on to know when to re-poll, or would this also only apply to certain versions/implementations?

[hanskroner]

That requires us to know beforehand that we're going to receive the update on the Lifeline Association. That should be possible with Association Group Information CC, but without we're tapping in the dark.

Not really. The Z-Wave Plus Info Command Class is sufficient to know the device supports a Lifeline. The Lifeline is required to point to the SIS, so if you're it that's all the information you need.

We can't know when using the factory default duration. Even if we do, we should probably be equipped to handle the case that we're receiving a transitional value. I've thought of a strategy for this a while ago: #452 (comment) - what are your thoughts on this?

This depends on the Command Class we're talking about. In general, if the device is not using a version of a Command Class that is aware of transitional values, you don't have that information. We discussed in another topic that it's just not possible to reliably provide the same feature set for devices that support newer versions of Command Classes and for older versions of the Command Classes - if it were possible, there wouldn't have been a need for a new Command Class.

Would there be a different path? Say, instead of trying to guess transition times and try to provide feature-parity with newer devices, is there a way to communicate to the library's clients that transitional values aren't supported for a command?

[AlCalzone]

The Z-Wave Plus Info Command Class is sufficient to know the device supports a Lifeline. The Lifeline is required to point to the SIS, so if you're it that's all the information you need.

Sorry, I have to ask again. I plan on providing this functionality some way or the other. When looking at the Z-Wave Plus Device Type Specification I find this:

A node MAY send additional commands via the lifeline association group to reflect the state of the node

That means we cannot rely on the presence of the Lifeline to verify that a SET command worked. Some devices send a report in that case, some don't. Ideally I would avoid polling if the report comes on its own.

[hanskroner]

That means we cannot rely on the presence of the Lifeline to verify that a SET command worked. Some devices send a report in that case, some don't. Ideally I would avoid polling if the report comes on its own.

I think I misunderstood what you meant in your follow-up. I think what you're trying to say here is "even if the Lifeline points back to me, there's no guarantees that it'll include a REPORT for a SET command I sent if its not on this list." If I got it right this time, yes - other than what's on that table, there are no guarantees. That also makes your comment about the Association Group Information Command Class make more sense.

In the context of the reporting we're talking about here, it'd be great if all REPORTs would be sent out the "Lifeline" but there's a reason the association got named "Lifeline" and not "Firehose" 😉

[AlCalzone]

Yes, that's what I meant! I'm looking for a way to determine (for the application layer) whether a poll is necessary to verify the change after a SET for a given device+CC combination.

[LordMike]

Could it be assumed that the node has accepted the new value, if it acknowledges the message?

Ie.. SET -> ACK from device -> zwave-js announces SET'd value.. ?

[AlCalzone]

No. You can send a SET with an invalid value and it will still acknowledge the receipt.

[LordMike]

In other places.. one might say... "garbage in, garbage out"... :)

[hanskroner]

Even if the values you send are valid for the Command Class used, a device could still refuse to change state. As I've mentioned before, the device could have an over-current protection mechanism that prevents it from turning on, or it might have over-temp. protection and refuse to turn on if it's too hot. It could also be locked by the Anti-Theft Command Class, and a bunch of other etc's. The ACK is Z-Wave's MAC layer saying "message received" - nothing more.

[Xrlls]

This very much depends on whether a device supports S2 (and thus supervision encapsulation), and whether it is Z-Wave Plus v.2

If the device supports supervision encapsulation, just use this, and the node will tell you if the command was carried out correctly.

Furthermore if the device is Z-Wave Plus v.2 it is required to send status updates to the lifeline destination, whenever the status was changed by anything else than the lifeline destination i.e. locally, or by another Z-Wave node.

For legacy devices, the the previous discussion applies :)

[AlCalzone]

If the device supports supervision encapsulation, just use this, and the node will tell you if the command was carried out correctly.

That sounds good, but it seems like there are devices (e.g. HomeSeer WD200+, there's an open issue) which support Supervision CC and Multilevel Switch CC but not supervision encapsulated Multilevel Switch CC.

[Xrlls]

@hanskroner told me about the homeseer device. This is really an issue with the device, that should be resolved by the manufacturer. For most controller developers there is a point where you will need to decide to which extent you will address issues with non-conforming devices on the controller side. Sometimes the workarounds may be destructive for other conforming devices.

[LordMike]

Can't you like, throw them out of the z-wave stuff if they don't behave? :D

[mmommo]

Just about to start a zwave project for my home as an experienced javascript frontend coder with limited nodejs knowledge. Still trying to figure out how to put things together.

May I ask

• what is Supervision CC/supervision encapsulation and where can I found more info about it?
• let's say, if the switch status is changed by the device(hardware), shouldn't it be reflected in my app in real-time?
• Where can I find some code examples for zwave-js? There're tons of examples of integrating this to HA, but I really would like to try something within js first.

Many thinks.

[AlCalzone]

* what is Supervision CC/supervision encapsulation and where can I found more info about it?

It is used to work around a shortcoming of Z-Wave where you don't know what the device does with a command.
See SDS13783 Z-Wave Transport-Encapsulation Command Class Specification

let's say, if the switch status is changed by the device(hardware), shouldn't it be reflected in my app in real-time?

Only if the device has a lifeline association to the controller which includes the corresponding CC.

Where can I find some code examples for zwave-js?

Very basic examples are in the docs: https://zwave-js.github.io/node-zwave-js/#/getting-started/quickstart
ioBroker.zwave2 is my reference implementation.
Or zwavejs2mqtt

[mmommo]

Thank you for your detailed reply. Will dive into your ioBroker project during the weekend to get some understanding.
On the other hand, how can I tell which device has a lifeline association to the controller which includes the corresponding CC? is there any label to indicate that? like z-wave plus?

Cheers!

[AlCalzone]

On the other hand, how can I tell which device has a lifeline association to the controller which includes the corresponding CC?

Not easily. Generally, every device has a lifeline association, except very old non-ZWave+ devices that do not support associations. What gets reported there gets queried with the Association Group Information CC (if supported), but not exposed in a nice way to applications yet. If that is interesting, please open a feature request.
Other than that, all you have to deduce this information is the device manual.