Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propose clarification covering "persistence" across reboots and the term volatile #1

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Propose clarification covering "persistence" across reboots and the term volatile #1

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
16d9a59
Propose clarification covering "persistence" across reboots.
robshakir Sep 9, 2024
dfe66f8
Address review comments.
robshakir Oct 17, 2024
faf19f2
Merge branch 'main' into clarifications
robshakir Oct 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 56 additions & 25 deletions docs/motivation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ can be triggered by an NOS-external application.

## Requirements

* **[R1]** The elements that are provided as a dynamic parameter should be _in
* **[R1]** The elements that are able to be changed more rapidly should be _in
robshakir marked this conversation as resolved.
Show resolved Hide resolved
addition_ to the current static “intended” state of a device. It should be
possible to fall back from the configuration that is provided to the current
static intended state of the device.
Expand All @@ -145,10 +145,10 @@ can be triggered by an NOS-external application.
* **[R3]** The API provided must be asynchronous – rather than the current
synchronous `Set` provided by gNMI. Each “write” should be responded to
individually with an ACK, or NACK.
* **[R4]** It must be possible to relate the dynamically provided state to that
* **[R4]** It must be possible to relate the rapidly-changing state to that
which is provided by current “configuration” APIs. Particularly, it should be
possible to understand what the dynamic state is, what the current intended
state is, and subsequently what the applied state is.
possible to understand what the rapidly-changing state is, what the current
intended state is, and subsequently what the applied state is.
* **[R5]** Authentication, and authorisation of the API should use standard
mechanisms that allow for both RBAC, as well as payload authentication (e.g.,
path-based authentication).
Expand Down Expand Up @@ -184,10 +184,9 @@ determines:
outside of the target, and the target is solely responsible for enforcing
that the `election_id` received in each `SetOperation` corresponds to the
`election_id `that is the current master
* Persistence mode – determining whether the failure of <span
style="text-decoration:underline;">this</span> controller’s RPC results in
the received input being flushed, or it is rather flushed only when specified
by an explicit signal.
* Persistence mode – determining whether the failure of _this_ controller’s
RPC results in the received input being flushed, or it is rather flushed
only when specified by an explicit signal.

In order to allow for removal of all entries in a failure case, a `Flush` RPC
will be provided to remove dynamically written state that is being persisted.
Expand Down Expand Up @@ -218,61 +217,73 @@ input format. This has advantages over `json_ietf_val` since:

* The on-the-wire data volume is significantly smaller if a subtree is
provided - since string values that make up the `path` are no longer
encoded.
encoded.
* A protobuf value can be generated from a YANG schema using the ygot suite
of tools, and this approach is well-proven in gRIBI.
* Since limited/no validation will need to occur in this scenario – and
control plane systems likely do not already support YANG-modelled data,
this increases the applicability of this interface beyond languages that
this increases the applicability of this interface beyond languages that
have existing YANG tooling ecosystems. The advantages of YANG over protobuf
tend to be in its ability to express more detailed constraints around the
tend to be in its ability to express more detailed constraints around the
data that is carried in the message. In the case that smaller subsets of
state are being updated, there is little to no need for this
state are being updated, there is little to no need for this
cross-validation within a wider tree.

It is not our initial expectation that every OpenConfig subtree be supported
via this new API – since clearly there are some cases where strong validation
is needed (e.g., dynamically being able to remove a linecard would mean also
invalidating many other entities that depend on that linecard and its
interfaces - so it does not seem a good candidate for the mechanisms
corresponding to dynamic updates described in this document). Therefore, we
propose that where payloads are generated for the `Modify` RPC, we give these a
new place within the OpenConfig schema, particularly, defining a new
`volatile`/`dynamic`/`ephemeral` subtree to correspond with the `config` and
described in this document). Therefore, we propose that where payloads are
generated for the `Modify` RPC, we give these a new place within the OpenConfig
schema, particularly, defining a new
`volatile`/`ephemeral` subtree to correspond with the `config` and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the design decisions for modeling should be outside the scope of the motivation doc?

That said, would an annotation be more appropriate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can be moved, yes.

I'm not sure how an annotation would work here -- what are you proposing? There are other PRs that cover how the content of these RPCs are proposed to be generated. PTAL at #6.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The volatile container could have an annotation so there is a property in yang identifying the node as volatile.
It might be useful so yang aware code can identify volatile leafs programmatically versus relying on a "magic" container name. (Although I think using "volatile" as a container name also makes sense from a style perspective. I would do both the annotation an the container name)

An analogous entity is using the yang config statement, but also as an OC style guide, we use a container named 'config'.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see -- yes, annotating as well as a container name makes sense. I think we must have a container name for consistency. The alternate suggestion of having an annotation on an existing config leaf would be possible but add more complexity of magic in the tooling to know that such an annotation means that there is another path for the leaf.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, agreed that we should have both: a container and an annotation such as config-volatile, as this is the most explicit.

`state` subtrees within the schema. Considering the case of an interface
`enabled` leaf, we would therefore propose to have:

* `/interfaces/interface/config/enabled` – today’s intended state, and
written by a standard gNMI client.
* `/interfaces/interface/state/enabled` – today’s applied state, indicating
the value that the system is currently running.
* `/interfaces/interface/dynamic/enabled` – a new path, indicating that this
* `/interfaces/interface/volatile/enabled` – a new path, indicating that this
path can be written to via the `Modify` RPC.

The latter path can be added to a YANG tree via an `augment` in a specific
dynamic module, allowing the subset of paths that are writable to be
discovered. Protobufs would be generated or written to correspond to the
`dynamic` path.
`volatile` path.

The `dynamic` path SHOULD be streamed via gNMI `Subscribe` as per the existing
The `volatile` path SHOULD be streamed via gNMI `Subscribe` as per the existing
expectations around the `config` path.

> [!NOTE]
> The naming volatile is chosen for this "more dynamic" configuration based on
> fact that:
> * This state can be cached in volatile storage (i.e., stored only in memory
> that is available when the device is powered on).
> * It corresponds with the idea of `volatile` variables in C-like programming
> languages, where a different reader/writer may access this value (mostly,
> we expect that volatile paths are written to by different systems than
> NMSes, namely control-plane entities).
> * Such state is expected to be liable to rapid change, corresponding to the
> common English definition of volatile.

#### Determining the System's applied state value

Today, the intended state (`config`) path is the value that a client expects
the target to be running for the specified leaf. With the introduction of the
`dynamic` path, there is a requirement to choose between the `config` and the
`dynamic` path.
`volatile` path, there is a requirement to choose between the `config` and the
`volatile` path.

It is proposed that the target adopts the rule of:

* if the `dynamic` leaf is present, this is the preferred value and should
* if the `volatile` leaf is present, this is the preferred value and should
be used,
* if no `dynamic` leaf is present, the intended state (`config` path) should
* if no `volatile` leaf is present, the intended state (`config` path) should
be used.

with a strict preference to prefer the dynamic configuration value. In the case
that the `dynamic` value is removed (e.g., due to an RPC failure with a
with a strict preference to prefer the volatile configuration value. In the case
that the `volatile` value is removed (e.g., due to an RPC failure with a
persistence mode that indicates that the values should be cleared on RPC
failure), the system should apply the `config` value if one is present.

Expand All @@ -296,6 +307,26 @@ assumes the existence of a coalescing queue on input. The client SHOULD NOT
assume that transactions for different paths are processed in the order in
which they are received.

### Persistence of Configuration

Like other control-plane protocols, the "volatile" configuration of a device
is not expected to be persisted across device reboots. Data that is written
via gNMI to a `config` path is expected to be persisted (saved to non-volatile
storage) after each transaction. Not requiring persistence of data across
reloads allows both for the target device to avoid the cost of writing the
configuration to persistent storage (potentially reducing throughput); as well
as models where a "default" value is written to a persistent `config` path,
and an amended value is written to the `volatile` path according to some
runtime condition of the wider network system.

Note that `persistence` within the service definition refers to persistence
across RPCs (i.e., clients connecting or disconnecting) not across device
reboots.
robshakir marked this conversation as resolved.
Show resolved Hide resolved

It is expected that volatile configuration persists across device control-plane
card switchovers (simililarly to gRIBI) such that a client switchover does not
require an external controller to re-push the volatile configuration.

### Security

The proposed service should:
Expand Down