Skip to content

Commit

Permalink
design-proposal: Feature lifecycle
Browse files Browse the repository at this point in the history 
Kubevirt requires a process with a clear policy on how features
are introduced, evaluated and finally graduated or dropped.

This proposal defines the steps and policies to follow in order
to manage a feature (lifecycle) in Kubevirt.

Signed-off-by: Edward Haas <[email protected]>
  • Loading branch information
@EdDev
EdDev committed Jan 17, 2024
1 parent 1d767d6 commit 37c1a75
Showing 1 changed file with 300 additions and 0 deletions.
300 changes: 300 additions & 0 deletions design-proposals/feature-lifecycle.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,300 @@
# KubeVirt Feature Lifecycle

## Summary

KubeVirt requires a clear policy on how features are introduced,
evaluated and finally graduated or dropped.

This proposal defines the steps and policies to follow in order
to manage a feature and its lifecycle in KubeVirt.

The proposal is focusing on introducing features in
a stable API (CRD) version.

## Overview

### Motivation
KubeVirt has grown into a matured virtualization management solution
with a large set of features.

New features are being proposed and added regularly to its portfolio.

With time, the challenge of supporting and maintaining such a large
set of features raised the need to re-examine their relevance.
It also raised the need to examine with more care features graduation.

The KubeVirt community has tried to control the flow of features
informally through feature-gates, similar to Kubernetes.
However, as time passed, several challenges presented themselves:
- Evaluated features rarely got graduated to GA or dropped, causing
feature consumption to be risky for users and a maintenance burden
for the project contributors.
- Features that have been targeted for graduation or dropping have
no agreed implementation steps. This in turn causes each feature
to take different approaches, possibly surprising users.

One possible conclusion is that the current use of FGs is insufficient
due to the lack of a well-defined processes and policy on how a feature
should progress in its lifecycle.

Eventually we would like to see features being evaluated carefully
before they are introduced, while they are experimented with and
proven to be actually in use (and useful) before graduating.

> **Note**: Once a feature graduates, it is included in a
> General Availability (GA) release with its functionality available
> to all users. GA features need to comply with [semver](https://semver.org/)
> which add constraints on their ability to change (including deprecation).
### Goals
- Define the process a feature needs to pass for graduation.
- Define the process a feature needs to pass in order to be dropped.
- Provide policies and rules on how to manage a feature during its
lifetime.

### Non Goals
- Implement enforcement tooling to keep features in sync with
the lifecycle rules.

### Definition Of Users
- Development contributors.
- Cluster operators.

### User Stories
- As a KubeVirt contributor, I would like to introduce a new useful
feature and follow it to graduation (GA).
- As a KubeVirt contributor, I would like to discontinue a feature
that has not yet reached (formal) graduation.
- As a KubeVirt contributor, I would like to discontinue a feature
that has already graduated.
> **Note**: Discontinuation of a GA feature is considered an exception.
> Strong arguments and a wide agreement is required for such an action.
- As a KubeVirt cluster operator, I would like to experiment with
an undergraduate feature.
- As a KubeVirt cluster operator, I would like to evaluate an
undergraduate feature.
- As a KubeVirt cluster operator, I would like to keep using a feature
that got graduated after I used it during the evaluation period.
- As a KubeVirt cluster operator, I would like to know that
an undergraduate feature is planned to be discontinued.
- As a KubeVirt cluster operator, I would like to stop using a feature
that got discontinued.

### Repos
This is a cross repo project policy under the
[kubevirt](https://github.com/kubevirt) organization.

## Proposal Design
The proposal on how to define a feature lifecycle is influenced by
processes and policies from the Kubernetes project.
These sources are scattered around, each focusing on different
aspects of a feature:
- [Feature Gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/)
- [Graduation](https://kubernetes.io/blog/2020/08/21/moving-forward-from-beta/)
- [Changing the API](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md)
- [Deprecation](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)

The proposal takes the top-down approach, starting with the high level
flow that a common feature will traverse through.
Continuing with actions that need to be taken and timeline suggestions.

Both feature graduation and discontinuation flows are covered.
Including the implications on users.

Depending on individual topics, follow-up proposals may extend the basic
points raised in this proposal.

### Feature Stages
A feature is expected to pass through the following stages:
1. Enhancement proposal.
2. Implementation.
3. Release as Alpha (experimental).
4. Release as Beta (pre-release for evaluation).
5. Release as General Availability (graduation).
6. Discontinuation.

Starting from the Alpha release, it can be discontinued with restrictions that
depend on the release stage (Alpha, Beta, GA).

[Discontinuation](#discontinuation) of features is widely discussed later
in this proposal.

#### Enhancement proposal
As the first step for introducing a new feature, a formal proposal is
expected to be shared for public review via mailinglist and
a [design proposal](https://github.com/kubevirt/community/tree/main/design-proposals).

This is the first opportunity to evaluate a new feature.
The proposal needs to include motivation, goals, implementation details
and phases. Review the [proposal template](https://github.com/kubevirt/community/blob/main/design-proposals/proposal-template.md)
for more information.

#### Implementation
The development work on the feature is expected to include coding,
testing, integration and documentation.

While most of the effort is expected to occur towards the Alpha stage,
work may start as a PoC during the design and as adjustments between
the release stages.

#### Releases
- **Alpha**:
An initial release of the feature for experimental purposes.
Recommended for non-production usages, evaluation or testing.

The API is considered unstable and may change significantly.

The period in which a feature can remain in Alpha is limited,
assuring features are not piling up without control.

The feature functionality is controlled using a Feature-Gate (FG) during
runtime. It must be specified for the feature to be active.

- **Beta**:
The first release that can be evaluated with care in production.
Acting as a pre-release, its main objective is to collect feedback
from users to assure its usefulness and readiness for graduation.
If there is no confidence of usage or usefulness, it may remain in
this stage for some time.
However, the period in which a feature can remain in Beta is limited,
assuring features are not piling up without control.

The API is considered stable with care no not break backward compatability
with previous beta releases.
This implies that only additional fields may be added during this stage
i.e. it is not allowed to change a field path/name or remove it.

The feature functionality is controlled using a Feature-Gate (FG) during
runtime. It must be specified for the feature to be active.

- **GA**:
The feature graduated to general-availability (GA) and is now part of
the core features.

The feature functionality is no longer controlled by a FG.

> **Warning**: A Feature Gate flag is solely intended to control
> feature lifecycle. It should not be confused and used as a cluster
> configurable enablement of the functionality.
> In cases where the cluster admin should control a functionality,
> regardless to the feature stage, dedicated configuration field/s
> should be included.
#### Discontinuation
When and if a feature is targeted for deprecation and retirement,
it needs to pass a deprecation process, depending on its current
release stage (Alpha, Beta, GA).

For more details, see [here](#deprecation-and-removal).

#### Release Stage Transition Table
The following table summarized the different release stages with their
transition requirements and restrictions.

| Stage | Period range | F.Gate | Discontinuation Availability |
|----------------|-----------------|--------|------------------------------|
| Alpha | 1 to 2 releases | YES | Between **minor** releases |
| Beta | 1 to 3 releases | YES | Between **minor** releases |
| GA | - | NO | Between **major** releases |

Through Alpha and Beta feature releases, a FG must be set in order
for the feature to function.
By default, no FG is specified, therefore the feature is by disabled.

If a feature is not able to transition to the next stage in the defined period,
it should be discontinued automatically.

> **Note**: Exceptions to the period range may apply
> if 2/3 of active maintainers come to agreement to prolong
> a specific feature.
### Deprecation and Removal
One reason for features to go through the Alpha and Beta stages,
is the opportunity to examine their usefulness and adoption.
Same goes with major releases that intentionally allow breaking
backward compatability (as specified by [semver](https://semver.org/)).

Therefore, it is only natural that some features will not graduate
between the stages, or will be found irrelevant after some time and be
discontinued when transitioning between major releases.

#### Major Releases
KubeVirt follows semver versioning, in which major versions may
break API compatability. Therefore, discontinuation of features
is somehow simpler when incrementing the major version.

However, this is not without a cost.
When a new major release is introduced, the previous one is still maintained
and supported, something that does not exist with minor releases.

#### The Deprecation Flow (for Minor releases)
Only Alpha and Beta features can be discontinued during a minor release.

These are the steps needed to deprecate & remove a feature:
- Proposal: Prepare a proposal to discontinue a feature with proper
reasoning, phases, exact timelines and functional alternatives (if any).
The proposal should be reviewed and approved.
- Notification: Notify the project community about the feature
discontinuation based on the approved proposal.
All details of the plan should be provided to allow users and possibly
down-stream projects to adjust.
Use all community media options to distribute this information
(e.g. mailing list, slack channel, community meetings).
- Deprecation warnings: Add deprecation warnings at runtime to warn users
that the feature is planned to be discontinued.
Warnings should be raised when:
- Feature API fields are accessed.
- Feature FG is activated.
- Behavior related to the feature is detected (optional).
- Removal: Feature removal involves dropping the core functionality
of a feature and its exposed API.
- The core implementation can be dropped in two steps:
- The FG is dropped by assuring it is never reported as set
(i.e. even if it is left by the operator configured, internally
it is ignored).
At this stage, the core implementation will follow the FG conditions
and therefore from the outside the feature is inactive.
- In case there are no side effects, the core implementation code can
be removed.
- The API types are not to be removed, as it may have implications
with the underlying storage which has already persisted them.
Kubernetes has not removed fields, it just kept them around with the
warning that they have been deprecated and no longer available.

While keeping fields around for a period of a release or two makes
sense, beyond a limited period it adds a burden on dragging leftover
fields around to eternity.

> **Note**: The only reference seen on why fields should not be removed
> was mentioned [here](https://github.com/kubernetes/kubernetes/issues/52185).
> But it is unclear if this is relevant for Alpha stage features.
### Exceptions
While the project strives to maintain a stable contract with its users,
there may be scenarios where the policy described here will not be a fit.

Therefore, it should be acceptable to have exceptions from time to time
given a very good reasoning and an agreement from 2/3 of the project
maintainers (also known as "approvers").

## Implementation Phases
- Add a section in the
[design proposal template](https://github.com/kubevirt/community/blob/main/design-proposals/proposal-template.md)
that describes the planned timelines for the feature stages.
Add a reference to the feature-lifecycle documentation to assure contributors
know the process and policy.
- Prepare a user-facing document that describes the usability implications of
this feature lifecycle.

## Open Issues
1. Removal of fields from the CRD: Is it relevant for Alpha
and Beta features?
2. The details on how exactly to mark an Alpha/Beta field.
Should we describe it in this proposal or leave it to a follow-up one?
3. Can Alpha features get introduced in z versions?
4. CI & E2E restrictions:
1. Run e2e tests of Alpha features in their own separate job
on the latest k8s version.
- Should such a job be gating or not?
2. Monitor Alpha features E2E execution results and use the data
as a gating (e.g. 95% success rate) to pass it to Beta stage.

0 comments on commit 37c1a75

Please sign in to comment.