diff --git a/design-proposals/feature-lifecycle.md b/design-proposals/feature-lifecycle.md new file mode 100644 index 00000000..9bd044b3 --- /dev/null +++ b/design-proposals/feature-lifecycle.md @@ -0,0 +1,300 @@ +# KubeVirt Feature Lifecycle + +## Summary + +KubeVirt requires a clear policy on how features are introduced, +evaluated and finally graduated or dropped. + +This proposal defines the steps and policies to follow in order +to manage a feature and its lifecycle in KubeVirt. + +The proposal is focusing on introducing features in +a stable API (CRD) version. + +## Overview + +### Motivation +KubeVirt has grown into a matured virtualization management solution +with a large set of features. + +New features are being proposed and added regularly to its portfolio. + +With time, the challenge of supporting and maintaining such a large +set of features raised the need to re-examine their relevance. +It also raised the need to examine with more care features graduation. + +The KubeVirt community has tried to control the flow of features +informally through feature-gates, similar to Kubernetes. +However, as time passed, several challenges presented themselves: +- Evaluated features rarely got graduated to GA or dropped, causing + feature consumption to be risky for users and a maintenance burden + for the project contributors. +- Features that have been targeted for graduation or dropping have + no agreed implementation steps. This in turn causes each feature + to take different approaches, possibly surprising users. + +One possible conclusion is that the current use of FGs is insufficient +due to the lack of a well-defined processes and policy on how a feature +should progress in its lifecycle. + +Eventually we would like to see features being evaluated carefully +before they are introduced, while they are experimented with and +proven to be actually in use (and useful) before graduating. + +> **Note**: Once a feature graduates, it is included in a +> General Availability (GA) release with its functionality available +> to all users. GA features need to comply with [semver](https://semver.org/) +> which add constraints on their ability to change (including deprecation). + +### Goals +- Define the process a feature needs to pass for graduation. +- Define the process a feature needs to pass in order to be dropped. +- Provide policies and rules on how to manage a feature during its + lifetime. + +### Non Goals +- Implement enforcement tooling to keep features in sync with + the lifecycle rules. + +### Definition Of Users +- Development contributors. +- Cluster operators. + +### User Stories +- As a KubeVirt contributor, I would like to introduce a new useful + feature and follow it to graduation (GA). +- As a KubeVirt contributor, I would like to discontinue a feature + that has not yet reached (formal) graduation. +- As a KubeVirt contributor, I would like to discontinue a feature + that has already graduated. + > **Note**: Discontinuation of a GA feature is considered an exception. + > Strong arguments and a wide agreement is required for such an action. +- As a KubeVirt cluster operator, I would like to experiment with + an undergraduate feature. +- As a KubeVirt cluster operator, I would like to evaluate an + undergraduate feature. +- As a KubeVirt cluster operator, I would like to keep using a feature + that got graduated after I used it during the evaluation period. +- As a KubeVirt cluster operator, I would like to know that + an undergraduate feature is planned to be discontinued. +- As a KubeVirt cluster operator, I would like to stop using a feature + that got discontinued. + +### Repos +This is a cross repo project policy under the +[kubevirt](https://github.com/kubevirt) organization. + +## Proposal Design +The proposal on how to define a feature lifecycle is influenced by +processes and policies from the Kubernetes project. +These sources are scattered around, each focusing on different +aspects of a feature: +- [Feature Gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) +- [Graduation](https://kubernetes.io/blog/2020/08/21/moving-forward-from-beta/) +- [Changing the API](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md) +- [Deprecation](https://kubernetes.io/docs/reference/using-api/deprecation-policy/) + +The proposal takes the top-down approach, starting with the high level +flow that a common feature will traverse through. +Continuing with actions that need to be taken and timeline suggestions. + +Both feature graduation and discontinuation flows are covered. +Including the implications on users. + +Depending on individual topics, follow-up proposals may extend the basic +points raised in this proposal. + +### Feature Stages +A feature is expected to pass through the following stages: +1. Enhancement proposal. +2. Implementation. +3. Release as Alpha (experimental). +4. Release as Beta (pre-release for evaluation). +5. Release as General Availability (graduation). +6. Discontinuation. + +Starting from the Alpha release, it can be discontinued with restrictions that +depend on the release stage (Alpha, Beta, GA). + +[Discontinuation](#discontinuation) of features is widely discussed later +in this proposal. + +#### Enhancement proposal +As the first step for introducing a new feature, a formal proposal is +expected to be shared for public review via mailinglist and +a [design proposal](https://github.com/kubevirt/community/tree/main/design-proposals). + +This is the first opportunity to evaluate a new feature. +The proposal needs to include motivation, goals, implementation details +and phases. Review the [proposal template](https://github.com/kubevirt/community/blob/main/design-proposals/proposal-template.md) +for more information. + +#### Implementation +The development work on the feature is expected to include coding, +testing, integration and documentation. + +While most of the effort is expected to occur towards the Alpha stage, +work may start as a PoC during the design and as adjustments between +the release stages. + +#### Releases +- **Alpha**: + An initial release of the feature for experimental purposes. + Recommended for non-production usages, evaluation or testing. + + The API is considered unstable and may change significantly. + + The period in which a feature can remain in Alpha is limited, + assuring features are not piling up without control. + + The feature functionality is controlled using a Feature-Gate (FG) during + runtime. It must be specified for the feature to be active. + +- **Beta**: + The first release that can be evaluated with care in production. + Acting as a pre-release, its main objective is to collect feedback + from users to assure its usefulness and readiness for graduation. + If there is no confidence of usage or usefulness, it may remain in + this stage for some time. + However, the period in which a feature can remain in Beta is limited, + assuring features are not piling up without control. + + The API is considered stable with care no not break backward compatability + with previous beta releases. + This implies that only additional fields may be added during this stage + i.e. it is not allowed to change a field path/name or remove it. + + The feature functionality is controlled using a Feature-Gate (FG) during + runtime. It must be specified for the feature to be active. + +- **GA**: + The feature graduated to general-availability (GA) and is now part of + the core features. + + The feature functionality is no longer controlled by a FG. + +> **Warning**: A Feature Gate flag is solely intended to control +> feature lifecycle. It should not be confused and used as a cluster +> configurable enablement of the functionality. +> In cases where the cluster admin should control a functionality, +> regardless to the feature stage, dedicated configuration field/s +> should be included. + +#### Discontinuation +When and if a feature is targeted for deprecation and retirement, +it needs to pass a deprecation process, depending on its current +release stage (Alpha, Beta, GA). + +For more details, see [here](#deprecation-and-removal). + +#### Release Stage Transition Table +The following table summarized the different release stages with their +transition requirements and restrictions. + +| Stage | Period range | F.Gate | Discontinuation Availability | +|----------------|-----------------|--------|------------------------------| +| Alpha | 1 to 2 releases | YES | Between **minor** releases | +| Beta | 1 to 3 releases | YES | Between **minor** releases | +| GA | - | NO | Between **major** releases | + +Through Alpha and Beta feature releases, a FG must be set in order +for the feature to function. +By default, no FG is specified, therefore the feature is by disabled. + +If a feature is not able to transition to the next stage in the defined period, +it should be discontinued automatically. + +> **Note**: Exceptions to the period range may apply +> if 2/3 of active maintainers come to agreement to prolong +> a specific feature. + +### Deprecation and Removal +One reason for features to go through the Alpha and Beta stages, +is the opportunity to examine their usefulness and adoption. +Same goes with major releases that intentionally allow breaking +backward compatability (as specified by [semver](https://semver.org/)). + +Therefore, it is only natural that some features will not graduate +between the stages, or will be found irrelevant after some time and be +discontinued when transitioning between major releases. + +#### Major Releases +KubeVirt follows semver versioning, in which major versions may +break API compatability. Therefore, discontinuation of features +is somehow simpler when incrementing the major version. + +However, this is not without a cost. +When a new major release is introduced, the previous one is still maintained +and supported, something that does not exist with minor releases. + +#### The Deprecation Flow (for Minor releases) +Only Alpha and Beta features can be discontinued during a minor release. + +These are the steps needed to deprecate & remove a feature: +- Proposal: Prepare a proposal to discontinue a feature with proper + reasoning, phases, exact timelines and functional alternatives (if any). + The proposal should be reviewed and approved. +- Notification: Notify the project community about the feature + discontinuation based on the approved proposal. + All details of the plan should be provided to allow users and possibly + down-stream projects to adjust. + Use all community media options to distribute this information + (e.g. mailing list, slack channel, community meetings). +- Deprecation warnings: Add deprecation warnings at runtime to warn users + that the feature is planned to be discontinued. + Warnings should be raised when: + - Feature API fields are accessed. + - Feature FG is activated. + - Behavior related to the feature is detected (optional). +- Removal: Feature removal involves dropping the core functionality + of a feature and its exposed API. + - The core implementation can be dropped in two steps: + - The FG is dropped by assuring it is never reported as set + (i.e. even if it is left by the operator configured, internally + it is ignored). + At this stage, the core implementation will follow the FG conditions + and therefore from the outside the feature is inactive. + - In case there are no side effects, the core implementation code can + be removed. + - The API types are not to be removed, as it may have implications + with the underlying storage which has already persisted them. + Kubernetes has not removed fields, it just kept them around with the + warning that they have been deprecated and no longer available. + + While keeping fields around for a period of a release or two makes + sense, beyond a limited period it adds a burden on dragging leftover + fields around to eternity. + +> **Note**: The only reference seen on why fields should not be removed +> was mentioned [here](https://github.com/kubernetes/kubernetes/issues/52185). +> But it is unclear if this is relevant for Alpha stage features. + +### Exceptions +While the project strives to maintain a stable contract with its users, +there may be scenarios where the policy described here will not be a fit. + +Therefore, it should be acceptable to have exceptions from time to time +given a very good reasoning and an agreement from 2/3 of the project +maintainers (also known as "approvers"). + +## Implementation Phases +- Add a section in the + [design proposal template](https://github.com/kubevirt/community/blob/main/design-proposals/proposal-template.md) + that describes the planned timelines for the feature stages. + Add a reference to the feature-lifecycle documentation to assure contributors + know the process and policy. +- Prepare a user-facing document that describes the usability implications of + this feature lifecycle. + +## Open Issues +1. Removal of fields from the CRD: Is it relevant for Alpha + and Beta features? +2. The details on how exactly to mark an Alpha/Beta field. + Should we describe it in this proposal or leave it to a follow-up one? +3. Can Alpha features get introduced in z versions? +4. CI & E2E restrictions: + 1. Run e2e tests of Alpha features in their own separate job + on the latest k8s version. + - Should such a job be gating or not? + 2. Monitor Alpha features E2E execution results and use the data + as a gating (e.g. 95% success rate) to pass it to Beta stage.