Generating CRD specifications with kube-derive without default values

[fjammes]

Problem Overview
We have encountered a challenge while working with CRDs (Custom Resource Definitions) in Kubernetes, particularly in environments synchronized via ArgoCD. The issue arises from the presence of default values in CRD specifications.

When a CRD with default values is applied in a Kubernetes cluster, the control plane removes fields containing default values before storing the CRD in etcd. This behavior is in line with Kubernetes' internal mechanics but creates a mismatch between the original CRD YAML and the stored CRD. As a result, ArgoCD, which relies on synchronization between declared and stored resources, flags this as a drift and cannot reconcile the resources properly.

To resolve this, I have manually removed the fields with default values (in my case spec.names.shortNames, spec.names.categories, and spec.versions.additionalPrinterColumns) from CRDs generated with kube-derive. While this approach addresses the ArgoCD synchronization issue, it raises questions about how not to generated this default fields with kube-derive.

Proposal and Questions
I am exploring whether kube-derive can support generating CRD specifications without default values. Specifically, we would like to understand:

Flexibility in kube-derive:
Is it possible to configure kube-derive to omit default values from the generated CRD specs.

We would greatly appreciate the team's insights on this matter and any recommendations or potential roadmaps for addressing this scenario.

Looking forward to your feedback and guidance!

[fjammes]

Here is a link to the issue: stackabletech/hdfs-operator#626, the associated PR show the yaml CRDS which have been patched in order to run ArgoCD sync successfully.

[fjammes]

Any idea please?

[clux]

This sounds like an argo cd or kubernetes issue tbh. The defaults you are listing (.spec.names.categories and .spec.names.shortNames and .spec.versions[].additionalPrinterColumns) are actual .spec types that are part of the custom resource definition and should not be omitted if set.

The propsal in the upstream issue is to remove them from templating, which is strange to me because then how do you actually get shortnames/printercoloumns etc? You can obviously elide the #[kube()] attributes that generate these, and maybe you could do this conditionally at compile time (or with some new kube switch), but both sound like overkill hacks for an argo diff problem (and worse than the existing ignoreDifferences based soln).

EDIT: slight clarifying edits

[clux]

There is a proposed improvement on the argocd side: argoproj/argo-cd#12686 which would be easy to plug in via #[kube(annotation...).

[clux]

Looking at your proposed PR upstream https://github.com/stackabletech/hdfs-operator/pull/639/files it seems you want to remove empty values rather than omit the fields in general. If we insert explicit empty vectors into optional vectors even if we do not set (e.g.) #[kube(category...)] then that is actually a bug on our side.

[clux]

yep, that should be fixable. raised a bug in #1680

[fjammes]

Thanks @clux , yes! Removing empty (more precisely 'default') values in generated yaml would be perfect and would allow all yaml generated with kube-dist to be compliant with argoCD. Feel free to ask for any additional information, or if you want somebody to test the bugfix #1680 with argoCD.