Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/hypershift/v1beta1/hostedcluster_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,7 @@ type Capabilities struct {
// +kubebuilder:validation:XValidation:rule=`self.platform.type == "Azure" ? self.services.exists(s, s.service == "Konnectivity" && s.servicePublishingStrategy.type == "Route") : true`,message="Azure platform requires Konnectivity to use Route service publishing strategy"
// +kubebuilder:validation:XValidation:rule=`self.platform.type == "Azure" ? self.services.exists(s, s.service == "Ignition" && s.servicePublishingStrategy.type == "Route") : true`,message="Azure platform requires Ignition to use Route service publishing strategy"
// +kubebuilder:validation:XValidation:rule=`has(self.issuerURL) || !has(self.serviceAccountSigningKey)`,message="If serviceAccountSigningKey is set, issuerURL must be set"
// +kubebuilder:validation:XValidation:rule=`!self.services.exists(s, s.service == 'APIServer' && has(s.servicePublishingStrategy.loadBalancer) && s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration) && has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert, cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))`, message="APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]"
// +kubebuilder:validation:XValidation:rule=`!self.services.exists(s, s.service == 'APIServer' && has(s.servicePublishingStrategy.loadBalancer) && s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration) && has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts) && has(self.configuration.apiServer.servingCerts.namedCertificates) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert, has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))`, message="APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered using optional types here to remove all the has blocks

Suggested change
// +kubebuilder:validation:XValidation:rule=`!self.services.exists(s, s.service == 'APIServer' && has(s.servicePublishingStrategy.loadBalancer) && s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration) && has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts) && has(self.configuration.apiServer.servingCerts.namedCertificates) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert, has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))`, message="APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]"
// +kubebuilder:validation:XValidation:rule=`!self.services.exists(s, s.service == 'APIServer' && has(s.servicePublishingStrategy.loadBalancer) && s.?servicePublishingStrategy.loadBalancer.hostname.orValue("") != "" && self.?configuration.apiServer.servingCerts.namedCertificates.orValue("[]").exists(cert, cert.?names.orValue("[]").exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))`, message="APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoelSpeed I tried using CEL optional types as you suggested; but then make update & CRD generation fails consistently on this cost-estimation error: OpenAPIv3 schema, spec.validation.openAPIV3Schema: Forbidden: x-kubernetes-validations estimated rule cost total for entire OpenAPIv3 schema exceeds budget by factor of more than 100x (try simplifying the rule, or adding maxItems, maxProperties, and maxLength where arrays, maps, and strings are declared)]

  • upon digging more in this context, it turned out that CEL optional types (.?/.orValue()) have significantly higher estimated cost in the Kubernetes CRD cost estimator compared to has() guards. Hence, the has() guards worked out properly in local testing & make update also didn't fail.
  • The HostedCluster CRD is massive with many validation rules, so it seem to be having very little cost headroom. We've had similar issue in this repo earlier -> NO-JIRA: fix(api): replace CEL url() validators with regex to fix CRD cost budget

So I think we're good to go with has() guards for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have significantly higher estimated cost

Do you have any sources/links to explain this? Since it's a presence check, it should be fixed (low) cost 🤔

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root cause is likely in the cel-go cost estimator (cel-go/checker/cost.go); let me know if you agree with this hypothesis:

Comparison

Aspect has() guards .?/.orValue()
AST kind SelectKind (test-only) CallKind
Path tracking Preserved Lost — no path for select_optional_field
List size for exists() Bounded by schema Max: math.MaxUint64
Nested loop cost bounded * bounded MaxUint64 * MaxUint64

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking, this feels like a bug in the estimator to me, lets continue with what you have for now

// +kubebuilder:validation:XValidation:rule="!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator) || !has(self.operatorConfiguration.clusterNetworkOperator.disableMultiNetwork) || !self.operatorConfiguration.clusterNetworkOperator.disableMultiNetwork || self.networking.networkType == 'Other'",message="disableMultiNetwork can only be set to true when networkType is 'Other'"
// +kubebuilder:validation:XValidation:rule="self.networking.networkType == 'OVNKubernetes' || !has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator) || !has(self.operatorConfiguration.clusterNetworkOperator.ovnKubernetesConfig)", message="ovnKubernetesConfig is forbidden when networkType is not OVNKubernetes"
type HostedClusterSpec struct {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6490,8 +6490,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6473,8 +6473,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6493,8 +6493,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6805,8 +6805,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6945,8 +6945,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6936,8 +6936,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6919,8 +6919,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6538,8 +6538,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6495,8 +6495,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6491,8 +6491,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6549,8 +6549,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7024,8 +7024,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6513,8 +6513,10 @@ spec:
- message: APIServer loadBalancer hostname cannot be in ClusterConfiguration.apiserver.servingCerts.namedCertificates[]
rule: '!self.services.exists(s, s.service == ''APIServer'' && has(s.servicePublishingStrategy.loadBalancer)
&& s.servicePublishingStrategy.loadBalancer.hostname != "" && has(self.configuration)
&& has(self.configuration.apiServer) && self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
&& has(self.configuration.apiServer) && has(self.configuration.apiServer.servingCerts)
&& has(self.configuration.apiServer.servingCerts.namedCertificates)
&& self.configuration.apiServer.servingCerts.namedCertificates.exists(cert,
has(cert.names) && cert.names.exists(n, n == s.servicePublishingStrategy.loadBalancer.hostname)))'
- message: disableMultiNetwork can only be set to true when networkType
is 'Other'
rule: '!has(self.operatorConfiguration) || !has(self.operatorConfiguration.clusterNetworkOperator)
Expand Down
Loading