| copyright |
|
||
|---|---|---|---|
| lastupdated | 2022-02-07 | ||
| keywords | openshift, http2, quota | ||
| subcollection | openshift |
{{site.data.keyword.attribute-definition-list}}
{: #openshift_limitations}
{{site.data.keyword.openshiftlong}} and the {{site.data.keyword.openshiftshort}} open source project come with default service settings and limitations to ensure security, convenience, and basic functionality. Some of the limitations you might be able to change where noted. {: shortdesc}
If you anticipate reaching any of the following {{site.data.keyword.openshiftlong_notm}} limitations, contact IBM Support and provide the cluster ID, the new quota limit, and the region in your support ticket. {: tip}
{: #tech_limits}
{{site.data.keyword.openshiftlong_notm}} comes with the following service limitations and quotas that apply to all clusters, independent of what infrastructure provider you plan to use. Keep in mind that the classic and VPC cluster limitations also apply. {: shortdesc}
To view quota limits on cluster-related resources in your {{site.data.keyword.cloud_notm}} account, use the ibmcloud oc quota ls command.
{: tip}
| Category | Description |
|---|---|
| API rate limits | 200 requests per 10 seconds to the {{site.data.keyword.openshiftlong_notm}} API from each unique source IP address. |
| App deployment | The apps that you deploy to and services that you integrate with your cluster must be able to run on the operating system of the worker nodes. |
| Container-native virtualization | The {{site.data.keyword.openshiftshort}} container-native virtualization add-on{: external} to run VM workloads alongside container workloads is not supported by IBM. If you choose to install the add-on yourself, you must use bare metal machines, not virtual machines. You are responsible for resolving any issues and impact to your workloads from using container-native virtualization. |
| Calico network plug-in | Changing the Calico plug-in, components, or default Calico settings is not supported. For example, don't deploy a new Calico plug-in version, or modify the daemon sets or deployments for the Calico components, default IPPool resources, or Calico nodes. Instead, you can follow the documentation to create a Calico NetworkPolicy or GlobalNetworkPolicy, to change the Calico MTU, or to disable the port map plug-in for the Calico CNI. |
| Cluster quota | You can't exceed 100 clusters per region and per infrastructure provider. If you need more of the resource, contact IBM Support. In the support case, include the new quota limit for the region and infrastructure provider that you want. |
| Free clusters | You can create only standard clusters, not free clusters. Instead, you can create a free Kubernetes cluster, and then redeploy the apps that you try out in the Kubernetes cluster to your {{site.data.keyword.openshiftshort}} cluster. |
| IAM access groups | You can't scope {{site.data.keyword.cloud_notm}} IAM service access roles to an IAM access group because the roles are not synced to the RBAC roles within the cluster. If you want to scope RBAC roles to a group of users, you must manually set up groups of users{: external} in your cluster instead of using IAM access groups. You can still manage individual users and service accounts with IAM service access roles. You can also still scope IAM platform access roles to IAM access groups to control actions like ordering worker nodes, because platform access roles are never synced to RBAC roles. |
| Kubernetes | Make sure to review the Kubernetes project limitations{: external}. |
| KMS provider | Customizing the IP addresses that are allowed to connect to your {{site.data.keyword.keymanagementservicefull}} instance is not supported. |
| {{site.data.keyword.openshiftshort}} | Make sure to review the OpenShift Container Platform limitations{: external} for your version. |
| Kubernetes pod logs | To check the logs for individual app pods, you can use the command line to run oc logs <pod name>. Do not use the Kubernetes dashboard to stream logs for your pods, which might cause a disruption in your access to the Kubernetes dashboard. |
| Logging |  {{site.data.keyword.openshiftshort}} 3.11 only: You can't run the Ansible playbook to deploy the OpenShift Container Platform Elasticsearch, Fluentd, and Kibana (EFK) stack{: external} because you can't modify the default configuration of the {{site.data.keyword.openshiftlong_notm}} cluster. In addition, collecting and forwarding API audit logs to {{site.data.keyword.la_full_notm}} is not supported. |
| Monitoring | - Because IBM manages your cluster master, event alerting for the master is disabled. IBM monitors your cluster master and fixes issues as they are detected. For this reason, in the Administrator perspective of the {{site.data.keyword.openshiftshort}}, you might see a Not available message for the control plane status. \n - The built-in Prometheus alert manager includes two rules that display as active alerts in a FIRING state: KubeControllerManagerDown and KubeSchedulerDown. These components are part of the IBM-managed cluster master, so you can ignore these alerts. |
| OpenShift Data Foundation | OpenShift Data Foundation is not supported. |
| Operating system | Worker nodes must run RHEL 7. You can't create a cluster with worker nodes that run different types of operating systems, such as {{site.data.keyword.openshiftshort}} on Red Hat Enterprise Linux and community Kubernetes on Ubuntu. |
| OperatorHub catalog | To use the OperatorHub catalog in private clusters that run {{site.data.keyword.openshiftshort}} version 4.6 or later, see Disabling and mirroring OperatorHub catalog source images. |
| Pod instances | You can run 110 pods per worker node. If you have worker nodes with 11 CPU cores or more, you can support 10 pods per core, up to a limit of 250 pods per worker node. The number of pods includes kube-system and ibm-system pods that run on the worker node. For improved performance, consider limiting the number of pods that you run per compute core so that you don't overuse the worker node. For example, on a worker node with a b3c.4x16 flavor, you might run 10 pods per core that use no more than 75% of the worker node total capacity. |
| Time-based one-time passcode (TOTP) | To use TOTP, make sure that you enable multifactor authentication (MFA) for your entire {{site.data.keyword.cloud_notm}} account. |
| Worker node quota | You can't exceed 500 worker nodes across all clusters in a region, per infrastructure provider. If you need more of the resource, contact IBM Support. In the support case, include the new quota limit for the region and infrastructure provider that you want. |
| Worker pool size | You must always have a minimum of 2 worker nodes in your cluster . Additionally, you can't scale down a worker pool below 2 worker nodes per zone. For more information, see What is the smallest size cluster that I can make?. You can't scale worker pools down to zero. Because of the worker node quota, you are limited in the number of worker pools per cluster and number of worker nodes per worker pool. For example, with the default worker node quota of 500 per region, you might have up to 500 worker pools of 1 worker node each in a region with only 1 cluster. Or, you might have 1 worker pool with up to 500 worker nodes in a region with only 1 cluster. |
| {: caption="{{site.data.keyword.openshiftlong_notm}} limitations"} |
{: #ocp4_limitations}
Review limitations that are specific to {{site.data.keyword.openshiftshort}} version 4 clusters. Keep in mind that the service and classic cluster or VPC cluster limitations also apply. {: shortdesc}
| Category | Description |
|---|---|
| Cluster autoscaling | The Red Hat {{site.data.keyword.openshiftshort}} cluster autoscaler from the {{site.data.keyword.openshiftshort}} Administration > Cluster Settings console or ClusterAutoscaler object from the autoscaling.openshift.io/v1 API is not supported. Instead, use the ibm-iks-cluster-autoscaler Helm plug-in. |
| Cluster updates | You must update your cluster by using the {{site.data.keyword.openshiftlong_notm}} API, CLI, or console tools. You can't update your cluster version from OpenShift Container Platform tools such as the {{site.data.keyword.openshiftshort}} web console. |
| Container logs | If you use a container logging operator such as Fluentd to send logs to an ElasticSearch stack, you must update the cluster logging deployment to use the ibmc-block-gold storage class. |
| Private clusters | Depending on the infrastructure provider, your options for private clusters are limited. \n - VPC: When you create your VPC cluster in the {{site.data.keyword.cloud_notm}} console, your cluster has both a public and a private cloud service endpoint. If you want only a private cloud service endpoint, you must create the cluster in the CLI instead, and include the --disable-public-service-endpoint flag. If you include this flag, your cluster is created with routers and Ingress controllers that expose your apps on the private network only by default. If you later want to expose apps to a public network, you must manually create public routers and Ingress controllers. \n - Classic: You can enable the public and private cloud service endpoint or the public cloud service endpoint only, but you can't enable the private cloud service endpoint only. After cluster creation, you can't later change the service endpoints. |
| Logging | To set up an OpenShift Container Platform Elasticsearch, Fluentd, and Kibana (EFK) stack{: external}, see installing the cluster logging operator. |
| Service catalog | The service catalog is not supported. Use Operators instead. Do not use the OperatorHub to install the service catalog. |
| Service mesh | The Istio managed add-on is not supported. Instead, use the Red Hat service mesh operator{: external}. Note: The default {{site.data.keyword.cloud_notm}} configuration of the routers enables host networking, which is not compatible with the service mesh network policy. For the service mesh ingress to work, apply a network policy{: external}. |
| {: caption="OpenShift Container Platform version 4 cluster limitations"} |
{: #classic_limits}
Classic infrastructure clusters in {{site.data.keyword.openshiftlong_notm}} are released with the following limitations.
{: shortdesc}
{: #classic_compute_limit}
Keep in mind that the service limitations also apply.
| Category | Description |
|---|---|
| Reserved instances | Reserved capacity and reserved instances are not supported. |
| Worker node flavors | Worker nodes are available in select flavors of compute resources. |
| Worker node host access | For security, you can't SSH into the worker node compute host. |
| {: caption="Classic cluster compute limitations"} |
{: #classic_networking_limit}
Keep in mind that the service limitations also apply.
| Category | Description |
|---|---|
| Ingress ALBs | \n - The Ingress application load balancer (ALB) can process 32,768 connections per second. If your Ingress traffic exceeds this number, scale up the number of ALB replicas in your cluster to handle the increased workload. \n - ALBs that run the {{site.data.keyword.openshiftlong_notm}} custom Ingress image only: HTTP/2 is not supported. \n - ALBs that run the [{{site.data.keyword.openshiftlong_notm}} custom Ingress image] (/docs/containers?topic=containers-ingress-types) only: The names of the ClusterIP services that expose your apps must be unique across all namespaces in your cluster. |
| Network load balancers (NLB) | - You can't create version 2.0 network load balancers (NLB 2.0) to expose your apps. \n - You can't create subdomains for private NLBs. \n - You can register up to 128 subdomains. This limit can be lifted on request by opening a support case. |
| {{site.data.keyword.openshiftshort}} web console | The web console cannot be exposed on the private network on clusters that have both public and private endpoints. If you want to expose the web console on the private network, you cluster cannot have a public endpoint enabled. |
| Private VLANs only | Private network load balancers (NLBs) can't be registered with the domain name server (DNS), so the cluster can't be created with only a private network interface. Worker nodes must be connected to both public and private VLANs. You can still create a private service to expose your apps on only the private network. |
| Service endpoints | When you create a cluster, you can enable the public and private cloud service endpoint or the public cloud service endpoint only, but you can't enable the private cloud service endpoint only. After cluster creation, you can't later change the service endpoints. |
| strongSwan VPN service | See strongSwan VPN service considerations. |
| Service IP addresses | You can have 65,000 IP addresses per cluster in the 172.21.0.0/16 range that you can assign to Kubernetes services within the cluster. |
| Subnets per VLAN | Each VLAN has a limit of 40 subnets. |
| {: caption="Classic cluster networking limitations"} |
{: #classic_storage_limit}
Keep in mind that the service limitations also apply.
| Category | Description |
|---|---|
| Volume instances | You can have a total of 250 IBM Cloud infrastructure file and block storage volumes per account. If you mount more than this amount, you might see an "out of capacity" message when you provision persistent volumes. For more FAQs, see the file and block storage docs. If you want to mount more volumes, contact IBM Support. In your support ticket, include your account ID and the new file or block storage volume quota that you want. |
| Portworx | Review the Portworx limitations. |
| File storage | Because of the way that {{site.data.keyword.cloud_notm}} NFS file storage configures Linux user permissions, you might encounter errors when you use file storage. If so, you might need to configure {{site.data.keyword.openshiftshort}} Security Context Constraints{: external} or use a different storage type. |
| {: caption="Classic cluster storage limitations"} |
{: #ks_vpc_gen2_limits}
VPC clusters in {{site.data.keyword.openshiftlong_notm}} are released with the following limitations. Additionally, all the underlying VPC quotas, VPC limits, VPC service limitations, and regular service limitations apply.
{: shortdesc}
{: #vpc_gen2_compute_limit}
Keep in mind that the service limitations also apply.
| Category | Description |
|---|---|
| Encryption | The secondary disks of your worker nodes are encrypted at rest by default by the underlying VPC infrastructure provider. However, you can't bring your own encryption to the underlying virtual server instances. |
| Location | VPC clusters are available only in select multizone metro locations. |
| Versions | VPC clusters must run {{site.data.keyword.openshiftshort}} version 4. |
| Virtual Private Cloud | See Limitations and Quotas. |
| v2 API | VPC clusters use the {{site.data.keyword.openshiftlong_notm}} v2 API. The v2 API is currently under development, with only a limited number of API operations currently available. You can run certain v1 API operations against the VPC cluster, such as GET /v1/clusters or ibmcloud oc cluster ls, but not all the information that a Classic cluster has is returned or you might experience unexpected results. For supported VPC v2 operations, see the CLI reference topic for VPC commands. |
| Worker node flavors | Only certain flavors are available for worker node virtual machines. Bare metal machines are not supported. |
| Worker node host access | For security, you can't SSH into the worker node compute host. |
| Worker node updates | You can't update or reload worker nodes. Instead, you can delete the worker node and rebalance the worker pool with the ibmcloud oc worker replace command. If you replace multiple worker nodes at the same time, they are deleted and replaced concurrently, not one by one. Make sure that you have enough capacity in your cluster to reschedule your workloads before you replace worker nodes. |
| {: caption="VPC cluster compute limitations"} |
{: #vpc_gen2_networking_limit}
Keep in mind that the service limitations also apply.
| Category | Description |
|---|---|
| App URL length | {{site.data.keyword.openshiftshort}} version 4.6 or later only: DNS resolution is managed by the cluster's virtual private endpoint (VPE), which can resolve URLs up to 130 characters. If you expose apps in your cluster with URLs, such as the Ingress subdomain or {{site.data.keyword.openshiftshort}} routes, ensure that the URLs are 130 characters or fewer. |
| Network speeds | VPC profile network speeds refer to the speeds of the worker node interfaces. The maximum speed available to your worker nodes is 16Gbps. Because IP in IP encapsulation is required for traffic between pods that are on different VPC worker nodes, data transfer speeds between pods on different worker nodes might be slower, about half the compute profile network speed. Overall network speeds for apps that you deploy to your cluster depend on the worker node size and application's architecture. |
| NodePort | You can access an app through a NodePort only if you are connected to your private VPC network, such as through a VPN connection. To access an app from the internet, you must use a VPC load balancer or Ingress service instead. |
| Pod network | VPC access control lists (ACLs) filter incoming and outgoing traffic for your cluster at the subnet level, and security groups filter incoming and outgoing traffic for your cluster at the worker nodes level. To control traffic within the cluster at the pod-to-pod level, you can't use VPC security groups or ACLs. Instead, use Calico and Kubernetes network policies, which can control the pod-level network traffic that uses IP in IP encapsulation. |
| Public gateway | If the public service endpoint is enabled, you must attach a public gateway to each VPC subnet so that your worker nodes can communicate on the public network. Default {{site.data.keyword.openshiftshort}} components, such as the web console and OperatorHub, require public network access. |
| Service endpoints | When you create your VPC cluster in the {{site.data.keyword.cloud_notm}} console, your cluster has both a public and a private cloud service endpoint. If you want only a private cloud service endpoint, you must create the cluster in the CLI instead, and include the --disable-public-service-endpoint flag. If you include this flag, your cluster is created with routers and Ingress controllers that expose your apps on the private network only by default. If you later want to expose apps to a public network, you must manually create public routers and Ingress controllers. |
| strongSwan VPN service | The strongSwan service is not supported. To connect your cluster to resources in an on-premises network or another VPC, see Using VPN with your VPC. |
| Subnets | \n - See VPC networking limitations. \n - Do not delete the subnets that you attach to your cluster during cluster creation or when you add worker nodes in a zone. If you delete a VPC subnet that your cluster used, any load balancers that use IP addresses from the subnet might experience issues, and you might be unable to create new load balancers. |
| VPC load balancer | See VPC load balancer limitations. |
| VPC security groups | VPC clusters that run {{site.data.keyword.openshiftshort}} version 4.4 or earlier only: You must allow inbound traffic requests to node ports on your worker nodes. |
| {: caption="VPC cluster networking limitations"} |
{: #vpc_gen2_storage_limit}
Keep in mind that the service limitations also apply.
| Category | Description |
|---|---|
| Storage class for profile sizes | The available volume profiles are limited to 2TB in size and 20,000 IOPS in capacity. |
| Supported types | You can set up {{site.data.keyword.cos_full_notm}} and {{site.data.keyword.databases-for}} only. |
| Unsupported types | NFS File Storage is not supported. |
| Volume attachments | See Volume attachment limits. |
| Portworx | Review the Portworx limitations. |
| {{site.data.keyword.block_storage_is_short}} | The default storage class in VPC clusters can not be changed. However, you can create a custom storage class. |
| {: caption="VPC cluster storage limitations"} |
{: #satellite_limits}
Review the following limitations for {{site.data.keyword.openshiftlong_notm}} clusters that you create in a {{site.data.keyword.satelliteshort}} location. Keep in mind that the service limitations also apply. {: shortdesc}
| Category | Description |
|---|---|
| Cluster add-ons | Review the unsupported managed add-ons for {{site.data.keyword.openshiftshort}} clusters in a {{site.data.keyword.satelliteshort}} location. For example, the cluster autoscaler and Istio are not supported. |
| Key management service (KMS) | Cluster integration with a key management service (KMS) provider like {{site.data.keyword.keymanagementserviceshort}} is not supported. |
| Locations | \n - {{site.data.keyword.openshiftshort}} clusters that are created in {{site.data.keyword.satelliteshort}} locations must use {{site.data.keyword.openshiftshort}} version 4.5 or later. \n - You must create your own {{site.data.keyword.satelliteshort}} location that is managed from select {{site.data.keyword.cloud_notm}} multizone metros. |
| Logging and monitoring | You can't currently use the {{site.data.keyword.openshiftlong_notm}} console or the observability plug-in CLI (ibmcloud ob) to enable logging and monitoring for {{site.data.keyword.satelliteshort}} clusters. Instead, you can manually deploy {{site.data.keyword.la_short}} agents and {{site.data.keyword.mon_short}} agents to your cluster to forward logs and metrics to {{site.data.keyword.la_full_notm}} and {{site.data.keyword.mon_full_notm}}. |
| Network | \n - The private cloud service endpoint is not supported for {{site.data.keyword.satelliteshort}} clusters. \n - Your {{site.data.keyword.satelliteshort}} clusters can't use Kubernetes load balancers. \n - The hosts that run the worker nodes for your cluster must meet the host networking and provider-specific requirements, such as for AWS, Azure, GCP, and {{site.data.keyword.cloud_notm}} (testing and demonstration purposes only). |
| Storage for worker node hosts | See Host storage and attached devices. |
| Storage for apps | No storage provider is installed in your {{site.data.keyword.satelliteshort}} clusters by default. Therefore, no pre-configured Kubernetes storage classes are set up by default in your clusters to store your application data in a Kubernetes persistent volume that is backed by storage device. For options to set up a storage provider, see Understanding {{site.data.keyword.satelliteshort}} storage templates. |
| Worker nodes | Worker nodes run on hosts in your own infrastructure environments. The hosts must meet host and provider-specific requirements, such as for AWS, Azure, GCP, and {{site.data.keyword.cloud_notm}} (testing and demonstration purposes only). You are responsible for managing the infrastructure lifecycle of your hosts, including adding and updating worker nodes. As such, worker node operations like ibmcloud oc worker add, update, replace, reload commands are not supported. |
| Worker pools | To use operations like resize, your worker pool uses host labels that must match available (unassigned) hosts in the {{site.data.keyword.satelliteshort}} location. |
| {: caption="{{site.data.keyword.satelliteshort}} cluster limitations"} |