diff --git a/docs/cloud/features/04_infrastructure/warehouses.md b/docs/cloud/features/04_infrastructure/warehouses.md index 7c842be1621..a07e2ccade3 100644 --- a/docs/cloud/features/04_infrastructure/warehouses.md +++ b/docs/cloud/features/04_infrastructure/warehouses.md @@ -19,17 +19,21 @@ import Image from '@theme/IdealImage'; ## What is compute-compute separation? {#what-is-compute-compute-separation} -Compute-compute separation is available for Scale and Enterprise tiers. +In ClickHouse Cloud, compute runs on dedicated CPU and memory clusters called **services**. Each ClickHouse Cloud service includes: -- A group of two or more ClickHouse nodes (or replicas) is required, but the child services can be single replica. +- ClickHouse nodes (referred to as **replicas**) - An endpoint (or multiple endpoints created via ClickHouse Cloud UI console), which is a service URL that you use to connect to the service (for example, `https://dv2fzne24g.us-east-1.aws.clickhouse.cloud:8443`). - An object storage folder where the service stores all the data and partially metadata: +The initial service you create is the primary service. Subsequently, you can create additional services that have access to the + :::note Child single services can scale vertically unlike single parent services. ::: +Compute-compute separation is available for Scale and Enterprise tiers. + Current service in ClickHouse Cloud
diff --git a/docs/use-cases/data_lake/polaris.md b/docs/use-cases/data_lake/polaris.md new file mode 100644 index 00000000000..60fe0b33998 --- /dev/null +++ b/docs/use-cases/data_lake/polaris.md @@ -0,0 +1,102 @@ +--- +slug: /use-cases/data-lake/polaris-catalog +sidebar_label: 'Polaris catalog' +title: 'Polaris catalog' +pagination_prev: null +pagination_next: null +description: 'In this guide, we will walk you through the steps to query + your data using ClickHouse and the Snowflake Polaris catalog.' +keywords: ['Polaris', 'Snowflake', 'Data Lake'] +show_related_blogs: true +doc_type: 'guide' +--- + +import BetaBadge from '@theme/badges/BetaBadge'; + + + +ClickHouse supports integration with multiple catalogs (Unity, Glue, Polaris, +etc.). In this guide, we will walk you through the steps to query your data +using ClickHouse and the [Apache Polaris Catalog](https://polaris.apache.org/releases/1.1.0/getting-started/using-polaris/#setup). +Apache Polaris supports Iceberg tables and Delta Tables (via Generic Tables). This integration only supports Iceberg tables at this time. + +:::note +As this feature is experimental, you will need to enable it using: +`SET allow_experimental_database_unity_catalog = 1;` +::: + +## Prerequisites {#prerequisites} + +To connect to the Polaris catalog, you will need: + +- Snowflake Open Catalog (hosted Polaris) or self-hosted Polaris Catalog +- Your Polaris catalog URI (for example, `https://..aws.snowflakecomputing.com/polaris/api/catalog/v1` or `http://polaris:8181/api/catalog/v1/oauth/tokens`) +- Catalog credentials (client ID and client secret) +- The OAuth tokens URI for your Polaris instance +- Storage endpoint for the object store where your Iceberg data lives (for example, S3) +- ClickHouse version 26.1+ + +For Open Catalog, Snowflake's managed Polaris offering, your URI will include `/polaris` while for self-hosted, it may not. + + + +## Creating a connection between Polaris and ClickHouse {#connecting} + +Create a database that connects ClickHouse to your Polaris catalog: + +```sql +CREATE DATABASE polaris_catalog +ENGINE = DataLakeCatalog('https:///api/catalog/v1') +SETTINGS + catalog_type = 'rest', + catalog_credential = ':', + warehouse = 'snowflake', + auth_scope = 'PRINCIPAL_ROLE:ALL', + oauth_server_uri = 'https:///api/catalog/v1/oauth/tokens', + storage_endpoint = '' +``` + +## Query the Polaris catalog using ClickHouse {#query-polaris-catalog} + +Once the connection is in place, you can query Polaris: + +```sql title="Query" +USE polaris_catalog; +SHOW TABLES; +``` + +To query a table: + +```sql title="Query" +SELECT count(*) FROM `polaris_db.my_iceberg_table`; +``` + +:::note +Backticks are required, for example, `schema.table`. +::: + +To inspect the table DDL: + +```sql +SHOW CREATE TABLE `polaris_db.my_iceberg_table`; +``` + +## Loading data from Polaris into ClickHouse {#loading-data-into-clickhouse} + +To load data from Polaris into a ClickHouse table, create the target table with your desired schema, then insert from the Polaris table: + +```sql title="Query" +CREATE TABLE my_clickhouse_table +( + -- define columns to match your Iceberg table + `id` Int64, + `name` String, + `event_time` DateTime64(3) +) +ENGINE = MergeTree +ORDER BY id; + +INSERT INTO my_clickhouse_table +SELECT * FROM polaris_catalog.`polaris_db.my_iceberg_table`; +``` +