Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ with (
```

>[!NOTE]
> Make sure to include a SAS token or use a managed identity to grant the service permission to access and download the blob.
> Make sure to include a SAS token or use a managed identity to grant the service permission to access and download the blob. See [storage connection strings](../../api/connection-strings/storage-connection-strings.md) for details.

### Ingest all blobs in a folder

Expand Down
16 changes: 2 additions & 14 deletions data-explorer/kusto/management/data-ingestion/list-blobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,23 +43,11 @@ You must have at least [Table Ingestor](../../access-control/role-based-access-c
> * We recommend using [obfuscated string literals](../../query/scalar-data-types/string.md#obfuscated-string-literals) for *SourceDataLocators* to scrub credentials in internal traces and error messages.
>
> * When used alone, `.list blob` returns up to 1,000 files, regardless of any larger value specified in *MaxFiles*.
>
> * The primary use of `.list blobs` is for queued ingestion which is done asynchronously with no user context. Therefore, [Impersonation](../../api/connection-strings/storage-connection-strings.md#impersonation) isn't supported.

[!INCLUDE [ingestion-properties](../../includes/ingestion-properties.md)]

## Authentication and authorization

Each storage connection string indicates the authorization method to use for access to the storage. Depending on the authorization method, the principal might need to be granted permissions on the external storage to perform the ingestion.

The following table lists the supported authentication methods and the permissions needed for ingesting data from external storage.

|Authentication method|Azure Blob Storage / Data Lake Storage Gen2|Data Lake Storage Gen1|
|--|--|--|
|[Shared Access (SAS) token](../../api/connection-strings/storage-connection-strings.md#shared-access-sas-token)|List + Read|This authentication method isn't supported in Gen1.|
|[Storage account access key](../../api/connection-strings/storage-connection-strings.md#storage-account-access-key)||This authentication method isn't supported in Gen1.|
|[Managed identity](../../api/connection-strings/storage-connection-strings.md#managed-identity)|Storage Blob Data Reader|Reader|

The primary use of `.list blobs` is for queued ingestion which is done asynchronously with no user context. Therefore, [Impersonation](../../api/connection-strings/storage-connection-strings.md#impersonation) isn't supported.

### Path format

The *PathFormat* parameter allows you to specify the format of the creation time for listed blobs. It consists of a sequence of text separators and partition elements. A partition element refers to a partition that is declared in the partition `by` clause, and the text separator is any text enclosed in quotes. Consecutive partition elements must be set apart using the text separator.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ To use the REST API for queued ingestion, you need:

- **Ingestor** role with **table** scope to ingest data into an existing table.
- **Database User** role to access the target database.
- **Storage Blob Data Reader** role on the blob storage container.

For more information, see [Role-based access control](../../access-control/role-based-access-control.md).

Expand Down Expand Up @@ -50,7 +49,7 @@ Each item in the `blobs` array must follow this structure:

|Field|Type|Required|Description|
|--|--|--|--|
|`url`|`string`| :heavy_check_mark: | The URL of the blob to ingest. The service performs light validation on this field.|
|`url`|`string`| :heavy_check_mark: | The URL of the blob to ingest. The service performs light validation on this field. The URL must be accessible by the service. For non-public blobs, include authentication information as part of the URL (for example, SAS token). See [storage connection strings](../../api/connection-strings/storage-connection-strings.md) for details|
|`sourceId`|`Guid`|No|An identifier for the source blob.|
|`rawSize`|`integer`|No|The size of the blob before compression (nullable).|

Expand Down