[DOC] add Databricks support matrix [skip ci]

[nvliyuan]

Closes #15059.

Description

This draft PR adds a Databricks support matrix to make runtime compatibility visible before users deploy the RAPIDS Accelerator on Databricks.

Changes include:

• Add docs/databricks-support.md with the current v26.06.0 Databricks runtime matrix covering Spark, Scala, JDK runtime ownership, CUDA jar variants, minimum driver, and runtime notes.
• Add a Databricks Delta Lake feature support table for DBR 14.3 and DBR 17.3, based on the related TME documentation MR: https://gitlab-master.nvidia.com/spark-rapids-tme/documentation/-/merge_requests/178
• Link the new support matrix from docs/download.md near the supported Databricks runtime list.
• Document caveats around Databricks runtime patching, binary compatibility errors such as NoSuchMethodError, and operation-specific Delta CPU fallback.

Validation:

• git diff --check -- docs/databricks-support.md docs/download.md
• IDE lints for docs/databricks-support.md and docs/download.md

Checklists

Documentation

• Updated for new or modified user-facing features or behaviors
• No user-facing change

Testing

• Added or modified tests to cover new code paths
• Covered by existing tests
  (Please provide the names of the existing tests in the PR description.)
• Not required

Performance

• Tests ran and results are added in the PR description
• Issue filed with a link in the PR description
• Not required

[greptile-apps]

Greptile Summary

This PR adds a new docs/databricks-support.md page and links it from docs/download.md, making Databricks runtime compatibility, Delta Lake GPU support, and known caveats visible to users before they deploy the RAPIDS Accelerator.

• Adds a runtime compatibility matrix for v26.06.0 covering DBR 14.3 and 17.3 with Spark, Scala, JDK, CUDA variant, and minimum driver columns.
• Adds a Delta Lake GPU/CPU-fallback support table for DBR 14.3 and 17.3 across reads, writes, DML, OPTIMIZE, auto compaction, deletion vectors, and liquid clustering.
• Updates docs/download.md front matter (has_children: true) and inserts a cross-link to the new matrix near the Databricks runtime list.

Confidence Score: 5/5

Documentation-only change with no code modifications; safe to merge.

Both changed files are Markdown documentation. The new support matrix correctly wires into Jekyll's just-the-docs parent/child navigation, the cross-link in download.md is accurate, and the technical content is internally consistent. The only observations are minor prose-level nits.

No files require special attention.

Important Files Changed

Filename	Overview
docs/databricks-support.md	New documentation page with runtime compatibility table, Delta Lake GPU support table, and caveats; version string is inconsistently written as "v26.06.0" vs "v26.06" within the same document
docs/download.md	Adds has_children: true to Jekyll front matter and inserts a cross-link to the new Databricks support matrix; changes are minimal and correct

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User wants to deploy\nRAPIDS Accelerator on Databricks] --> B{Which DBR runtime?}
    B --> C[DBR 14.3 ML LTS GPU]
    B --> D[DBR 17.3 ML LTS GPU]
    C --> E[Spark 3.5.0 / Scala 2.12\nCUDA 12 or CUDA 13 jar\nDriver R525+]
    D --> F[Spark 4.0.0 / Scala 2.13\nCUDA 12 or CUDA 13 jar\nDriver R525+]
    E --> G{Delta Lake features?}
    F --> H{Delta Lake features?}
    G --> I[Reads: GPU\nDV reads: CPU fallback\nWrites append/overwrite/CTAS/RTAS: GPU\nMERGE incl. liquid clustering: GPU\nOPTIMIZE: CPU fallback]
    H --> J[Reads: GPU\nDV reads: GPU with row-index pushdown\nWrites append/overwrite: GPU\nMERGE non-liquid-clustered: GPU\nOPTIMIZE non-DV non-liquid: GPU\nLiquid clustering: CPU fallback]
    I --> K[Verify with\nspark.rapids.sql.explain=NOT_ON_GPU]
    J --> K

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[User wants to deploy\nRAPIDS Accelerator on Databricks] --> B{Which DBR runtime?}
    B --> C[DBR 14.3 ML LTS GPU]
    B --> D[DBR 17.3 ML LTS GPU]
    C --> E[Spark 3.5.0 / Scala 2.12\nCUDA 12 or CUDA 13 jar\nDriver R525+]
    D --> F[Spark 4.0.0 / Scala 2.13\nCUDA 12 or CUDA 13 jar\nDriver R525+]
    E --> G{Delta Lake features?}
    F --> H{Delta Lake features?}
    G --> I[Reads: GPU\nDV reads: CPU fallback\nWrites append/overwrite/CTAS/RTAS: GPU\nMERGE incl. liquid clustering: GPU\nOPTIMIZE: CPU fallback]
    H --> J[Reads: GPU\nDV reads: GPU with row-index pushdown\nWrites append/overwrite: GPU\nMERGE non-liquid-clustered: GPU\nOPTIMIZE non-DV non-liquid: GPU\nLiquid clustering: CPU fallback]
    I --> K[Verify with\nspark.rapids.sql.explain=NOT_ON_GPU]
    J --> K

Loading

_{Reviews (2): Last reviewed commit: "[DOC] address Databricks support matrix ..." | Re-trigger Greptile}

[sameerz]

I hope our work in 26.08 might eliminate the need for an extra page on Databricks. In the meantime I added some changes needed for the document as it is.

[sameerz]

We are removing support for Databricks 17.3 in 26.08.

Suggested change

	| RAPIDS Accelerator | Databricks Runtime | Apache Spark | Scala | JDK runtime | CUDA jar variants | Minimum NVIDIA driver | Notes |
	|--------------------|--------------------|--------------|-------|-------------|-------------------|-----------------------|-------|
	| v26.06.0 | 13.3 ML LTS GPU | 3.4.1 | 2.12 | Databricks runtime default | CUDA 12, CUDA 13 | R525+ | Supported Databricks 13.3 runtime line. |
	| v26.06.0 | 14.3 ML LTS GPU | 3.5.0 | 2.12 | Databricks runtime default | CUDA 12, CUDA 13 | R525+ | Supported Databricks 14.3 runtime line. |
	| v26.06.0 | 17.3 ML LTS GPU | 4.0.0 | 2.13 | Databricks runtime default | CUDA 12, CUDA 13 | R525+ | Spark 4 / Scala 2.13 Databricks runtime line. |
	| RAPIDS Accelerator | Databricks Runtime | Apache Spark | Scala | JDK runtime | CUDA jar variants | Minimum NVIDIA driver | Notes |
	|--------------------|--------------------|--------------|-------|-------------|-------------------|-----------------------|-------|
	| v26.06.0 | 14.3 ML LTS GPU | 3.5.0 | 2.12 | Databricks runtime default | CUDA 12, CUDA 13 | R525+ | Supported Databricks 14.3 runtime line. |
	| v26.06.0 | 17.3 ML LTS GPU | 4.0.0 | 2.13 | Databricks runtime default | CUDA 12, CUDA 13 | R525+ | Spark 4 / Scala 2.13 Databricks runtime line. |

[nvliyuan]

updated

[sameerz]

We will add liquid clustering support to 17.3 in 26.08 per #14599 .

[nvliyuan]

updated

[sameerz]

Can we make this a sub-page of the Download page?

[nvliyuan]

updated: https://nvliyuan.github.io/yuali-spark-rapids/databricks-support.html

[nvliyuan]

I hope our work in 26.08 might eliminate the need for an extra page on Databricks. In the meantime I added some changes needed for the document as it is.

Sure, I’ll keep this as a Download sub-page for now and apply the requested doc updates. We can fold/remove it later if the 26.08 work makes the separate page unnecessary. thx