[!DO NOT MERGE!][DOC] Pipelined shuffle dependency & concurrent stage scheduling spec discussion#57092

Open

jerrypeng wants to merge 10 commits into

apache:masterfrom

jerrypeng:pipelined-shuffle-spec

Commits on Jul 7, 2026

[SPARK-XXXXX][DOC] Pipelined shuffle dependency & concurrent stage scheduling spec
jerrypeng
committed
[SPARK-XXXXX][DOC] Address review round 1 (cloud-fan): admission, commit, failure framing
jerrypeng
committed
[SPARK-XXXXX][DOC] Address review round 2 (tgravescs): gang-admission scoping, per-dependency shuffle manager
jerrypeng
committed

Commits on Jul 8, 2026

addressing @cloud-fan comments
jerrypeng
committed

Commits on Jul 9, 2026

Commits on Jul 11, 2026

[SPARK-XXXXX][DOC] Address review round 4 (mridulm follow-up): fan-out deferred, reuse layers, checkpoint, admission retry, commit reset
jerrypeng
committed

Commits on Jul 13, 2026