Skip to content

[Spike] Pipeline generation: Core abstractions and GitHub Actions scheduling#15643

Draft
mitchdenny wants to merge 4 commits intomainfrom
spike/pipeline-generation
Draft

[Spike] Pipeline generation: Core abstractions and GitHub Actions scheduling#15643
mitchdenny wants to merge 4 commits intomainfrom
spike/pipeline-generation

Conversation

@mitchdenny
Copy link
Copy Markdown
Member

Summary

Spike/proof-of-concept for pipeline generation — the ability to generate CI/CD workflow YAML files (e.g., GitHub Actions) from the Aspire application model.

What's Included

Core Abstractions (Aspire.Hosting)

  • IPipelineEnvironment — Marker interface extending IResource for CI/CD environments
  • IPipelineStepTarget — Interface for scheduling targets (workflow jobs implement this)
  • PipelineEnvironmentCheckAnnotation — Annotation-based environment relevance check
  • LocalPipelineEnvironment — Internal fallback for local execution
  • PipelineStep.ScheduledBy — Property linking steps to workflow jobs
  • GetEnvironmentAsync() — Resolution of active pipeline environment from model

GitHub Actions Package (Aspire.Hosting.Pipelines.GitHubActions)

  • GitHubActionsWorkflowResourceResource + IPipelineEnvironment representing a workflow
  • GitHubActionsJob — Job within a workflow, implements IPipelineStepTarget
  • GitHubActionsWorkflowExtensionsbuilder.AddGitHubActionsWorkflow(name) extension
  • SchedulingResolver — Projects step DAG onto job dependency graph with cycle detection

Tests (29 total)

  • 7 environment resolution tests (fallback, single match, ambiguity, late-added resources)
  • 9 workflow resource model tests
  • 13 scheduling resolver tests (fan-out, fan-in, diamond, cycles, default job, mixed)

Spec Document

  • docs/specs/pipeline-generation.md — Full architecture, API primitives, and future roadmap

Example Usage

var workflow = builder.AddGitHubActionsWorkflow("deploy");
var publishJob = workflow.AddJob("publish");
var deployJob = workflow.AddJob("deploy");

builder.Pipeline.AddStep("build-images", BuildImagesAsync, scheduledBy: publishJob);
builder.Pipeline.AddStep("deploy-app", DeployAppAsync,
    dependsOn: "build-images", scheduledBy: deployJob);

Not Yet Implemented

  • YAML generation (aspire pipeline init)
  • State upload/download between jobs
  • aspire do --continue --job <jobId> CLI support
  • Azure DevOps provider

Related

See docs/specs/pipeline-generation.md for full design details and open questions.

…uling

Add core pipeline environment abstractions (IPipelineEnvironment, IPipelineStepTarget,
PipelineEnvironmentCheckAnnotation) to Aspire.Hosting, along with a new
Aspire.Hosting.Pipelines.GitHubActions package implementing workflow resource model
and scheduling resolver.

Key changes:
- IPipelineEnvironment marker interface for CI/CD environments
- IPipelineStepTarget for scheduling steps onto workflow jobs
- PipelineStep.ScheduledBy property for step-to-job assignment
- GetEnvironmentAsync() with annotation-based environment resolution
- GitHubActionsWorkflowResource and GitHubActionsJob types
- SchedulingResolver projecting step DAG onto job dependency graph
- 29 unit tests covering environment resolution and scheduling

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 27, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 15643

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 15643"

@github-actions
Copy link
Copy Markdown
Contributor

Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
GitHub was asked to rerun all failed jobs for that attempt, and the rerun is being tracked in the rerun attempt.
The job links below point to the failed attempt jobs that matched the retry-safe transient failure rules.

Mitch Denny and others added 3 commits March 27, 2026 21:34
- Add WorkflowYaml model types (WorkflowYaml, JobYaml, StepYaml, etc.)
- Add hand-rolled WorkflowYamlSerializer (no external dependencies)
- Add WorkflowYamlGenerator: scheduling result → complete workflow YAML
  - Boilerplate steps: checkout, setup-dotnet, install Aspire CLI
  - State artifact upload/download between dependent jobs
  - Per-job aspire do --continue --job <jobId> execution
- Add TryRestoreStepAsync on PipelineStep for CI/CD state restore
  - Executor calls restore callback before Action
  - If restore returns true, step is skipped (already completed)
- Add 9 YAML generator unit tests
- Add 4 Verify snapshot tests with complete YAML output
- Add 5 step state restore integration tests
- Update spec doc with YAML generation and state restore design

Total: 47 tests passing (12 hosting + 35 GitHub Actions)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds ScheduleStep(string stepName, IPipelineStepTarget target) to
IDistributedApplicationPipeline. This allows consumers to schedule
built-in steps (e.g., WellKnownPipelineSteps.Build) onto CI/CD jobs
without having to create them — useful when integrations register
steps and the AppHost just needs to assign them to workflow jobs.

- Interface and implementation in Aspire.Hosting
- 7 unit tests: basic scheduling, override, built-in steps, null guards, error on missing step

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant