Flyte 2 WIP by EngHabu · Pull Request #6583 · flyteorg/flyte

EngHabu · 2025-08-22T06:38:23Z

Add flyteidl and minimal publishing workflows

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

added: For new features.
changed: For changes in existing functionality.
deprecated: For soon-to-be-removed features.
removed: For features being removed.
fixed: For any bug fixed.
security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

main
- Flyte 2 WIP #6583 👈

* Add flyteidl and minimal publishing workflows Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * pretend version for buf python Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-22T06:38:58Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

* python version fix and use OIDC to authenticate crates Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Fix python versioning Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Set version before setting up env Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * echo version Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Write to GITHUB_ENV Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Use env: Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * buf generate Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-22T18:38:23Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-25T20:39:37Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

* Mark go package as /v2 to be able to import both at the same time Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Upgrade to go 1.24 Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Rename python, rust and npm to flyteidl2 Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-26T01:51:47Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

…6610) * trying idl2 Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update trace def Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor: rename idl2 to flyteidl2 Signed-off-by: machichima <nary12321@gmail.com> * generate -> buf Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor: rename idl2 to flyteidl2 Signed-off-by: machichima <nary12321@gmail.com> * build: re-gen Signed-off-by: machichima <nary12321@gmail.com> * fix: exclude python venv Signed-off-by: machichima <nary12321@gmail.com> * Add translator service Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Regen Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * buf gen Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * remove node execution service proto Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Breakup task service into a separate package Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Help strings for make file targets Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update TaskLog with read bool Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Trying the updated protovalidate plugin Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * protovalidate with semver Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * try claude :shrug Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * add buf Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * update Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * update deps Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * build: remove generate validate for python Signed-off-by: machichima <nary12321@gmail.com> * build: use managed mode on buf.gen.yaml Signed-off-by: machichima <nary12321@gmail.com> * Update validate go plugin Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update imagebuild Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Fix oneof buf lint Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Add deps to generated python code Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Be explicit about the index Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * updated idl Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Catch up to main Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Generate mocks for workflow package Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * generate mocks for other packages Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * buf improvements Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * updates Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * feat: generate mock for common Signed-off-by: machichima <nary12321@gmail.com> * use make gen in GH Actions Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Do not setup python env in generation CI Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * setup uv Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * set working directory Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Try to reference files by path Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Explicitly move .venv to root Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * try exclude in buf.yaml Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * set UV_PROJECT_ENVIRONMENT Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * download tooling Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * add which mockery Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com>

flyte-bot · 2025-09-22T20:23:04Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

* Add Trigger APIs Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * update triggers Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

bito-code-review · 2025-09-30T18:23:19Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* Fix cargo build and add CI check Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * lint lib.rs Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update Readme and contributing guides Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * signed? Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix build target Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix build-crate target Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix included ts files in npm Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

bito-code-review · 2025-09-30T22:29:13Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

bito-code-review · 2025-10-03T18:55:43Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* Add device class to `GpuAccelerator` Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * make gen Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Add HABANA_GAUDI as supported device class Signed-off-by: Jeev B <jeevb@users.noreply.github.com> --------- Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

bito-code-review · 2025-10-04T00:01:40Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

bito-code-review · 2025-10-04T04:21:41Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

bito-code-review · 2025-10-07T18:11:43Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

bito-code-review · 2025-10-08T19:51:43Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* Clean up triggers api Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> * make gen Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> --------- Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

bito-code-review · 2025-10-09T18:11:16Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* refactor: define all mockery in .mockery.yaml - Centralize mockery configuration in .mockery.yaml - Remove //go:generate mockery comments from interface files - Add configurations for flyteplugins and flytestdlib packages - Add global filename, inpackage, and outpkg configurations - Remove all 'case: underscore' settings (use global filename template instead) This change consolidates all mock generation configuration into a single location, making it easier to maintain and ensuring consistency across the codebase. This ensures all mock files: - Are named in snake_case via filename template - Are placed in mocks/ directories with package mocks - Follow consistent naming conventions across the codebase Fixes #6815 Signed-off-by: majiayu000 <1835304752@qq.com> * Run make gen Signed-off-by: majiayu000 <1835304752@qq.com> * Update tests for new mockery output Signed-off-by: majiayu000 <1835304752@qq.com> * Regenerate mocks after rebase Signed-off-by: majiayu000 <1835304752@qq.com> * build: move with-expecter to global settings Signed-off-by: machichima <nary12321@gmail.com> * fix: build error in flyteplugins Signed-off-by: machichima <nary12321@gmail.com> * test: convert On to EXPECT in flyteplugins tests Signed-off-by: machichima <nary12321@gmail.com> * test: more On to EXPECT in flyteplugins Signed-off-by: machichima <nary12321@gmail.com> * test: more On to EXPECT in flyteplugins Signed-off-by: machichima <nary12321@gmail.com> * test: On to EXPECT in executor/ Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: majiayu000 <1835304752@qq.com> Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com>

* test: add API test for ListTasks endpoint Signed-off-by: Ryan Lo <wenchih@apache.org> * make gen Signed-off-by: Ryan Lo <wenchih@apache.org> * Revert "make gen" This reverts commit 9ae4c78. Signed-off-by: Ryan Lo <wenchih@apache.org> * chore: make gen & add test tasks Signed-off-by: Ryan Lo <wenchih@apache.org> --------- Signed-off-by: Ryan Lo <wenchih@apache.org>

* build: ignore Cargo.lock Signed-off-by: machichima <nary12321@gmail.com> * build: update Makefile Signed-off-by: machichima <nary12321@gmail.com> * fix: ignore Cargo.lock in dockerignore Signed-off-by: machichima <nary12321@gmail.com> * build: git rm cargo lock Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com>

* feat: add data proxy proto for CreateUploadLocationRequest Signed-off-by: machichima <nary12321@gmail.com> * feat: add skeleton of data proxy service Signed-off-by: machichima <nary12321@gmail.com> * feat: update file exists and create storage path logic Signed-off-by: machichima <nary12321@gmail.com> * build: gen mockery for metadata Signed-off-by: machichima <nary12321@gmail.com> * feat: move out validation function Signed-off-by: machichima <nary12321@gmail.com> * test: add unit test Signed-off-by: machichima <nary12321@gmail.com> * build: add main.go to run dataproxy Signed-off-by: machichima <nary12321@gmail.com> * docs: add config example for data proxy and data storage Signed-off-by: machichima <nary12321@gmail.com> * docs: add development/readme and minio setup for develop locally Signed-off-by: machichima <nary12321@gmail.com> * docs: add example script Signed-off-by: machichima <nary12321@gmail.com> * docs: update config example YAML Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> * fix: add AddContentMd5Metadata back and make compatible with flyteidl Signed-off-by: machichima <nary12321@gmail.com> * fix: AddContentMD5Metadata and base64 encode ContentMD5 Signed-off-by: machichima <nary12321@gmail.com> * feat: run all service (include dataproxy) in same port Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> * fix: update mockery Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com>

Signed-off-by: WangWang0226 <eeha8834@gmail.com>

* feat: add data proxy proto for CreateUploadLocationRequest Signed-off-by: machichima <nary12321@gmail.com> * feat: add skeleton of data proxy service Signed-off-by: machichima <nary12321@gmail.com> * feat: update file exists and create storage path logic Signed-off-by: machichima <nary12321@gmail.com> * build: gen mockery for metadata Signed-off-by: machichima <nary12321@gmail.com> * feat: move out validation function Signed-off-by: machichima <nary12321@gmail.com> * test: add unit test Signed-off-by: machichima <nary12321@gmail.com> * build: add main.go to run dataproxy Signed-off-by: machichima <nary12321@gmail.com> * docs: add config example for data proxy and data storage Signed-off-by: machichima <nary12321@gmail.com> * docs: add development/readme and minio setup for develop locally Signed-off-by: machichima <nary12321@gmail.com> * docs: add example script Signed-off-by: machichima <nary12321@gmail.com> * docs: update config example YAML Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> * fix: add AddContentMd5Metadata back and make compatible with flyteidl Signed-off-by: machichima <nary12321@gmail.com> * fix: AddContentMD5Metadata and base64 encode ContentMD5 Signed-off-by: machichima <nary12321@gmail.com> * feat: run all service (include dataproxy) in same port Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> * fix: update mockery Signed-off-by: machichima <nary12321@gmail.com> * feat: add gateway Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com>

* Add test script for GetTaskDetails Signed-off-by: aniket2405 <aniketshaha2001@gmail.com> * Remove Cargo.lock Signed-off-by: aniket2405 <aniketshaha2001@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: aniket2405 <aniketshaha2001@gmail.com> Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com>

Signed-off-by: machichima <nary12321@gmail.com>

Signed-off-by: Kevin Su <pingsutw@apache.org>

Signed-off-by: Carina Ursu <carina@union.ai>

…ServiceLinks (#6892) Signed-off-by: Kevin Su <pingsutw@apache.org>

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Kevin Su <pingsutw@apache.org>

Signed-off-by: Kevin Su <pingsutw@apache.org>

Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Kevin Su <kevin@union.ai>

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* feat(workflow): add ActionsService proto for action state APIs Introduce a new flyteidl2/workflow/actions_service.proto that defines an ActionsService consolidating state and queue functionality for actions. Add RPCs to update and retrieve action state (UpdateActionStatus, GetActionState), plus EnqueueAction, AbortQueuedAction, and Watch to support queuing and observation. Define request/response messages for UpdateActionStatus and GetActionState, including validation rules and fields for action identifiers, attempt number, status, and serialized NodeStatus state. This centralizes action state/queue operations into a single service intended to replace parts of the existing StateService and QueueService, simplifies APIs for workers, and provides at-least-once semantics for state watches. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(proto): validate action attempt is greater than zero and regen pb - Add buf.validate rule to require attempt > 0 for action request and response in actions_service.proto. This enforces that action attempts start from 1 and prevents zero-valued attempts from being accepted. - Regenerate Go protobuf bindings (actions_service.pb.go) to reflect the added validation annotations and updated descriptor bytes. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(actions): rename ActionsService RPCs and messages Rename several RPCs and corresponding request/response messages to simplify the ActionsService API and provide clearer, shorter names. - Rename EnqueueAction -> Enqueue for queueing new actions. - Rename Watch -> WatchForUpdates to clarify streaming semantics. - Rename UpdateActionStatus -> Update and its request/response types to UpdateRequest/UpdateResponse. - Rename GetActionState -> GetLatestState and its request/response types to GetLatestStateRequest/GetLatestStateResponse. - Rename AbortQueuedAction -> Abort while keeping the original abort request/response payload types. - Update comments to match the new names and keep deprecation notes. These changes make the service surface more concise and consistent, improving readability and aligning RPC names with their intent. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(actions): extract ActionsService and add action RPC msgs Move ActionsService into a dedicated flyteidl2.workflow.actions package and adjust Go package path accordingly. Replace fully-qualified RPC types with local message names for Enqueue, WatchForUpdates, and Abort RPCs to simplify imports and proto usage. Introduce new request/response messages for Enqueue: - EnqueueRequest with action_id, optional parent_action_name, run_spec, input/run output paths, group, subject and a required oneof spec for Task/Trace/Condition actions. - EnqueueResponse as an empty ack. Add WatchForUpdatesRequest/WatchForUpdatesResponse messages to support streaming updates with filtering by parent_action_id and payloads that carry either ActionUpdate or ControlMessage. Minor changes: - Import flyteidl2/task/run.proto for RunSpec. - Add buf.validate annotations and field constraints for required fields. These changes centralize action-related APIs, improve type locality in the proto, and prepare the ActionsService to replace previous StateService and QueueService behaviors. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(generated): add gRPC ActionsService stubs and servicer Add generated gRPC Python bindings for the ActionsService into the workflow actions package. This creates ActionsServiceStub with unary and streaming RPC methods (Enqueue, GetLatestState, WatchForUpdates, Update, Abort) wired to the protobuf serializers/deserializers, and an ActionsServiceServicer skeleton that returns UNIMPLEMENTED for each method. These generated files provide the client and server class templates required to implement and call the new ActionsService, which consolidates state and execution management for actions (replacing previous StateService and QueueService patterns). Keeping the generated stubs in tree enables service integration and local development until the service implementations are provided. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * chore(docker): bump Rust base image to 1.85.0 Update the Dockerfile build ARG for Rust from 1.84.0 to 1.85.0. This brings the base Rust toolchain in line with the latest stable patch release, addressing compatibility and security fixes in the compiler and standard library. It ensures reproducible builds use the newer toolchain without changing other language versions. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * refactor(actions): move actions proto package and go_package Change the proto package from flyteidl2.workflow.actions to flyteidl2.actions and update the Go package option accordingly. This reduces unnecessary nesting, aligns the proto package with the repository layout, and ensures generated Go code is placed under gen/go/flyteidl2/actions rather than gen/go/flyteidl2/workflow/actions. These changes simplify imports and reflect that the service belongs to the actions domain instead of the workflow subpackage. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * fix(actions): update grpc imports and service paths for ActionsService Update generated gRPC Python stubs to reference the correct module path and fully-qualified service names for ActionsService. Replace incorrect imports and symbol names under flyteidl2.workflow.actions with the canonical flyteidl2.actions module. Adjust all RPC method paths and (de)serializers to use flyteidl2_dot_actions_dot_actions__service__pb2. This fixes mismatches between generated stubs and the actual package layout so gRPC clients and servers import the right protobuf symbols and register the correct RPC endpoints. Signed-off-by: Sergey Vilgelm <sergey@union.ai> --------- Signed-off-by: Sergey Vilgelm <sergey@union.ai>

* add avg time to run phase field in task group Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com>

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Add app crate Signed-off-by: Kevin Su <pingsutw@apache.org> * rename the module from auth to app Signed-off-by: Kevin Su <pingsutw@apache.org> * fix Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>

…se it (#6916) * feat(actions): introduce generic Action message Replace the previous Run-centric enqueue types with a generic Action message that represents units of work (task, trace, or condition). Reorder and renumber fields for the new Action message, adding clear inline comments for task/trace/condition specs and moving validation constraints accordingly. Add a compact EnqueueRequest that wraps Action and restores the optional RunSpec for propagation to downstream actions. Improve WatchForUpdatesResponse comments to clarify action_update vs control_message and fix minor formatting. These changes enable more flexible action types beyond runs and improve API clarity for enqueuing and update streaming. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(proto): update actions_service generated files Action message and refactor Enqueue/Update descriptors - Replace previous serialized descriptor blob for actions_service.proto with an updated serialized file that introduces a new top-level Action message and shifts several fields/indices (action_id, parent_action_name, input_uri, run_output_base, group, subject, task, trace, condition, spec). This reflects a proto schema change in actions_service.proto. - Adjust Enqueue, Update, GetLatestState, WatchForUpdates and Abort service descriptors to align with the regenerated proto bytes. - Regenerate the generated Python module gen/python/flyteidl2/actions/ actions_service_pb2.py to incorporate the new descriptor contents, ensuring Python bindings match the updated proto definitions. Why: - The proto schema for Flyte actions changed (new Action message / reorganized fields). Regenerating the Python protobuf ensures code uses the correct wire format and field layout for serialization / deserialization and stays in sync with the updated .proto. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * fix(proto): simplify Action message comment and correct wording Clarify the Action protobuf comment to be more concise and correct the wording. Replace the longer explanation with a shorter line that lists the kinds of things an Action can represent (task executions, traces, and conditions). This improves readability and avoids an awkward sentence construction. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * fix(protocol): rename WatchForUpdatesResponse comment Update the proto comment for WatchForUpdatesResponse to correctly describe the message as a response for watching action state updates. This fixes a misleading comment that previously named it WatchForUpdatesRequest, improving clarity and aligning documentation with the message name. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * generate Update generated protobuf/TS/Rust code comments to clarify and fix inaccurate message descriptions introduced by code generation. - Change Action comment to a concise description: "Action represents a unit of work to be executed. Theses can be task executions, traces, or conditions." to avoid the misleading "more generic version of the Run concept" phrasing. - Rename WatchForUpdatesRequest comment to WatchForUpdatesResponse in generated Go and TS to match the actual message type and purpose. - Trim a truncated Rust doc comment for Action to reflect the corrected description. These edits are limited to generated files' comments to improve readability and correctness of the inline documentation without changing runtime behavior. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * docs(proto): clarify Action differs from workflow Action Add a clarifying comment to the Action message in the actions_service.proto file to explicitly note that this Action is distinct from the Action protobuf defined in flyteidl2/workflow/run_definition.proto. This prevents confusion for developers reading the proto files and helps maintainers avoid accidental mixing of the two conceptually similar but separate message types. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * geenrate Add a note to generated TypeScript, Go, and Rust protobuf files clarifying that the flyteidl2.actions.Action type is distinct from the Action protobuf declared in flyteidl2/workflow/run_definition.proto. This change prevents confusion when developers read generated code by explicitly calling out the difference. It also updates the Rust file's embedded FILE_DESCRIPTOR_SET bytes (regenerated content) to reflect the new generated output. Signed-off-by: Sergey Vilgelm <sergey@union.ai> --------- Signed-off-by: Sergey Vilgelm <sergey@union.ai>

* change proto Signed-off-by: Alex Wu <c.alexwu@gmail.com> * fix proto Signed-off-by: Alex Wu <c.alexwu@gmail.com> * re generate idl Signed-off-by: Alex Wu <c.alexwu@gmail.com> --------- Signed-off-by: Alex Wu <c.alexwu@gmail.com>

Add Abort APIs (#6590)

82a41b6

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Vendor in python buf dependencies (#6651)

1993238

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

EngHabu added 5 commits October 3, 2025 21:20

wip

7bb0db0

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

queue service implementation

075f324

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Support sqllite

987d536

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Runs service

aa24751

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Connect RunService to QueueService

3ef9544

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Triggers API adjustments (#6655)

83acecf

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

RunService.CreateRun from_automation flags (#6653)

cda285a

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

Cleanup triggers api (#6666)

a480286

* Clean up triggers api Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> * make gen Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> --------- Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

majiayu000 and others added 19 commits January 13, 2026 04:58

apply mockery in queue_service_test (#6859)

ef0c2ed

Signed-off-by: WangWang0226 <eeha8834@gmail.com>

feat: add phase counts (#6862)

4c8cc6f

Signed-off-by: machichima <nary12321@gmail.com>

build: pin openapiv2 version to prevent panic (#6870)

54f428b

Signed-off-by: machichima <nary12321@gmail.com>

feat: include run name in recent statuses (#6871)

98c0dac

Signed-off-by: machichima <nary12321@gmail.com>

Add auth and identity proto (#6850)

494cca8

Signed-off-by: Kevin Su <pingsutw@apache.org>

Add connector plugin (#6861)

d22313a

Signed-off-by: Kevin Su <pingsutw@apache.org>

Always set enableServiceLinks to false (#6884)

efa4f95

Signed-off-by: Kevin Su <pingsutw@apache.org>

JsonValuesToLiterals (#6878)

50644f7

Signed-off-by: Carina Ursu <carina@union.ai>

Build SubmitterPodTemplate with default toleration and disable Enable…

83d8cf3

…ServiceLinks (#6892) Signed-off-by: Kevin Su <pingsutw@apache.org>

Stable sort ImagePullSecrets to avoid spec change errors (#6886)

19c2c00

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Typed interface migrate (#6721)

e12798b

Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Kevin Su <pingsutw@apache.org>

update log context (#6894)

ad5838e

Signed-off-by: Kevin Su <pingsutw@apache.org>

Fix RayJob submitter pod affinity to use default config (#6897)

05e39be

Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Kevin Su <kevin@union.ai>

This was referenced Feb 8, 2026

Fix vet issues #6901

Open

Implement state service on top of etcd. #6902

Draft

Initial executor <> Plugins integration #6903

Draft

Switch to pflags #6904

Draft

wild-endeavor and others added 7 commits February 9, 2026 08:48

V2 - auth and project service crate updates (#6907)

bce8228

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

Add app crate (#6913)

8e48016

Signed-off-by: Kevin Su <pingsutw@apache.org>

Add permission action proto (#6898)

b142044

* change proto Signed-off-by: Alex Wu <c.alexwu@gmail.com> * fix proto Signed-off-by: Alex Wu <c.alexwu@gmail.com> * re generate idl Signed-off-by: Alex Wu <c.alexwu@gmail.com> --------- Signed-off-by: Alex Wu <c.alexwu@gmail.com>

Comments

Conversation

EngHabu commented Aug 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

Uh oh!

flyte-bot commented Aug 22, 2025

Uh oh!

flyte-bot commented Aug 22, 2025

Uh oh!

flyte-bot commented Aug 25, 2025

Uh oh!

flyte-bot commented Aug 26, 2025

Uh oh!

flyte-bot commented Sep 22, 2025

Uh oh!

bito-code-review bot commented Sep 30, 2025

Uh oh!

bito-code-review bot commented Sep 30, 2025

Uh oh!

bito-code-review bot commented Oct 3, 2025

Uh oh!

bito-code-review bot commented Oct 4, 2025

Uh oh!

bito-code-review bot commented Oct 4, 2025

Uh oh!

bito-code-review bot commented Oct 7, 2025

Uh oh!

bito-code-review bot commented Oct 8, 2025

Uh oh!

bito-code-review bot commented Oct 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

EngHabu commented Aug 22, 2025 •

edited by github-actions bot

Loading