Open
Conversation
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: rustco <ruster@111.com>
…ion (#5124) * Add allowedAudience to flyte-core external auth deployment documentation Signed-off-by: Mark Waylonis <mwaylonis@gmail.com> * Update auth_setup.rst Signed-off-by: Eduardo Apolinario <653394+eapolinario@users.noreply.github.com> --------- Signed-off-by: Mark Waylonis <mwaylonis@gmail.com> Signed-off-by: Eduardo Apolinario <653394+eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <653394+eapolinario@users.noreply.github.com>
* feat: add `template` field in three Notification message proto Signed-off-by: peterxcli <peterxcli@gmail.com> * feat: add support for custom email templates in email notifications Signed-off-by: peterxcli <peterxcli@gmail.com> * chore: codegen Signed-off-by: peterxcli <peterxcli@gmail.com> * feat: add support for custom email templates in email notifications Signed-off-by: peterxcli <peterxcli@gmail.com> * avoid direct access to proto value emailNotification.template -> emailNotification.GetTemplate Signed-off-by: peterxcli <peterxcli@gmail.com> --------- Signed-off-by: peterxcli <peterxcli@gmail.com>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.5. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.4...3.1.5) --- updated-dependencies: - dependency-name: jinja2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: sjtucoder <sjtucoder@icloud.com>
* Ignore generated code from codecov reports Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Ignore mocks Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Ignore mocks - the right way Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
* [flyteagent] add reserved field in Get Request Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> --------- Signed-off-by: Future-Outlier <eric901201@gmail.com>
Bumps [github.com/go-jose/go-jose/v3](https://github.com/go-jose/go-jose) from 3.0.3 to 3.0.4. - [Release notes](https://github.com/go-jose/go-jose/releases) - [Changelog](https://github.com/go-jose/go-jose/blob/main/CHANGELOG.md) - [Commits](go-jose/go-jose@v3.0.3...v3.0.4) --- updated-dependencies: - dependency-name: github.com/go-jose/go-jose/v3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Update glog 1.2.0 -> 1.2.4 - CVE-2024-45339 Signed-off-by: ddl-ebrown <ethan.brown@dominodatalab.com> * Update go.sum via make generate Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: ddl-ebrown <ethan.brown@dominodatalab.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
* Bump golang.org/x/net and github.com/mattn/go-sqlite3 Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Set glog to correct version Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.5...3.1.6) --- updated-dependencies: - dependency-name: jinja2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add aws secret mamager support to env_var Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com> * Smaller diff Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com> --------- Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Attila Nagy <nagy.attila@gmail.com>
Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
* upstream bound inputs support Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * lint Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> --------- Signed-off-by: Paul Dittamo <pvdittamo@gmail.com>
Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Chris <chris.chan@gridmatic.com> Co-authored-by: Chris <chris.chan@gridmatic.com>
…er launch execution (#6312) * [WIP][flytepropeller][flyteagent] watch agent support task type by go routine Signed-off-by: Future-Outlier <eric901201@gmail.com> * List Agent Timeout Settings Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> * updateTimeOut Signed-off-by: Future-Outlier <eric901201@gmail.com> --------- Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.5.1 to 4.5.2. - [Release notes](https://github.com/golang-jwt/jwt/releases) - [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md) - [Commits](golang-jwt/jwt@v4.5.1...v4.5.2) --- updated-dependencies: - dependency-name: github.com/golang-jwt/jwt/v4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 tasks
Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
* feat(workflow): add ActionsService proto for action state APIs Introduce a new flyteidl2/workflow/actions_service.proto that defines an ActionsService consolidating state and queue functionality for actions. Add RPCs to update and retrieve action state (UpdateActionStatus, GetActionState), plus EnqueueAction, AbortQueuedAction, and Watch to support queuing and observation. Define request/response messages for UpdateActionStatus and GetActionState, including validation rules and fields for action identifiers, attempt number, status, and serialized NodeStatus state. This centralizes action state/queue operations into a single service intended to replace parts of the existing StateService and QueueService, simplifies APIs for workers, and provides at-least-once semantics for state watches. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(proto): validate action attempt is greater than zero and regen pb - Add buf.validate rule to require attempt > 0 for action request and response in actions_service.proto. This enforces that action attempts start from 1 and prevents zero-valued attempts from being accepted. - Regenerate Go protobuf bindings (actions_service.pb.go) to reflect the added validation annotations and updated descriptor bytes. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(actions): rename ActionsService RPCs and messages Rename several RPCs and corresponding request/response messages to simplify the ActionsService API and provide clearer, shorter names. - Rename EnqueueAction -> Enqueue for queueing new actions. - Rename Watch -> WatchForUpdates to clarify streaming semantics. - Rename UpdateActionStatus -> Update and its request/response types to UpdateRequest/UpdateResponse. - Rename GetActionState -> GetLatestState and its request/response types to GetLatestStateRequest/GetLatestStateResponse. - Rename AbortQueuedAction -> Abort while keeping the original abort request/response payload types. - Update comments to match the new names and keep deprecation notes. These changes make the service surface more concise and consistent, improving readability and aligning RPC names with their intent. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(actions): extract ActionsService and add action RPC msgs Move ActionsService into a dedicated flyteidl2.workflow.actions package and adjust Go package path accordingly. Replace fully-qualified RPC types with local message names for Enqueue, WatchForUpdates, and Abort RPCs to simplify imports and proto usage. Introduce new request/response messages for Enqueue: - EnqueueRequest with action_id, optional parent_action_name, run_spec, input/run output paths, group, subject and a required oneof spec for Task/Trace/Condition actions. - EnqueueResponse as an empty ack. Add WatchForUpdatesRequest/WatchForUpdatesResponse messages to support streaming updates with filtering by parent_action_id and payloads that carry either ActionUpdate or ControlMessage. Minor changes: - Import flyteidl2/task/run.proto for RunSpec. - Add buf.validate annotations and field constraints for required fields. These changes centralize action-related APIs, improve type locality in the proto, and prepare the ActionsService to replace previous StateService and QueueService behaviors. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(generated): add gRPC ActionsService stubs and servicer Add generated gRPC Python bindings for the ActionsService into the workflow actions package. This creates ActionsServiceStub with unary and streaming RPC methods (Enqueue, GetLatestState, WatchForUpdates, Update, Abort) wired to the protobuf serializers/deserializers, and an ActionsServiceServicer skeleton that returns UNIMPLEMENTED for each method. These generated files provide the client and server class templates required to implement and call the new ActionsService, which consolidates state and execution management for actions (replacing previous StateService and QueueService patterns). Keeping the generated stubs in tree enables service integration and local development until the service implementations are provided. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * chore(docker): bump Rust base image to 1.85.0 Update the Dockerfile build ARG for Rust from 1.84.0 to 1.85.0. This brings the base Rust toolchain in line with the latest stable patch release, addressing compatibility and security fixes in the compiler and standard library. It ensures reproducible builds use the newer toolchain without changing other language versions. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * refactor(actions): move actions proto package and go_package Change the proto package from flyteidl2.workflow.actions to flyteidl2.actions and update the Go package option accordingly. This reduces unnecessary nesting, aligns the proto package with the repository layout, and ensures generated Go code is placed under gen/go/flyteidl2/actions rather than gen/go/flyteidl2/workflow/actions. These changes simplify imports and reflect that the service belongs to the actions domain instead of the workflow subpackage. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * fix(actions): update grpc imports and service paths for ActionsService Update generated gRPC Python stubs to reference the correct module path and fully-qualified service names for ActionsService. Replace incorrect imports and symbol names under flyteidl2.workflow.actions with the canonical flyteidl2.actions module. Adjust all RPC method paths and (de)serializers to use flyteidl2_dot_actions_dot_actions__service__pb2. This fixes mismatches between generated stubs and the actual package layout so gRPC clients and servers import the right protobuf symbols and register the correct RPC endpoints. Signed-off-by: Sergey Vilgelm <sergey@union.ai> --------- Signed-off-by: Sergey Vilgelm <sergey@union.ai>
* add avg time to run phase field in task group Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> * build: make gen Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
…chema translation. The converter handles bidirectional conversion between Flyte LiteralType/Literal and JSON schema representations, supporting simple types, blobs, collections, maps, enums, unions, and structs. Remove stale binary `cmd` file.
Signed-off-by: Kevin Su <pingsutw@apache.org>
* Add app crate Signed-off-by: Kevin Su <pingsutw@apache.org> * rename the module from auth to app Signed-off-by: Kevin Su <pingsutw@apache.org> * fix Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
…se it (#6916) * feat(actions): introduce generic Action message Replace the previous Run-centric enqueue types with a generic Action message that represents units of work (task, trace, or condition). Reorder and renumber fields for the new Action message, adding clear inline comments for task/trace/condition specs and moving validation constraints accordingly. Add a compact EnqueueRequest that wraps Action and restores the optional RunSpec for propagation to downstream actions. Improve WatchForUpdatesResponse comments to clarify action_update vs control_message and fix minor formatting. These changes enable more flexible action types beyond runs and improve API clarity for enqueuing and update streaming. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * feat(proto): update actions_service generated files Action message and refactor Enqueue/Update descriptors - Replace previous serialized descriptor blob for actions_service.proto with an updated serialized file that introduces a new top-level Action message and shifts several fields/indices (action_id, parent_action_name, input_uri, run_output_base, group, subject, task, trace, condition, spec). This reflects a proto schema change in actions_service.proto. - Adjust Enqueue, Update, GetLatestState, WatchForUpdates and Abort service descriptors to align with the regenerated proto bytes. - Regenerate the generated Python module gen/python/flyteidl2/actions/ actions_service_pb2.py to incorporate the new descriptor contents, ensuring Python bindings match the updated proto definitions. Why: - The proto schema for Flyte actions changed (new Action message / reorganized fields). Regenerating the Python protobuf ensures code uses the correct wire format and field layout for serialization / deserialization and stays in sync with the updated .proto. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * fix(proto): simplify Action message comment and correct wording Clarify the Action protobuf comment to be more concise and correct the wording. Replace the longer explanation with a shorter line that lists the kinds of things an Action can represent (task executions, traces, and conditions). This improves readability and avoids an awkward sentence construction. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * fix(protocol): rename WatchForUpdatesResponse comment Update the proto comment for WatchForUpdatesResponse to correctly describe the message as a response for watching action state updates. This fixes a misleading comment that previously named it WatchForUpdatesRequest, improving clarity and aligning documentation with the message name. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * generate Update generated protobuf/TS/Rust code comments to clarify and fix inaccurate message descriptions introduced by code generation. - Change Action comment to a concise description: "Action represents a unit of work to be executed. Theses can be task executions, traces, or conditions." to avoid the misleading "more generic version of the Run concept" phrasing. - Rename WatchForUpdatesRequest comment to WatchForUpdatesResponse in generated Go and TS to match the actual message type and purpose. - Trim a truncated Rust doc comment for Action to reflect the corrected description. These edits are limited to generated files' comments to improve readability and correctness of the inline documentation without changing runtime behavior. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * docs(proto): clarify Action differs from workflow Action Add a clarifying comment to the Action message in the actions_service.proto file to explicitly note that this Action is distinct from the Action protobuf defined in flyteidl2/workflow/run_definition.proto. This prevents confusion for developers reading the proto files and helps maintainers avoid accidental mixing of the two conceptually similar but separate message types. Signed-off-by: Sergey Vilgelm <sergey@union.ai> * geenrate Add a note to generated TypeScript, Go, and Rust protobuf files clarifying that the flyteidl2.actions.Action type is distinct from the Action protobuf declared in flyteidl2/workflow/run_definition.proto. This change prevents confusion when developers read generated code by explicitly calling out the difference. It also updates the Rust file's embedded FILE_DESCRIPTOR_SET bytes (regenerated content) to reflect the new generated output. Signed-off-by: Sergey Vilgelm <sergey@union.ai> --------- Signed-off-by: Sergey Vilgelm <sergey@union.ai>
* change proto Signed-off-by: Alex Wu <c.alexwu@gmail.com> * fix proto Signed-off-by: Alex Wu <c.alexwu@gmail.com> * re generate idl Signed-off-by: Alex Wu <c.alexwu@gmail.com> --------- Signed-off-by: Alex Wu <c.alexwu@gmail.com>
…ue and watch pipelines. Pass inputUri and runOutputBase as parameters to CreateRun instead of computing them after DB insert. Fix WatchActions to subscribe before listing existing actions to avoid missing events. Stop dropping the live actions table during migrations. Extract extractShortName helper for deriving display names from task template IDs.
machichima
reviewed
Feb 23, 2026
manager/cmd/main.go
Outdated
Comment on lines
110
to
121
Member
There was a problem hiding this comment.
Rather than hard coded config here, I think we should use managerconfig.GetConfig() instead?
manager/config/config.go
Outdated
Comment on lines
52
to
61
Member
There was a problem hiding this comment.
I think we should follow how we did in other config.go file, for example:
var configSection = config.MustRegisterSection(configSectionKey, defaultConfig)
// GetConfig returns the parsed data proxy configuration
func GetConfig() *DataProxyConfig {
return configSection.GetConfig().(*DataProxyConfig)
}
Comment on lines
91
to
93
| opts.BindFlags(flag.CommandLine) | ||
| flag.Parse() | ||
|
|
Member
There was a problem hiding this comment.
Seems like the zap flag is being ignored after the change. I draft a patch to add it back: machichima@3a60375
executor/cmd/main.go
Outdated
Member
There was a problem hiding this comment.
There's multiple places under run() that use os.Exit() rather than return error. It would be better to return error in case we want to add defer block/tests in the future (os.Exit() will directly kill the process)
Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: machichima <nary12321@gmail.com>
rely fully on task action conditions Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: machichima <nary12321@gmail.com>
…/exec-plgns # Conflicts: # gen/rust/Cargo.lock
# Conflicts: # executor/pkg/controller/taskaction_controller.go
EngHabu
force-pushed
the
enghabu/use-pflags
branch
from
ffbdafa to
15c2d4c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Haytham Abuelfutuh haytham@afutuh.com
Tracking issue
Why are the changes needed?
What changes were proposed in this pull request?
How was this patch tested?
Labels
Please add one or more of the following labels to categorize your PR:
This is important to improve the readability of release notes.
Setup process
Screenshots
Check all the applicable boxes
Related PRs
Docs link
main