-
Notifications
You must be signed in to change notification settings - Fork 149
Feature: Implement Garbage Collection (GC) Management Commands #624
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
intojhanurag
wants to merge
27
commits into
goharbor:main
Choose a base branch
from
intojhanurag:feat/gc-commmand-implementation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
949beba
gc commands structure created
intojhanurag e0eb17b
fix:convert dry run coloumn in boolean
intojhanurag f09cc1e
minor_fixing
intojhanurag 7d1c58a
docs(gc): generate GC command documentation
intojhanurag 1500649
fix:Add copywrite headers in all file
intojhanurag d8970a7
feat: Add interactive prompt in update_schedule
intojhanurag 15daff1
feat: Add stop command for running gc
intojhanurag 1ace4f5
feat: add docs by dagger
intojhanurag 7353b71
All edge cases covered
intojhanurag af08e36
feat: Add pagination in list command
intojhanurag 849aa29
fix: bug in the codebase
intojhanurag 5fac257
fix: update_schedule custom command not working
intojhanurag 8f18671
add test file for upate_schedule_test
intojhanurag 1c8587c
fix: address review feedback on GC commands
intojhanurag c434c3f
fix: added docs
intojhanurag 3151948
fix: add missing GC query keys and interactive selection for gc log c…
intojhanurag fd7812e
fix:add docs
intojhanurag f535910
feat: add output-format support for gc list command
intojhanurag ae64877
refactor: migrate gc selection views to base/selection model and prom…
intojhanurag dd0f453
fix: address review feedback for gc commands
intojhanurag 2797bab
fix: use server-side filtering for running GC jobs in stop command
intojhanurag 1680db7
fix: remove committed binary and return error from ListGC instead of …
intojhanurag c6c09af
fix: Apply minor fixes and bugs
intojhanurag b7ff352
fix: use correct API query key 'status' instead of 'job_status' for G…
intojhanurag e37fa48
fix: add cron expressions for predefined GC schedule types (hourly/…
intojhanurag dc63310
fix: regenerated docs
intojhanurag d4de77e
fix: Apply feedback
intojhanurag File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "github.com/spf13/cobra" | ||
| ) | ||
|
|
||
| func GC() *cobra.Command { | ||
| cmd := &cobra.Command{ | ||
| Use: "gc", | ||
| Short: "Manage Garbage Collection", | ||
| Long: "Manage Garbage Collection in Harbor (schedule, history, logs)", | ||
| } | ||
|
|
||
| cmd.AddCommand( | ||
| ListGCCommand(), | ||
| GetGCLogCommand(), | ||
| ViewGCScheduleCommand(), | ||
| UpdateGCScheduleCommand(), | ||
| RunGCCommand(), | ||
| StopGCCommand(), | ||
| ) | ||
|
|
||
| return cmd | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "fmt" | ||
|
|
||
| "github.com/goharbor/harbor-cli/pkg/api" | ||
| "github.com/goharbor/harbor-cli/pkg/utils" | ||
| "github.com/goharbor/harbor-cli/pkg/views/gc" | ||
| log "github.com/sirupsen/logrus" | ||
| "github.com/spf13/cobra" | ||
| "github.com/spf13/viper" | ||
| ) | ||
|
|
||
| var validGCSortFields = []string{ | ||
| "creation_time", | ||
| "update_time", | ||
| "id", | ||
| "job_status", | ||
| } | ||
|
|
||
| var validGCQueryKeys = []string{ | ||
| "id", | ||
| "status", | ||
| } | ||
|
|
||
| func ListGCCommand() *cobra.Command { | ||
| var ( | ||
| opts api.ListFlags | ||
| sort []string | ||
| fuzzy []string | ||
| match []string | ||
| ranges []string | ||
| ) | ||
|
|
||
| cmd := &cobra.Command{ | ||
| Use: "list", | ||
| Short: "List GC history", | ||
| Long: `List GC (Garbage Collection) history in Harbor. | ||
|
intojhanurag marked this conversation as resolved.
|
||
|
|
||
| This command displays a list of GC executions with their status, creation time, | ||
| and other details. You can control the output using pagination flags and format options. | ||
|
|
||
| Examples: | ||
| # List GC history with default pagination (page 1, 10 items per page) | ||
| harbor gc list | ||
|
|
||
| # List GC history with custom pagination | ||
| harbor gc list --page 2 --page-size 20 | ||
|
|
||
| # List GC history with sorting by creation time (newest first) | ||
| harbor gc list --sort -creation_time | ||
|
|
||
| # List GC history with multiple sort fields | ||
| harbor gc list --sort creation_time --sort -update_time | ||
|
|
||
| # Filter GC history by status (exact match) | ||
| harbor gc list --match status=Success`, | ||
| Args: cobra.ExactArgs(0), | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| if opts.PageSize > 100 { | ||
| return fmt.Errorf("page size should be less than or equal to 100") | ||
| } | ||
|
|
||
| if len(sort) > 0 { | ||
| sortParam, err := utils.BuildSortParam(sort, validGCSortFields) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| opts.Sort = sortParam | ||
| } | ||
|
|
||
| if len(fuzzy) != 0 || len(match) != 0 || len(ranges) != 0 { | ||
| q, qErr := utils.BuildQueryParam(fuzzy, match, ranges, validGCQueryKeys) | ||
| if qErr != nil { | ||
| return qErr | ||
| } | ||
| opts.Q = q | ||
| } | ||
|
|
||
| history, err := api.GetGCHistory(opts) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get GC history: %v", utils.ParseHarborErrorMsg(err)) | ||
| } | ||
|
|
||
| if len(history) == 0 { | ||
| log.Info("No GC history found") | ||
| return nil | ||
| } | ||
|
|
||
| formatFlag := viper.GetString("output-format") | ||
| if formatFlag != "" { | ||
| err = utils.PrintFormat(history, formatFlag) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| } else { | ||
| if err := gc.ListGC(history); err != nil { | ||
| return err | ||
| } | ||
| } | ||
| return nil | ||
| }, | ||
| } | ||
|
|
||
| flags := cmd.Flags() | ||
| flags.Int64VarP(&opts.Page, "page", "p", 1, "Page number") | ||
| flags.Int64VarP(&opts.PageSize, "page-size", "s", 10, "Size of per page") | ||
| flags.StringSliceVar(&sort, "sort", nil, "Sort the resource list (e.g. --sort creation_time --sort -update_time)") | ||
| flags.StringSliceVar(&fuzzy, "fuzzy", nil, "Fuzzy match filter (key=value)") | ||
| flags.StringSliceVar(&match, "match", nil, "Exact match filter (key=value)") | ||
| flags.StringSliceVar(&ranges, "range", nil, "Range filter (key=min~max)") | ||
|
|
||
| return cmd | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "fmt" | ||
|
|
||
| "github.com/goharbor/harbor-cli/pkg/api" | ||
| "github.com/goharbor/harbor-cli/pkg/prompt" | ||
| "github.com/spf13/cobra" | ||
| ) | ||
|
|
||
| func GetGCLogCommand() *cobra.Command { | ||
| var gcID int64 | ||
|
|
||
| cmd := &cobra.Command{ | ||
| Use: "log", | ||
| Short: "Get GC job log", | ||
| Long: `Get the log of a specific GC (Garbage Collection) job. | ||
|
|
||
| If no GC job ID is provided via the --id flag, an interactive selector | ||
| will be displayed to choose from available GC jobs. | ||
|
|
||
| Examples: | ||
| # Get GC log by specifying the job ID | ||
| harbor gc log --id 42 | ||
|
|
||
| # Get GC log interactively (select from list) | ||
| harbor gc log`, | ||
| Args: cobra.ExactArgs(0), | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| var err error | ||
|
|
||
| if gcID < 0 { | ||
| return fmt.Errorf("invalid GC job ID: %d. ID must be a positive number", gcID) | ||
| } | ||
|
|
||
| if gcID == 0 { | ||
| gcID, err = prompt.GetGCJobIDFromUser() | ||
| if err != nil { | ||
| return err | ||
| } | ||
| } | ||
|
|
||
| logData, err := api.GetGCJobLog(gcID) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get GC log: %v", err) | ||
| } | ||
|
|
||
| fmt.Println(logData) | ||
| return nil | ||
| }, | ||
| } | ||
|
|
||
| cmd.Flags().Int64Var(&gcID, "id", 0, "ID of the GC job to get logs for") | ||
|
|
||
| return cmd | ||
| } | ||
|
intojhanurag marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "fmt" | ||
|
|
||
| "github.com/goharbor/go-client/pkg/sdk/v2.0/models" | ||
| "github.com/goharbor/harbor-cli/pkg/api" | ||
| "github.com/goharbor/harbor-cli/pkg/utils" | ||
| log "github.com/sirupsen/logrus" | ||
| "github.com/spf13/cobra" | ||
| ) | ||
|
|
||
| func RunGCCommand() *cobra.Command { | ||
| var dryRun, deleteUntagged bool | ||
| var workers int | ||
|
|
||
| cmd := &cobra.Command{ | ||
| Use: "run", | ||
| Short: "Run Garbage Collection manually", | ||
|
intojhanurag marked this conversation as resolved.
|
||
| Args: cobra.ExactArgs(0), | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| scheduleObj := models.ScheduleObj{ | ||
| Type: "Manual", | ||
| } | ||
|
|
||
| params := map[string]interface{}{ | ||
| "dry_run": dryRun, | ||
| "delete_untagged": deleteUntagged, | ||
| "workers": workers, | ||
| } | ||
|
|
||
| scheduleBody := &models.Schedule{ | ||
| Schedule: &scheduleObj, | ||
| Parameters: params, | ||
| } | ||
|
|
||
| err := api.CreateGCSchedule(scheduleBody) | ||
|
intojhanurag marked this conversation as resolved.
intojhanurag marked this conversation as resolved.
|
||
| if err != nil { | ||
| return fmt.Errorf("failed to start GC: %v", utils.ParseHarborErrorMsg(err)) | ||
| } | ||
| log.Info("GC started successfully") | ||
| return nil | ||
| }, | ||
| } | ||
|
|
||
| cmd.Flags().BoolVarP(&dryRun, "dry-run", "", false, "Simulate GC without deleting artifacts") | ||
| cmd.Flags().BoolVarP(&deleteUntagged, "delete-untagged", "", true, "Delete untagged artifacts") | ||
| cmd.Flags().IntVar(&workers, "workers", 1, "Number of workers for GC job") | ||
|
|
||
| return cmd | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.