Skip to content

Add config option#294

Closed
agyoungs wants to merge 5 commits intoosrf:mainfrom
agyoungs:add-config-option
Closed

Add config option#294
agyoungs wants to merge 5 commits intoosrf:mainfrom
agyoungs:add-config-option

Conversation

@agyoungs
Copy link
Contributor

Added a command line option to pass a config file with the command line arguments. I've been using this for a while and have found it handy and I didn't see a way that already existed to allow for this.

@agyoungs agyoungs requested a review from tfoote as a code owner October 30, 2024 22:05
Copy link
Collaborator

@tfoote tfoote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks this has been something that I've wanted to do overall. I have a behavior tweak request.

And to merge this we definitely need to add unit tests.

Why did you pick YAML versus using INI or TOML?

And to that matter Potentially ConfigArgParse might be a good solution to this problem: https://pypi.org/project/ConfigArgParse/ and will provide the different layers automatically for us.

parser.add_argument('image')
parser.add_argument('command', nargs='*', default='')
parser.add_argument('--config', help='''Optional yaml file to handle command line arguments
(except positional args) as a config file. This config will override any other command line
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change the order of precedence here. The command line should be higher priority than the config file. It's closer to the user and has been explicitly typed out on the command line so should take precedence.

This will be really convenient to have standard configurations. And then you can override or extend for a specific need on the commandline reusing the saved config file that's closest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I 100% agree. I spent a little while fiddling with it before submitting this PR but couldn't get it to work and I gave up. Do you know of a way to determine which args from argparse are explicitly passed? I've seen a few methods out there (namely creating a second namespaced parser to compare against), but none of them work as-is with the non-trivial argument options that rocker has so it requires some more digging.

One way I had initially thought of was to check for default values in the argument list, but if you explicitly want to override a config option with a default value, you won't be able to. If you don't have any immediate ideas, I'll find another way to make it work

# Load config file if provided
if args.config:
with open(args.config, 'r') as f:
config = yaml.safe_load(f)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we should be doing some validation here.

And providing some visibility as to what's being loaded and from where as well. Think about how one would debug a typo in a config value or confirm that something is being selected.

@agyoungs
Copy link
Contributor Author

Why did you pick YAML versus using INI or TOML?

Habit. I don't have a strong preference so I'll update that to one of your suggestions

@tfoote
Copy link
Collaborator

tfoote commented Oct 30, 2024

Habit. I don't have a strong preference so I'll update that to one of your suggestions

Can you see how ConfigArgParse would work for this case? It looks like the is_config_file could be pretty powerful. For now skipping the default config file paths is probably worth it.

If the documentation is as I think we might be able to just use
parser.add('-c', '--config', required=False, is_config_file=True, help='config file path for default values')

Or maybe using the file references in ArgParse: https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars

It would be great to leverage other's work in this area.

@agyoungs
Copy link
Contributor Author

@tfoote I've had a few partial implementations sitting around on my machine for this, but I'd like to make an effort to actually clean it all up and get it working upstream. ConfigArgParse is the right tool for this so I went ahead and opened a new pull request with those changes. It'll remain a work in progress until ConfigArgParse itself is updated (it's a few versions out of date, but I'm working to try to push forward the maintenance on that project).

@agyoungs agyoungs closed this Mar 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants