Reasonable defaults in config

[jlibovicky]

With many languages, many metrics, many models to evaluate, it might be tedious to write the full config from scratch. A nice feature might something like: do all available metrics with default params, consider all txt files in directories etc.

[varisd]

I am thinking that we can implement some intermediate "testsuite" ("metricsuite" ? :) object that would have a specific set of parametrized metric instances to avoid having these "default" specifications directly in config.

In config, you would then (optionally) select a testsuite and possibly additional metric instances (or overrides). However, it only means that these spec would move from configs to Python code (which might be less feasible for users who want to share their "testsuites"