feat: Online tuning for hipblaslt gemm

[Z-Y00]

No description provided.

[RuibinCheung]

I think the reuse of set_auto_tune will cause confused. I suggest that you can add another an API named GlobalBackendManager.set_tune_level() to control tuning level.

1. When set_auto_tune(False), disable auto tune.
2. When set_auto_tune(True) and set_tune_level(1), use backend selection only
3. When set_auto_tune(True) and set_tune_level(2), use backend selection and hipblaslt multi-algo tuning.

[RuibinCheung]

Also add an environmental variable to control auto tune level.

[xiaobochen-amd]

The new feature added in primus_turbo/pytorch/core/backend.py::AutoKernelDispatcher is too biased toward hipblaslt. It’s not recommended to add it here. With a different backend, it may no longer be an algo_index.

[xiaobochen-amd]

This benchmark file can be removed. We should try to consolidate everything into the existing bench_gemm_turbo.py as much as possible.

[xiaobochen-amd]

Can we remove hipblaslt_gemm_algo_count?

We could replace it with something like:
online_tune_hipblaslt_gemm(..., max_num_tune=50), which directly returns the best algo_index (feel free to decide the exact function name).

This would mean that when Level 2 is enabled, it triggers online_tune_hipblaslt_gemm, obtains the best result, and writes it into the cache.

The idea is that the upper-level framework doesn’t need to be aware of how many algos each backend has. It only needs to specify a maximum tuning budget and retrieve the best-performing result for the current backend.