Stateless Inference, LM-Eval Integration, Multiple Encoding Schemes, and Device Drift Retention Support

[osama-usuf]

Summary

This PR introduces several enhancements to improve inference flexibility, evaluation workflows, and inference accelerator state management for large models:

Features

• Added support for stateless inference, enabling inference execution without persistent session state.
• Integrated lm-eval for streamlined large language model evaluation and benchmarking.
• Added support for multiple encoding schemes, allowing greater compatibility across model configurations and deployment environments.
• Implemented device retention and drift handling, improving device state persistence and robustness across inference runs.

Benefits

• Simplifies deployment scenarios requiring stateless execution.
• Enables standardized benchmarking using the lm-eval ecosystem.
• Increases flexibility through support for diverse encoding formats.
• Improves reliability and consistency of inference accelerator state management.

Testing

• Verified stateless inference execution paths.
• Validated lm-eval integration against supported benchmark tasks.
• Tested compatibility across supported encoding schemes.
• Confirmed device retention and drift behavior under state transition scenarios.