Skip to content

Add quantization capabilities for all models. #13

@EricApgar

Description

@EricApgar

All models have hooks for quantization but don't currently support it. This is a valuable add, especially if the hardware isn't capable of loading the whole model in memory.

Ask

Integrate model quantization for all models.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions