Release of Half-Walrus pretrained weights

[gcouairon]

Hi, first of all, thanks for the amazing work !

I was wondering if that would be possible to release the weights for the Half-Walrus pre-trained model shown in the paper ? It will be easier to experiment with. Thank you !

[mikemccabe210]

Hi Guillaume, just going to repeat what we discussed here for other readers since I'm going through the issues right now.

We're not planning on releasing the HalfWalrus weights because those were something we considered part of an experiment rather than full model release - it's undertrained and only trained a subset of the full training data.

However, we are happy to release the finetuned weights on the pretrained datasets and that process is working its way through the system now.