Add to the repo:
- The remedial pretrain script.
- The instruct fine tune scipt.
- documentation of the 3 stage curriculum, rationale, ... links to the best instruct fine tune model and recommended remedial - pretrained base model, examples of chat sessions from the 512 seq len instruct model.
Instruct fine tuning:
instruct_fine_tuning_script_rough_draft.py
Remedial pre-train:
remedial-pretrain.py
Add to the repo:
Instruct fine tuning:
instruct_fine_tuning_script_rough_draft.py
Remedial pre-train:
remedial-pretrain.py