If I want to train to generate Japanese, do I need to change the entire words.txt file to Japanese as well?

[XIONGPEILIN]

In the pre_glyph YAML file, it doesn't mention modifying this file, but I see that the datasets use this file to generate font images.

[XIONGPEILIN]

the training process uses a pretrained OCR model, ABINet, which is only trained on English.

[weichaozeng]

Yes, the corpus in words.txt file should be changed to Japanese as well. And there are several other places should be adjusted for training on other languages:

• In 3.1 Data Preparation, adjust the corpus in texts.txt and font files in font_dir to support certain languages.
• In 3.3 Text Glyph Pretraining, adjust the corpus in words.txt, font files in font_path and text generation rules in dataloader to support certain languages. Besides, change the visual_encoder to certain language OCR model's vision backbone and modify the code and config for alignment.
• In 3.4 Prior Guided Training, change the ocr model (ABINet) or re-train it to support linguistic supervison on certain language.

[XIONGPEILIN]

thanks

[XIONGPEILIN]

preglyph/modules.py
self.character = string.printable[:-6]
self.num_cls = len(self.character) + 1

This should also be adapted to Japanese.

I made some modifications to the CLIP training, adding embeddings for all Japanese fonts. The training process proceeded normally, but the generation of Japanese kanji is not very good. To be precise, it's not good on the test set, while it looks quite good on the training set.

train
vail

oh i forgot to switch account，it is me XIONGPEILIN

[weichaozeng]

Regarding these cases, I can only provide some suggestions based on speculation. Firstly, verify whether the incorrect character is properly rendered by the font files during the pre-training phases, as the other character in val image seems to be fine. Besides, ensure that there is sufficient data available for training all the characters.

[XIONGPEILIN]

Many of the images in the training set are like this, with only half of the image present. At first, I thought it was due to image augmentation. Now that I've finished training, I modified the inference file to add mangaocr. I tried using it to generate images, but the generated characters only have half of them.

Should the training set include images that are only half present?

[weichaozeng]

No, the training set requires intact and legible text images to ensure accurate model training.

[weichaozeng]

BTW, if you are using SRNet-Datagen to generate data, the problem where only half of the character image is displayed may be caused by insufficient padding during text rendering. You can try modifying the pad parameter in render_text_mask.py to see if it resolves the generation issue

[ManyDragons]

May I ask if words.txt and texts.txt can be used interchangeably? In other words, do the two use the same txt file

[weichaozeng]

May I ask if words.txt and texts.txt can be used interchangeably? In other words, do the two use the same txt file

They dont need to be the same txt file, but they can be used interchangeably (as long as they exclude the illegal character).

[ManyDragons]

Could you please confirm if the charset file referenced in the configuration file should also be replaced? Thank you!