how can i use predictor.py?

my code is bellow, but someting wrong.

    mpredictor = Im2SeqPredictor(args.encoder, args.decoder, args.vocabulary)

    img_path = '../../imgtest/' + args.img
    img = Image.open(img_path)
    ans = mpredictor.predict(img)
    print(ans)

[AIRLegend]

Any error?

Any error?

I modify some code of the predictor.py
from:
self.decoder = Decoder(40, 256, len(self.vocabulary))
to:
self.decoder = Decoder(40, 256, len(Vocabulary.load_from(self.vocabulary)))

and i hava another error:

  File "D:\study\modules\MathOCR\src\models\predict_.py", line 33, in <module>
    ans = mpredictor.predict(img)
  File "D:\study\modules\MathOCR\src\models\predictor.py", line 44, in predict
    formula = self._predict_greedy(image)
  File "D:\study\modules\MathOCR\src\models\predictor.py", line 124, in _predict_greedy
    encoded = self.encoder(image)
  File "C:\Users\Icarus\Anaconda3\envs\MathOCR\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1012, in __call__
    outputs = call_fn(inputs, *args, **kwargs)
  File "D:\study\modules\MathOCR\src\models\encoder.py", line 29, in call
    x = self.conv1(x)
  File "C:\Users\Icarus\Anaconda3\envs\MathOCR\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 998, in __call__
    input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
  File "C:\Users\Icarus\Anaconda3\envs\MathOCR\lib\site-packages\tensorflow\python\keras\engine\input_spec.py", line 259, in assert_input_compatibility
    ' but received input with shape ' + display_shape(x.shape))
ValueError: Input 0 of layer conv2d is incompatible with the layer: expected axis -1 of input shape to have value 1 but received input with shape (1, 75, 300, 3)

could you please give me some example to use predictor.py?

[AIRLegend]

It's been a while since I worked on this, but my guess is that you're loading a 3-channel image.

I made some utilities for loading images, take a look at https://github.com/AIRLegend/MathOCR/blob/master/src/utils/image.py.

For loading and predicting you should do something like this:

from models import predictor
from utils import image as improc
from utils.latexlang import Vocabulary

vocab = Vocabulary.load_from(PATH_TO_VOCABULARY.txt)

enc_weights = 'PATH_TO_ENCODER_WEIGHTS'
dec_weights = 'PATH_TO_DECODER_WEIGHTS'

pred = predictor.Im2SeqPredictor(enc_weights, dec_weights, vocab)

im = improc.load_png(PATH_TO_PNG)

formula = pred.predict(im, beam=True)

print(''.join(formula))

Hope this can help you.

It's been a while since I worked on this, but my guess is that you're loading a 3-channel image.

I made some utilities for loading images, take a look at https://github.com/AIRLegend/MathOCR/blob/master/src/utils/image.py.

For loading and predicting you should do something like this:

from models import predictor
from utils import image as improc
from utils.latexlang import Vocabulary

vocab = Vocabulary.load_from(PATH_TO_VOCABULARY.txt)

enc_weights = 'PATH_TO_ENCODER_WEIGHTS'
dec_weights = 'PATH_TO_DECODER_WEIGHTS'

pred = predictor.Im2SeqPredictor(enc_weights, dec_weights, vocab)

im = improc.load_png(PATH_TO_PNG)

formula = pred.predict(im, beam=True)

print(''.join(formula))

Hope this can help you.

thank you so much! It really help me!
But i found that it only print )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) .The imgs I test were in the data set and something unrelated like cat. Have you ever meet this problem?
there may be some difference between our model. I train my hand written model by 1 epoch bucause is print loss=0.001 and acc = 1 in validation test by the end of 1 epoch. Will it raise problem?

[AIRLegend]

Have you trained the model? If so, try setting beam to False.

It still has work to do. I remember managed to train it quite well over the generated dataset, but it didn't work outside it. Data augmentation should be added to the training program and maybe fine tune it on your custom dataset.

Have you trained the model? If so, try setting beam to False.

It still has work to do. I remember managed to train it quite well over the generated dataset, but it didn't work outside it. Data augmentation should be added to the training program and maybe fine tune it on your custom dataset.

I am very sorry that I have not recovered for so long. I did not train the model. I changed my research direction according to the teacher’s suggestion. Thank you very much for your help. Your project is very useful. I will do other research later. Thank you again.