Rekognition not returning label instances

[aaroneden]

I am successfully processing images using your plugin, and appreciate it! I am running into an issue where the response coming back from Rekognition does not include the Label->Instances Array. The photo I sent for processing includes 3 cars.

Here is the message I get back from AWS ----
{"_msgid":"c8d786cc224722aa","payload":{"Labels":[{"Name":"Car","Confidence":99.86408233642578},{"Name":"Automobile","Confidence":99.86408233642578},{"Name":"Vehicle","Confidence":99.86408233642578},{"Name":"Transportation","Confidence":99.86408233642578},{"Name":"Tire","Confidence":95.65074920654297},{"Name":"Alloy Wheel","Confidence":94.08968353271484},{"Name":"Wheel","Confidence":94.08968353271484},{"Name":"Spoke","Confidence":94.08968353271484},{"Name":"Machine","Confidence":94.08968353271484},{"Name":"Sports Car","Confidence":91.68609619140625},{"Name":"Car Wheel","Confidence":87.5404281616211},{"Name":"Car Dealership","Confidence":70.07891845703125},{"Name":"Car Show","Confidence":64.47376251220703},{"Name":"Hot Rod","Confidence":61.91443634033203},{"Name":"Coupe","Confidence":61.532676696777344},{"Name":"Convertible","Confidence":56.90849685668945}]},"statusCode":200,"headers":{"content-type":"image/jpeg","content-length":"153809","x-frame-options":"SAMEORIGIN","connection":"close","date":"Sun, 11 Sep 2022 18:21:31 GMT","x-node-red-request-node":"3b8188a6"},"responseUrl":"http://192.168.86.57:88/cgi-bin/CGIProxy.fcgi?cmd=snapPicture2","redirectList":[],"retry":0,"_event":"node:a242ad21ac2a969d","Image":{"Bytes":[removed to save space]}}

If you look at the responses from the Rekognition demo here: https://us-east-1.console.aws.amazon.com/rekognition/home?region=us-east-1#/label-detection you'll see that each label should have instances and bounding boxes returned...

"Labels": [
        {
            "Name": "Car",
            "Confidence": 99.8634262084961,
            "Instances": [
                {
                    "BoundingBox": {
                        "Width": 0.27277201414108276,
                        "Height": 0.8030486106872559,
                        "Left": 0.0016274770023301244,
                        "Top": 0.18348804116249084
                    },
                    "Confidence": 99.8634262084961
                },

All suggestions welcome!

[aaroneden]

Any suggestions for how to fix this?

[Daniel-t]

Hmm, not sure. This module is a direct wrapper of the AWS api, so anything it returns should come up.

If could be that aws isn't providing bounding boxes, it doesn't always, see https://stackoverflow.com/questions/70951377/how-to-return-image-with-bounding-box-using-aws-rekognition

I'm trying to update the aws api version used at the moment, but I'm on holidays and have no wifi and very low signal, so may need to wait until I get home. That may help as well. [You could try this yourself by installing this package from github and forcing it to upgrade

[aaroneden]

Thank you @Daniel-t -- The strange thing is that the sample response I provided above was from the Rekognition web site using exactly the same photo that I sent via NodeRed. I'll keep digging and ensure I'm not sending anything weird to AWS that's causing it to exclude the bounding boxes.