specialized case - Object Color Transfer

[alwinsiemensDSDS]

Hi,
Maybe someone can help me.
I want to use the modflows-model to transfer object color. I have 250 objects per color. For two colors it would be 500 images. The images have all the same background color, but the object could be in any position. Is that possible to train flows on this special case with my not so large dataset? The pretrained models don't give me good enough results, therefore I want to train on myself. At the end I want to change the object color based on a reference image

[maria-larchenko]

to use the modflows-model to transfer object color

Hi, I was actually also thinking about this. The hard part is that color transfers typically do not separate objects and do not use semantic information.

If it is important to work with photos, you can try smth like this: https://github.com/andersource/image-color-replacement, but reference image is not supported here, colors should be set manually.

If it is okay to generate images, consider using inpainting with IP-Adapter and Control-Net: set desired reference image as IP-Adapter input, remove your objects and inpaint them back with Control-Net + Adapter. This may work with clothes.

I also would really appreciate if you provide some examples of objects and references, then I could test my pipeline that combines segmentation and modflows color transfer. It is generally possible but depends on your data, specifically whether it is possible to separate objects from background reliably.

[alwinsiemensDSDS]

Hi,
Thanks a lot for your reply.
I will review the proposed github repo and the idea with controlnet + ipadapter in the next days for my use case ;)

Specifically, I want to change the color of a part of an object. Like the color of plastic housings in the following examples. All other Details shall be preserved.

Unfortunately the data that I use is confidential, so it is currently not possible to share it.

[maria-larchenko]

I see, so it is not clothes and there are several different objects...
Also I have found a very recent method for texture transfer, may be useful: https://github.com/ControlGenAI/MaterialFusion

[alwinsiemensDSDS]

Hi,
Thanks a lot for your advice, the paper seems very interesting.
I Think I should add, that I only want to change the color of one object at a time.

So from this (forget the yellow part in the corner).

To this

That is my goal. Hopefully my idea is now clearer.

Best Greetings.