Cuda OOM when run inference

[dmxj]

when run inference refer to ModFlows_demo.ipynb，An OOM error occurred， my gpu is V100 have 32GB memories

[maria-larchenko]

Hi! What is the size of your image? In general, it is possible to run this code on any size, but it is not supported in the demo jupyter

[alobashev]

@dmxj

Please check the notebook with batched inference. https://github.com/maria-larchenko/modflows/blob/main/ModFlows_demo_batched.ipynb

You can set batch_size_pixels to control the GPU memory consumption.

styled_im = run_inference_modflows_batched(
    encoder, 
    device, 
    content_im, 
    style_im, 
    enc_steps=10, 
    strength=0.999,
    batch_size_pixels=4096,
)

[dmxj]

@dmxj

Please check the notebook with batched inference. https://github.com/maria-larchenko/modflows/blob/main/ModFlows_demo_batched.ipynb

You can set batch_size_pixels to control the GPU memory consumption.

styled_im = run_inference_modflows_batched(
    encoder, 
    device, 
    content_im, 
    style_im, 
    enc_steps=10, 
    strength=0.999,
    batch_size_pixels=4096,
)

@alobashev thank you, when I change to run_inference_modflows_batched, it's worked!!
but the color transfer result looked a bit color disturbance and distortion. I set strength=0.5, others params is default.

content image:

style image:

result with B6 type:

result with B0 type:

[maria-larchenko]

result looked a bit color disturbance and distortion

Thank you for sharing the result! Yes, the color transfer with this method could be too strong and also it highly depends on a reference image. In this case I would recommend setting enc_steps=40 and strength=0.3 to get less artifacts

Actually it is a hard task for color transfer to handle this style image, because the blue sky takes a huge part of the photo, and color proportions are important for color transfer. In some cases you can just crop the style image and get better results

Original style image

Cropped style

Also it is possible to get greener trees without a blue road, if a style image contains good match for black and grey colors of the road (so the encoder will not transfer it to blue)

Result with the new style image

[maria-larchenko]

@dmxj by the way, these photos are the good example, may we use them in additional tutorial for the encoder in the future?

[dmxj]

@dmxj by the way, these photos are the good example, may we use them in additional tutorial for the encoder in the future?

@maria-larchenko Of course! Thank you very much for your reply and guidance.