How to achieve "4.5s end-to-end generation" as in the readme?

[lingcong-k]

Hi,
Thanks for the model.
I am trying to run the inference with the new 1.1 version.
But it is quite slow even with "--optimized " on as described in the readme.

I am running on L40s GPU with
"--optimized" on and
with "pipe.load_lora_weights(
"FireRedTeam/FireRed-Image-Edit-1.0-Lightning", weight_name="FireRed-Image-Edit-1.0-Lightning-8steps-v1.0.safetensors"
)" ,
with 8 steps, the speed is 18s per image, far more than 4.5s per generation as described?

I ran a batch of images, even disregard of the first image being slower due to warm up. The other images were not faster either.
Anything am missing?
maybe @Maycbj can help me here? Thanks

[Maycbj]

Sorry for the late reply.

The default inference.py does not use distilled inference. When using Lightning LoRA, two changes are needed:

1. args.true_cfg_scale should be set lower.
2. args.num_inference_steps should be set to 8.

We ran our tests on an H800, so the speed was a bit faster. On an L40, it’s estimated to be around 6–7 seconds.

The community has also provided some faster demos and inference implementations—you may want to check them out.
https://github.com/PRITHIVSAKTHIUR/FireRed-Image-Edit-1.0-Fast
https://huggingface.co/spaces/prithivMLmods/FireRed-Image-Edit-1.0-Fast