need help

[WZHUZHZUZHU]

I’m currently unable to reproduce the results in your paper using your model. What should I do, and which part should I modify? I’d really appreciate any guidance you could offer.

[CeyaoZhang]

What specific problem did you encounter?

[WZHUZHZUZHU]

I followed the README you provided for reproduction, downloaded the model as you instructed, and reproduced the metrics in Table 1. The code can run normally now. I ran five algorithms five times each within one layout, but the final results are still worse than those in Table 1.

For example, in the Cramped Room layout, the first row of Table 1 shows proagent 197.3 ± 6.1, but the results I obtained are 167.2 ± 10.0. The data includes values like 180, 20, 180, 180, 200, which shows instability—there may be a 20, but even if I replace it with 200, the metrics still can’t reach the values shown in the table.

In the Coordination Ring layout, your reported metric is 175.3 ± 29, but the results I get are 108.8 ± 14.6.

I don’t know what went wrong. I would appreciate any guidance you can provide. Thank you very much for your help and your response.

[CeyaoZhang]

Which LLM did you use, and what was the prompt level you selected?
Getting 20 points in Cramped Room Layout is definitely problematic; you can check where you're stuck.