Feedback: Add all required files in the guide

[kaiserdan]

A lot of the guides use files as examples that are not made available in the guide, it would be great to extend the guides with all the required configuration files required to replicate them.

For example in extending_containers_with_venv it uses a minimal_pytorch.yml but its unclear what this should look like.

By not providing an example its completely unclear if it should include some rocm specific dependencies, and how one should find the correct version of these without having access to a rocm environment to create this conda env.

Also it's unclear how one should build on a container on a base container that already contains some dependencies pre-installed (eg lumi-pytorch-rocm-6.2.1-python-3.12-pytorch-20240918-vllm-4075b35.sif)

[gregordecristoforo]

Hi Daniel,

This repository is part of the LUMI AI workshop that we organize roughly twice a year and is not meant to work as a set of standalone guides. We wrote the LUMI AI Guide for exactly this purpose, so please have a look there. If you want to have a look at the lectures of our last AI workshop, you'll find all recordings and slides in our training material.

About your questions:
The minimal_pytorch.yml file is mentioned in the previous chapter

Also it's unclear how one should build on a container on a base container that already contains some dependencies pre-installed (eg lumi-pytorch-rocm-6.2.1-python-3.12-pytorch-20240918-vllm-4075b35.sif)

We don't recommend building a container using the cotainr tool with a container that already has PyTorch and lots of other packages installed (see lecture 6), you can however extend any container with a virtual environment.

Cheers,
Gregor

[kaiserdan]

Thanks for the great and quick feedback Gregor, I appreciate it!

Do you by chance have the "recipe" for the vllm custom image available somewhere so I could update/modify it with extra dependencies (and in the future newer vllm versions) and rebuild it from scratch?

I will try the Overlay solution today in the meanwhile, but if I could rebuild the vllm base container (which probably includes the dependencies/config for performance optimized multi node LLM inference) that would be best.

[gregordecristoforo]

The containers are build by our contact person at AMD, Samuel Antao, and he keeps his recipes here and here. That said, I wouldn't recommend building a PyTorch container for LUMI from scratch, if you run into any problems we won't be able to help you. Samuel will update and test the containers regularly, so I wouldn't spend much time trying to build the container yourself.

Extending existing containers with additional python packages is a good idea. Keep in mind that this overlay solution that you linked is in the NRIS documentation and not in LUMI's documentation. This means that it is not maintained by anyone working for LUMI. I had a quick glance at it and it appears to do a similar thing to what we use in the AI guide, so I think it works. Personally, I would however follow the approach that we, the LUMI User Support Team, provide as information in our documentation and training material which includes the AI workshop material as I would expect these resources to be better maintained.

[kaiserdan]

Thank you so much for these references!

[gregordecristoforo]

You're welcome, feel free to reach out to us if you have any more questions!