feat: add AI agent guidance for users

[marlon-tobaben]

Hi,

based on internal discussions at CSC, but this requires input from some people.

I think it would be maybe good to emphasize more the terms of use and what you should not do.

Preview is here: https://lumi-supercomputer-docs-preview.rahtiapp.fi/origin/ai-agent-guide/development/ai-tools/ai-agent-guide/

Related PR in terms of infrastructure is #285.

Best,
Marlon

[marlon-tobaben]

Want to additionally add other people that took part in the discussion. Waiting on their GitHub names.

[csctale]

We asking asking a lot from users here, familiarity with concepts like "compromised repository" or understanding providers' data and privacy policies that have never been fully truthful. My view is that people will use these tools, there are risks that cannot be fully mitigated, and we should focus on actionable guidance and the Terms of Use as said earlier. Right now none of these tools are clearly prohibited.

About specific points in the guide:

• Item 2. "Save your work frequently. LUMI admins may have to kill processes if they affect system stability." -- While this is technically true, it's a rare occurrence now that we have automatic process cleanup on login nodes. I think point 5 basically covers misuse and harmful actions.
• Item 3. "Run your AI agent in a container to explicitly decide which folders it can access." -- Have we tested this? I don't know if the editors support this.
• Login node availability -- There have been vscode extensions that blindly launch processes that get stuck, but it's really hard as a user to be aware when these things are happening. In my view this isn't actionable concern to the users or specific to AI agents.

The original Triton docs also raise the ethical concerns. We can think LUMI is better in this regard so that it's not worth mentioning, but at least mention that these models are not running on LUMI. It's also another way of driving home the point that your data is leaving the system.

[klust]

I'm not sure how I should collaborate on this in the best way.

My suggestion:

1. Elaborate a little bit on potential Slurm issues. We have had workflow tools taking down Slurm already and this is a very annoying experience for the sysadmins and for other users. So in my suggestion, I've put a frequency on the polling of squeue, sacct or related APIs.

2. Tapio (LUMI sysadmin) warned us already 5 months ago (message from November 24 to the LUST members in a chat channel) for the risks that all those tools pose. Part is the load on the login nodes, and we already had some discussion within LUST and started looking at what other sites do. SURF (The Netherlands), e.g., requires that even VScode is run in the context of jobs and not on the login nodes. (https://servicedesk.surf.nl/wiki/spaces/WIKI/pages/30660616/Visual+Studio+Code+for+remote+development) So it looks they have already run into load issues. Better warn that users may be confronted with stricter rules in the future.

3. The other thing Tapio warned us for is that some of those tools break the LUMI Terms of Use. I've tried to write some text based on the discussion that we had then in Rocket Chat and also list the tools that he mentioned as certainly not allowed (but that list is non-exhaustive).

As I don't know how to enter those changes in a way that you can select which ones you like and which ones not (as they depend on one another and will need further editing if not all are selected), I have for now made the changes in a separate branch (https://github.com/Lumi-supercomputer/lumi-userguide/tree/ai-agent-guide-remarks-kurt) that builds on top of this one. Page preview is https://lumi-supercomputer-docs-preview.rahtiapp.fi/origin/ai-agent-guide-remarks-kurt/development/ai-tools/ai-agent-guide/ .

[marlon-tobaben]

Thanks @csctale and @klust - these remarks are very helpful. I think maybe next week we can iterate further.

[PramodMunaweera]

1. “Run in a container” is underspecified - we should provide clear guidance and method on how to do this (Agree with Tapio)
2. Clearer distinction between “recommended” and “required”?

[galfthan]

I think this looks quite good, as a first version of guidance. It takes up key things, such as you being responsible for everything it does.

This is more of a policy than a hands on guide for how to use containers and how to ensure slurm is not overloaded. I think this is not a big problem, the practical guide could be a separate follow up to this, giving more info on the "how". We could also provide suitable skills or text to add to claude.md (and others), with guidance on how to be nice to slurm, lustre, etc.

[frroberts-csc]

I wonder if it would be worth adding a row to the table that is something like: the agent not understanding the system limits. Like storage quotas in different spaces, and even some slurm limits. And that the user should double check what issues the agent runs into before just throwing its output at the user support people asking for changes. Not sure if it would have an actual effect on the user tickets we will get but it might be worth a try.

[PramodMunaweera]

Let’s proceed with this. We can refine and improve it later.