Hytale server instance fails after host reboot

[MergathFin]

Operating System

Debian 13

AMP Version and Build Date

2.6.5.2 - 20260128.2

AMP Release Stream

Mainline

I confirm that

• I have searched for an existing bug report for this issue.
• I am using the latest available version of AMP.
• my operating system is up-to-date.

Intended Action

Automatically update and start Hytale server instance after rebooting the host.

Expected Behaviour

Hytale instance updates, starts and server authenticates.

Actual Behaviour

Running instance is unable to complete update tasks and stops with error "unable to run", server software can not authenticate and hangs with connection errors in console.

Reproduction

Clean debian server and AMP install. Install Hytale instance normally and authenticate server, "Create in Docker Containers" is set to true in settings under Deployment Defaults. This starts the instance normally with initial authentication working.

Reboot host and instance tries to start but fails to update and start server due to authentication/communication and unresolvable address errors with Hytale OAuth servers and hangs. Setting startup mode to only start allows the instance and server to start running but it hangs on authenticating, no players are able to join the server due to serverside authentication error. Auth persistence is set to "Encrypted" in Hytale console during inital install. Restarting the server software inside instance does not help, in some cases this will show the instance as running even with the server still failing on auth errors. Manually stopping and starting the full instance after failure allows the instance+server to update and start normally without errors.

Possibly related to container starting too fast, before network is properly up and resolving addresses. Maybe related to this issue #1385 with startup delay not working? Tested and as a workaround issue does not appear to happen if a small (tested with 10s) startup delay is added manually to the docker system service, which allows the network to settle before the containers are started. Possible fix would be to add delay/wait or online check in container or before containerized instances start in order for the network to be properly ready?

[p0t4t0sandwich]

This would be different to that start-delay issue, since that setting is used to configure how much time you want between each AMP instance's start.

This is caused by some weird Docker cold-start behavior where the JVM starts inside the Docker container before some certs are loaded properly (often seen with MC as well).
You could try this fix here to see if it sorts things out:
https://discourse.cubecoders.com/t/connectivity-problem-with-application-deployment-all-servers-go-offline-every-saturday-night-at-00-00/36298/4?u=greelan

[MergathFin]

Thanks, I tested the suggested fix and removing the manual delay from docker.service and it didnt fix the problem in this case.

However, tested adding the delay override to ampinstmgr.service instead and that allows the Hytale instance to start normally on host reboot, same result if i added the delay in docker.service. Maybe this is some strange docker/networking issue with the services being up but networking not being properly online and still doing some starting up in background. I guess the amount of delay can be specific to the host system, but in my case 15sec seems to work stable.

Here is the steps i did to apply the 15sec delay workaround in ampinstmgr.service:

1. As root or with sudo systemctl edit ampinstmgr.service
2. Added following:

[Service]
ExecStartPre=/bin/sleep 15

3. Reboot host

[MergathFin]

As an additional note, i think adding also the fix you suggested wouldnt be a bad idea as it might fix other possible issues while running instances with "Create in Docker Containers" enabled. I think by default ampinstmgr.service doesent require docker service to be up for it to run, so that might cause other strange behaviour while trying to run instances as containers as the linked post suggests.

[p0t4t0sandwich]

Looks like the odd cold-start issue is happening with Podman as well (which can't use that same fix), and I've encountered a user running into the issue with MC Bedrock https://discord.com/channels/266012086423912458/1482226081871302706