General

This project is meant to reproduce the results presented by David Ha and Jürgen Schmidhuber in "World Models" paper with different games, architectures and training algorithms.

URL to original paper: https://worldmodels.github.io/

Usage

How to train RNN:

Note that all scripts should be run from this repository root (World-models-RL)

• In rnn_dataset_from_vae.ipynb set
  • VAE latent dim
  • How many datapoints to collect
  • Output dirrectory
  • VAE model path
  • Run it using ipython and then %run rnn/rnn_dataset_from_vae.ipynb
• Execute rnn_training.ipynb by typing in terminal:
  • ipython
  • %run rnn/rnn_training.ipynb
  • Model with the best validation loss will automatically be saved
• After training is finished you can run inference using:
  • python3 -m rnn.rnn_inference for inference in which all states are generated inside model's "dream"
  • python3 -m rnn.rnn_inference_one_step for inference in which real game is played and RNN is used only for next state prediction

Models

Models are currently stored on my google drive. Download them using this link: https://drive.google.com/drive/folders/163vQpkmIZHRwZEKTqnKS24iOwkLx9YgA?usp=sharing

Tips

How to run script on server and leave it running

If you want to run training script on a server and exit ssh connection you can use tmux so that training doesn't stop. On server run:

• tmux new -s session_name
• run training script or anything you need
• Ctrl+B and then D (this will detach the session)
• Now you can exit SSH connection.
• When logged back use tmux attach -t session_name to connect back to this session
• To close session just use Ctrl+D from session

How to open windows while being connected to server

You might want to run scripts that open some windows (like cv2 images) from SSH conenction. To do this you can use ssh user@ip_address -X. On Linux no additional steps should be needed, on MacOS you will probably need to install XQuartz.

On Linux it usually works without any delay. On MacOS i noticed that it sometimes laggs and than plays so if you don't have playback but also no error occured be patient and it will probably play.

If everything was setup correctly game should run. If something it's not error like this will appear:

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/wilk/World-models-RL/.venv/lib/python3.10/site-packages/cv2/qt/plugins" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb.