Anthony Wang edited this page 2024-02-28 15:04:05 +00:00

You can run general purpose GPU computations using our AMD Radeon 6600XT graphics card. To monitor GPU usage, use the command /opt/rocm/bin/rocm-smi or btop (press 5 to show detailed GPU usage information).


You can install PyTorch for ROCm using these instructions. Run your PyTorch programs with the environment variable HSA_OVERRIDE_GFX_VERSION=10.3.0.

Text generation

We have a llama.cpp instance. Go to /opt/llama.cpp and run it with ./main -ngl 100 -p "Your prompt here". You can also run it in interactive mode (similar to ChatGPT) using the flags -i -ins --color instead of -p, or in server mode with ./server -ngl 100.

Image generation

We have also have a Stable Diffusion web UI instance.

  1. First connect to exozyme using SSH port forwarding with ssh -L 8000:/home/USER/.cache/sd.sock USER@exozy.me
  2. Go to /opt/stable-diffusion-webui and run the server with ./webui.sh.
  3. You can now visit the web interface on your local machine at localhost:8000.