16
GPGPU
Anthony Wang edited this page 2024-02-28 15:04:05 +00:00
Table of Contents
You can run general purpose GPU computations using our AMD Radeon 6600XT graphics card. To monitor GPU usage, use the command /opt/rocm/bin/rocm-smi
or btop
(press 5 to show detailed GPU usage information).
PyTorch
You can install PyTorch for ROCm using these instructions. Run your PyTorch programs with the environment variable HSA_OVERRIDE_GFX_VERSION=10.3.0
.
Text generation
We have a llama.cpp instance. Go to /opt/llama.cpp
and run it with ./main -ngl 100 -p "Your prompt here"
. You can also run it in interactive mode (similar to ChatGPT) using the flags -i -ins --color
instead of -p
, or in server mode with ./server -ngl 100
.
Image generation
We have also have a Stable Diffusion web UI instance.
- First connect to exozyme using SSH port forwarding with
ssh -L 8000:/home/USER/.cache/sd.sock USER@exozy.me
- Go to
/opt/stable-diffusion-webui
and run the server with./webui.sh
. - You can now visit the web interface on your local machine at localhost:8000.