AMD_instructions.txt

Setting up the Ollama container and downloading models

docker run -d --restart always --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm

docker exec -it ollama bash

ollama pull {model name}



Running OpenWebUI (Not essential, but I would recommend it)

docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main



Setting up the ROCM docker container (thanks to Trade Mamba for the initial instruction https://medium.com/@trademamba/pytorch-using-amd-gpus-for-the-average-data-science-joe-389bf0878e95)

#You can use this for initial setup, then stop and start it using just the name. Adding a volume means that your work will be preserved, even if the container needs to be updated or rebuilt from scratch

docker volume create rocmjupytervolume

docker run -d \
    -p 8888:8888 \
    -p 3000:3000 \
    -p 5000:5000 \
    --cap-add=SYS_PTRACE \
    --security-opt seccomp=unconfined \
    --device=/dev/kfd \
    --device=/dev/dri \
    --group-add video \
    --net=host \
    --ipc=host \
    --shm-size 8G \
    --volume rocmjupytervolume:/MachineLearning \
    --name rocmjupyter \
    rocm/pytorch:latest \
    bash -c "sleep infinity"

docker exec -it rocmjupyter bash

#This only needs to be run once
pip install jupyterlab

#I'd suggest adding the following to a bash script, so you can quickly get it running
nohup jupyter-lab --ip=0.0.0.0 --port=8888 --no-browser --allow-root \
--NotebookApp.token='mytoken'

#The link your browser will use to connect
http://<your-server-ip>:8888/?token=mytoken