How To Setup AI Chat and AI Image Generation To Run On AMD GPUs With Linux

0 38

Written by

FamousM1

Free Speech and Privacy Enthusiast

1 year ago

In community: Linux and Open Source Software(5ac1)

Topics: Technology, Education

KoboldAI Section:

This guide should be mostly fool-proof if you follow it step by step. After I wrote it, I followed it and installed it successfully for myself.

1. Install Linux distro 22.04 (Quick Dual boot Tutorial at end)

2. Go to the driver page of your AMD GPU at amd.com or search something like “amd 6800xt drivers”

download the amdgpu .deb for ubuntu 22.04
double clicking the deb file should bring you to a window to install it, install it

3. Go to Terminal and add yourself to the render and video groups using

     sudo usermod -a -G render YourUsernameHere      
     sudo usermod -a -G video YourUsernameHere

4. Confirm you have python 3 installed by typing into terminal

    python3 –version

it should return the version number, mine is 3.10.6

take the first 2 version numbers and edit the next line to yours.
(I added a few version examples, only enter the one you have installed)

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.10 5

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.9 5

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.8 5

this allows the command “python” to be used for your python3 package by increasing the priority of the python3 package to level 5.

5. Verify it by typing “python --version”, a version 3 should come up.

python --version

6. go to Terminal and type

sudo amdgpu-install --usecase=rocm –no-dkms

this installs only the machine learning package and keeps the built in AMD gpu drivers

7. REBOOT your computer

8. Check that ROCM is installed and shows your GPU by opening terminal and typing:

rocminfo

9. Next steps, type:

sudo apt-get install git

If you have python 3.10 enter this
```
apt install python3.10-venv    
```

if you have a different version, enter
“python -m venv venv” and the error message should show which package is available for your python version.

10. after you have the venv package installed, install pip and update it

sudo apt install python3-pip           
python -m pip install --upgrade pip wheel

11. Next is installing the PyTorch machine learning library for AMD:

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/rocm5.2

after that’s installed, check your version numbers with the command

pip list | grep 'torch'

the 3 version numbers that come back should have ROCM tagged at the end.
any others without ROCM can be removed with
“pip uninstall torch==WrongVersionHere”

12. OPTIONAL STEP: Upgrading to the latest stable Linux kernel
I recommend upgrading to the latest linux kernel especially for people on newer GPUs because it added a bunch of new drivers for GPU support. It increased my Stable Diffusion iteration speed by around 5%

Download the Ubuntu Mainline Kernel Installer GUI https://github.com/bkw777/mainline
DEB file in releases, more installation instructions on the github page

Go to start menu and search “Ubuntu Mainline” and open “Ubuntu Mainline Kernel Installer”
click the latest kernel (one on top) for me its 6.1.10, and press Install
reboot after install and you’ll automatically be on the latest kernel

13. OPTIONAL STEP 2: Download CoreCtrl to control your GPU fans and allow GUI overclocking
These commands adds the repo and signals to get the stable version instead of developmental releases:

sudo add-apt-repository ppa:ernstp/mesarc
sudo apt update
sudo sh -c "echo '
Package: *
Pin: release o=LP-PPA-ernstp-mesarc 
Pin-Priority: 1
Package: corectrl
Pin: release o=LP-PPA-ernstp-mesarc
Pin-Priority: 500
' > /etc/apt/preferences.d/corectrl"
sudo apt install corectrl

You can open up CoreCtrl from start menu or terminal

Your computer is now prepared to run KoboldAI or Stable Diffusion

14. Now we’re ready to get KoboldAI:

Create a directory for pygmalion wherever you want. In my case I have a directory just called "Pygmalion"
Go to the directory in Terminal and type

git clone https://github.com/henk717/KoboldAI

Create a text file named “Start.sh” and add this to it:

cd KoboldAI           
git pull &            
source play-rocm.sh --model PygmalionAI/pygmalion-6b --revision main --cacheonly”

Make a text file named Stop.sh and add this to it:
```
pkill -9 -f "aiserver"           
```

15. Run Start.sh

Wait for everything to download and load up, when its ready it should open a webpage to 127.0.0.1:5000

KoboldAI should be running!

Quick Dual Boot tutorial:
- Be extremely careful here and its best practice to keep data backups. Search how to do this on YouTube. Search windows for “Disk Management” program, open it, find a hard drive with at least 100-200gb free space and right click on it in the boxes along the bottom then click Shrink Volume. Shrink it by 100-200gb and process it. You should now have a Free Space partition available on your Harddrive Download the Linux ISO you want, I used Linux Mint Cinnamon. Any Debian based distro like Ubuntu should work. Get a flash drive and download a program called “Rufus” to burn the .iso onto the flashdrive as a bootable drive. Once its finished burning, shut down your pc (don’t restart). Then start it again, access your Bios Boot menu and select the Flash drive. This will start the linux installation disk and from the install menu when it asks where to install it, select the 100-200gb free space partition, press the Plus to create a partition, use the default Ext4 mode and make the mount point “/”. If it asks where to install the bootloader, put it on the same drive youre installing the OS on. Finish thru Install steps

Section 2 Stable Diffusion

Since read.cash wouldn't let me post both articles, I've combined them into one.

This guide should be mostly fool-proof if you follow it step by step. After I wrote it, I followed it and installed it successfully for myself.

1. Install Linux distro 22.04 (Quick Dual boot Tutorial at end)

2. Go to the driver page of your AMD GPU at amd.com or search something like “amd 6800xt drivers”

download the amdgpu .deb for ubuntu 22.04
double clicking the deb file should bring you to a window to install it, install it

3. Go to Terminal and add yourself to the render and video groups using

     sudo usermod -a -G render YourUsernameHere      
     sudo usermod -a -G video YourUsernameHere

4. Confirm you have python 3 installed by typing into terminal

    python3 –version

it should return the version number, mine is 3.10.6

take the first 2 version numbers and edit the next line to yours.
(I added a few version examples, only enter the one you have installed)

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.10 5

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.9 5

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.8 5

this allows the command “python” to be used for your python3 package by increasing the priority of the python3 package to level 5.

5. Verify it by typing “python --version”, a version 3 should come up.

python --version

6. go to Terminal and type

sudo amdgpu-install --usecase=rocm –no-dkms

this installs only the machine learning package and keeps the built in AMD gpu drivers

7. REBOOT your computer

8. Check that ROCM is installed and shows your GPU by opening terminal and typing:

rocminfo

9. Next steps, type:

sudo apt-get install git
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
cd stable-diffusion-webui

If you have python 3.10 enter this
```
apt install python3.10-venv    
```

if you have a different version, enter
“python -m venv venv” and the error message should show which package is available for your python version.

10. after you have the venv package installed, install pip and update it

sudo apt install python3-pip           
python -m venv venv
source venv/bin/activate
python -m pip install --upgrade pip wheel

11. Next is installing the PyTorch machine learning library for AMD:

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/rocm5.2

after that’s installed, check your version numbers with the command

pip list | grep 'torch'

the 3 version numbers that come back should have ROCM tagged at the end.
any others without ROCM can be removed with
“pip uninstall torch==WrongVersionHere”

12. Next you’ll need to download the models you want to use for Stable Diffusion,
SD v1.5 CKPT: https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.ckpt
also download
Stable Diffusion v1.5 inpainting CKPT: https://huggingface.co/runwayml/stable-diffusion-inpainting/resolve/main/sd-v1-5-inpainting.ckpt

Once those have downloaded, cut and paste them both to your Stable Diffusion model folder which should be located in your home folder:
```
“~/stable-diffusion-webui/models/Stable-diffusion”
```

13. OPTIONAL STEP: Upgrading to the latest stable Linux kernel
I recommend upgrading to the latest linux kernel especially for people on newer GPUs because it added a bunch of new drivers for GPU support. It increased my Stable Diffusion iteration speed by around 5%

Download the Ubuntu Mainline Kernel Installer GUI https://github.com/bkw777/mainline
DEB file in releases, more installation instructions on the github page

14. OPTIONAL STEP 2: Download CoreCtrl to control your GPU fans and allow GUI overclocking
These commands adds the repo and signals to get the stable version instead of developmental releases:

sudo add-apt-repository ppa:ernstp/mesarc
sudo apt update
sudo sh -c "echo '
Package: *
Pin: release o=LP-PPA-ernstp-mesarc 
Pin-Priority: 1
Package: corectrl
Pin: release o=LP-PPA-ernstp-mesarc
Pin-Priority: 500
' > /etc/apt/preferences.d/corectrl"
sudo apt install corectrl

You can open up CoreCtrl from start menu or terminal

Your computer is now prepared to run KoboldAI or Stable Diffusion

15. Now we’re ready to get AUTOMATIC1111's Stable Diffusion:

If you did not upgrade your kernel and haven’t rebooted, close the terminal you used and open a new one
Now enter:

cd stable-diffusion-webui
python -m venv venv 
source venv/bin/activate

From here there’s a few options of running Stable diffusion for AMD: if you have a newer gpu with large amount of VRAM, try:

python launch.py

If you try to generate images and get a green or black screen, press Ctrl+C in the terminal to terminate and relaunch with these arguments:

python launch.py --precision full --no-half

if you want to reduce vram usage add "--medvram"

python launch.py --precision full --no-half --medvram

pick one and press enter, it should start up Stable Diffusion on 127.0.0.1:7860.

Open up 127.0.0.1:7860 in a browser, on the top left you can choose your models. For text to images use the normal pruned-emaonly file, for editing parts of already created images, use the inpainting model

Each time you want to start Stable Diffusion, you’ll enter these commands (adjusted to what works for you):

cd stable-diffusion-webui
python -m venv venv 
source venv/bin/activate
python launch.py

Stable Diffusion should be running!

Quick Dual Boot tutorial:
- Be extremely careful here and its best practice to keep data backups. Search how to do this on YouTube. Search windows for “Disk Management” program, open it, find a hard drive with at least 100-200gb free space and right click on it in the boxes along the bottom then click Shrink Volume. Shrink it by 100-200gb and process it. You should now have a Free Space partition available on your Harddrive Download the Linux ISO you want, I used Linux Mint Cinnamon. Any Debian based distro like Ubuntu should work. Get a flash drive and download a program called “Rufus” to burn the .iso onto the flashdrive as a bootable drive. Once its finished burning, shut down your pc (don’t restart). Then start it again, access your Bios Boot menu and select the Flash drive. This will start the linux installation disk and from the install menu when it asks where to install it, select the 100-200gb free space partition, press the Plus to create a partition, use the default Ext4 mode and make the mount point “/”. If it asks where to install the bootloader, put it on the same drive youre installing the OS on. Finish thru Install steps

Sources for additional reading:
https://rentry.org/pygmalion-local
https://hub.tcno.co/windows/wsl/desktop-gui/
Big thanks to this thread for the original basis, I had to change a few things to work out the kinks and get it to work for me

$ 0.00