RX 580 Vulkan AI Benchmarks (llama.cpp + SD.cpp)

Short answer (what this article helps you do)

An AMD Radeon RX 580 8GB can run modern GGUF LLMs and even Stable Diffusion locally if you use Vulkan-backed builds (llama.cpp Vulkan backend and stable-diffusion.cpp Vulkan) and pick the right quantization. You’ll get the most repeatable results on Linux with a working Vulkan stack.

The big rule for RX 580 multi-GPU rigs: scaling works best by running one model per GPU as independent workers (one request per GPU). Trying to shard one model across cards (tensor parallelism) usually fails due to PCIe bandwidth and latency limits.

Who this is for

• You want to run an LLM on RX 580 without ROCm.
• You have an old Polaris card and want a clear, repeatable setup.
• You want a benchmark plan with exact commands so your numbers are real, not guesses.
• You’re thinking about a small “cluster” made from used mining GPUs.

Quickstart checklist (60–90 minutes)

1. Install the Linux graphics stack (Mesa + RADV is the easiest place to start).
2. Confirm Vulkan works with vulkaninfo.
3. Build llama.cpp with Vulkan enabled.
4. Download a small GGUF model (start with 3B–8B, Q4).
5. Run llama-bench (or llama-cli) with a fixed prompt.
6. Build stable-diffusion.cpp with Vulkan enabled.
7. Run one Stable Diffusion image test at 512x512 and record time.

Checkpoint question: can your machine run vulkaninfo without errors and list your RX 580 as a device?

Test rig notes (what changes your results)

RX 580 results can swing a lot. The biggest knobs are driver/backend, model and quantization, context length (KV cache VRAM), GPU offload level, and CPU/RAM speed if you spill to system memory.

• Driver/backend: Mesa RADV vs AMDVLK.
• Model + quant: Q4 is smaller and often faster; Q8 is larger and often slower.
• Context length: longer context needs more KV cache, which uses more VRAM.
• GPU offload: how many layers you put on GPU changes speed and memory.
• CPU + RAM: CPU fallback can become the bottleneck.

Reality check: Windows, WSL, ROCm, and what actually works

RX 580 (Polaris) is in a tough spot for ROCm. Many users hit a wall with ROCm support, especially on Windows. Plan around Vulkan (or sometimes OpenCL) instead.

• Windows: ROCm isn’t a reliable option. Vulkan builds may work for some tools, but Linux is usually more repeatable.
• WSL: don’t count on ROCm on RX 580. Treat it as a “maybe later” option.
• Linux: typically the strongest path for RX 580 + Vulkan because drivers are mature.
• Alternative: some users use OpenCL (clBLAST) for LLMs, but this article focuses on Vulkan.

Step 1: Install Vulkan drivers (RADV vs AMDVLK)

On Linux, you’ll usually see two Vulkan driver paths: Mesa RADV (open-source, common default) and AMDVLK (AMD’s Vulkan driver). If you can, test both and record results using the same models and settings.

Debian/Ubuntu: install tools to verify Vulkan

sudo apt update
sudo apt install -y mesa-vulkan-drivers vulkan-tools
vulkaninfo | head -n 50

If vulkaninfo fails, fix this first. Don’t move on until Vulkan is stable.

How to capture versions (for reproducible benchmarks)

Record versions in a notes file next to your benchmark results. This makes your results reproducible and comparable later.

# GPU + driver
lspci | grep -i vga
uname -a
vulkaninfo --summary

# Mesa packages (Debian/Ubuntu)
apt-cache policy mesa-vulkan-drivers

# If using AMDVLK, record that package too
apt-cache policy amdvlk

Step 2: Build llama.cpp with Vulkan

This is the key piece for running GGUF models on RX 580 via Vulkan. You want GGML_VULKAN enabled.

Build (CMake)

sudo apt install -y git cmake build-essential
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
# Record the exact commit for your benchmark notes
git rev-parse HEAD
cmake -B build -DGGML_VULKAN=ON
cmake --build build -j

Quick smoke test (run a small model)

Start with a GGUF model that fits easily in 8GB (example: 3B or 7B in Q4). Then run a short prompt to confirm GPU offload works.

./build/bin/llama-cli -m /path/to/model.gguf -p "Write a short list of 5 GPU tips." -n 128 -ngl 99

-ngl controls GPU layers. -ngl 99 means “try to put as much as possible on the GPU.” If you hit out-of-memory, lower it.

Benchmark tool: llama-bench

llama-bench is the cleanest way to measure tokens per second. Keep settings constant so results are comparable.

./build/bin/llama-bench -m /path/to/model.gguf -ngl 99 -c 2048 -n 256

Step 3: Build stable-diffusion.cpp with Vulkan

For Stable Diffusion on RX 580 without ROCm, a Vulkan build of stable-diffusion.cpp is a common approach. Build steps can change between releases, so record the commit and the Vulkan flag you used.

Build (example flow)

sudo apt install -y git cmake build-essential
git clone https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp
git rev-parse HEAD
cmake -B build -DSD_VULKAN=ON
cmake --build build -j

If your project uses a different flag than SD_VULKAN, check its README and record the exact flag you used.

One simple Stable Diffusion benchmark run

Start with 512x512. It’s the easiest baseline on 8GB cards and helps confirm end-to-end GPU usage.

./build/bin/sd -m /path/to/model.safetensors -p "a small cabin in the woods" -W 512 -H 512 --steps 20

Record total time and whether it used the GPU for the full run.

Model fit guide: what can you fit in RX 580 8GB?

VRAM limits drive what runs well. If you overfill VRAM you may crash (OOM) or spill to CPU (slow). The biggest drivers are quantization size, context length (KV cache), and GPU offload level.

LLM model fit table (rules of thumb)

Model size	Good starting quant	Typical context goal	Notes for RX 580 8GB
3B	Q4_K_M	2048+	Safest start; easiest to keep more on GPU.
7B	Q4_K_M (or Q5_K_M if it fits)	1024–2048	Often usable when tuned; watch KV cache VRAM use.
8B	Q4_K_M	1024–2048	Often solid when offload is high; depends on driver and build.
13B	Q4_K_M	512–1024	May fit with careful settings; speed drops quickly if you spill to CPU.

What makes you run out of VRAM?

• Bigger quant (Q8 uses more memory than Q4).
• Longer context (KV cache grows with context length).
• Too many GPU layers for the model size.

If you’re stuck, reduce (1) context length, then (2) GPU layers, then (3) choose a smaller quant/model.

Benchmark methodology (so your results mean something)

To get clean data, keep conditions steady between runs. Close other GPU-heavy apps, use consistent power settings, run each test 3 times, and take the median.

• Record: model, quant, context length, GPU layers, driver, and commit hashes.
• Keep prompt and token count fixed across runs.

LLM benchmark settings to lock down

• Prompt: fixed text prompt (same every run).
• n_predict: same token count (example: 256).
• Context (-c): test 512 and 2048 as two baselines.
• GPU layers (-ngl): test 0, a mid value, and “max” (99) to see scaling.

Stable Diffusion benchmark settings to lock down

• Resolution: 512x512 first, then 768x768 if it fits.
• Steps: fixed (example: 20 and 30).
• Sampler: pick one and don’t change it for the test set.

20-test benchmark matrix

This suite is designed to answer one question: what model and settings give the best speed on RX 580 Vulkan? It defines 20 LLM tests; Stable Diffusion tests are optional extras.

How it hits 20 tests: 5 LLM tests x 4 variants (context/GPU layers) = 20.

LLM tests (20)

Test #	Model	Quant	Context	GPU layers	Command	Result (t/s)
1	3B	Q4_K_M	512	0	llama-bench -c 512 -ngl 0
2	3B	Q4_K_M	512	99	llama-bench -c 512 -ngl 99
3	3B	Q4_K_M	2048	0	llama-bench -c 2048 -ngl 0
4	3B	Q4_K_M	2048	99	llama-bench -c 2048 -ngl 99
5	3B	Q5_K_M	2048	99	llama-bench -c 2048 -ngl 99
6	7B	Q4_K_M	512	0	llama-bench -c 512 -ngl 0
7	7B	Q4_K_M	512	99	llama-bench -c 512 -ngl 99
8	7B	Q4_K_M	2048	0	llama-bench -c 2048 -ngl 0
9	7B	Q4_K_M	2048	99	llama-bench -c 2048 -ngl 99
10	7B	Q5_K_M	1024	99	llama-bench -c 1024 -ngl 99
11	8B	Q4_K_M	512	0	llama-bench -c 512 -ngl 0
12	8B	Q4_K_M	512	99	llama-bench -c 512 -ngl 99
13	8B	Q4_K_M	2048	0	llama-bench -c 2048 -ngl 0
14	8B	Q4_K_M	2048	99	llama-bench -c 2048 -ngl 99
15	8B	Q8_0	1024	99	llama-bench -c 1024 -ngl 99
16	13B	Q4_K_M	512	0	llama-bench -c 512 -ngl 0
17	13B	Q4_K_M	512	99	llama-bench -c 512 -ngl 99
18	13B	Q4_K_M	1024	0	llama-bench -c 1024 -ngl 0
19	13B	Q4_K_M	1024	99	llama-bench -c 1024 -ngl 99
20	13B	Q5_K_M	512	99	llama-bench -c 512 -ngl 99

Expected performance ranges (sanity check)

These are broad ranges because drivers, clocks, and CPU matter. Use them as a smoke alarm, not a promise.

• 3B Q4: often comfortable and can feel snappy.
• 7B/8B Q4: often usable when GPU offload is high; slow runs may be spilling to CPU or using a weaker backend.
• 13B Q4: can work with careful settings; expect slower output, especially at longer contexts.

If you see very low tokens/sec, check your Vulkan driver, confirm -ngl is applied, and ensure you’re not accidentally running CPU-only.

Multi-GPU RX 580: what works (and what usually fails)

It’s tempting to think “I have 5 cards, so I can run a model 5x bigger.” Most of the time, that doesn’t work well on RX 580 rigs. You usually get better results from parallel throughput rather than sharding one model.

Why tensor parallelism usually fails here

• Many mining boards and low-end CPUs have weak PCIe layouts.
• Inter-device communication is slow and adds latency.
• LLMs need frequent cross-device synchronization when sharded, and latency kills speed.

Worker-queue blueprint (one request per GPU)

• Queue: Redis or RabbitMQ holds jobs.
• Workers: one worker process per GPU, pinned to a device.
• Rule: one model per worker, one job at a time.
• Result: throughput scales with GPU count, even if single-request latency does not.

Example layout:

• GPU0: LLM 7B Q4 worker
• GPU1: LLM 7B Q4 worker
• GPU2: Stable Diffusion 512x512 worker
• GPU3: LLM 13B Q4 worker

Tiered escalation (fast first, slow only when needed)

This pattern is common in batch pipelines. It reduces cost by only running expensive settings when you need them.

1. Try tier 1: Q4, low context, fast.
2. If output is bad, try tier 2: Q8 or bigger context.
3. If still bad, try tier 3: higher quality settings on a stronger worker.

Run it as a service (llama-server + systemd)

If you want a local endpoint (for apps, bots, or a home lab), run a server process and keep it alive with systemd. Review the unit before enabling it, and update paths, user, and model location.

Example systemd unit

sudo nano /etc/systemd/system/llama-vulkan.service

# Paste this (edit paths):
[Unit]
Description=llama.cpp Vulkan server
After=network.target

[Service]
Type=simple
User=YOUR_USER
WorkingDirectory=/home/YOUR_USER/llama.cpp
ExecStart=/home/YOUR_USER/llama.cpp/build/bin/llama-server -m /models/7b-q4.gguf -ngl 99 -c 2048
Restart=always
RestartSec=2

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now llama-vulkan.service
sudo systemctl status llama-vulkan.service

Next step: add a simple health check and log rotation if you plan to run it 24/7.

Troubleshooting matrix (common RX 580 Vulkan problems)

Problem	What it usually means	Fix
Vulkan device not found	Driver stack isn’t working	Install vulkan-tools, run vulkaninfo, and confirm RADV/AMDVLK is installed.
OOM / crash when loading model	VRAM is full (model + KV cache + layers)	Lower -c, then lower -ngl, then use a smaller quant (Q4).
Very slow tokens/sec	CPU mode or heavy spill to system RAM	Confirm -ngl is set, check logs, reduce context, and avoid huge quants.
Speed changes a lot between runs	Thermals/clocks or background load	Monitor temps, fix fan curve, close background apps, and use median-of-3 runs.
Stable Diffusion runs but is painfully slow	Resolution/steps are too high for 8GB	Start at 512x512 and 20 steps; scale only after you record a baseline.

FAQ

Is there a best choice between AMDVLK vs RADV for RX 580?

There isn’t one winner for everyone. Start with Mesa RADV because it’s common and easy. Then test AMDVLK and record both results.

Can I combine VRAM across multiple RX 580 cards?

Not in a simple way for these tools. Treat each GPU as its own worker.

What is the best GGUF quantization for RX 580 8GB: Q4 vs Q5 vs Q8?

Start with Q4_K_M. It’s usually the best balance for 8GB. Move to Q5 or Q8 only if you still fit the model and context without heavy CPU spill.

Can RX 580 run Stable Diffusion without ROCm?

Yes. A Vulkan build of stable-diffusion.cpp is a common approach, and 512x512 is the usual starting point for 8GB cards.

Can I do a multi-GPU RX 580 setup for one big model?

Most setups do better with many small workers than one sharded model. Use a queue and run one request per GPU.

When you should stop tuning and just upgrade

RX 580 is a good choice if you already own it or can get it cheap. If you’re buying hardware mainly for AI, a card with more VRAM headroom can reduce “fit” problems and constant tuning.

Use RX 580 when you want low cost, learning, privacy, and “good enough” local runs. Upgrade when you need bigger models, longer context, or more stable performance with less tweaking.

What to do next

1. Run the 20-test matrix and save results with driver and commit versions.
2. Test RADV vs AMDVLK on the same model and settings.
3. If you have multiple RX 580s, build a worker queue and scale throughput horizontally.

If you want help debugging performance, share your OS, Vulkan driver (RADV or AMDVLK), and your llama.cpp commit hash.