Blog

vLLM Now Supports Running GGUF on AMD Radeon GPU

This guide shows the impact of Liger-Kernels Training Kernels on AMD MI300X. The build has been verified for ROCm 6.2.

Read Full Article

vLLM Now Supports Running GGUF on AMD Radeon GPU

All Blog Posts

By EmbeddedLLM Team • 2 mins

Dec 1, 2024

vLLM Now Supports Running GGUF on AMD Radeon GPU

This guide shows the impact of Liger-Kernels Training Kernels on AMD MI300X. The build has been verified for ROCm 6.2.

Liger Kernels Leap the CUDA Moat: A Case Study with Liger, LinkedIn's SOTA Training Kernels on AMD GPU

By EmbeddedLLM Team • 8 mins

Nov 5, 2024

Liger Kernels Leap the CUDA Moat: A Case Study with Liger, LinkedIn's SOTA Training Kernels on AMD GPU

This guide shows the impact of Liger-Kernels Training Kernels on AMD MI300X. The build has been verified for ROCm 6.2.

See the Power of Llama 3.2 Vision on AMD MI300X

By EmbeddedLLM Team • 5 mins

Oct 28, 2024

See the Power of Llama 3.2 Vision on AMD MI300X

This blog post shows you how to run Meta's powerful Llama 3.2-90B-Vision-Instruct model on an AMD MI300X GPU using vLLM. We provide the Docker commands, code snippets, and a video demo to help you get started with image-based prompts and experience impressive performance

By EmbeddedLLM Team • 8 mins

Oct 11, 2024

How to Build vLLM on MI300X from Source

This guide walks you through the process of building vLLM from source on AMD MI300X. The build has been verified for ROCm 6.2.

By EmbeddedLLM Team • 7 mins

Oct 27, 2023

High throughput LLM inference with vLLM and AMD: Achieving LLM inference parity with Nvidia

EmbeddedLLM has ported vLLM to ROCm 5.6, and we are excited to report that LLM inference has achieved parity with Nvidia A100 using AMD MI210.

vLLM Now Supports Running GGUF on AMD Radeon GPU

All Blog Posts

vLLM Now Supports Running GGUF on AMD Radeon GPU

Liger Kernels Leap the CUDA Moat: A Case Study with Liger, LinkedIn's SOTA Training Kernels on AMD GPU

See the Power of Llama 3.2 Vision on AMD MI300X

How to Build vLLM on MI300X from Source

High throughput LLM inference with vLLM and AMD: Achieving LLM inference parity with Nvidia

Legal

Email Us