A server for the ultimate homelab, multi-3090 GPU LXC and Docker host has turned into a really cool rig. I'll show you the complete step-by-step setup from Proxmox to Nvidia drivers to the Nvidia Container Toolkit. We then set up an LXC to house a Docker that handles GPU passthrough at bare metal speeds. Then I install Ollama and Openwebui for us and test the speeds of Llama 3.1 compared to bare metal. With this tutorial you are completely set up!

Written article with all steps – https://digitalspaceport.com/proxmox-multi-gpu-passthru-for-lxc-and-docker-ai-homelab-server/

Video on how to build an AI server – https://youtu.be/JN4EhaM7vyw

AI Server Build
GPU Rack Frame https://geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 motherboard https://geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) https://geni.us/GPU3090
256GB DDR4 2400 RAM https://geni.us/256GB_DDR4_RAM
PCIe4 Riser (x4) https://geni.us/PCIe4_Riser_Cable
EPYC 7702 https://geni.us/EPYC_7702p
Corsair H170i ELITE CAPELLIX https://geni.us/iCUE_H170i_Capellix
SP3/TR4/5 Retention Kit https://geni.us/ELITE_RetentionKit
ARCTIC MX4 thermal paste https://geni.us/Arctic_ThermalPaste
CORSAIR HX1500i power supply https://geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) https://geni.us/SFF8654_to_SFF8654
HDD Rack Screws for Fans https://geni.us/HDD_RackScrews

Chapter
0:00 All in one Homelab Ai Server
1:34 Install Proxmox
4:28 Configure host
5:49 Proxmox GPU Passthru Options
8:42 NVIDIA drivers for HOST
11:27 Install Docker LXC
13:26 Proxmox LXC GPU Docker setup
20:58 Install Dockge and OpenwebUI
25:20 Setting up Ollama and testing Lama 3.1
34:13 Summary

Subscribe to us to receive more content like this!

Join this channel https://www.youtube.com/@DigitalSpaceport/join

Please share this video to spread the word and leave a comment below with your thoughts or questions. Thanks for watching!

Digital Spaceport Website
https://digitalspaceport.com

Shop (channel members receive 3% or 5% discount)
Find great deals on hardware and merchandising at https://Shop.DigitalSpaceport.com.

*****
As an Amazon Associate I earn from qualifying purchases.

When you click on links to various merchants on this website and make a purchase, this may result in this website earning a commission. Affiliate programs and partnerships include, but are not limited to, the eBay Partner Network.

Other affiliate partners on this site include Newegg and Best Buy. I receive a commission when you click links and make a purchase from the retailer.
*****

Please take the opportunity to connect with your friends and family and share this video with them if you find it useful.