I'm learning and sharing my Home Server Ai quest in real time. This video is a step-by-step guide focused on installing a complete self-hosted Ollama on a WebUI powerhouse. I'll also show you how to install Docker and Nvidia Container Toolkit as well as run some tests on the new 8b and 70b Meta Llama 3.1. These models are awesome! This Homelab Mega Ai server was built to help me understand and deploy the most modern self-hosted LLM Ai tools.

We'll start by installing Ubuntu and Docker with Nvidia Container Toolkit. Then we'll set up WebUI and Ollama. Next, I'll show you how to download your first models and load them into the client. Finally, I'll show you a bit about the speed of the llama3.1 70b model. This tutorial is based on our Quad 3090 GPU rig that we built in the last video.

Ollama with open WebUI and LLAMA 3.1 70b in Docker on Ubuntu 22.04

Written part and instructions for GPU rack modification
https://digitalspaceport.com/ollama-gpu-3090-home-server-quad-gpu-ai-training-rig/

GPU Rack Frame https://geni.us/GPU_Rack_Frame
Gigabyte MZ32-AR0 motherboard https://geni.us/mz32-ar0_motherboard
RTX 3090 24GB GPU (x4) https://geni.us/GPU3090
512GB DDR4 2400 RAM KIT https://geni.us/512GB_DDR4_RAM
PCIe4 Riser (x4) https://geni.us/PCIe4_Riser_Cable
AMD EPYC 7702p https://geni.us/EPYC_7702p
iCUE H170i ELITE CAPELLIX https://geni.us/iCUE_H170i_Capellix
ELITE Retention Kit https://geni.us/ELITE_RetentionKit
ARCTIC MX4 thermal paste https://geni.us/Arctic_ThermalPaste
CORSAIR HX1500i power supply https://geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4) https://geni.us/SFF8654_to_SFF8654
HDD Rack Screws for Fans https://geni.us/HDD_RackScrews
Subscribe to us to receive more content like this!

Join this channel https://www.youtube.com/@digitalspaceport/join

Chapter
0:00 Ollama Server Setup Guide
0:30 Quad GPU AI Rig Updates
1:40 Setting up Ubuntu 22.04 Desktop for Nvidia
10:05 Install Docker Nvidia Toolkit
11:33 Open WebUI and Ollama setup
14:24 How to install LLAMA 3.1 70b
18:52 LAMA 3.1 405b
20:59 LAMA 3.1 8b
22:48 LLAMA 3.1 Statistics
24:32 The Ultimate Ollama Setup Guide for Beginners

Please share this video to spread the word and leave a comment below with your thoughts or questions. Thanks for watching!

Digital Spaceport Website
https://digitalspaceport.com

Shop (channel members receive 3% or 5% discount)
Find great deals on hardware and merchandising at https://Shop.DigitalSpaceport.com.

*****
As an Amazon Associate I earn from qualifying purchases.

When you click on links to various merchants on this website and make a purchase, this may result in this website earning a commission. Affiliate programs and partnerships include, but are not limited to, the eBay Partner Network.

Other affiliate partners on this site include Newegg and Best Buy. I receive a commission when you click links and make a purchase from the retailer.
*****

Please take the opportunity to connect with your friends and family and share this video with them if you find it useful.