KNOWLEDGE BASE & REFERENCE

Local AI FAQ & Hardware Glossary

Clear, plain-English answers to common beginner questions and an essential dictionary of terms like VRAM, memory bandwidth, and quantization.

GLOSSARY TERMSoftware

Open WebUI

Open WebUI is a self-hosted, browser-based interface that gives your local AI models the exact look, feel, and everyday features of ChatGPT—running completely offline on your own hardware or home server.

Open Definition & Link ↗
GLOSSARY TERMhardware

VRAM (Video Random Access Memory)

VRAM is the high-speed, dedicated memory built directly onto a graphics card (GPU). While regular system RAM holds your operating system, browser tabs, and everyday apps, VRAM acts as a private, ultra-fast workspace exclusively for graphics and AI calculations.

Open Definition & Link ↗
GLOSSARY TERMhardware

Unified Memory

In a traditional desktop PC, the processor (CPU) and the graphics card (GPU) live in separate worlds with separate memory pools—system RAM for the CPU and VRAM for the GPU—forcing data to be copied back and forth between them. Unified Memory eliminates that wall by placing a single, shared pool of high-speed memory directly into the main chip package so both the CPU and GPU can read from the exact same space simultaneously.

Open Definition & Link ↗
GLOSSARY TERMhardware

Discrete GPU (dGPU)

In plain English: A standalone graphics card with its own dedicated processor, cooling fans, and private pool of high-speed VRAM—such as an NVIDIA GeForce RTX 3090, RTX 5090, or AMD Radeon RX 9070 XT. Unlike integrated graphics built directly inside a standard laptop or desktop CPU that have to share slow motherboard RAM, a discrete GPU plugs into a motherboard's `PCIe` expansion slot (or sits as a separate chip inside a gaming laptop) and handles heavy parallel math completely on its own.

Open Definition & Link ↗
GLOSSARY TERMhardware

Memory Bandwidth (GB/s)

Memory bandwidth measures how much data your memory can feed to the processor each second, expressed in gigabytes per second (`GB/s`). Picture VRAM capacity as the size of a warehouse, and memory bandwidth as the width of the highway connecting that warehouse to the factory floor—no matter how fast the workers are, they can only assemble products as fast as trucks deliver the parts.

Open Definition & Link ↗
GLOSSARY TERMhardware

HBM3e (High Bandwidth Memory)

`HBM3e` is an ultra-fast type of memory built by stacking thin memory dies vertically on top of one another like floors in a skyscraper, rather than laying them out flat side-by-side on a circuit board. Thousands of microscopic vertical wires called Through-Silicon Vias (`TSVs`) link those stacked layers together and sit right beside the GPU die on a shared silicon interposer, creating an extraordinarily wide data highway while drawing far less power than traditional graphics memory.

Open Definition & Link ↗
GLOSSARY TERMHardware

TDP (Thermal Design Power)

TDP stands for Thermal Design Power, measured in Watts (W). It tells you the maximum amount of heat a processor or graphics card generates when running at full tilt—and how beefy your cooling system needs to be to keep it from overheating.

Open Definition & Link ↗
GLOSSARY TERMHardware

PCIe (Peripheral Component Interconnect Express)

PCIe is the high-speed slot on your motherboard where you plug in a discrete graphics card. Think of it as the data highway connecting your GPU to your CPU and the rest of your computer.

Open Definition & Link ↗
GLOSSARY TERMHardware

SXM5

SXM5 is a specialized server socket used in enterprise AI hardware—such as NVIDIA H100 and H200 rigs—that mounts graphics processors directly onto the server motherboard instead of plugging them into standard PCIe slots.

Open Definition & Link ↗
GLOSSARY TERMConcepts

Parameter Count (7B, 8B, 32B, 70B)

When you see labels like 7B, 8B, 32B, or 70B next to an AI model, the B stands for Billion parameters. Parameters are the internal adjustable dials—called weights—that the model tuned during its training to learn language, logic, coding, and facts.

Open Definition & Link ↗
GLOSSARY TERMConcepts

Quantization (Q4, Q4_K_M, Q8, Unquantized/FP16)

Quantization is the process of shrinking an AI model so it fits into consumer graphics cards without losing much of its intelligence. It works by rounding off the precision of the numbers (weights) inside the model—much like saving a massive raw photo as a high-quality JPEG.

Open Definition & Link ↗
GLOSSARY TERMConcepts

Weights

Weights are the billions of learned numerical values stored inside a model file—such as a `.gguf` or `.safetensors` download—that represent everything the AI knows.

Open Definition & Link ↗
GLOSSARY TERMConcepts

Context Window

The context window is an AI model's short-term working memory. Measured in tokens (roughly ¾ of a word), it defines the maximum amount of text the model can read, hold in mind, and reference at one time during a single conversation.

Open Definition & Link ↗
GLOSSARY TERMConcepts

KV Cache (Key-Value Cache)

The KV Cache (Key-Value Cache) is the extra slice of VRAM your system reserves to remember your ongoing conversation so the AI does not have to re-read the entire chat from scratch after every single word it writes.

Open Definition & Link ↗
GLOSSARY TERMConcepts

Inference vs. Fine-Tuning

Inference and Fine-Tuning are the two main ways you interact with an AI model, and they place completely different demands on your hardware.

Open Definition & Link ↗
GLOSSARY TERMMetrics

Tokens & Tokens per Second (t/s or tok/s)

AI models do not read or write whole words the way humans do; they break text down into chunks called tokens. In English, one token equals roughly ¾ of a word (so 100 tokens is around 75 words). Common short words are single tokens, while long technical terms, punctuation, and code indentation get split into multiple pieces.

Open Definition & Link ↗
GLOSSARY TERMMetrics

£/GB (Cost per Gigabyte of Memory)

£/GB (Cost per Gigabyte of Memory) is the core value metric used across our hardware index. You calculate it by taking the total purchase price of a graphics card or complete system and dividing it by the amount of usable, high-bandwidth memory (VRAM or Apple Unified Memory) you get.

Open Definition & Link ↗
GLOSSARY TERMMetrics

CPU Offloading

CPU Offloading is a clever safety net built into local AI engines like Ollama and LM Studio that lets you run a model even when it is slightly too big to fit inside your graphics card's VRAM.

Open Definition & Link ↗
GLOSSARY TERMSoftware

CUDA

CUDA is NVIDIA's proprietary software platform that lets developers tap directly into the thousands of parallel cores inside GeForce, RTX, and datacenter graphics cards for heavy math instead of just gaming graphics.

Open Definition & Link ↗
GLOSSARY TERMSoftware

Metal & ROCm

Metal and ROCm are the two main alternatives to NVIDIA's CUDA that allow Apple and AMD hardware to accelerate local AI workloads.

Open Definition & Link ↗
GLOSSARY TERMSoftware

Local Inference Engines (Ollama, LM Studio, Jan.ai)

Local inference engines are the free desktop apps and background services that turn raw, unplayable AI model files into a working private assistant on your own computer.

Open Definition & Link ↗
GLOSSARY TERMSoftware

GGUF

GGUF (GPT-Generated Unified Format) is the universal single-file format used by local AI tools like Ollama, LM Studio, and `llama.cpp` to store and run quantized language models.

Open Definition & Link ↗
GLOSSARY TERMSoftware

ComfyUI

ComfyUI is the go-to open-source interface for running local image and video generation models like Flux.1, Stable Diffusion XL (SDXL), and Wan on your own graphics card.

Open Definition & Link ↗
Rent Cloud GPU Power

Not Ready to Buy a Full Rig?

Don't want to spend £3,500+ upfront? Rent the GPU power instead.

Buying a dedicated AI workstation is a serious investment—and if you aren't ready to spend that much right now, you don't have to. Renting cloud GPUs gives you instant access to high-VRAM hardware like RTX 4090s and H100s on demand. Spin up an instance for fine-tuning or running massive 70B+ models, pay strictly for the minutes you use, and avoid the upfront hardware cost, electricity bills, and maintenance.

Vast.ai Global GPU Marketplace

Lowest $/hr
Unbeatable $/hr rates on community-hosted RTX 4090 & 3090 rigs
Deploy Pod →

RunPod Community & Secure Cloud

Community & Secure Cloud
RTX 4090 from $0.34/hr • H100 SXM from $1.99/hr (Billed in USD)
Deploy Pod →

FindBestRig independently benchmarks cloud GPU providers to compare rental economics against physical hardware.