One of the hardest parts of running local LLMs is knowing what your hardware can actually handle. A 70B model sounds impressive, but if your GPU only has 12 GB of VRAM, it will not load without extreme quantization. That is where the hardware wizard comes in.
What the wizard scans
On first launch, OpenLLM Studio builds a complete profile of your machine. It looks at the specs that actually determine which models you can run.
- GPU model and available VRAM
- System RAM and memory pressure
- CPU cores and threads
- Operating system and supported backends
- Available storage space
How it matches models to hardware
The wizard does not just check if a model fits. It scores models by quality, speed, and stability on your exact hardware profile. It then picks the best quantization level to balance performance and capability.
- 1Identify models that fit within your VRAM or system RAM budget.
- 2Filter for backends your hardware supports: CUDA, Vulkan, Metal, or CPU.
- 3Rank by expected tokens per second and quality.
- 4Recommend a quantization that maximizes quality without running out of memory.
Example: on an RTX 4090 with 24 GB VRAM, the wizard might recommend Llama 3.3 70B at Q5_K_M for the best quality that still fits entirely on the GPU.
Why this matters
Without a wizard, you are stuck reading model cards, calculating quantization sizes, and hoping your guess is right. With the wizard, you pick from a curated list of models that are guaranteed to run well on your machine. It turns a 30-minute research task into a 30-second decision.
Always in your control
The recommendation is a starting point, not a rule. You can override the model, change the quantization, or load a custom GGUF from Hugging Face. The wizard just removes the friction from the first step.
Supported backends
- CUDA for NVIDIA GPUs on Windows and Linux.
- Vulkan for AMD and Intel GPUs on Windows and Linux.
- Metal for Apple Silicon on macOS.
- CPU fallback for any machine that lacks a compatible GPU.