How to Run LLMs Fully Offline: A Practical Guide

Running an LLM offline is one of the strongest reasons to choose a local model over a cloud API. Once the model is downloaded, you can disconnect from the internet and still chat, write code, and reason through documents. This guide walks you through what you need, how to verify that nothing is leaking, and which workflows work best offline.

What you need

• A machine with a modern CPU, or preferably a GPU with at least 8 GB of VRAM.
• A local LLM runtime like llama.cpp, or an app that bundles it.
• A GGUF model that fits inside your available memory.
• Optional: a firewall rule or sandbox to block outbound network traffic.

Pick the right model

For most users, a 7B or 8B model at Q5_K_M is the sweet spot. It fits on an 8 GB VRAM GPU, runs fast, and is capable enough for coding, writing, and reasoning. If you have 24 GB of VRAM, you can run a 70B model at Q4_K_M entirely on the GPU.

Download while online

The only online step is the initial model download. Use a tool with a Hugging Face browser to find a GGUF version of the model you want, download it, and cache it locally. After that, the model file lives on your disk.

Verify zero outbound calls

To prove the app is truly offline, put your machine in airplane mode or block the app with a firewall. Then try chatting, running the coding agent, and loading a model. Everything should continue to work.

Offline workflows that work well

• Writing and refactoring code with a local coding agent.
• Summarizing and querying local documents.
• Brainstorming and drafting long-form content.
• Learning and experimenting with open models without API costs.

Limitations to know

Offline models cannot browse the web or access live data. They also cannot download new models or updates without a connection. Plan ahead: keep a few quantized variants cached for different tasks.