# How to run AI locally

Updated: 2026-10-15 | Cluster: Coding and dev | Canonical: https://theailoadout.com/best-ai-setup/run-ai-locally/

The easiest way to run AI locally is Ollama: install it, then run one command such as ollama run gemma4:e4b. Gemma 4 E4B (about 7 GB) is the best starting model for a 16 GB laptop. With 24 to 32 GB, step up to Gemma 4 26B or Qwen 3.5 27B. Prefer a desktop app? LM Studio does the same with a model browser and a built-in chat.

| Slot | Pick |
|---|---|
| Tool | [Ollama](https://ollama.com/download) |
| Model | [Gemma 4 E4B](https://theailoadout.com/models/gemma-4-local/) |
| Skill | [Agent Project Manager](https://theailoadout.com/packs/agent-pm/) |
| Persona | [The Local Assistant](https://theailoadout.com/best-ai-setup/run-ai-locally/#copy-this-setup) |

## Why each pick
- Tool: Ollama installs on Mac, Windows and Linux, pulls a model with one command and serves a local API on port 11434 that most coding agents can use. Its library lists download sizes and context windows for every tag.
- Model: Gemma 4 is Apache 2.0, takes image input on every size and ships in sizes from E2B to 31B, so you can move up without changing tools. E4B is a 6.6 GB download at Q4 with a 128K context window, which fits a 16 GB laptop.
- Skill, Agent Project Manager: Routes easy tasks to your local model and only sends hard ones to a paid cloud model, so local AI saves real money.
- Persona: Small local models do better with short, explicit instructions.

## Comparison
| Option | Best for | Model(s) | Price | Free plan |
|---|---|---|---|---|
| Gemma 4 (E4B to 31B) | Best all-rounder at every size | Google, Apache 2.0 | Free, 6.6 GB (E4B Q4) | Yes |
| Qwen 3.5 (0.8B to 122B) | Coding and many languages, widest size range | Alibaba, Apache 2.0 | Free, 6.6 GB (9B Q4) | Yes |
| Qwen 3.8 27B | Newest Qwen for agentic work on 32 GB | Alibaba, Apache 2.0 | Free, 18 GB (Q4) | Yes |
| gpt-oss 20B | Reasoning on a 16 GB GPU or Mac | OpenAI, Apache 2.0 | Free, 14 GB | Yes |
| Muse Glimmer 30B | Always-on local agents with tools | Meta, Apache 2.0 | Free, 18 GB (Q4) | Yes |
| Devstral Small 2 24B | Local coding agent, 384K context | Mistral, Apache 2.0 | Free, 15 GB (Q4) | Yes |
| Mistral Small 3.2 24B | General assistant with vision | Mistral, Apache 2.0 | Free, 15 GB (Q4) | Yes |
| Ministral 3 (3B to 14B) | Low-memory laptops | Mistral, Apache 2.0 | Free, 3.0 GB (3B Q4) | Yes |
| DeepSeek-R1 8B | Step-by-step reasoning and math | DeepSeek, MIT | Free, 5.2 GB (Q4) | Yes |
| Llama 3.3 70B | Proven general model for 48 GB and up | Meta, Llama 3.3 Community License | Free, 43 GB (Q4) | Yes |

## Persona prompt
```
You are a concise local assistant. Answer in plain language. If a task needs facts you may not know or data newer than your training, say so instead of guessing. Keep answers under 200 words unless asked for more. No em dashes.
```

## What changed
- 2026-10-15: Page launched with initial picks. Sources are listed at the bottom of the page.

## FAQ
### Is running AI locally free?
Yes. Ollama, LM Studio and the models listed here cost nothing to download and use. You pay only for your hardware and electricity.

### What is the best local model right now?
For most people, Gemma 4: E4B on a 16 GB laptop and 26B A4B on a 32 GB machine, as of Oct 15, 2026. Qwen 3.5 and Qwen 3.8 are the main alternatives, especially for coding.

### How much RAM do I need?
Roughly the model download size plus 2 to 4 GB. A 7 GB model wants about 10 GB free; on a Mac that is shared memory, on a PC it should be GPU VRAM for good speed.

### Ollama or LM Studio?
Ollama if you live in the terminal or want an API for agents. LM Studio if you want a desktop app with a model browser. Both serve an OpenAI compatible API, so tools work with either.

### Are local models as good as ChatGPT or Claude?
Not for the hardest tasks. The best cloud models still lead on complex coding and reasoning. Local models are strong for private data, offline work, summaries and routine edits.

## Sources
- [Ollama library: Gemma 4](https://ollama.com/library/gemma4)
- [Ollama library: Qwen 3.5](https://ollama.com/library/qwen3.5)
- [Ollama library: gpt-oss](https://ollama.com/library/gpt-oss)
- [LM Studio: Gemma 4](https://lmstudio.ai/models/gemma-4)
- [LM Studio docs: OpenAI compatibility endpoints](https://lmstudio.ai/docs/developer/openai-compat)
- [Ollama docs: API](https://docs.ollama.com/api/chat)