Skip to content

Local AI on a mini PC or GPU: which models fit where?

You can also run an AI model on your own computer, so your text does not go to someone else's server. What you can run depends mainly on memory: the model has to fit in it. Below, for each mini PC and GPU, you can see which models fit, what they are for, what they cost in electricity and where local AI falls short.

How it works

GPU
The model fits in VRAMIt then sits entirely on the card, which according to Ollama works best. If it does not fit, Ollama puts part of it in ordinary system RAM, and responses may then be slower.
Mini PC
The model fits in RAMWithout a dedicated GPU, the model uses ordinary system RAM. The system itself needs some of it too: Windows 11 requires at least 4 GB.
Context
More text needs more memoryThe more text the model holds at once (the context window), the more memory it needs. That is why, with less than 24 GB of VRAM, Ollama defaults to a short window of about 4,000 tokens (pieces of words).

What today's computers can do

By type of device: which models fit entirely in memory, and what they are for according to Ollama.

GPU, VRAM

GPU with 8 GB

For example, the RTX 4060 and RTX 5060.

Good for: text, images, tools

Fits
  • Gemma 3 4B
  • Llama 3.1 8B

GPU with 12 or 16 GB

12 GB: RTX 4070 and RTX 5070. 16 GB: RTX 5080, plus the RTX 4060 Ti and RTX 5060 Ti in their 16 GB versions.

Good for: text, images, tools, reasoning

Fits
  • Gemma 3 4B
  • Llama 3.1 8B
  • Gemma 3 12B
  • Qwen3 14B

GPU with 24 or 32 GB

24 GB: RTX 4090. 32 GB: RTX 5090.

Good for: text, images, tools, reasoning, coding

Fits
  • Gemma 3 4B
  • Llama 3.1 8B
  • Gemma 3 12B
  • Qwen3 14B
  • gpt-oss 20b
  • Gemma 3 27B
  • Qwen3-Coder 30B
Tight
  • Qwen3 32B

Mini PC, RAM

Mini PC with 16 GB

For example, the Umbrel Home.

Good for: text, images, tools, reasoning

Fits
  • Gemma 3 4B
  • Llama 3.1 8B
  • Gemma 3 12B
  • Qwen3 14B

Mini PC with 32 GB

For example, the GEEKOM A9 Max in its 32 GB configuration.

Good for: text, images, tools, reasoning, coding

Fits
  • Gemma 3 4B
  • Llama 3.1 8B
  • Gemma 3 12B
  • Qwen3 14B
  • gpt-oss 20b
  • Gemma 3 27B
  • Qwen3-Coder 30B
  • Qwen3 32B

Mini PC with 64 GB

For example, the GEEKOM A6 and A8: two memory slots for up to 64 GB in total.

Good for: text, images, tools, reasoning, coding

Fits
  • Gemma 3 4B
  • Llama 3.1 8B
  • Gemma 3 12B
  • Qwen3 14B
  • gpt-oss 20b
  • Gemma 3 27B
  • Qwen3-Coder 30B
  • Qwen3 32B
  • Llama 3.3 70B

Mini PC with 128 GB of unified memory

For example, the GEEKOM A9 Mega with the AMD Ryzen AI Max+ 395. The processor and graphics chip share the memory.

Good for: text, images, tools, reasoning, coding

Fits
  • Gemma 3 4B
  • Llama 3.1 8B
  • Gemma 3 12B
  • Qwen3 14B
  • gpt-oss 20b
  • Gemma 3 27B
  • Qwen3-Coder 30B
  • Qwen3 32B
  • Llama 3.3 70B
  • gpt-oss 120b

A 70B model such as Llama 3.3 does not fit on any of the dedicated GPUs in this overview; for that you need a mini PC with 64 GB or more. Searching your own documents works on everything above: you run a small search model such as EmbeddingGemma (622 MB) alongside a chat model.

What the labels mean

  • Images: the model describes pictures and answers questions about them.
  • Tools: the model can call other programs and use their output. AI assistants that carry out tasks on their own (agents) are built on this.
  • Reasoning: the model first thinks step by step, separately from its answer.
  • Coding: built for code and programming tasks.

Categories based on Ollama's library and documentation.

All models: what fits where

The size is what you download in Ollama, the program for running open models on your own computer, for the default version of each model. Whether a model fits is our own calculation from that size: we leave a fifth of the memory free for the context and everything else, and on a mini PC we first set aside 4 GB for the system. It is a guide, not a measurement.

8B means 8 billion parameters: the numbers the model calculates with. Larger models are meant for heavier tasks, but need more memory.

Mini PC, RAM
ModelDownload16 GB32 GB64 GB128 GB
Gemma 3 4B3.4 GBfitsfitsfitsfits
Llama 3.1 8B4.9 GBfitsfitsfitsfits
Gemma 3 12B8.2 GBfitsfitsfitsfits
Qwen3 14B9.3 GBfitsfitsfitsfits
gpt-oss 20b14 GBdoes not fit entirelyfitsfitsfits
Gemma 3 27B17 GBdoes not fit entirelyfitsfitsfits
Qwen3-Coder 30B19 GBdoes not fit entirelyfitsfitsfits
Qwen3 32B20 GBdoes not fit entirelyfitsfitsfits
Llama 3.3 70B43 GBdoes not fit entirelydoes not fit entirelyfitsfits
gpt-oss 120b65 GBdoes not fit entirelydoes not fit entirelydoes not fit entirelyfits
GPU, VRAM
ModelDownload8 GB12 GB16 GB24 GB32 GB
Gemma 3 4B3.4 GBfitsfitsfitsfitsfits
Llama 3.1 8B4.9 GBfitsfitsfitsfitsfits
Gemma 3 12B8.2 GBdoes not fit entirelyfitsfitsfitsfits
Qwen3 14B9.3 GBdoes not fit entirelyfitsfitsfitsfits
gpt-oss 20b14 GBdoes not fit entirelydoes not fit entirelytight: only with a short context windowfitsfits
Gemma 3 27B17 GBdoes not fit entirelydoes not fit entirelydoes not fit entirelyfitsfits
Qwen3-Coder 30B19 GBdoes not fit entirelydoes not fit entirelydoes not fit entirelyfitsfits
Qwen3 32B20 GBdoes not fit entirelydoes not fit entirelydoes not fit entirelytight: only with a short context windowfits
Llama 3.3 70B43 GBdoes not fit entirelydoes not fit entirelydoes not fit entirelydoes not fit entirelydoes not fit entirely
gpt-oss 120b65 GBdoes not fit entirelydoes not fit entirelydoes not fit entirelydoes not fit entirelydoes not fit entirely
  • fits
  • tight: only with a short context window
  • does not fit entirely

If a model does not fit entirely in VRAM, Ollama puts the rest in system RAM. It still works, but responses may be slower. On a mini PC, it means there is not enough memory.

Snapshot from 23 October 2026; models and sizes change. OpenAI says 16 GB of memory is enough for gpt-oss 20b. On a 16 GB GPU that is tight, and on a mini PC with 16 GB too little is left once the system has taken its share. For coding help and agents, Ollama recommends a context window of at least 64,000 tokens; in that case, go one size up.

VRAM and electricity per card

The VRAM is fixed on the card; you cannot upgrade it. The RTX 4060 Ti and RTX 5060 Ti come with either 8 or 16 GB, so check which version you are buying.

The electricity figures assume two hours a day at full power, at the average household price of €0.29 per kWh (Eurostat).

CardVRAMPower under full loadElectricity per year
RTX 40608 GB115 W€24.3584 kWh
RTX 5060 Ti8 / 16 GB180 W€38.11131 kWh
RTX 407012 GB200 W€42.34146 kWh
RTX 507012 GB250 W€52.93183 kWh
RTX 508016 GB360 W€76.21263 kWh
RTX 409024 GB450 W€95.26329 kWh
RTX 509032 GB575 W€121.73420 kWh

According to NVIDIA, for NVIDIA's own version of each card. Cards from other brands may differ.

GPU or mini PC?

GPU

According to Ollama, a model works best when it sits entirely on the GPU. You do need a desktop PC with a hefty power supply to go with it: for a PC with an RTX 5090, NVIDIA specifies 1,000 W. The VRAM is fixed, from 8 to 32 GB on the cards above.

Mini PC

Small, and you can choose more memory than any GPU has: up to 64 GB in slots, or 128 GB of unified memory. That means it can also hold models that do not fit on any dedicated card. They do run from system RAM, though, and responses may then be slower. Your Bitcoin node can run on the same mini PC; in that case, count the memory for both.

Idle consumption and mini PCs

If you leave the PC on day and night, its idle consumption counts too. According to NVIDIA, an RTX 4090 draws 19 W at idle; around the clock that comes to 166 kWh a year, not counting the rest of the PC.

For a mini PC, the manufacturer usually states only the processor's design power (TDP): 45 to 120 W for the GEEKOM A9 Mega. That is not a measurement at the wall socket. Measure it with a plug-in power meter and work out the cost.

Work it out yourself

Where local AI falls short

Not enough for everything

Even GEEKOM says of its A9 Max that for larger models and for questions that call for a lot of reasoning or knowledge, you are better off using a cloud service alongside it.

No fixed speed

How quickly responses come depends on the card, the model, the length of the text and the software. That is why we do not quote speeds.

Short context window

With less than 24 GB of VRAM, Ollama defaults to about 4,000 tokens. For web search, agents and coding help, Ollama recommends at least 64,000 tokens, and that takes more memory.

You choose the memory when you buy

You cannot add VRAM, and on some mini PCs, such as the GEEKOM A9 Mega, the RAM is also soldered to the board. It is better to buy a little more than you need now.

Electricity becomes heat

According to NVIDIA, an RTX 5090 draws up to 575 W under full load. That electricity ends up as heat in your room.

Frequently asked questions

How much VRAM do I need for local AI?

For small models of around 8 billion parameters (about 5 GB), 8 GB is enough. With 12 or 16 GB, models of up to about 14 billion parameters fit, including ones that handle images, and gpt-oss 20b is a tight fit on 16 GB. For models of around 30 billion parameters, such as Qwen3-Coder for coding, you need 24 or 32 GB. A 70-billion-parameter model does not fit on any of the dedicated GPUs in our overview.

Can I run local AI on a mini PC without a dedicated GPU?

Yes. Ollama then uses system RAM, but responses may be slower than on a GPU the model fits on. With 32 GB, models of up to about 20 GB fit; with 64 GB, a 70B model fits as well; and with 128 GB of unified memory, so does gpt-oss 120b. Whether Ollama uses your mini PC's graphics chip depends on the chip and the operating system; Ollama keeps a list of supported hardware.

Is a local model as good as a cloud AI service?

Not for everything. Small models are more likely to fall short on difficult questions, and even GEEKOM recommends using a cloud service as well for larger models. The advantage of running locally: your text stays on your own device, and each question costs you only the electricity.

Further reading

Calculate electricity costsA Bitcoin node on a mini PCMini PCs & GPUs

Sources