AI Stack Library · Models
Local vs cloud AI models: which jobs go where
Send hard reasoning, long writing and customer-facing work that must be excellent to a cloud model, and keep private data, high-volume small jobs and anything that must work offline on a local model on your own machine. Most businesses end up with both, and the split is decided job by job, not once for the whole stack.
| If you are | Start with |
|---|---|
| A business with no private data and light volume | A dedicated always-on machine, then cloud models for everything and a local model added later |
| A practice whose notes and documents are private | A dedicated always-on machine with enough memory for a local model, and a rule that private jobs stay local |
| A business with thousands of small jobs a day, such as tags, titles and sorting | A dedicated always-on machine running a small local model for the bulk work, with a cloud model for the hard jobs |
| A developer building for clients | A dedicated always-on machine for each client, then a written job-by-job table of local and cloud |
Decide per job, not once
There is no single answer for a whole business. Summarizing a private meeting and writing a sales page are different jobs, and they often belong on different models.
Ask four questions about each job, in this order. The first yes decides.
A list of four questions asked in order. Private data leads to a local model. Must work offline leads to a local model. Hard reasoning or customer-facing work leads to a cloud model. Many small jobs a day lead to a local model. Anything else can go either way.
- Does the job touch private data?Yes Local model
- Must it work with no internet connection?Yes Local model
- Does it need hard reasoning, or will a customer read the result?Yes Cloud model
- Is it a small job you run many times a day?Yes Local model
- None of the aboveEither. Test both and keep the cheaper one that passes
An eight-row table of common jobs, whether each touches private data, its volume, the quality it needs, and its usual home: local, cloud or either.
| Job | Private data | Volume | Quality needed | Usual home |
|---|---|---|---|---|
| Drafting customer replies | Sometimes | Medium | High | Cloud, or local when the thread is private |
| Summarizing meeting notes | Often | Medium | Medium | Local |
| Tags and titles | Rarely | High | Low | Local |
| Questions over private documents | Yes | Medium | Medium | Local |
| Coding | Sometimes | Medium | High | Cloud |
| Embeddings for search | Often | High | Medium | Local |
| Long reports | Sometimes | Low | High | Cloud |
| Image and video generation | Rarely | Low | High | Cloud |
Judgements, not measurements. Your own test on your own work decides.
What cloud models do best
Hard reasoning, long documents, careful use of tools, and writing that goes out under your name. The strongest models run in the cloud. You pay per use through an API, or per seat through a plan, and prices change often enough that you should read them on the provider's page on the day you choose.
What local models do best
Private data, work that has to carry on offline, bulk small jobs such as tags, titles and sorting, and embeddings for search over your own files. You pay once for the hardware and then for electricity, so there is no bill per call.
The quality gap, honestly
For a narrow job, a local model that passes your test is good enough for that job, whatever the benchmarks say about it in general.
Build the test from your own work: a small set of real examples, each with the answer you would accept, run on both models. A chat that feels clever is not a test.
Memory decides which local models fit
A local model has to fit in the machine's memory, with room left for everything else the stack runs. On a Mac with Apple silicon, the main processor and the graphics processor share one pool of memory, so the memory size is the number to check.
On 2026-10-22, Apple's spec pages listed the Mac mini with up to 64GB of unified memory and the Mac Studio with up to 512GB. Quantization, which stores a model with less precision, lets a larger model fit in less memory, at some cost in quality.
Private data: catch it first, keep it local
Decide what counts as private before any job runs: client names, health details, money, anything under a confidentiality agreement. Then set a rule that sends any job holding it to a local model, whatever model was asked for, and write down every place data goes when it does leave the machine.
Check where your local runner actually runs each model. Ollama, for example, offers models in the cloud as well as on your own machine, so an app on your desk is not proof that the model is.
A harness is what makes a local model useful
A local model on its own is a chat box. The harness around it gives it tools, memory and rules, and a local server lets every agent on the machine call it the way it would call a cloud model.
LM Studio, for example, can run a local server that accepts requests in the same format as OpenAI's and Anthropic's APIs, so tools built for those can point at it instead. Ollama runs on macOS, Linux and Windows. Under most runners sits an engine such as llama.cpp, which treats Apple silicon as a first-class target, or Apple's own MLX framework.
Routing between the two
Once each job has a home, routing does the rest. Every call passes the private-data check first, then goes to the cheapest model that passed your test for that job.
A job enters a private-data check. Private work goes straight to a local model; the rest goes to a router that picks the cheapest model that passed the test for that job, and the routing log records the choice and the cost.
- A job comes in
- Private-data checkprivate work goes straight to a local model
- Routerpicks the cheapest model that passed your test for this job
- Local or cloud modelthe routing log records which, and the cost
What goes wrong
- A local model so large it leaves no memory for the rest of the stack, and everything slows down.
- Private notes sent to a cloud model just once, to see if it does better.
- Local models judged on how a chat feels, instead of on your own jobs.
- A runner set to a cloud model when you thought the work was staying local.
Questions
Is a local model as good as ChatGPT?
Not at everything. For a narrow job, such as tagging or summarizing your own notes, a local model can pass the same test a cloud model passes. Test it on your work and decide job by job.
Do you need a graphics card for local models?
On a Mac with Apple silicon there is no separate card: the graphics processor shares the main memory, so memory size decides which models fit. On a PC, the memory on the graphics card usually decides it.
Terms on this page
Sources
- Ollama, checked 2026-10-22
- Ollama: Download, checked 2026-10-22
- LM Studio: Developer docs, checked 2026-10-22
- llama.cpp, checked 2026-10-22
- MLX, checked 2026-10-22
- Apple: Mac mini tech specs, checked 2026-10-22
- Apple: Mac Studio tech specs, checked 2026-10-22