Local LLMs on macOS: Ollama, LM Studio, MLX, llama.cpp

Viacheslav Dubrov

2025-05-10 · Updated 2026-08-10 · 6 min · Model Engineering · Developer Tooling

Engineers running local models on an Apple Silicon Mac need to size the complete workload, not just choose a model that downloads. Apple Silicon can run everyday prompts without a remote API because the CPU and GPU share one unified memory pool, but model weights compete there with the KV cache, runtime workspace, and macOS.

Once the intended model, context, and memory headroom fit, choose the control surface that matches the workflow: Ollama for a managed local service, LM Studio for desktop inspection, llama.cpp for direct GGUF execution, or MLX-LM for Apple-native Python. That keeps model-quality testing separate from tool choice and gives you a fair, memory-aware benchmark for the stack you will run.

TL;DR. Use Ollama for a managed local service, LM Studio for desktop model exploration and its local APIs, llama.cpp for direct control over GGUF execution, and MLX-LM for Apple-native Python experimentation. None is universally fastest. Benchmark the exact model, quantization, context, and workload with memory headroom.

For the compact decision table, see Local LLM Tools on macOS in 2026.

Start with a memory envelope

Raw quantized weight size is only the first term:

peak memory ≈ model weights
            + KV cache
            + runtime workspace
            + multimodal components
            + application and OS memory

Context length, cache dtype, parallel requests, and model architecture change the result. A nominal four-bit 7B or 8B model can still be uncomfortable on an 8 GB Mac because the operating system cannot give the runtime all installed memory.

Use Activity Monitor or the runtime’s own metrics while testing. Apple defines memory pressure using free memory, swap rate, wired memory, and cached files. Leave enough headroom to avoid sustained swap; a model that loads but pushes the system into memory pressure is not a good interactive fit.

Also separate local inference from offline operation. Prompts can stay on the machine while the application still reaches the network for model downloads, updates, or optional features. Ollama documents local execution, model downloads, and optional cloud features separately in its FAQ. Treat offline operation as a workflow requirement to verify per application: download artifacts first, disconnect the network, and test the complete workflow.

The four tools solve different workflow problems

This is a responsibility table, not a speed ranking. Several tools may use related kernels or formats, and performance changes with model support and release.

Ollama: managed local service

Ollama manages model downloads, templates, process lifecycle, and a localhost API. It is useful when application code should target a stable local service rather than own inference flags.

MODEL=llama3.2
ollama pull "$MODEL"
ollama run "$MODEL" "Explain unified memory."

curl http://localhost:11434/api/chat \
    -H 'Content-Type: application/json' \
    -d '{
      "model": "'"$MODEL"'",
      "messages": [{"role": "user", "content": "Explain unified memory."}],
      "stream": false
    }'

Inspect the model manifest and context configuration rather than assuming a short library name identifies an immutable checkpoint. Pin or record the exact artifact for evaluations.

Ollama trades some low-level visibility for lifecycle convenience. Drop to llama.cpp or another runtime when you need to control a GGUF file, chat template, cache setting, or new backend feature directly.

LM Studio: desktop exploration and local APIs

LM Studio is useful when model discovery, load configuration, chat inspection, and side-by-side human evaluation belong in one workflow. It now exposes native Python and TypeScript SDKs, OpenAI-compatible endpoints, structured output, tool use, and a headless daemon, so it is no longer only a desktop GUI.

The current Python SDK connects to a model you have already downloaded:

import lmstudio as lms

MODEL_KEY = "ibm/granite-4-micro"

with lms.Client() as client:
    model = client.llm.model(MODEL_KEY)
    response = model.respond("Write one sentence about local inference.")
    print(response)

Start the local API from the Developer tab or with:

lms server start

LM Studio can serve on localhost or a local network and supports API tokens in its API authentication settings. Keep it on loopback unless remote access is deliberate. If you expose it on a LAN, require its documented API-token option, apply a host firewall rule, and review tool and integration access.

llama.cpp: direct GGUF execution

llama.cpp is the reference path when the artifact is GGUF and you want to see the runtime boundary directly. It supports Apple Metal along with CPU and other hardware backends.

brew install llama.cpp

# Download through the Hugging Face integration and select a quantization.
MODEL_REPO=ggml-org/gemma-3-1b-it-GGUF
QUANT=Q4_K_M
llama-cli -hf "$MODEL_REPO:$QUANT"

# Or start an OpenAI-compatible local server.
llama-server -hf "$MODEL_REPO:$QUANT"

The repository’s current -hf path can download a matching multimodal projector when available. Model support, templates, and CLI flags change quickly, so pin a known build and keep the launch command with the evaluation record.

Choose llama.cpp when direct control is the goal, not because “lower-level” automatically means faster. A managed tool may choose good defaults; direct flags can also make performance worse.

MLX-LM: Apple-native Python work

MLX-LM builds on Apple’s MLX array framework. It supports generation, chat, conversion, quantization, and parameter-efficient fine-tuning for compatible models.

MODEL=mlx-community/Llama-3.2-3B-Instruct-4bit
uv add mlx-lm
uv run mlx_lm.generate \
    --model "$MODEL" \
    --prompt "Explain Metal acceleration in one paragraph."

Python exposes the model and tokenizer directly:

from mlx_lm import generate, load

MODEL = "mlx-community/Llama-3.2-3B-Instruct-4bit"

model, tokenizer = load(MODEL)
messages = [{"role": "user", "content": "Give one local-LLM benchmark rule."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=80))

MLX-LM’s HTTP server is documented as a development server with basic security checks, not a production service. Use it for local experimentation or place a reviewed application boundary in front of it.

A fair benchmark takes less time than a bad download

Test the same checkpoint family and comparable quantization where formats allow. Use a small prompt set containing:

• one short interactive prompt
• one long prompt near the intended context
• structured output or tool calls if the application needs them
• a representative generation length
• one repeated request to distinguish cold load from warm inference

Record:

Do not compare one tool’s 4-bit model with another tool’s full-precision model and attribute the difference to the runtime.

Security and privacy checklist

1. Bind APIs to loopback unless network access is required.
2. If LAN access is required, use a host firewall and either the tool’s documented authentication or an authenticated reverse proxy in front of the service. API-token support is tool-specific: LM Studio documents API tokens, while Ollama’s documented exposure path uses host binding and proxying.
3. Treat model files as third-party artifacts; record source, revision, license, and hash.
4. Avoid executing unreviewed custom model code.
5. Confirm whether optional document, tool, update, or analytics features make network calls.
6. Do not assume local generation makes retrieved documents, logs, or tool side effects safe.

Key takeaways

1. Choose the workflow boundary first: managed service, desktop and headless development, direct GGUF control, or Apple-native Python.
2. Ollama and LM Studio both expose local APIs. Compare lifecycle control, SDKs, structured output, tools, and runtime visibility.
3. Model weights are only part of the memory budget. Include KV cache, workspace, multimodal components, the application, and macOS headroom.
4. Local inference is not automatically offline or private. Verify network calls, binding, authentication, artifacts, logs, and tool side effects.
5. Benchmark the same task, checkpoint family, context, and comparable quantization before attributing a result to the runtime.

References

• Ollama documentation and source
• LM Studio developer documentation and Python SDK
• llama.cpp documentation and source
• MLX-LM documentation and source
• Apple MLX documentation