This week’s Mac tips already covered Ollama and LM Studio. A third local path sits closer to Apple silicon itself: mlx-lm, the Python package on top of Apple’s MLX framework.
MLX is built for unified memory on M-series Macs. The mlx-lm README says install with pip or conda, then generate or chat from Terminal. Intel Macs are out of scope here. Use Ollama or another GGUF runner if the chip is Intel.
Install mlx-lm in a Python venv
Apple silicon Mac, Python 3.9 or newer. Keep packages out of the system Python. Make a venv, activate it, then install:
python3 -m venv ~/mlx
source ~/mlx/bin/activate
pip install -U mlx-lm
That pulls MLX as a dependency. Conda users can run conda install -c conda-forge mlx-lm instead. First download of a model needs network. After weights are cached, chat can stay offline.
Generate and chat from Terminal
Official quick start for one-shot text:
mlx_lm.generate --prompt "How tall is Mt Everest?"
Default model in the current README is mlx-community/Llama-3.2-3B-Instruct-4bit. Override with --model. For a multi-turn REPL:
mlx_lm.chat
Context stays for that Terminal session. Run mlx_lm.generate -h or mlx_lm.chat --help for sampling flags. Hugging Face’s mlx-community org hosts many ready MLX weights. One documented example on the hub page is mlx_lm.generate --model mlx-community/Qwen3-4B-Instruct-2507-4bit --prompt "hello".
Pick a 4-bit instruct build first. Bigger weights need more unified memory. If Activity Monitor shows memory pressure, unload and try a smaller file. Do not invent token rates from a blog. Measure on your own Mac.
Local server on port 8080
mlx-lm also ships an HTTP server meant to look like OpenAI’s chat API. Docs start it with:
mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit
Default bind is localhost port 8080. Clients POST to /v1/chat/completions with a messages array. The same docs warn the server is not for production and only has basic security checks. Keep it on the Mac unless you mean to expose it.
That is the practical split versus cloud ChatGPT or Claude desktop apps. Local weights stay on disk. Cloud sign-in still leaves the machine. Apple’s own assistant stack is a different pipe, closer to the Siri AI English beta and today’s Camera Siri / Visual Intelligence notes than a self-hosted MLX weight.
Large models and wired memory
The mlx-lm README has a Large Models note. On macOS 15 or newer, MLX can wire model memory to speed big loads. If a warning says the model needs more than available RAM allows, shrink the model or raise the wired limit with the documented sysctl iogpu.wired_limit_mb value. N should sit above the model size in megabytes and below total system memory. That knob is optional. Skip it on a first smoke test.
After a big OS train, re-check the venv still imports. Today’s firmware brief on iOS 27.2 beta 24B5084k is a reminder toolchains keep moving. mlx-lm itself tracks the MLX release, not the iPhone build number.
Install the venv, run mlx_lm.generate once with the default 3B 4-bit model, then try chat. Add the local server only after that works with Wi‑Fi off.
Photo: Apple
Source: mlx-lm README and SERVER.md (ml-explore/mlx-lm), Hugging Face mlx-community quick start, Apple Newsroom (MacBook Air with M5, MacBook Pro with M5 Pro and M5 Max, macOS 27 software platforms)
