What you’ll be able to do

Run a proper AI model, right there on your own laptop or desktop, with nothing leaving the machine. No account, no per-message bill, no internet needed once it’s set up. You type something in, the model on your hard drive answers you. That’s it.

It’s not going to out-think ChatGPT. But it’s yours, it’s private, and it works on a train with no signal. For a lot of tasks, that’s plenty.

Before you start

First thing: check what you’re working with. Open your system settings and look at RAM, that’s the number that matters most here. 8GB will run small models. 16GB gives you more room. 32GB or more, and you can start looking at the bigger open-weight models properly. Storage matters too, some of these downloads run into several gigabytes each, so make sure you’ve got space free before you start.

Second thing, and this is the bit people skip: understand what you’re actually getting. A local model is not a smaller ChatGPT. It’s a different, generally weaker, model that happens to run entirely on your machine instead of on someone else’s servers. It’ll be slower to respond on modest hardware. It’ll make more mistakes. It won’t have the same breadth of knowledge or reasoning as the frontier tools you’re used to.

So why bother? Three reasons, really. Privacy: nothing you type gets sent anywhere, which matters if you’re working with anything sensitive. Cost: once it’s downloaded, it’s free, no subscription, no per-message meter running. And offline: it works with no connection at all, which is genuinely useful more often than you’d think.

There’s a fourth reason too, less practical but worth saying: running a model locally teaches you what a model actually is. Not a magic box, just weights and a program reading them. Worth seeing up close at least once.

Get set up

The easiest way in is Ollama. It’s free, it works on Mac, Windows and Linux, and you run it from the terminal. If a command line makes you nervous, LM Studio does much the same job with a proper chat window, no typing required beyond your prompts.

I’d start with Ollama, honestly. It’s quick, the commands are simple, and once it’s running you can always add LM Studio on top for a nicer interface later. Here’s the process:

  1. Go to Ollama’s website and download the installer for your operating system.
  2. Install it like you would any other app.
  3. Open a terminal (Terminal on Mac, PowerShell or Command Prompt on Windows).
  4. Pull a model and run it in one go.

That last step looks like this:

ollama run llama3.2

That downloads Meta’s Llama model (a smaller version suited to ordinary hardware) and drops you straight into a chat prompt. First run takes a few minutes depending on your connection, the file’s a few gigabytes. After that it’s instant, because it’s just sitting on your disk.

If you want to try a different model family, the pattern’s the same. Google’s Gemma, Mistral, Alibaba’s Qwen, and OpenAI’s own open-weight model called gpt-oss are all available the same way, just swap the name:

ollama run gemma2

Pick one that fits your RAM. Smaller models, a few billion parameters, run fine on an ordinary laptop. The bigger ones want a serious GPU and a lot more memory, so if your machine’s a few years old, start small and work up.

Prefer a window over a terminal? Download LM Studio instead, it has a model browser built in, you click download, then chat in a window that looks a lot like any other chat app.

Try it yourself

Once you’ve got a model running, throw some real prompts at it. Here are three that test different things.

Writing:

Write a short, friendly email to a client explaining that a project deadline needs to move back by one week. Keep it to three sentences.

Analysis:

Here is a paragraph of text: [paste your text]. Summarise the main argument in two sentences, then list any claims that aren't backed up with evidence.

Practical or coding help:

Write a Python function that takes a list of numbers and returns the average, ignoring any negative numbers in the list.

Run all three. Notice the differences from what you’re used to. Response might be slower, especially on the first go while the model loads into memory. The writing might be flatter, less nuanced. The code might work first time or might need a nudge. That’s normal, and it’s the whole point of this exercise: you’re finding out what your setup can actually do, not what it should theoretically do.

Check the result

Don’t just read the output and nod. Test it properly.

For the email, ask yourself: would you actually send this? Is the tone right, does it sound like something a person wrote? If it’s stiff or repetitive, try again with more detail in the prompt, local models often need more explicit instructions than ChatGPT does.

For the summary, go back to the original text and check the claims it flagged. Did it actually catch anything, or did it just restate the paragraph? This is a good way to see the limits of a smaller model, they can be weaker at picking apart nuance than the big cloud services.

For the code, run it. Actually run it, don’t just eyeball it. Feed it a list with negative numbers in and see if the average comes out right. If it errors, that tells you something useful, feed the error back into the same chat and see if it can fix itself.

If any of these are clearly wrong or nonsensical, that’s not a fault exactly, it’s a reminder: local models, especially the small ones, are genuinely weaker than the frontier services. Some tasks just need bigger tools. Knowing where that line sits is most of what this whole exercise teaches you.

If it doesn’t work

Model won’t download? Check your internet connection first, then check disk space, these files are large and a half-finished download is often just a full hard drive in disguise.

Ollama command not recognised? Restart your terminal after installing, sometimes it needs a fresh window to pick up the new install.

Machine grinding to a halt or the model crashing partway through a response? That’s usually memory. Close other applications, try a smaller model, or check the model’s listed requirements before you pull it. If you’ve only got 8GB of RAM, don’t reach for the biggest model in the list and expect it to behave.

Now, the bit that matters regardless of whether you’re running local or cloud: think about what you’re actually typing in. Local models keep things off the internet, which is genuinely useful, but that doesn’t mean you should get careless. Don’t paste in passwords, banking details, medical records, or anything that identifies another real person without their say-so, even into a model running on your own machine. It’s not really an AI risk at that point, it’s just good sense with any document, on any device.

One more thing worth saying plainly: never download model files from random sites or forum links promising some miracle jailbroken version. Stick to Ollama’s own library or LM Studio’s built-in browser, or official pages from the model makers themselves. A model file is just data, but there’s no good reason to take one from somewhere you don’t trust.

Keep exploring

Once you’ve got a model running comfortably, there’s more worth trying. If you write code regularly, have a look at “How to get AI to write code”, it goes into using local and cloud models for that specifically. If your inbox is a mess, “How to get AI to handle my emails” covers that. And if you’re after ways to take repetitive admin off your plate entirely, “How to get AI to automate repetitive tasks” is the one to read next.

Sources and review notes

Review date: 15 September 2026 — every source above was opened and checked against the text on that date.