You do not need a subscription to use AI. With a free tool called Ollama, you can run powerful open source models right on your Windows computer. This guide walks you through every step, even if you have never touched a command line before.
Right now, most people access AI through services like ChatGPT or Claude. You type a message into a website, it goes to a server somewhere, an AI processes it, and the response comes back. That works well, but it has a cost: a subscription, and the fact that your messages are traveling to a company's computers.
Ollama is different. It is a free, open source tool that lets you download AI models and run them entirely on your own computer. No subscription. No internet connection required once the model is downloaded. No one else reading your conversations.
ChatGPT is like calling a very smart person on the phone. Ollama is like hiring that person to live in your house. The person in your house might not be quite as sharp, but they are always available, completely private, and free after day one.
Ollama is the tool that sits between you and the AI model. Think of it as the engine that loads a model onto your computer and lets you talk to it through a simple interface. You choose which model to use, and Ollama handles all the technical heavy lifting.
This connects directly to things we have already covered on this site. If you read the Hermes Agent article, you may remember that Hermes lets you plug in a local model as its brain instead of paying for a cloud service. Ollama is exactly how you do that. And if you want to understand more about how these models actually work under the hood, the How to Build an LLM article covers the full picture.
When you use ChatGPT, the AI does not run on your computer. Your words travel over the internet to a massive data center owned by OpenAI, get processed by their computers, and the response travels back to you. This is called a cloud-based or remote model. The AI lives somewhere else.
When you use Ollama, the model file downloads to your hard drive and runs entirely on your own hardware. This is called a local model. The AI lives on your machine.
| What you care about | Cloud (ChatGPT, Claude) | Local (Ollama) |
|---|---|---|
| Cost | Monthly subscription or pay per use | Free after hardware you already own |
| Privacy | Your prompts go to a company's servers | Nothing leaves your computer |
| Quality | Best available, constantly updated | Good to very good, but behind frontier models |
| Speed | Fast, runs on powerful data center hardware | Depends entirely on your machine |
| Internet needed | Yes, always | Only to download the model initially |
| Customization | Limited to what the company allows | Full control, connect to anything |
Local models are not as capable as the frontier models you get from paid services. GPT-4.5 and Claude Sonnet are trained on vastly more compute and data than any model you will run locally. The gap is real and worth knowing about. What you gain is privacy, zero cost, and the ability to connect these models to your own tools and workflows. For many tasks, a good local model is more than enough.
This is the part most guides skip over or bury. Your hardware determines what you can run and how fast it will feel. Here is what you need to know before you download anything.
The key component is your GPU, which stands for Graphics Processing Unit. This is the chip inside your computer originally designed for video games and graphics. It turns out AI models run dramatically faster on GPUs than on regular processors (called CPUs) because of how GPUs handle parallel math operations. If you want to understand why, the LLM article covers the systems side of this.
Press Windows key + R, type dxdiag, and press Enter. Click the Display tab. The card listed under "Name" is your GPU. If it says NVIDIA GeForce or AMD Radeon followed by a model number, you have a dedicated GPU. If it says Intel UHD or AMD Radeon Graphics with no model number, you are running on integrated graphics, which is much slower for AI.
Here is a plain-English breakdown of what to expect at each hardware tier:
| Tier | Your hardware | What you can run | What to expect |
|---|---|---|---|
| CPU Only | No dedicated GPU, or older GPU with less than 4GB VRAM | Phi-3 Mini (2.3GB), Gemma 2B (1.7GB) | Slow but works. Expect 5-15 words per second. Fine for experimenting, frustrating for real use. |
| Mid-Range GPU | NVIDIA RTX 3060 / RTX 4060, or AMD RX 6700 XT and up (8GB+ VRAM) | Llama 3.2 8B (5GB), Mistral 7B (4.1GB) | Responsive and genuinely useful. 30-60 words per second. This is the sweet spot for most people. |
| Enthusiast GPU | NVIDIA RTX 3090 / RTX 4080 / RTX 4090 (16-24GB VRAM) | Llama 3.3 70B (40GB), large coding models | Fast and capable. 40-80 words per second on large models. Approaches frontier model quality. |
You also need at least 8GB of system RAM, though 16GB is better. And leave a good chunk of free disk space before you start: model files range from about 1.7GB for the smallest models up to 40GB or more for the largest ones. Download only what your hardware can actually run.
Ollama requires Windows 10 version 22H2 or newer. Since Windows 10 reached end of life in October 2025, upgrading to Windows 11 is the right move regardless. Microsoft provides a free upgrade path here if you are still on Windows 10. If your PC cannot run Windows 11, it is also unlikely to run AI models well.
Installing Ollama is genuinely easy. No technical knowledge required. Here is the complete process:
By default, Ollama saves downloaded models to your main drive in your user folder. If your main drive is short on space, you can tell Ollama to store models on a different drive. Search for "environment variables" in Windows Settings, click "Edit environment variables for your account," and create a new variable called OLLAMA_MODELS with the path to wherever you want models stored. For example: D:\OllamaModels.
If you have an NVIDIA GPU, make sure your drivers are up to date. Go to nvidia.com/drivers and check for the latest version. Ollama requires driver version 531 or newer for GPU support. AMD Radeon users should check amd.com/en/support for the latest Radeon drivers. If your drivers are current, Ollama will automatically use your GPU without any extra configuration.
To use Ollama, you need to open something called the command line, also called the terminal or command prompt. If you have never used one before, this section is for you.
Your computer has two ways to take instructions. The first is the graphical interface: windows, buttons, icons, and menus. You click things and things happen. The second is the command line: a plain text window where you type instructions directly and the computer executes them. It looks old-fashioned but it is extremely powerful. Most developer tools, including Ollama, work through the command line. Learning a few basic commands is one of the highest-value things you can do as someone getting into tech.
Press the Windows key on your keyboard, type cmd, and press Enter. A black window will open with white text. This is the Command Prompt. Alternatively, you can search for PowerShell or Windows Terminal, both work just as well.
With the terminal open, type the following command and press Enter. This tells Ollama to download a model called Llama 3.2 (a good general-purpose starting point):
C:\Users\you> ollama pull llama3.2
You will see a progress bar as the model downloads. For Llama 3.2, that is about 2GB, so depending on your internet speed, it may take a few minutes. When it finishes, you are ready to chat.
C:\Users\you> ollama run llama3.2
The terminal will show a prompt that says >>>. Type anything and press Enter. The model will respond right there in the terminal. To end the session, type /bye and press Enter.
C:\Users\you> ollama list
C:\Users\you> ollama rm llama3.2
C:\Users\you> ollama help
The terminal is how you install and manage models. But for actually chatting with them, several free graphical apps wrap around Ollama so you get a proper chat interface. Open WebUI is the most popular one. It runs in your browser and looks similar to ChatGPT. We will cover that setup in a future article. *** How to set up Open WebUI with Ollama (coming soon)
Once Ollama is installed, you can download any model from the Ollama model library. There are dozens. Here is a curated starting point organized by what your hardware can handle and what you want to do.
There are two broad types of local models worth knowing about. Chat models are designed for conversation and general questions, similar to how you use ChatGPT. Coding models are fine-tuned specifically for writing and debugging code. They understand programming languages and can help you build things. We will cover using coding models with an IDE (a code editor) in a future article. *** Using local coding models in VS Code (coming soon)
Start small. These models are specifically designed to run on limited hardware. Responses will be slow, but they work.
| Model | Size | Pull command | Good for |
|---|---|---|---|
| Phi-3 Mini by Microsoft |
2.3 GB | ollama pull phi3 |
General chat, simple questions. Surprisingly capable for its size. |
| Gemma 2B by Google |
1.7 GB | ollama pull gemma:2b |
Quick answers, basic summarization. Fastest option for weak hardware. |
This is where local AI starts feeling genuinely useful. These models hold a conversation well and handle most everyday tasks.
| Model | Size | Pull command | Good for |
|---|---|---|---|
| Llama 3.2 8B by Meta |
5.0 GB | ollama pull llama3.2 |
General conversation, summarizing, writing assistance, answering questions. |
| Mistral 7B by Mistral AI (France) |
4.1 GB | ollama pull mistral |
Strong reasoning and instruction following. Great all-around model. |
| Phi-4 Mini by Microsoft |
2.5 GB | ollama pull phi4-mini |
Efficient and fast on mid-range hardware. Good at coding and math. |
If you have the hardware, these are the strongest open source models available right now. They are not all from US companies, but as we explain in the next section, that matters less than you might think for locally running models.
| Model | Size | Pull command | Best for |
|---|---|---|---|
| Llama 3.3 70B by Meta |
40 GB | ollama pull llama3.3 |
The best open source general chat model available. Approaches GPT-4 quality on many tasks. Needs 24GB+ VRAM. |
| Qwen 2.5 32B by Alibaba |
20 GB | ollama pull qwen2.5:32b |
Exceptional at reasoning and instruction following. Strong multilingual support. |
| DeepSeek Coder V2 by DeepSeek |
8.9 GB | ollama pull deepseek-coder-v2 |
Among the best open source coding models available. Handles multiple languages well. |
| Qwen 2.5 Coder 7B by Alibaba |
4.7 GB | ollama pull qwen2.5-coder:7b |
Top-tier coding model for mid-range hardware. Beats many larger models on code tasks. |
It is a fair question. Qwen comes from Alibaba, a Chinese company. DeepSeek is also Chinese. Some people feel uncomfortable with that, and you should make whatever decision feels right for you.
Here is the technical reality, stated plainly:
When you run a model locally through Ollama, you are running a file of mathematical weights on your own computer. Once downloaded, the model does not connect to any server. It does not send your prompts anywhere. It cannot phone home. The model file is just data, the same way an MP3 file cannot call anyone.
The legitimate concern with foreign-developed AI tools is with cloud-hosted versions, where your prompts travel to servers you do not control. That risk is real for the cloud versions. For a locally running open source model, it does not apply in the same way.
That said: open source models are still "black boxes" in the sense that their training data and full alignment details are not always fully audited. If you are working with sensitive business information, classified material, or anything you would not want anyone to potentially see, use a vetted cloud service with a proper data processing agreement rather than any locally running model.
For personal use, creative projects, learning, and building automations, the risk of running any of these models locally is minimal and well understood by the security community. The consensus among researchers is that the local-model risk is low. Use your own judgment for anything beyond that.
Having a free, private AI running on your own machine is more useful than it might first appear. Here are three concrete starting points.
Everything you type stays on your computer. That changes what you are willing to ask. Medical questions. Financial situations. Personal journals you want help thinking through. Sensitive work documents you want to summarize without sending to a third party. A local model is the only kind of AI where none of that leaves your machine.
Once you have Ollama running, you can connect it to VS Code (the free code editor from Microsoft) and get code suggestions and explanations without paying for GitHub Copilot. We will walk through that setup in an upcoming article. The coding models listed above, especially Qwen 2.5 Coder and DeepSeek Coder, are genuinely good at this. *** Using local coding models in VS Code (coming soon)
This is where things get interesting for anyone following this site's arc. If you have read the Hermes Agent article, you know that Hermes lets you plug in any model as its brain. Ollama is exactly the local option it supports. That means you can run a persistent AI agent on your own machine, with its own memory and skills, without paying a single cent per message. Pair that with n8n for workflow automation and you have a genuinely powerful private AI stack. *** Connecting Ollama to Hermes Agent (coming soon)
Local models are improving fast. Six months ago, the gap between local and cloud models was larger than it is today. The trajectory is toward parity on most everyday tasks. You are not just learning a tool. You are getting ahead of a shift that is still in progress. The people who understand how to use these things when they are slightly rough will be very well positioned when they become excellent.
You have everything you need to get started right now. Download Ollama, open a terminal, pull a model, and have a conversation. The water is warm.