I Finally Let Local AI Run Part of My Digital Life — Here’s What Happened
For years, I’ve been hesitant to fully embrace artificial intelligence. It’s not that I don’t see the appeal — I do. It’s that using AI has traditionally meant handing over my most personal data to cloud services operated by distant companies with opaque policies. My emails, my calendar, my documents, my browsing habits — all of it would have to flow through servers controlled by someone else. That has always felt like too high a price to pay for convenience.
But recently, something has changed. It’s now increasingly possible to run powerful AI models directly on your own computer. No cloud, no subscription, no third-party overlord peering into your digital life. Just a model sitting on your hard drive, waiting for instructions. That shift has made me curious enough to finally start experimenting with local AI on a regular basis.
I should be clear: I’m not an AI expert. I’m coming at this as someone who has been skeptical, cautious, and frankly a little overwhelmed by the pace of change in this space. But I’ve decided to dive in and figure out whether this technology is genuinely useful enough to justify spending serious money on a computer with a massive amount of RAM. Is local AI worth the investment? That’s the question I’m trying to answer — and I’ll be documenting my journey along the way.
A big part of Apple’s current pitch for its newest Mac desktops is how well they handle local AI workloads. There’s also an entire new lineup of Windows machines powered by Nvidia’s RTX Spark chips, some with up to 128GB of RAM, designed specifically for agentic AI. Local AI and AI agents are having a moment right now, to put it mildly.
The idea of having a little digital gofer living entirely inside the box on my desk — someone who answers only to me, not to OpenAI, Google, Microsoft, or Anthropic — is genuinely appealing. It hits differently than trusting a cloud chatbot with my personal information or subscribing to another AI service. Don’t get me wrong: I’m still not going to let AI write my articles, edit my prose, or generate ugly fliers for events. But with privacy safeguards in place, maybe I can finally find useful busywork to trust it with.
As part of my ongoing testing of the M5 Ultra Mac Studio — a machine with 256GB of unified memory — I’ve installed some local large language models and started tinkering. I’ve also been experimenting on other Mac and Windows systems. Here’s how things are going so far.
I started by installing Hermes Agent, an open-source, self-hosted AI agent desktop app that works on macOS, Windows, and Linux. It’s completely free to use if you stick with local LLMs. Since the M5 Ultra Mac Studio I’m testing has 256GB of unified memory, I can run just about any model I want. That’s both exciting and overwhelming.
There are more models available than I can count, and many of them have specialized strengths and use cases. Generally speaking, models with more billions of parameters are capable of doing more than smaller models with only a few billion. And because I’m running everything locally, I don’t have to worry about paying per token. So I figured I’d start big.
Using Hermes’ straightforward onboarding interface and its built-in model picker, I chose Qwen 3.8 Flash Next — a 125-billion-parameter model that takes up around 105GB of storage. It’s a serious model, and it gave me a sense of what local AI can really do when you have the hardware to support it.
I also plan to spend time with much smaller Qwen models on other devices. I’ll be testing them on an M6 Mac Mini, an M5 MacBook Air, an Asus TUF Gaming A14 with an AMD Strix Halo processor, and eventually some RTX Spark machines. I’m curious to see how the experience differs across hardware tiers, and whether you really need a maxed-out workstation to get value out of local AI.
Getting up and running with Hermes and Qwen was surprisingly quick. The setup process was smooth, and I was even able to make the agent controllable from my phone by connecting it to a Telegram bot. That was a nice touch. But then came the moment I always dread when using AI: the blank text box.
“What do I do with this thing?” I asked myself.
It’s one thing to have a powerful tool at your disposal. It’s another thing entirely to figure out what to actually use it for. So I started small. I followed some helpful YouTube advice and had Hermes create a daily morning briefing for me. It was a simple cron job — a scheduled task that runs automatically — designed to make sure everything was working properly.
The briefing scans my email and calendar, tells me anything that might need my immediate attention, and gives me a short weather report. It’s not exactly life-changing, and you definitely don’t need a $12,000 computer to build something like it. But it’s a foundation I can build on over time.
The briefing actually kept failing to generate at first. I couldn’t figure out why. Eventually, I realized that macOS couldn’t be asleep when the job was supposed to fire at 7:30 in the morning. That was the problem. Once I solved that, it started working like clockwork. Now I need to think of more information for it to pull in so the results become more genuinely useful.
A more practical task came when I decided to reorganize my Steam library. I have over 400 games on Steam, and I’ve been meaning to sort them into categories so I can find something to play when I’m in the mood for a specific genre. Steam doesn’t categorize your games for you. You have to do it manually, one game at a time, which is incredibly tedious.
This felt like the kind of thing an agentic AI assistant could handle. So I asked Hermes, “Can you reorganize my Steam library?” It was able to see most of my games by looking at the already installed Steam client. It quickly came up with some organization options for me. I decided to have it organize my games by genre while preserving some of my own specialized categories — favorites, co-op titles, party games, and games I like to play with my wife.
Of course, giving an AI agent the ability to take actions on your machine requires granting it some permissions. In this case, I had to register a Steam Web API key and give it to Hermes. Once it had that access, it took only a few minutes to churn through all of my games and sort them into categories. The API key was easy to revoke right afterward, since I didn’t need it anymore.
The whole point was for Hermes to do the heavy lifting once, and then I could file new games into the appropriate categories as I buy them in the future. It did a fine job overall. My Steam library is now much easier to browse, and that alone felt like a genuinely useful win.
I also used Hermes for some simple data analysis. I had it look at financial records and crunch some numbers for me. I also had it create a spec comparison spreadsheet for a laptop I was considering. These are small tasks, but they’re both things I cannot or will not trust with any cloud service. I don’t want to upload sensitive financial data to the cloud, and in the case of the laptop, the specs were under embargo. Keeping those tasks on my own computer made all the difference. It was the reason I was willing to use the tool at all.
The bigger automation project I’m currently working on is a script to run a series of laptop benchmark tests. Right now, I run all of our benchmarks manually. That means tending to a laptop while it runs a dozen or more tests — and then doing it all over again three times to take an average. It’s incredibly time-consuming. I’d love to automate at least part of that process.
Even just walking Hermes through all of the required procedures and getting it to generate vibe-coded Python scripts has been an endeavor. It’s still very much a work in progress. But I’m hoping to make some positive headway soon. If I can get that automation working, it would save me hours of repetitive work.
I should mention that Apple recently demoed a cluster of four Mac Studios wired up for local AI at a briefing I attended. That setup was nearly $50,000 worth of compute. It was impressive, but it also underscored how expensive this kind of power can be when you scale it up. Most people won’t need anything close to that. But it’s a reminder of where this technology is heading.
It’s still early days for me with Hermes, and I’m hoping this is just the tip of the iceberg in terms of useful things I can get it to do. But it’s also becoming clear that even the most powerful local models don’t magically turn into a flawless assistant that can do anything. Things still break. My daily briefing has already failed multiple times, for various reasons. Local AI is not a magic wand.
I view Hermes and local AI as just a tool — a computer program running on my computer. I don’t want a digital friend. I’m not going to say “thank you” or “please” to it, and I’m not going to type “sorry” when I change my mind about something. I’m remaining as stone-cold and indifferent when ordering it around as I am when clicking a mouse around an operating system.
I hope to remain vigilant and cautious while I continue testing and learning with local AI. There are real questions about privacy, security, and reliability that still need to be answered. But so far, the experience has been more positive than I expected. The ability to run powerful AI models locally, without handing my data to a cloud service, is a genuinely compelling proposition. Let’s see where it takes me.
