I finally started using local AI — and it didn’t take long to find real, practical uses
I’ll be honest: I’ve been avoiding AI for a long time. Not because I don’t think it’s useful, but because I’ve never been comfortable with the trade-off that most AI services require. To use a cloud-based chatbot or agent, you’re typically expected to hand over your data — your email, your documents, your personal habits, your private thoughts — to someone else’s servers. For me, that’s been a non-starter. I’m not an AI expert by any stretch of the imagination, and this is all going to be a learning process for me. But after seeing how much the landscape has changed, I’ve decided it’s finally time to dive in.
What changed? Local AI. It’s now possible to run powerful language models on your own computer, entirely offline, without sending your personal information to a cloud service. And there’s been a big push behind this idea lately. Apple has made local AI performance a major part of its pitch for the latest Mac desktops. There are also new Windows machines, including a wave of RTX Spark laptops, designed from the ground up for running these kinds of workloads. Some of them come with up to 128GB of RAM, and they’re aimed squarely at the growing interest in so-called agentic AI — systems that can take actions on your behalf instead of just generating text in a chat window.
I’ll be the first to admit that the idea of having my own little AI agent living on my desk, answering only to me, is genuinely appealing. There’s something reassuring about the thought of a digital gofer that isn’t reporting back to a large tech company. I don’t want my queries and personal data going through OpenAI, Google, Microsoft, or Anthropic. And while I still have no interest in using AI to write my articles, edit my copy, or design any kind of promotional material, I can see the potential for it to handle busywork. With privacy as a safeguard, maybe I can finally trust an AI with some useful tasks.
So, as part of my ongoing testing of the M5 Ultra Mac Studio, I’ve started installing local Large Language Models and trying to put them to work. I’m also experimenting on other Mac and Windows systems. I thought I’d share how things are going so far.
Getting started with Hermes Agent
The first thing I did was install an open-source tool called Hermes Agent. It’s a self-hosted AI agent desktop app that works on macOS, Windows, and Linux. And the best part? It’s completely free to use if you stick with local LLMs. Even better, the Mac Studio I’m testing has 256GB of unified memory, which means I can run just about any model I can find. That said, this quickly becomes overwhelming. There are so many models available now that it’s impossible to keep track of them all. Some are specialized for certain tasks, while others are more general purpose. The general rule is that larger models — those with many billions of parameters — are more capable than smaller ones. And because running a model locally doesn’t require paying per token, I decided to start big.
Using Hermes’ onboarding interface and model picker, I chose Qwen 3.8 Flash Next. That’s a 125-billion-parameter model, coming in at around 105GB in size. It’s a hefty download, but the Mac Studio handled it without issue. I also plan to spend some time with much smaller Qwen models on other devices, including an M6 Mac Mini, an M5 MacBook Air, an Asus TUF Gaming A14 with an AMD Strix Halo processor, and eventually an RTX Spark machine. If you’re already using local models and hardware, I’d love to hear what setups are working for you. Feel free to leave a comment.
Getting Hermes up and running with Qwen was surprisingly smooth. I also set it up so I could control the agent remotely from my phone through a Telegram bot. That was neat. But then came the moment I knew was coming. I was sitting there, staring at a text box, and the same question hit me as always: “What do I actually do with this thing?”
Building a daily briefing
To start small, I followed some advice I found on YouTube and decided to set up a daily morning briefing. The idea was simple: use a cron job to have Hermes run a scheduled task every morning. The agent would scan my email and calendar, highlight anything that needs immediate attention, and give me a short weather report. It’s a pretty basic use case, and you definitely don’t need a $12,000 computer to make it work, but it was a good way to make sure the system was functioning.
I ran into a problem at first. The briefing kept failing to generate. It took me a while to figure out why — the Mac Studio was asleep when the scheduled job was supposed to fire at 7:30am. Of course. Once I made sure the computer was awake, it started working like clockwork. The daily briefing isn’t life-changing, but it’s a foundation I can build on. I’m already thinking about what other information I can feed it to make the reports more useful.
Taming a massive Steam library
One of the more genuinely useful tasks I found was reorganizing my Steam library. I’ve accumulated more than 400 games over the years, and I’ve been meaning to sort them for a long time. The problem is that Steam doesn’t categorize games for you. You have to do it manually, one game at a time. With a library that large, the thought of spending hours clicking and dragging was demoralizing.
I asked Hermes, “Can you reorganize my Steam library?” And it turned out that it could. Because I already had the Steam client installed, Hermes was able to see most of my games by looking at the installed application. It quickly offered me a few different organizational strategies. I decided to have it arrange my games by genre, but I asked it to leave my existing custom categories alone — things like favorites, co-op titles, party games, and the games I like to play with my wife.
To make this work, I had to give Hermes permission to interact with my Steam account. That meant registering a Steam Web API key and providing it to the agent. It took a few minutes for Hermes to work through the entire library and sort all the games into appropriate categories, but the result was genuinely good. Once the task was finished, I revoked the API key. I didn’t need it anymore. My Steam library is now organized and easy to browse, and the AI only did the heavy lifting that I didn’t want to do myself. New games I buy in the future will be easy to file into the existing categories.
Handling sensitive data and embargoed specs
Another area where local AI made a difference was data analysis. I had Hermes take a look at some financial records and crunch a bunch of numbers for me. I also asked it to build a spec comparison spreadsheet for a laptop I was evaluating. These might seem like small tasks, but they’re exactly the kind of things I would never trust a cloud-based service with. In one case, the information was sensitive personal data. In the other, the details were still under embargo and I couldn’t risk an external service seeing them. The fact that everything stayed on my own computer made the difference between whether I’d use AI at all.
Automating laptop benchmarks
The bigger project I’m working on now is automation of laptop benchmark testing. At work, we run all of our benchmarks manually. That means setting up a laptop, running a dozen or more tests, and repeating the entire process three times to calculate an average. It’s incredibly time-consuming. I’ve been hoping to automate at least some of that process, and I’m trying to get Hermes to help me write the scripts.
I’ll be honest: getting it to understand all of the required procedures and then produce working Python code has been a challenge. It’s definitely still a work in progress. But I’m making headway, and even if the entire process doesn’t become fully automated, every little bit of assistance helps.
Not magic, just a tool
It’s still early days. I’ve only been working with Hermes for a little while, and I expect that the useful tasks will multiply as I learn more. But it’s already clear that even powerful local models don’t turn into a magical assistant that can do everything. Things still break. My daily briefing has failed multiple times. Some tasks take longer than expected. The agent can be stubborn. That’s all fine. I don’t expect perfection.
I’m trying to keep a clear-eyed perspective. To me, Hermes and local AI are just tools — computer programs running on my computer. I’m not looking for a digital friend. I’m not going to say “please” and “thank you,” and I’m certainly not going to apologize when I change my mind halfway through a request. I’m going to treat it the same way I treat any other piece of software: with a certain level of detachment. I click a mouse, I give commands, and I expect the machine to do its job.
At the same time, I want to remain cautious. The technology is fascinating, but it’s not something to trust blindly. I’ll keep testing, keep learning, and keep an eye on what’s working and what isn’t. Local AI has a lot of potential, and for the first time, I feel like I can explore it on my own terms. Let’s see where this journey takes me.
