My data stays in this room

On a regular Thursday evening, I realized ChatGPT knew more about my family's minor health issues than what I wanted to share. Thanks to tens of questions I had asked about symptoms I had observed on my children or myself over the past years, it guessed their age range and started to make connections I was uncomfortable to see. Buddy, since when do you know that much about us? You're very good at this game, or I really did a poor job with my temporary chats and "anonymized" questions. I'm very cautious with AI in general and do not overshare or rely on it for everything, but despite my efforts, it was still able to build a solid knowledge graph around my profile. It made me think about how much AI companies know about the average users who talk to AI every day like they would with their doctor/therapist/friend...

AI is an incredible technology that I use daily, mostly at work. However, I can't help but question how the tech is delivered to its users. I mean another cloud technology run by a few tech giants relentlessly collecting user data and dictating AI behavior according to their agendas and principles. Should we continue with these models where we have no control over our data and actually don't possess what we pay for?
Think about it: we are already renting our music and movies, storing our pictures and personal family documents on cloud servers scattered across the globe. Everything is subscription today.

I've personally been using OneDrive for 10+ years to back up my pictures, I'm pretty sure AI models were trained with all my family's faces.

I've been thinking about leaving big tech for a while now but it requires a strong motivation and some effort. Cloud services have at least 2 strong retention points:
1) Convenience: Camera roll auto-synced to the cloud, unlimited on-demand movies at the tap of a finger, not having to think about data storage and all the technical stuff.
2) Price: Most consumer cloud services are cheap and/or offer generous "free" tiers (you pay with your private data, not cash). For the average user who wants to save money and "doesn't have anything to hide", it's a big win. Those more concerned about data privacy have built their digital ecosystem with various things ranging from VPNs, big tech alternatives to mirrored hard drives where they store family or work files locally.
I'm leaning more towards the second category of people described here. Naturally I had already asked myself if I was okay giving more personal data to big tech's AI. The recent experience of finding out ChatGpt knew a thing or two about my family's health convinced me to explore private AI alternatives.

I started researching the topic and happily went down the rabbit hole of local AI. Then, I quickly realized it intersect with PC Gaming as the hardware requirements to build a Gamer PC are very close to those of AI PCs. The journey through articles, Youtube videos and AI chats to pick the right hardware was enjoyable and a bit nostalgic (I owned several PCs for gaming through my teenage years).
I finally picked top-tier components to build a PC not only for AI but also for coding, media hosting, photo and files backup. I was finally going to escape big tech's grip on my data, and yes it comes at a hefty price. Building this machine cost close to €5,000, the GPU alone was €2,500. It's the most important part of an AI PC. I went for a 32Gb Nvidia RTX Pro 4500 Blackwell, along with my 64GB RAM; it's enough to run smart open-source models without internet.

I built everything myself and it was a fun experience. The reality check? Local AI is expensive because the recent AI race has caused RAM and GPU prices to skyrocket. Not everyone can afford it. Local models will never match frontier cloud intelligence; that runs on multi-million dollar datacenters. And you need to know what you're doing: downloading models, installing software, coaxing the GPU into cooperation. But what you get in return is this: free, open source models running 100% locally, no internet required, no subscription, no one watching. This opens up multiple use cases I wanted to explore like teaching my kids to use AI safely, building a family AI agent, coding my side projects with free AI assistance, exploring creative ideas and more.

Honestly, since I set up this machine, I haven't done as much as I anticipated. I still work a demanding day job and family time comes first.

But I'm proud of one thing I built: a family agent that knows us a little. Ask it what brand of cereals we usually buy, or what's on the school timetable next week, and it will tell you. Small things but ours and private. Knowing I can freely ask personal questions and that the answers never leave this room is kind of precious.

No training run somewhere that includes my family's health history. Owning such powerful technology without external dependency and restrictions is something to be treasured, I believe.

For all the philosophy about data sovereignty and open source AI, it comes down to this: I can ask my machine what cereals my kids prefer and nobody else will ever know I asked. €5,000 well spent, if you ask me.

This is part 0 of a series on building and living with a local AI: the hardware, the models, the real use cases, and the bigger question of who should own this technology.