Historical experiment and deployment correction — October 11, 2026: This is our February 27 experiment, with the October 6 residency/cost correction retained. The roughly $700/month is a reported rental cost; the 80% task share, user-load examples and enterprise price ranges lack a matched acceptance/throughput or current quote dataset. A rented server has a third-party operator. Bedrock routing profiles and Google model/endpoint residency commitments need checking for the actual workload. This diary is not current compliance guidance or proof that regulated data is safe to deploy.
TL;DR:
- I recorded illustrative high-end hosting estimates, not a matched quote and acceptance dataset establishing a universal monthly cost or what is worthwhile for most businesses.
- Cloud-hosted models (AWS Bedrock, Google Vertex, Azure) are the underrated middle ground — you can evaluate hosted models with service-specific contracts, routing and data controls; an account alone is not a residency guarantee.
- I reported a rented AI setup costing roughly $700/month and informally estimated about 80% of our routine use. The account below does not establish measured task coverage or the right setup for another business.
My team and I just spent a few days building our own AI system from scratch. Hosted it on our own cloud server. Ran it alongside the AI tools we use every day — Claude, ChatGPT, all of them.
The goal wasn't to replace those tools. It was to answer a question that's been nagging me: what does it actually take to own your AI and your data?
Short answer: it's more accessible than you think, less powerful than you'd hope, and the real play is somewhere in the middle that almost nobody is talking about.
Let me walk you through everything we found.
The Experiment: What We Actually Built
We rented a server from Google Cloud in the Toronto region (Canada — this matters for where your data physically lives), equipped with a specialized AI chip.
On that server, we installed an open-source AI model called Qwen3 — think of it as a free, publicly available alternative to ChatGPT — and put a familiar chat interface on top so anyone on the team could use it like they'd use ChatGPT or Claude.
Total setup time: a few hours. Total monthly cost: about $700.
And honestly? I was shocked. It just worked.
Not "worked" as in it competes with Claude or GPT frontier models. It doesn't. This is a smaller, less capable model. On complex reasoning, deep analysis, nuanced writing — the top-tier tools are clearly better.
But here's the thing. The routine tasks we informally estimated at about 80% of our own use — drafting emails, summarizing documents, sorting through data, answering internal questions, basic code generation — it handles them fine. Responses come back quickly. Quality is good enough.
I think most of us who work with AI daily have gotten spoiled. We use the most powerful models on the market for everything because the subscription is flat-rate. There's no extra cost to asking the most advanced AI ever built to sort your inbox. But you don't need it for that. You're driving a Ferrari to the grocery store.
Even Anthropic — the company behind Claude — figured this out. Their coding tool uses their smallest, cheapest model as a helper for routine work. If you've seen how fast and efficient that is, you already know: smaller models are shockingly capable for everyday tasks.
We take these frontier models for granted because we're paying flat-rate. But when you actually break it down by task, most of what we do doesn't require the best model on earth.
Why I Even Started Thinking About This
I subscribe to every major AI platform. Claude, ChatGPT, Perplexity, Gemini, Grok — if it exists, I'm probably paying for it.
And I've gotten comfortable giving my data to all of them. Maybe too comfortable.
Here's what sits in the back of my mind: with social media, we gave our data to Google, Facebook, TikTok. We accepted it because what we were sharing — photos, status updates, likes — felt low-stakes.
With AI, it's different. Way different.
We're sharing company strategies. Financial models. Hiring plans. Investor conversations. Legal questions. Personal health concerns. The intimacy of what goes into these chat windows is on a completely different level than anything we've shared with tech platforms before. And we're doing it casually, without thinking twice.
Yes, I opt out of training wherever I can. But let me be real — the risks are there. These companies retain your data. Moderators may be flagging and reading conversations that trigger certain keywords. Humans might be reviewing your chats without you knowing. Retention and access depend on the product, settings and contract. This essay does not establish that every possible review practice is lawful or authorized for a particular workload.
Even on enterprise plans — the expensive corporate versions — your data still sits on their servers. More guardrails, sure. But the data doesn't vanish. It's still there, on someone else's infrastructure, under someone else's control.
So I wanted to know: what does it actually cost to keep it all under your own roof?
The Brutal Truth About Full Self-Hosting
Let me be real — if you want to host an AI that actually competes with the best commercial tools, the math is ugly.
Here's why: the AI models you use through a $20 or $200/month subscription are running on massive clusters of specialized chips that cost tens of thousands of dollars each. You're sharing those chips with millions of other users. The economics work because the cost is spread across everyone.
When you self-host, you're renting or buying those chips just for you. And the bill reflects that.
During this research I considered an illustrative high-end configuration of 8 to 16 GPUs running continuously. Without an exact model, quantization, accepted-task comparison and throughput target, it does not establish what the closest model universally requires:
Minimal setup: historical research estimate of $15,000–$25,000 per month; the exact configuration and dated quote were not supplied here.
Full production setup: historical research estimate of $25,000–$50,000 per month; accepted-task throughput and total running costs were not measured here.
I'll admit — I underestimated these numbers before we started researching. I knew it was expensive. I didn't know it was that expensive.
And that's just for the system to be running. There's another problem most people don't think about at all.
The Speed Problem Nobody Mentions
This one caught me off guard too.
When one person is using a self-hosted AI, it's fast. Responses stream back in seconds.
When 15 people use it at the same time? Still usable. A bit slower, but fine.
50 people simultaneously? You start feeling it. Responses take noticeably longer.
100 people at once? Everyone's waiting. The system is crunching a lot in total, but each individual person's experience gets worse. Noticeably worse.
So if you're a company of 500 employees and 50–100 of them are actively using AI at any given time, you need multiple clusters of high-end chips to keep things feeling responsive. That pushes costs to $50,000–$120,000 per month.
Now compare that to enterprise AI plans from the major providers. Based on reported pricing, OpenAI's enterprise plan runs around $60 per user per month. Anthropic's Claude enterprise is similar. At the illustrative $60 per user per month, 100 users cost $72,000 per year before other charges; actual enterprise terms require a quote. Not per month. Per year.
Those research estimates made enterprise subscriptions look less expensive for the workload I had in mind. They were not matched quotes and accepted-output measurements proving that outcome for every deployment.
What you gain is data sovereignty. Where conversations, documents and logs go depends on the complete deployment and integrations; renting cloud hardware still involves a third-party operator. For most companies, that trade-off doesn't make financial sense. For some, it's non-negotiable. Know which one you are.
The Middle Ground Nobody Talks About
Okay, this is the part I think matters most. And the part almost every guide on this topic skips entirely.
There's a whole tier between "send everything to OpenAI's servers and hope for the best" and "spend $50K a month building your own AI fortress." It's called cloud-provider hosted models. And for most businesses, this is probably the right answer.
Here's the concept in plain terms:
The Big Cloud Providers Now Host AI Models For You
Amazon, Google, and Microsoft have each partnered with the major AI companies. Hosted-model services can provide a different contractual and deployment path. Processing location and controls still depend on the exact model, endpoint, routing profile and service, rather than the account alone.
Amazon Web Services + Anthropic (Claude):
When you use Claude through AWS, the model runs inside Amazon's data centers, not Anthropic's. You pick the region — I'm in Toronto, so I select Canada. Selecting a Canadian source region alone does not prove all processing stays in Canada: model availability, inference profiles and service-specific routing must be verified. Amazon states that customer content is not used to train the AI models. Verify the particular service’s provider access, retention and processing commitments rather than infer them from a cloud account.
Google Cloud + Various AI Models:
Same idea. You deploy AI inside your Google Cloud account, in your chosen region. You control access. Storage and processing commitments depend on the model, endpoint and service; a project alone does not establish a country boundary.
Microsoft Azure + OpenAI:
Same arrangement. You access OpenAI's models through Microsoft's infrastructure without data going to OpenAI directly. If something goes wrong, Microsoft is your provider with enterprise contracts and accountability. You have someone to call.
What This Actually Gives You
Verify where each model request, stored artifact and log is processed or retained, together with access controls, encryption and contract terms. A hosted-model service can offer a different contractual and processing path. The exact model’s provider access and processing locations still require verification; the account choice alone is not a no-provider-access guarantee.
Is it the same as owning the physical hardware yourself? No. The cloud provider still operates the platform. And yes — there's a theoretical risk. A US government data request could potentially reach AWS or Google infrastructure even if it's hosted in a Canadian region. That's real, and worth knowing about.
But whether those controls are sufficient depends on the data, contract, threat model and applicable obligations; this experiment did not measure a 99% coverage rate. Compare accepted-task quality, actual processing boundaries and total cost. Neither residency nor a particular saving follows from the account choice alone.
This is the option I think most businesses should seriously look at. If you're a government entity or defense contractor with strict sovereignty requirements, you might need full self-hosting. It is one option to evaluate, not an established answer for almost every business.
I've worked directly with both AWS and Google Cloud on setups like this. The experience is good. This isn't experimental — it's production-ready infrastructure that major enterprises are already using today.
What Our $700/Month Setup Actually Looks Like
Back to our experiment. We weren't trying to replace the best AI tools. We wanted a private AI layer — something for sensitive work that we don't want flowing through any third-party; hosting on rented infrastructure still involves its operator and our integrations.
Here's the setup:
- Where it runs: Google Cloud, Toronto (Canada) region
- Hardware: A single server with a mid-tier AI chip
- AI model: Qwen3 — an open-source model (for context, the top commercial models are roughly 10-50x larger)
- Interface: An open-source chat interface that looks and feels like ChatGPT
- Monthly cost: ~$700
For a small team of 3–5 people, it handles daily workloads without issues. The responses aren't as sophisticated as Claude or ChatGPT at their best, but for most routine work? They don't need to be.
We use this for things where we want the data staying on our servers. We use frontier models for everything else. Best of both worlds.
That's the key insight here — you don't have to pick one. Run both. Use the right tool for the task. Stop using the Ferrari for groceries.
The Even Cheaper Path: Just Buy the Hardware
If you want to go even further, skip the cloud entirely.
Apple's Mac Mini Pro ($1,500–$2,000) can run capable open-source AI models locally, 24/7, right from your desk or office. Responses will be slower — maybe 10–30 seconds for complex questions — but it works. After purchase, budget for electricity, engineering, review, maintenance, backups and eventual replacement; the amount depends on measured use.
You can also build a dedicated AI server with a specialized chip for about $3,000–$4,000. That may provide a machine you own, depending on the configuration and workload. Electricity, maintenance, connectivity and any external services still have ongoing costs; buying hardware does not guarantee permanent availability or eliminate all providers.
Compare that to $700/month in cloud costs: $3,000–$4,000 equals about 4.3–5.7 months of $700 rental charges before running costs and workload differences. That is a simple charge comparison, not measured break-even or free operation. Owning the hardware changes the cost and control tradeoff; it does not make operation free or independently establish every data-access boundary.
We're planning to try this next — building a physical AI server in-house. It's part of what we're exploring with Sovereign Cloud. If that interests you, I'll share what we learn.
So What Should You Actually Do?
Here's my honest framework. No fluff.
If you're a small team (under 10 people) doing general work:
For our small-team general work, I preferred commercial tools alongside the experiment. The quoted subscription range was historical; compare current terms, limits, accepted-task quality and review time for your own use.
If you handle sensitive client data, financial models, or legal work:
Look at the cloud-hosted options — AWS Bedrock, Google Vertex, Azure OpenAI. Compare the exact model’s quality, routing, retention, access and contract against the actual regulated workload; these products do not establish compliance merely by being enabled in your account. Seriously — look at this. It's what I'd do if I were running a 50-person firm.
If you want a private AI alongside your existing subscriptions:
Do what we did. Spin up a small open-source model on a cloud server or buy the hardware outright. $700/month or a one-time $3,000–$4,000 investment. Our 80%/20% split was informal. Measure accepted outputs, rejected outputs, review time and actual task coverage for your workload before choosing a split.
If you're a government entity or in a strict regulatory environment:
Identify the actual legal and security requirements first. On-premises or specialized hosted options may be relevant, but this experiment cannot establish that every government or regulated organization has only one option.
If you're a company with 50–500+ employees:
Compare actual enterprise quotes with the complete self-hosted cost and accepted-task throughput for your workload. Assess sensitive data against the exact contracts, routing and access requirements. My experiment did not establish that one combination is cheaper for every organization or covers almost everything.
Why This Matters Right Now
Look — AI is becoming the operating system for how companies run. Strategy, communications, code, decisions — more and more of it is flowing through AI. And right now, most of that is flowing through a handful of providers who can see everything.
I'm not saying that's evil. These are good companies building incredible products. I pay for them. I'll keep paying for them.
But the question of who controls your AI infrastructure is going to matter. A lot. It's the same shift we saw with cloud computing a decade ago. The companies that understood it early and moved strategically had an advantage. The ones that waited scrambled to catch up when they had no choice.
Same thing is happening now with AI. Except the stakes are higher, because the data is more sensitive.
Figure this out now. Don't wait until the first breach or regulation forces your hand. By then, you're already behind.
Own your AI, or be owned by whoever does.