When I teased “private AI workloads on Moya” in my first post, a few people asked me the same question: why bother?<\/em> Cloud AI is one API call away, it’s cheap, and somebody else deals with the GPUs catching fire. Fair question. Here’s my honest answer, plus what’s actually running.<\/p>\n\n\n\n My whole philosophy with StagNode is tearing things down to understand them. You can’t do that with a hosted model. You send a prompt, you get a response, and the actual mechanics \u2014 the weights, the context handling, the quantization tradeoffs \u2014 are somebody else’s black box. I wanted the black box on my own hardware, where I could open it.<\/p>\n\n\n\n There’s also the boring practical stuff: I didn’t want my home automation, personal notes, or half-finished writing routed through a third party’s servers just to get a summary or a rewrite. Moya already handles my daily production traffic. Adding inference felt like the natural extension of “this is my data, on my infrastructure.”<\/p>\n\n\n\n Moya is my production cluster \u2014 three mini PCs plus two desktop-class machines. The mini PCs handle the lightweight stuff: home automation, internal services, the usual Kubernetes day-to-day. The two desktops are where the real horsepower lives, and they’re the nodes doing all the AI work.<\/p>\n\n\n\n Right now that means:<\/p>\n\n\n\n It’s not trying to compete with frontier hosted models on raw capability. It’s optimized for “good enough, instant, and mine.”<\/p>\n\n\n\n Running this in Kubernetes instead of just SSHing into a box and running a model directly adds real overhead \u2014 scheduling GPU resources properly, making sure pods land on the right nodes, dealing with model files that are tens of gigabytes sitting on persistent volumes backed by Pilot. None of that is free, and there were a few weeks early on where I questioned whether the GitOps purity was worth the friction versus just running it bare-metal.<\/p>\n\n\n\n It was. Mostly because when a node needs to be rebuilt or I want to replicate the setup on Talyn for testing, ArgoCD just handles it. I described this in the first post and it holds true for AI workloads more than almost anything else I run \u2014 model serving configs are exactly the kind of thing you do not want to hand-configure twice.<\/p>\n\n\n\n In the next post I’ll get into the actual mechanics: how I’m handling GPU scheduling on bare-metal Kubernetes nodes, the quantization tradeoffs I landed on, and a few mistakes I made that cost me an entire weekend. If you’re trying to do something similar on your own cluster, that one’s for you.<\/p>\n","protected":false},"excerpt":{"rendered":" When I teased “private AI workloads on Moya” in my first post, a few people asked me the same question: why bother? Cloud AI is one API call away, it’s cheap, and somebody else deals with the GPUs catching fire. Fair question. Here’s my honest answer, plus what’s actually running. The “How Does That Work?” … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10,4,6],"tags":[],"class_list":["post-65","post","type-post","status-publish","format-standard","hentry","category-ai","category-home-lab","category-moya"],"_links":{"self":[{"href":"https:\/\/www.stagnode.com\/index.php?rest_route=\/wp\/v2\/posts\/65","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.stagnode.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.stagnode.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.stagnode.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.stagnode.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=65"}],"version-history":[{"count":1,"href":"https:\/\/www.stagnode.com\/index.php?rest_route=\/wp\/v2\/posts\/65\/revisions"}],"predecessor-version":[{"id":72,"href":"https:\/\/www.stagnode.com\/index.php?rest_route=\/wp\/v2\/posts\/65\/revisions\/72"}],"wp:attachment":[{"href":"https:\/\/www.stagnode.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=65"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.stagnode.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=65"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.stagnode.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=65"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}The “How Does That Work?” Problem<\/h2>\n\n\n\n
What’s Actually Running on Moya<\/h2>\n\n\n\n
\n
The Tradeoffs Nobody Tells You About Upfront<\/h2>\n\n\n\n
What’s Next<\/h2>\n\n\n\n