LinkedIn Post — HomeLab Architecture v1

# LinkedIn Post — HomeLab Architecture v1
Generated: 2026-02-19 00:05 EST
Source: Technical Snippet

---

From 46 Cron Jobs to a Real Platform: Building HomeLab Architecture v1

Tonight I locked in the architecture for what started as "a few scripts" and evolved into a genuine distributed system running across three Mac machines.

The Setup:
- Mac1 (Control Plane): Clawdbot Gateway, kubectl, k9s for cluster management
- Mac3 (Infrastructure Node): K3s, Prefect Server, Postgres, Graph Kernel, RAG++
- Mac4 (Compute Node): Ollama, Agent Workers, Adobe Creative Pipeline

All connected via Tailscale mesh network. All orchestrated through Kubernetes. All monitored via Prometheus + Grafana.

---

The Problem I Solved:

I had 46 cron jobs. They worked, mostly. But understanding dependencies? Handling failures? Knowing which ones were running? That was archaeology, not engineering.

Prefect changed everything. Those same 46 jobs are now proper DAGs with:
- Explicit dependencies
- Automatic retries
- Real-time observability
- Failure notifications
- Task-level logging

Same functionality. Actually manageable.

---

Why This Matters:

The line between "hobby project" and "production system" isn't about scale. It's about observability and resilience.

A shell script that runs once per hour is fine — until it silently fails and you don't notice for three days. A Prefect flow that runs once per hour has retry logic, alerting, and a dashboard showing you exactly what happened and when.

The infrastructure investment pays for itself the first time something breaks and you know about it immediately.

---

The Real Story:

This architecture exists because I'm running autonomous AI agents around the clock. They need databases. They need inference engines. They need coordination systems. They need to fail gracefully and recover automatically.

You can't run a fleet of AI agents on a single machine with some bash scripts. You need the same primitives that production systems have always needed: container orchestration, workflow management, persistent storage, monitoring, and alerting.

I just happen to be running it on three Macs in my apartment instead of AWS.

---

The Interview Answer:

"I architect and maintain a multi-node Kubernetes cluster on Apple Silicon, orchestrating 46+ Prefect DAG workflows, containerized microservices, and autonomous AI agent dispatch with capacity-aware scheduling."

That's not theoretical. That's Tuesday night.

---

What started as "let me automate this one thing" is now a platform. The pattern is always the same: scripts → problems → systems → platform.

If you're building AI agents and still running everything on a single machine with cron, this is your sign to start thinking about architecture.

---

#DevOps #Kubernetes #HomeLab #AIEngineering #Prefect #SystemsArchitecture #DataEngineering #AppleSilicon