Wycliffe Rotich

Wycliffe Rotich

AI infra's part nobody wants to build

About

Backend and distributed systems engineer based in Nairobi, 7+ years across enterprise systems, cloud platforms, and AI infrastructure. I build things that stay correct under real failure conditions, not just in the happy path. Currently building AetherGrid, an open-source distributed AI workload orchestrator with lease-based execution ownership, real subprocess execution, and reconciliation that actually recovers from crashed workers and dead nodes. I also evaluate LLM-generated code for correctness and architecture as an AI training specialist. Always up for talking distributed systems, clean architecture, or anything that breaks under concurrency.

Badges

Tastemaker
Tastemaker

Forums

Building AetherGrid: an AI orchestrator that survives real worker failures

I've spent years chasing race conditions in production systems, revenue systems handling real money, Kafka worker queues, and now reviewing LLM-generated backend code for a living. The pattern is always the same: code works fine until something fails mid-execution, then two processes think they own the same job.

AetherGrid is my answer to that. Job execution ownership runs on time-bound leases that renew continuously for the job's whole runtime, not a boolean flag that lies to you the moment a worker crashes or a heartbeat drops. Real subprocess execution with enforced timeouts, reconciliation that reclaims abandoned work, 280 tests behind it, including one that genuinely kills a process that ignores SIGTERM.

Live now on Product Hunt, launching August 21st. Would love feedback from anyone who's dealt with execution ownership or lease-based systems in their own infra, and happy to answer anything about the architecture.

Repo: https://github.com/wycliffRotich...

AetherGrid - Open-source AI workload orchestrator, lease-based execution

AetherGrid is an open-source AI workload orchestrator built to survive real failure, not demo well. Jobs run under time-bound, continuously-renewed leases, not a simple assignment flag, so a crashed worker can't cause duplicate execution. Real subprocess execution with enforced timeouts, reconciliation that reclaims abandoned work, and 280 tests proving it, including one that force-kills a process ignoring SIGTERM. Clean Architecture, Domain-Driven Design,26 ADRs, live with CI/CD on every merge.
View more