Hey Product Hunt! I built Ghostlink to solve a problem I kept hitting while running local LLMs: my compute was scattered across different devices (CPUs, GPUs, and NPUs), but existing orchestration stacks were either too heavy or hard to route across mixed hardware. Ghostlink is a high-performance, open-source distributed inference fabric that routes workloads across local, heterogeneous clusters. Key Highlights: Zero-Copy SPSC Ring Buffers: Sub-microsecond handoff primitives for low latency. Hardware-Aware Placement: Auto-detects and routes across CPU, CUDA, DirectML, Metal, and NPUs. OpenAI-Compatible API: Plug-and-play drop-in replacement for existing LLM tools and workflows. Ghostlink Studio GUI: Modern frontend with real-time hardware telemetry and built-in MCP tool integration. It’s 100% open source under the MIT license. Check out the GitHub repo or run launch.bat / launch.sh to try it out locally. Would love to hear your feedback, thoughts on distributed local inference, or feature requests!

Ghostlink

Run local LLMs across heterogeneous clusters

Run local LLMs across heterogeneous clusters